This is very exciting! (I had suspected it would slip to 114)
WebGPU implementations are still pretty immature, but certainly enough to get started with. I've been implementing a Rust + WebGPU ML runtime for the past few months and have enjoyed writing WGSL.
That said, matmuls are still pretty handicapped in the browser (especially considering the bounds checking enforced in the browser). From my benchmarking I've struggled to hit 50% of theoretical FLOPS, which is cut down to 30% when the bounds checking comes in. (Benchmarks here: https://github.com/FL33TW00D/wgpu-mm)
I look forward to accessing shader cores as they mentioned in the post.
Burn looks interesting... so I notice in the sample code you mention:
use burn_tch::TchBackend;
I don't see "TchBackend" mentioned in the docs. Is Tch = torch and Burn can be used as a Torch (or tch-rs) wrapper? Or am I imagining too much from a miscellaneous three letter prefix?
I've been looking for a good way to do standard modern AI stuff in Rust, and was leaning towards tch-rs because it exists and seems reasonably frequently maintained, but if you have some sort of reason to use Burn instead then I'm curious to hear it.
That's right, Tch stands for torch. We use tch-rs (bindings to torch++).
The selling point of burn is that it offers the full level of deep learning framework with ability to swap backends. This is useful, for example, I can train with Torch backend on GPUs but if I want to deploy, lets say to embedded device, I can use NDArray backend, which supports pure rust code (also with various BLAS acceleration if needed). You can also use NDArray backend with Accelerate (iOS or MacOS blass). You can even compile for WASM because Burn supports inference with no_std (you can try this online demo https://burn-rs.github.io/demo).
The framework is well written and follows Rust's conventions and best practices. It's still evolving, however.
Give it a try. If you find problems, you can file a ticket or join discord chat.
It’s better to compare against an ML framework than to maximum theoretical flops because sometimes it’s not possible to reach. These models are often limited by memory bandwidth rather than flop capability.
Something more than 30 years ago, I had the privilege of working as a young compiler writer for [a supercomputer designer]'s penultimate start-up. I once naively asked him why the machine he was working on couldn't have more memory bandwidth, since the floating-point functional units were sometimes starved for operand data and it was hard to hit the peak Flop/sec figures. And his response has stuck with me ever since; basically, it makes better sense for the memory to be fully utilized, not the floating-point units, because the memory paths were way more expensive than the floating-point units. And this was something you could actually physically see through the transparent top of the system's case. I guess the lesson would be: Don't let a constraint that would be fairly cheap to overdesign be the limiting factor in a system's performance.
In my opinion, ONNX is more complex than necessary. Therefore, I opted to convert it to an intermediate representation (IR) first, which is then used to generate source code. A key advantage of this approach is the ease of merging nodes into corresponding operations, since ONNX and Burn don't share the same set of operators.
Actually WONNX also transforms to an IR first (early versions did not and simply translated the graph 1:1 to GPU shader invocations in topographically sorted order of the graph). In WONNX the IR nodes are (initially) simply (copy-on-write references to) the ONNX nodes. This IR is then optimized in various ways, including the fusion of ONNX ops (e.g. Conv+ReLU->ConvReLU). The newly inserted node still embeds an ONNX node structure to describe it but uses an internal operator.
oh cool! will this be numpy-like or will it have autograd as well? We're looking around for a web backend for shumai[1] and the former is really all we need :)
I guess all the AI type use cases are front seat here but the performance boost increase to Immersive Web (WebXR) and general moves towards webpages as 3D UI/UX for applications -- that is where I hope to see expansion into new creative territory.
what would it take to python -> wasm -> webgpu for the entire existing webgpu ecosystem (all of the libraries around neural networks, torch, yada yada)
The Apache TVM machine learning compiler has a WASM and WebGPU backend, and can import from most DNN frameworks. Here's a project running Stable Diffusion with webgpu and TVM [1].
Questions exist around post-and-pre-processing code in folks' Python stacks, with e.g. NumPy and opencv. There's some NumPy to JS transpilers out there, but those aren't feature complete or fully integrated.
> WebGL was getting really old by now. I do wonder whether WebGPU is a bit late too though (e.g. right now Vulkan decides that PSOs maybe are not a great idea lol)
> As in, WebGPU is very much a "modern graphics API design" as it was 8 years ago by now. Better late than never, but... What's "modern" now seems to be moving towards like: bindless everything (like 3rd iteration of what "bindless" means), mesh shaders, raytracing, flexible pipeline state. All of which are not in WebGPU.
I'm not that versed on details, but would interesting to hear what are the advantages of this modern bindless way of doing things.
Aras is right, but the elephant in the room is still shitty mobile GPUs.
Most of those new and fancy techniques don't work on mobile GPUs, and probably won't for the foreseeable future (Vulkan should actually have been two APIs: one for desktop GPUs, and one for mobile GPUs - and those new extensions are doing exactly that - splitting Vulkan into two more or less separate APIs, one that sucks (for mobile GPUs) and one that's pretty decent (but only works on desktop GPUs).
WebGPU cannot afford such a split. It must work equally well on desktop and mobile from the same code base (with mobile being actually much more important than desktop).
I think it unrealistic management of expectations that desktop and mobile must or should be equal. There is plenty of web applications use cases one would like to run on a desktop, but they are irrelevant for mobile, for many other reasons as well. E.g. think editing spreadsheets.
WebGPU says the baseline should be what is supported on both desktop+mobile, and that extensions (in the future) should enable the desktop-only use cases.
Others seemingly argue that mobile should be ignored entirely, that WebGPU shouldn't work there, or that it should only work on bleeding-edge mobile hardware.
This is an odd analogy. We should reduce the API space for mobile so devs don't make mobile spreadsheets? I mean...what is this arguing exactly? UX is different, sure, but how does that translate into something this low level?
Can you explain what the split is supposed to be? I'm fairly confused because mobile GPUs (tile based) are creeping into the desktop space. The Apple Silicon macs are closer to tile based mobile GPUs than traditional cards.
What APIs are supposed to be separate, why, and what side of the fence is the M1 supposed to land on?
In places where Vulkan feels unnecessarily restrictive, the reason is mostly some specific mobile GPU vendor which has some random restrictions baked into their hardware architecture.
AFAIK it's mostly not about tiled renderers but about resource binding and shader compilation (e.g. shader compilation may produce different outputs based on some render states, and the details differ between GPU vendors, or bound resources may have all sorts of restrictions, like alignment, max size or how shader code can access them).
Apple's mobile GPUs are pretty much top of the crop and mostly don't suffer from those restrictions (and any remaining restrictions are part of the Metal programming model anyway, but even on Metal there are quite a few differences between iOS and macOS, which even carried over to ARM Macs - although I don't know if these are just backward compatibility requirements to make code written for Intel Macs also work on ARM Macs).
It's mostly on Android where all the problems lurk though.
Ah ok, so its not so much the mobile architecture as the realities of embedded GPUs and unchanging drivers compared to more uniform nVidia/AMD desktop drivers.
This is a real problem but I'm not sure splitting the API is a solution. If a cheap mobile GPU has broken functionality or misreports capabilities, I'm not sure the API can really protect you.
Uh, no ; it power and heat management so battery and fire risk that limits SFF -- It would be good for mobile devices to have external GPU/battery attachments via a universal connector... this will boost efficacy of devices... but you may not always need the boost provided by the umbilical - but when you do need it - just put it outside the machine, and connect it when needed...
> with mobile being actually much more important than desktop
How so?
I always thought the more common use case for GPU acceleration on the web for mobile were 2D games (Candy crush etc). Even on low end devices these are already plenty fast with something like Pixi, no?
We live in a bubble where we don't notice it, but desktop as a platform is... not dying exactly, but maybe returning to 90s levels of popularity. Common enough, but something tech-minded people use, and not necessarily for everybody. Mobile is rapidly becoming the ubiquitous computing paradigm we all thought desktop computers would be. In that world, WebGPU is much more important on mobile than on desktop.
I genuinely think personal computing has been severely hamstrung over the past decade+ due to the race to be all-encompassing. Not everything has to be for everyone. It's ok to focus on tools that only appeal to other people in tech. It really is.
A chromebook, internally, is more a mobile device than a "real" computer. Plenty of high school kids today will own their first real computer when they go to college. Until then, most of their computing is done their iPhone or iPad, and perhaps their school-issued chromebook.
We see this issue with kids of their generation entering the workforce with a lack of basic computer skills, or CS students in college who have to be explained the concept of a hierarchical file/directory structure.
> A chromebook, internally, is more a mobile device than a "real" computer
How is that? And if so how am I typing this on an Intel i5 Chromebook with 16G RAM that is hosting a Linux VM? If upgradeability is the issue, Framework's Chromebook is completely upgradeable.
In general, WebGL has more CPU overhead under the hood than WebGPU, so the same rendering workload may be more energy efficient when implemented with WebGPU, even if the GPU is essentially doing the same work.
In bindless (pointers) you say "at this GPU memory location I have a texture with this params".
In non-bindless you say "API create a texture with these params and give me a handle I will later use to access it".
Bindless gives you more flexibility, but it's also harder to use since it's now your responsability to make sure those pointers point at the right stuff.
It's a bit more complex than that. In classical OpenGL (and thus WebGL) "bindless" is more significant: You had to bind resources to numbered stages like TEXTURE2 in order to render, so every object with a unique texture required you to make a bunch of API calls to switch the textures around. People rightly rejected that, which led to bindless rendering in OpenGL. Even then however you still had to create textures, the distinction is that you no longer had to make a billion API calls per object in order to bind them.
Critically however, things like vertex buffers and fragment/vertex shaders are also device state in OpenGL, and bindless textures don't fix that. A fully bindless model would allow you to simply hand the driver a bundle of handles like 'please render vertices from these two vertex buffers, using this shader, and these uniforms+textures' - whether or not you have to allocate texture handles first or can provide the GPU raw texture data is a separate question.
How badly can you wreck state in bindless? Badly enough to see the pointers of another process or detect a lot of high-detail information on what computer is running the program?
If so, that'd be a non-starter for a web API. Web APIs have to be, first and foremost, secure and protect the user's anonymity.
“The web” should not first and foremost protect anonymity. It should do what humans need it to do ideally while keeping users private and secure. If there’s a concern, my browser should ask me if I’m willing to share potentially sensitive information with a product or service. I fucking hate this weird angsty idea that the web is only designed for anonymous blobs and trolls.
Letting advertisers identify you through some web accessible GPUs interface so they can track your every move and sell the data to all comers … won’t help you fight anonymous online trolls.
All of this is in the context of a browser. If a misbehaving web app uses pointers for memory from another process, that should be blocked by all of the same things that prevent non-privileged apps from doing the same thing.
Yeah, WebGPU unfortunately ended up becoming an outdated mobile phone graphics API on arrival. Still better than WebGL, but not quite what I would have liked it to be.
This is a huge milestone. It's also part of a much larger journey. In my work on developing Vello, an advanced 2D renderer, I have come to believe WebGPU is a game changer. We're going to have reasonably modern infrastructure that runs everywhere: web, Windows, mac, Linux, ChromeOS, iOS, and Android. You're going to see textbooks(*), tutorials, benchmark suites, tons of sample code and projects to learn from.
WebGPU 1.0 is a lowest common denominator product. As 'FL33TW00D points out, matrix multiplication performance is much lower than you'd hope from native. However, it is possible to run machine learning workloads, and getting that performance back is merely an engineering challenge. A few extensions are needed, in particular cooperative matrix multiply (also known as tensor cores, WMMA, or simd_matrix). That in turn depends on subgroups, which have some complex portability concerns[1].
Bindless is another thing everybody wants. The wgpu team is working on a native extension[2], which will inform web standardization as well. I am confident this will happen.
The future looks bright. If you are learning GPU, I now highly recommend WebGPU, as it lets you learn modern techniques (including compute), and those skills will transfer to native APIs including Vulkan, Metal, and D3D12.
Disclosure: I work at Google and have been involved in WebGPU development, but on a different team and as one who has been quite critical of aspects of WebGPU.
(*): If you're writing a serious, high quality textbook on compute with WebGPU, then I will collaborate on a chapter on prefix sums / scan.
This depends entirely on your goals. If you're researching the actual machine learning algorithms, then use a framework like TensorFlow or Torch, which provides all the tensor operations and abstracts away the hardware. If you're trying to get maximum performance on hardware today, stick with Nvidia and use CUDA. If you're interested in deploying across a range of hardware, or want to get your hands dirty with the actual implementation of algorithms (such as wonnx), then WebGPU is the way to go.
If you're eager to learn WebGPU, consider checking out Mach[0] which lets you develop with it natively using Zig today very easily. We aim to be a competitor-in-spirit to Unity/Unreal/Godot, but extremely modular. As part of that we have Mach core which just provides Window+Input+WebGPU and some ~19 standalone WebGPU examples[1].
Currently we only support native desktop platforms; but we're working towards browser support. WebGPU is very nice because it lets us target desktop+wasm+mobile for truly-cross-platform games & native applications.
Closer to 3.5 years now, maybe I'll do an update at 4 years.
The 2.5 year initial duration (when I posted that article) was like working two full-time jobs, 100h/week, 80% writing code.
Past year has been more like 85h/week, my dayjob has required more time (48h -> ~60h), now with only 30% of that being /writing code/. ~25h/week going to my gamedev passion projects, of which a good chunk has also gone to helping others with their code/issues.
The -15h/week 'loss' has gone to caring for pets, and a gentle touch more travel/sleep/gardening.
Hey, it's the man himself! I'd love to read a followup post if you have the time (hehe) to write it. I'd also be curious to read about how you spend your non-working hours in more detail!
I'm actually keeping a close eye on mach after seeing your talk about gkurve. That has made GPU accelerated 2D graphics look much more approachable to me.
I plan to experiment with that after I get a better understanding of the WebGPU c API.
> WebGPU is a new API for the web, which exposes modern hardware capabilities and allows rendering and computation operations on a GPU, similar to Direct3D 12, Metal, and Vulkan. Unlike the WebGL family of APIs, WebGPU offers access to more advanced GPU features and provides first-class support for general computations on the GPU.
And to prevent device fingerprinting, all the operations are specified to deterministically produce the same bit-exact results on all hardware, and the feature set is fixed without any support for extensions, right?
Or is this yet another information leak anti-feature that we need to disable?
Just one example: A script which runs many different types of computations. Each computation will take a certain amount of time depending on your hardware and software. So you will get a fingerprint like this:
Just put WebGL/WebGPU behind permission and the problem is solved. I don't understand why highly paid Google and Firefox developers cannot understand such a simple idea.
For a user to correctly answer a permissions dialog, they need to learn programming and read all the source code of the application. To say nothing of the negative effects of permission dialog fatigue.
In practice, no-one who answers a web permissions dialog truly knows if they have made the correct answer.
Asking the user a question they realistically can't answer correctly is not a solution. It's giving up on the problem.
I think browsers should distinguish more aggressively between "web application", "web site", and "user hostile web site".
Many APIs should be gated behind being a web application. This itself could be a permission dialog already, with a big warning that this enables tracking and "no reputable web site will ask for it unless it is clear why this permission is needed - in doubt, choose no".
Collect opt-in telemetry. Web sites that claim to be a web application but keep getting denied can then be reclassified as hostile web sites, at which point they not only lose the ability to annoy users with web app permission prompts, but also other privileges that web sites don't need.
Clearly if we knew how to perfectly identify user hostile websites we'd not need permissions dialogs at all.
Distinguishing between site and app, e.g. via an installation process, is equivalent to a permissions dialog, except that you're now advocating for one giant permission dialog instead of fine-grained ones, which seems like a step backwards.
Yes, if we knew how to do it perfectly, we wouldn't need them. But we can identify some known-good and known-bad cases with high confidence. My proposal mainly addresses the "fatigue" aspect: it allows apps to use some of the more powerful features without letting every web site use them, and it prevents random web sites from declaring themselves an app and spamming users with the permission request just so they can abuse the users more.
The new permission dialog wouldn't grant all of the finer-grained permissions - it would be a prerequisite to requesting them in the first place.
Curating known good would equate to some sort of app store. There are probably initiatives to make one for web apps, but it kind of makes me sad to think of applying that to the web, which is supposed to be a free and open commons (although I suppose Google already de facto controls enough of it to be considered a bit of a gatekeeper).
Making the user the arbiter of "known good", ie reliance on permissions dialogs, is not perfect but it's what we have. Yet I fail to see how your proposal of "just add ANOTHER dialog" improves the situation.
Do you have something specific in mind with your opening paragraph?
Because defining what is a web site and what's an app, strikes me as particularly impractical idea. You correctly point out that yes, there are a number of powerful APIs that should be behind permissions. But there are a number of permissions already, so we need to start bundling them and also figure out how to present all this to the regular user.
Frankly, I wouldn't know where to begin with all this.
News sites are a particular category that I expect to spam people with permission prompts, as they did when notifications became a thing. Without the deterrent of possibly landing in the naughty box, they'd all do it. With it, I still expect some of them to try until they land in the box.
> In practice, no-one who answers a web permissions dialog truly knows if they have made the correct answer.
Counterpoint: if webpage with latest news (for example) immediately asks me to allow notification, access to webcamera and location I definitely know what is correct answer to these dialogs.
"Do you want to allow example.com to send you notifications" is way more understandable to a layperson than "do you want to allow access to WebGPU" or "do you want to allow access to your graphics card". Especially because they would still have access to canvas and WebGL.
Permission prompts are a HUGE user education issue and also a fatigue issue. Rendering is widely used on websites so if users get the prompt constantly they're going to tune it out.
They don't need to learn programming. Just write that this technology can be used for displaying 3D graphics and fingerprinting and let user decide whether they take the risk.
They're going to be confused if you say "display 3D graphics", because canvas and WebGL will still work. The website will just be laggier and burn their battery faster. That's not going to make sense to them.
"Fingerprinting" is a better approach to the messaging, but is also going to be confusing since if you take that approach, almost all modern permissions are fingerprinting permissions, so now you have the problem of "okay, this website requires fingerprinting class A but not fingerprinting class B" and we expect an ordinary user to understand that somehow?
Most of them will say, "I need to see this site, who cares about fingerprints." Some will notice that they're on their screen anyway, a few will know what it's all about.
Maybe "it can be used to display 3D graphics and to track you", but I expect that most people will shrug and go on.
The page implies it no longer requires permissions, but I just tested and you definitely get a permissions popup, just a different one.
WebHID, WebUSB and Filesystem Access are IIRC, "considered harmful" so they won't get implemented. And Sensor support was removed after sites started abusing battery APIs.
I'm not. It's a bit of a sarcasm (?) listing a subset of APIs that browsers implement (or push forward against objections like hardware APIs) and that all require some sort of permission.
> but this is exactly the path Firefox was advocating
Originally? Perhaps. Since then Firefox's stance is very much "we can't just pile on more and more permissions for every API because we can't properly explain to the user what the hell is going on, and permission fatigue is a thing"
Everything except WebGL and WebGPU allows the system to change more state than what is rendered on a screen.
Users already expect browsers to change screen contents. That's why WebGPU / WebGL aren't behind a permission block (any moreso than "show images" should be... Hey, remember back in the day when that was a thing?).
Saturating the user with permissions requests for every single website they visit is a dead-end idea. We have decades of browser development and UI design history to show that if you saturate the user with nag prompts that don't mean anything to them, they will just mechanically click yes or no (whichever option makes the website work).
Permission popups can be replaced with an additional permission toolbar or with a button in the address bar user needs to click. This way they won't be annoying and won't require a click to dismiss.
Like the site settings page on Chrome, which is in the address bar (clicking the lock icon)? You can set the permissions (including defaults) for like 50 of these APIs.
We already have extensions for websites that spam the user with unwanted popups and other displays. Those just need to be extended to cover permission abuse and be included by default in all webbrowsers.
I do this since forever, but I have to give explicit permission to load and run JS, which solves a lot of other problems as well. Letting any site just willy-nilly load code from whereever and run it on your machine is insane, and it's well worth the effort to manually whitelist every site.
It's not that they don't understand it, it's that they don't want the average user to have a convenient way to control this setting. Prompting the user for permission would give the user a very convenient way to keep it disabled for most websites. It's as simple as that.
Think about it this way: Which is more tedious: going into the settings and enabling and disabling webGPU every time you need it or a popup? Which way would see you keeping it enabled?
"Completely co-incidentally", it's in Google's best interest to be able to fingerprint everyone.
So, changing it to actually be privacy friendly while they have the lion's share of the market doesn't seem like it's going to happen without some major external intervention. :/
It's running on Chrome. Google doesn't need fingerprinting. By making it harder for others to fingerprint it actually cements Google position in the ad market.
Just don’t use Chrome. There are plenty of alternative web browsers you can choose that are more privacy oriented. You are not Chrome’s customer unless you pay for it - or you have 100% money back guarantee. Demanding features on free product is never going to go anywhere.
You can reduce clock precision, which has already been done to mitigate speculative execution attacks. You can delay network requests to prevent the JS from using the server as a more precise clock. In addition to random delays, you can quantize execution times by only responding in 100ms increments, for example. You can do lots of things to mitigate fingerprinting, if not completely prevent it.
But then you could also just omit features that have no reason to exist in the first place.
You only get fingerprinting from your method if the variation of the “fingerprint“ between two different runs by the same user is lower than the difference you get between two different users. This is far from obvious since it depends a lot on the workload running on the machine at the time.
I'm not aware of a single fingerprinting tool that primarily use this king of timing attack rather than more traditional fingerprinting methods.
We would have to make examples of what Computation1 is and what Computation2 is to make a prediction if certain types of workloads will impact the ratio of their performance.
Example:
s=performance.now();
r=0;
for (i=0; i<1000000; i++) r+=1;
t1=performance.now()-s;
s=performance.now();
r=0;
for (i=0; i<1000000; i++) r+="bladibla".match(/bla/)[0].length;
t2=performance.now()-s;
console.log("Ratio: " + t2/t1);
For me, the ratio is consistently larger in Chrome than in Firefox. Which workload would reverse that?
Fingerprinting in the usual sense the term isn't about distinguishing Chrome from Firefox, it's about distinguishing user A from user B, … user X reliably in order to be able to track the user across website and navigation sessions.
Your example is unlikely to get you far.
Edit: in a quick test, I got a range between 8 and 49 in Chrome, and between 1.27 and 51 (!) on Firefox, on the same computer, the results are very noisy.
> Chrome and Firefox here are an example for "Two users who use exactly the same hardware but different software".
But it's also the most pathological example one can think of, yet the results are extremely noisy (while being very costly, which means you won't be able to make a big number of such test without dramatically affecting the user's ability to just browse your website).
Sure. And nobody actually wants that, because it would be so restrictive in practice that you might as well just limit yourself to plain text.
The horse bolted long ago; there's little sense in trying to prevent future web platform features from enabling fingerprinting, because the existing surface that enables it is way too big to do anything meaningful about it.
Here are a couple of more constructive things to do:
- Campaign to make fingerprinting illegal in as many jurisdictions as possible. This addresses the big "legitimate" companies.
- Use some combination of allow-listing, deny-listing, and "grey-listing" to lock down what untrusted websites can do with your browser. I'm sure I've seen extensions and Pi-hole type products for this. You could even stop your browser from sending anything to untrusted sites except simple GET requests to pages that show up on Google. (I.e. make it harder for them to smuggle information back to the server.)
- Support projects like the Internet Archive that enable viewing large parts of the web without ever making a request to the original server.
This would essentially mean that every computation would have to run as slow as the slowest supported hardware. It would completely undermine the entire point of supporting hardware acceleration.
I’m sympathetic to the privacy concerns but this isn’t a solution worth considering.
Everything that can be used for fingerprinting should be behind a permission. Almost all sites I use (like Google, Hacker News or Youtube) need none of those technologies.
Main thing that ought to be behind a permission is letting Javascript initiate connections or modify anything that might be sent in a request. Should be possible, but ought to require asking first.
If the data can't be exfiltrated, who cares if they can fingerprint?
Letting JS communicate with servers without the user's explicit consent was the original sin of web dev, that ruined everything. Turned it from a user-controlled experience to one giant spyware service.
If javascript can modify the set of URLs the page can access (e.g. put an image tag on the page or tweak what images need to be downloaded using CSS) then it can signal information to the server. Without those basic capabilities, what's the point of using javascript?
No video driver is actually going to implement fixed-time rendering. So you'd have to implement it in user-space, and it would be even slower than WebGL. Nobody wants that. You're basically just saying the feature shouldn't ship in an indirect way (which is a valid opinion you should just express directly.)
I don't mean to prescribe the way to stop fingerprinting, just throwing out a trivial existence proof, and maybe a starting point of thinking, that it's not impossible like was suggested.
Also, WebGPU seems to conceptually support software rendering ("fallback adapter"), where fixed time rendering would seem to be possible even without getting cooperation from HW drivers. Being slower than WebGL might still be an acceptable tradeoff at least if the alternative WebGL API avenue of fingerprinting could be plugged.
Could you explain what techniques would make this possible? I can see how it's possible in principle, if you, say, compile JS down to bytecode and then have the interpreter time the execution of every instruction. I don't immediately see a way to do it that's compatible with any kind of efficient execution model.
The rest would be optimization while keeping the timing sidechannel constraint in mind, hard to say what the performance possibilities are. For example not all computations have externally observable side effects, so those parts could be executed conventionally if the runtime could guarantee it. Or the program-visible clock APIs might be keeping virtual time that makes it seem from timing POV that operations are slower than they are, combined with network API checkpoints that halt execution until virtual time catches up with real time. Etc. Seems like a interesting research area.
>not all computations have externally observable side effects
You can time any computation. So they all have that side effect.
Also, from Javascript you can execute tons of C++ code (e.g. via DOM manipulation). There's no way all of that native code can be guaranteed to run with consistent timing across platforms.
Depends on who you mean by "you". In context of fingerprinting resistance the timing would have to be done by code in certain limited ways using browser APIs or side channels that transmit information outside the JS runtime.
Computations that call into native APIs can be put in the "has observable side effects" category (but in more fine grained treatment, some could have more specific handling).
In this case the runtime would not be able to guarantee that the timing has no externally observable side effects (at least if you do something with t). It would then run in the fixed execution speed mode.
Lots of code accesses the current time. So I think you'd end up just running 90% of realistic code in the fixed execution speed mode, which wouldn't be sufficiently performant.
It's hard to reason about how much noise is guaranteed to be enough, because it depends on how much measurement the adversary has a chance to do, there could be collusion beween several sites, etc. To allow timing API usage I'd be more inclined toward the virtual time thing I mentioned upthread.
I wish all information-leaky browser features were turned off by default and I could easily turn them on on demand when needed. Like, the browser could detect that a webpage accesses one of them and tells me that I am currently experiencing a degraded experience which I could improve by turning this slider on.
I've set up my Firefox with resistFingerprinting but without an auto deny on canvas access.
It's sickening to see how often web pages still profile you, but the setting seems to work.
Similarly, on Android there's a Chromium fork called Bromite that shows JIT, WebRTC, and WebGL as separate permissions, denied by default. I only use it for when broken websites don't work right on Firefox, but websites seem to function fine without all those permissions being enabled by default.
Competent websites will tell you the necessary settings ("WebGL is not available") so making the websites work isn't much trouble. I'd much rather see those error messages than getting a "turn on canvas fingerprinting for a better experience" popup from my browser every time I try to visit blogs or news websites.
Right. But I don't want to have to dig into settings hierarchies for those knobs. The threashold for that is too high and almost nobody will bother and do that. Something easier with simple sliders would be much better.
The trade-off for extracting maximum performance from a user's hardware is that it becomes much easier to fingerprint.
Judging by the history of the web this is a trade-off that probably isn't worth making.
I track population frequency of WebGPU extensions/limits here: https://web3dsurvey.com. The situation currently is much better when WebGL1/WebGL2 but there is still a lot of surface area.
Interesting. The data shows that WEBGL_debug_rendered_info [1] which allows sites to know the name of your graphic card, is supported almost in 100% of browsers. Seems that better fingerprinting support is really a priority among all browser vendors.
This is sadly a requirement for the time-honored game development tradition of "work around all the bugs in end user drivers", which also applied to WebGL while it was still immature.
At this point there's probably no excuse for continuing to expose that info though, since everyone* just uses ANGLE or intentionally offers a bad experience anyway.
Fingerprinting is a very difficult and unreliable way of identifying users. You would not bank on fingerprinting to protect your money. You cannot bank on it to protect user info. You can just wish that you are targeting the right person.
The animals already fled the barn on that one, WebAssembly floating point is not specified to be bit-exact, so you can use WASM FP as a fingerprinting measure (theoretically - I don't know under which configurations it would actually vary.)
As long you can keep it off or turn it off then I think this is a good option to have.
I too would prefer to have the Web split into 2 parts, documents and apps , then I could have a browser that optimizes for JS , GPU speed and a simple safe browser for reading Wikipedia and articles.
I am sure there will be browsers that will not support this or keep it off so at worst you need to give up on Chrome and use a privacy friendly browser.
> all the operations are specified to deterministically produce the same bit-exact results on all hardware,
I want this so badly. A compiler flag perhaps, that enables running the same program with the exact output bit for bit on any platform, perhaps by doing the same thing as a reference platform (any will do), even if it has a performance penalty.
I'm surprised people accept non-bit-identical output. Intel did a lot of damage here with their wacky 80-bit floating point implementation, but really it should be the norm for all languages.
Trying to reproduce is a good point, but at the same time it’s usually a pretty bad idea to do tests of the form assert(result == expected) with a floating point result though. You’re just asking for trouble in all but the simplest of cases. Tests with floating point should typically allow for LSB rounding differences, or use an epsilon or explicit tolerance knob.
There’s absolutely no guarantee that a computation will be bit-identical even if the hardware primitives are, unless you use exactly the same instructions in exactly the same order. Order of operations matters, therefore valid code optimizations can change your results. Plus you’ll rule out hardware that can produce more accurate results than other hardware if we demand everything be bit-identical always, it will hold us back or even regress. Hardware with FMA units are an example that produce different results than using MUL and ADD instructions, and the FMA is preferred, but hardware without FMA cannot match it. There are more options for similar kinds of hardware improvements in the future.
> Order of operations matters, therefore valid code optimizations can change your results.
This is exactly why optimizations that change the order of operations of floating points aren't valid! And many other optimizations, like (I learned this just recently) transforming x + 0.0 into x: those are not the same thing when x is -0.0. In other news, -ffast-math produces broken code.
Current programming languages enable writing 100% deterministic floating point code just fine (even with compiler optimizations, as long as they are not buggy). The trouble is writing cross-platform deterministic floating point code, that works the same in every machine, but with great care it still can be done, as in https://rapier.rs/docs/user_guides/rust/determinism/ (well this project does this for every platform that supports IEEE 754-2008)
Cross-platform bit-matching determinism is a tradeoff. It’s not a correctness or accuracy issue. It’s one of many goals one might have, and it comes with advantages and disadvantages. Like I pointed out above, you may be trading away higher accuracy in order to achieve cross-platform determinism. You also trade away performance almost certainly.
You say “aren’t valid” and “broken code” as though it’s somehow factual, when in reality you’re making opinionated assumptions about your choice of tradeoff. Those opinions are only true if you assume that only bit-matched results are “valid”. This hyperbolic wording breaks down a little once we start talking about the accuracy of floating point calculations and how bit-matching FP calculations on two different machines is just making two wrong values agree, and there’s nothing “exact” about it.
It is 100% absolutely fine to have bit-matching determinism as a goal, and I’m in favor of compilers supporting it. I’m not suggesting anyone shouldn’t, but I hope you recognize your language is implicitly demanding that everyone must care about floating point determinism just because you do. Some people have serious floating point calculations where they want cross-platform determinism, but -ffast-math exists precisely because many people do not need it, or because they simply prioritize performance over bit-matching, or because they engineered with epsilons instead of unrealistic expectations. There are good reasons why Rapier’s cross platform determinism is not the default, right?
Generally speaking, even the people who have strong reasons to want bit-matching results on different hardware, because they understand the nature of floating point and the reality of the hardware landscape, do not depend on it to be true, they still write their tests using tolerances.
The easiest option to prevent fingerprinting is to disable WebGPU. Or even better, which is already the option for many today, use one of privacy focused web browsers instead of Chrome.
Meanwhile there is a large audience who will benefit from WebGPU features e.g. gamers and this audience is in the numbers of hundreds of millions.
Google are the people paying for Chrome, they do not benefit in any way from this kind of fingerprinting. To the contrary, it decreases the value of their browser monopoly.
They don't benefit from fingerprinting, because the browser has all sorts of easier to track mechanisms available by default. Fingerprinting is for browsers that don't actively enable tracking.
Even supposing that Google do benefit from it in that manner, there would be far simpler ways for them to make fingerprinting easier. It's extremely unlikely that this is a significant motivation for adding WebGPU. Not to mention that a lot of the fingerprinting you can potentially do with WebGPU can already be done with WebGL.
Google has many hands in many pots. It's not that they are necessarily looking for easier ways to do fingerprinting. But they sure as hell wouldn't put up a fight to make it harder.
Chrome has to walk a fine line between what it does for privacy and what is says it does. So you have the protection against fingerprinting and at the same time you have the FLoC fiasco
The simplest explanation is that the Chrome developers genuinely want to protect privacy and also genuinely want to add features. Every browser has to make that trade off. There are plenty of fingerprinting vulnerabilities in Firefox and Safari too.
The reasoning here seems to be something like "Google is evil; X is an evil reason for doing Y; therefore Google must have done Y because of X". It's not a great argument.
I can only quote Johnathan Nightingale, former executive of Mozilla, from his thread on how Google was sabotaging Firefox [1]:
"The question is not whether individual sidewalk labs people have pure motives. I know some of them, just like I know plenty on the Chrome team. They’re great people. But focus on the behaviour of the organism as a whole. At the macro level, google/alphabet is very intentional."
That whole Twitter thread says nothing about fingerprinting or privacy. The first comment is close to gibberish, but seems to be mostly about some kind of Google office development project in Toronto.
You are literally following the parody argument schema that I mentioned in my previous comment. You make some vague insinuations that Google is evil, then attribute everything it does to non-specific evil motivations. Even if Google is evil, this kind of reasoning is completely unconvincing.
> That whole Twitter thread says nothing about fingerprinting or privacy.
I should've been more clear. In this case I was responding to this: "The reasoning here seems to be something like "Google is evil; X is an evil reason for doing Y; therefore Google must have done Y because of X". It's not a great argument."
> You are literally following the parody argument schema that I mentioned in my previous comment.
Because you have to look at the behaviour of the organism as a whole. If the shoe fits etc.
What is not acceptable is the use of opaque or hidden techniques that transfer data about individual users and allow them to be tracked in a covert manner, such as fingerprinting. We believe that any attempts to track people or obtain information that could identify them, without their knowledge and permission, should be blocked. We’ll continue to take a strong position against these practices. -- https://blog.google/products/ads-commerce/improving-user-pri...
Google Ads, 2021-03-03:
Today, we’re making explicit that once third-party cookies are phased out, we will not build alternate identifiers to track individuals as they browse across the web, nor will we use them in our products. -- https://blog.google/products/ads-commerce/a-more-privacy-fir...
(I used to work on ads at Google, speaking only for myself)
1. In the context of browsers Google's competitors are Safari and Firefox. And in this context Google is always consistently behind: either unwilling to implement the same privacy protections, or implementing them years later, or coming up with non-solutions
2. It's funny how you link to a Google propaganda piece on FLoC. Whereas Google's competitors (context: browsers) actually try to reduce fingerprinting, tracking, and thrid-party cookies, Google is trying to have the cake and eat it too with FLoC. Which was such a blatant attempt to keep fingerprinting and tracking alive that everyone immediately disabled it within months of Google's experiments with it.
Edit: Tracking and fingerprinting is Google's bread and butter, literally: 80% of its money comes from targeted advertising.
If you're trying to understand what Google's doing here and what their incentives are, it's important to distinguish between tracking in general and specifically using fingerprinting to track. They're very interested in showing people relevant ads based on their history, but only in ways where users have some control. With the traditional approach of third-party cookies, for example, the user can clear some or all cookies, open a private browsing window, or use extensions to limit what cookies are sent/received where. With fingerprinting, however, the user has no control: if I clear cookies I'll still have the same fingerprint, and I can't tell the web to forget me anymore. Same if I open a private browsing window, close it, and open it again. We started this thread with the question of whether Chrome adding an API that increased the fingerprinting surface benefited Google, and I've been arguing no: as shown in my quotes above Google has committed not to use fingerprinting.
Your (1) and (2) are about tracking in general and not fingerprinting. On (1), I agree that Google is behind. This is explicitly a strategy to (a) protect ads monetization and (b) avoid a situation where you turn off third party cookies only to have advertisers move to something worse (see: being anti-fingerprinting):
After initial dialogue with the web community, we are confident that with continued iteration and feedback, privacy-preserving and open-standard mechanisms like the Privacy Sandbox can sustain a healthy, ad-supported web in a way that will render third-party cookies obsolete. Once these approaches have addressed the needs of users, publishers, and advertisers, and we have developed the tools to mitigate workarounds, we plan to phase out support for third-party cookies in Chrome. -- https://blog.chromium.org/2020/01/building-more-private-web-...
On (2), while FLoC abandoned the successor, Topics, is still moving forward: https://developer.chrome.com/docs/privacy-sandbox/topics/ Note that unlike FLoC it only observes pages where the page calls "document.browsingTopics()". I don't see how FLoC or Topics represent trying to "have the cake and eat it too" -- they're explicitly attempts to move user interest tracking from the server to the client, to address some of the privacy issues people have with server-side tracking.
On "literally: 80% of its money comes from targeted advertising" that's wrong? The vast majority of Google's income is from ads, yes, but it's mostly from search ads, which aren't targeted.
(I used to work in this area at Google; speaking only for myself)
I believe webGL "cryptojacking" as it's called, indeed is a thing. Not sure on prevalence though or to what extent this introduction makes it more viable for malicious actors.
I'm not sure if lots of hashing algos are gpu-ready or optimized either.
Sadly all of this comes at cost of necessaty to lear new webgl and new API whic isn't good. I would expect to have something similar to a Vulkan one. This is not good.
I remember when WebRTC was introduced, it was found that it became very popular and was used by almost every page. But closer inspection showed that its use was to get user's IP address for better fingerprinting.
I predict that WebGL/WebGPU will be mainly used for the same purposes. Nobody needs new fancy features, what people really need is more reliable fingerprinting (this is proven by number of uses of WebRTC for fingerprinting vs number of uses for communication).
People tend to forget that the good part about Flash was that it offered all this (there was an API for low-level high-performance 3D graphics, Stage3D), except there was also an explicit boundary between the "document" and the "app" and most browsers offered an option to only load Flash content when you click on its area. I'm thus still convinced that turning browsers themselves into this application platform kind of thing is a disaster. The industry is going very wrong and dangerous way right now.
It's never too late to delete all this code and pretend it never happened though. I want to see a parallel reality where Flash became an open standard, with open-source plugins and all that. It is an open standard and there is at least one project of an open source Flash player (Ruffle), but it's too late. I still hope Flash makes a comeback though.
> and most browsers offered an option to only load Flash content when you click on its area
> there is at least one project of an open source Flash player (Ruffle)
Just so you know, Ruffle supports (optionally) autoplay without clicking (this has some restrictions, like no sound until it gets a user interaction) _and_ we do plan to try running on webgpu very soon - so unfortunately we rely on this one way or another :(
(also there are other projects, Ruffle just happened to get the best word-of-mouth PR)
Last I knew, the Chrome team was made out of people. If the people working on a new technology are excited about that technology, it probably means that they are excited about that technology. It doesn't say much about how excited the ad people may or may not be about it, other than indirectly by knowing that the technology people are allowed to spend time on it.
That is the link I believe you're referring to, but my guess is that it's going to be fairly tenuous. The set of things that the technology people get to work on probably has more to do with the set of things that they try to persuade the powers that be are important, and that set is more likely to be driven by their own interests and visions of what could and should be done on the web, than it is to be driven by what will make their paymasters happiest.
Sorry, I'm sure there is a much more brief way to say that.
> But closer inspection showed that its use was to get user's IP address for better fingerprinting.
Maybe that's why it fell to the wayside: scripts are no longer allowed to get the local IP address (taking with it the most useful aspect of WebRTC, true serverless p2p without internet[1]).
[1] I'm not saying that I disagree with the decision, but still sad that we can't have nice things :(
i want the chrome apps model to come back - put your permission requests in a manifest, and when the user clicks and install button the app gets its permissions. so "web apps" that the user cares enough about to install get useful features, but pages that you just visit briefly don't.
> i want the chrome apps model to come back - put your permission requests in a manifest, and when the user clicks and install button the app gets its permissions.
So you will get the Android case where flashlight apps where asking for everything, including location data and contact access, and people were giving it to them
you can't protect everybody from themselves all the time.
some people are just gonna agree to everything, and you can't stop it. don't ruin apps for everybody just because some guy who couldn't care less shared some data with an app you think they shouldn't have.
> you can't protect everybody from themselves all the time.
It's not an either-or thing. There are multiple levels to security, and just shoving everything into one big prompt, and letting users deal with it ain't it.
> taking with it the most useful aspect of WebRTC, true serverless p2p without internet[1].
Would you mind elaborating? In any case WebRTC always needed some kind of third party server to connect the peers together (sure, such a server can be on your local network), and then they replaced the local IP in ICE candidate with mDNS addresses which serve the same purpose and allow for direct P2P communication between the two peers without going through the internet.
I was reading that WebGPU leaves you open to fingerprinting by your hardware info, particularly your GPU. It seems that it is even possible to distinguish between GPUs of the same model through tolerances in frequency.
Honestly I wish we just had a good fingerprinting API that people could opt in to so that we didn't poison all these other good use cases with cringy “it’s gotta be anonymous so I can troll” requirements.
This is HUGE news. Webgpu solves the incentives problem where all actors tries to lock you in to their graphics/compute ecosystem by abstracting over them. It's already the best way to code cross-platform graphics outside the browser. This release in Chrome ought to bring lots more developer mindshare to it, which is an awesome thing.
"Webgpu solves the incentives problem where all actors tries to lock you in"
Or flipped around, it creates an incentive problem where none of the vendors see much benefit in doing R&D anymore, because browsers aren't content to merely abstract irrelevant differences, they also refuse to support vendor extensions except for their own. No point in adding a cool new feature to the GPU if Chrome/Safari insists on blocking it until your competitors all have it too.
Luckily the GPU industry is driven by video games and nowadays AI, industries that don't write code using the web stack. So they'll be alright. Still, that same incentives problem exists in other areas of computing that sit underneath the browser (CPUs, hardware extensions, operating systems, filesystems, etc). Abstractions don't have to erase all the differences in what they abstract, but people who create them often do so anyway.
It doesn't pay off for most apps to run on GPUs because of the fragmentation, bad dev experience, sad programming language situation, proprietary buggy sw stacks and so on. Reining in the cambrian explosion to let non-niche applications actually use GPUs would be a good tradeoff.
(But of course like you say this doesn't detract from the native code apps that are content being nonportable, or want to debug, support, bug-workaround, perf-specialcase their app on N operating systems x M GPU hardware archs x Z GPU APIs, x Y historical generations thereof)
You could say the same about working off a common html, browser, language spec. There are tradeoffs, but industry consensus hasn't really stifled innovation in a meaningful way.
Why can't a common graphics API evolve through well researched and heavily scrutinized proposals? The amount of societal loss of efficiency in having competing specs that do the same thing in slightly different ways, but require immense effort to bridge between is truly vast.
The incentive to innovate comes from limitations with the current spec. That doesn't change just because there's consensus on a common base spec
There are tradeoffs, but industry consensus hasn't really stifled innovation in a meaningful way.
Stifled innovation in what context? I wouldn't describe the web stack as an industry consensus given the rejection of it (for apps) by mobile devs, mobile users, video game devs, VR devs and so on. If you mean within the web stack, then, well, there have been plenty of proposals that died in the crib because they couldn't get consensus for rather obscure reasons that are hard to understand from the outside. Certainly, there were innovations in the earlier days of the web when it was more open that have never been replicated, for example, Flash's timeline based animation designer died and nothing really replaced it.
Fundamentally we can't really know what cool things might have existed in a different world where our tech stack is more open to extensions and forking.
Why can't a common graphics API evolve through well researched and heavily scrutinized proposals?
Why can't everything be done that way? It's been tried and results are a tragedy of the commons. These ideas don't originate in the web world. The incentive to develop the tech isn't actually limits in web specs, that's a decade+ after the innovation happens. What we see is that the web stuff is derivative and downstream from the big players. WebGPU traces its history through Vulkan to Metal/D3D12 and from there to AMD's Mantle, where it all started (afaik). So this stuff really starts as you'd expect, with some risk taking by a GPU designer looking for competitive advantage, then the proprietary OS design teams pick it up and start to abstract the GPU vendors, and then Khronos/Linux/Android realizes that OpenGL is going to become obsolete so they'd better have an answer to it, and then finally the HTML5 people decide the same for WebGL and start work on making a sandboxed version of Vulkan (sorta).
N.B. what made Mantle possible is that Windows is a relatively open and stable environment for driver developers. Ditto for CUDA. They can expose new hardware caps via vendor-specific APIs and Microsoft don't care/encourage it. That means there's value in coming up with a radical new way to accelerate driver performance. In contrast, Apple write their own drivers / APIs, Linux doesn't want proprietary drivers, ChromeOS doesn't even have any notion of an installable vendor driver model at all.
Good and valid point. Effort will naturally focus on the lowest common denominator. Any new features would have to be implemented in software in the WebGPU layer for targets without API support. That's probably going to slow development of new extensions.
However, I think what GPU programming needs at the moment is better ergonomics, and code portability is a step in the right direction, at least. Currently it's quite a bit more annoying than CPU programming.
Yeah, but that's because performance always takes priority. The Vulkan/DX12/Metal APIs are lower level and much more tedious than OpenGL and shader based OpenGL was itself a step backwards in usability from the fixed function pipeline (when doing equivalent tasks). So the trend has been towards more generality and performance at the cost of usability for a long time in graphics APIs. Prefixing it with "Web" won't change that. I'm not saying that's a bad trend - the vendors take the perspective that people who need usability are using game engines anyway so "easy but low level" like classic GL is a need whose time has passed. There's probably a lot of truth to that view.
Which is fine, IMO, as long as there are user-friendly abstractions on top of those foundations. I'd "just" like to write high-level code that runs on GPUs without having to worry about vendor/OS differences (yay, WebGPU) or writing boilerplate (nay, WebGPU). Currently it's all a bit too annoying for normal programmers to bother with. Every month there's a new programming language for CPUs that intends to make programming easier. I'd like this state of things, but for GPUs.
That's fine. They can make the special features for native apps (aka video games) while common things on the web get GPU acceleration. Right now the GPU compatibility situation is pretty bad for the average user despite most of the feature sets being essentially the same.
There are lots of ways to write code cross-platform graphics outside the browser, it has been available for decades, it is called middleware.
WebGPU outside of the browser not only suffers the pain of being yet another incompatible shading language, it is also offers the additional possibility of having code incompatible with browsers WebGPU support, due to the use of extensions not available on the browsers.
These differences are smaller than the differences between desktop GL and WebGL though. It's just another option in the low-level vs high-level 3D API zoo. As far as wrapper APIs go, the native WebGPU implementation libraries are a pretty good option, because being also used in browsers there's a ton of testing and driver bug workarounds going into them which other 3D API wrapper projects simply can't afford to do.
It already made sense to have a SPIRV- and SPIRVCross-based shader pipeline in cross-platform engines with WebGL support though (because you'd want to author shaders in one shading language only, and then translate them to various target languages, like some desktop GLSL version, then separate versions for GLES2/WebGL and GLES3/WebGL2, and finally HLSL and MSL, WGSL is just another output option in such an asset pipeline).
And we all know how well it works when features don't map across languages, WebGPU meeting minutes is full of such examples, hence why threejs is adopting a graphical node editor for easing the transition between GLSL and WGSL.
> This is HUGE news. Webgpu solves the incentives problem where all actors tries to lock you in to their graphics/compute ecosystem by abstracting over them.
Instead we have a third graphics standard (after canvas and WebGL) fully incompatible with the previous two that does exactly that: abstracts the OS graphics stack in a new layer
WebGPU brings some important improvements that are simply impossible to implement with the WebGL programming model (such as moving all the expensive dynamic state validation from the draw loop into the initialization phase).
It's not perfect mind you (e.g. baked BindGroups may be a bit too 'rigid'), but it's still a massive improvement over WebGL in terms of reduced CPU overhead.
Further, in some ways it's the "lowest common denominator" (so that it can be easily implemented on top of each "underlying" graphics API), with union of all their downsides and only intersection of upsides.
Except unlike WebAssembly this one is actually useful. WASM enters a domain full of existing universal bytecodes that all failed to become truly universal because the actual assembly is never the hard part to get portable, it's the API surface. Which WASM doesn't even pretend to attempt to handle, unlike a Java or C# which ship pretty fat bundled libraries to abstract most OS differences.
Meanwhile on the GPU front you don't have much in the way of large, good middlewares to abstract away the underlying API. OpenGL used to be OK at this, but now neither platforms nor Khronos wants to touch it. Vulkan is "portable" but it's like writing assembly by hand, it's ludicrously complicated. WebGPU fills the role of essentially just being GLES 4.0. It won't be fast, it won't be powerful, but it doesn't actually need to for the long tail of things that just want to leverage a GPU for some small workloads.
Unfortunately WebGPU is also an essentially a 5 year old mobile phone graphics API, which means for most desktop features younger than 10 years, you'll still need DirectX, Vulkan, or Metal.
Still, this comes at cost of new complexity. They just createa and abstraction over different operating system APIs. This isn't something cool really. Now they trying to obsolete Webgl. What a joke..
Not really. I don't see how any of the vendors could break it without being extremely overt about it. And it has no direct competition. Nothing else runs on any OS and any GPU, except WebGL, which is abandoned.
It becoming a stale standard is maybe the biggest threat, yes. It has the issue all standards have since several actors have to play along, except this time it isn't Nvidia, Apple, and AMD, but Wgpu/Mozilla and Google. Their incentives are hopefully better aligned with users than those of the hardware vendors.
I suspect WebGL was different, since it was based on the old OpenGL/DX9 way of doing things, so a clean break was desireable. But I honestly am not that knowledgeable in neither graphics programming nor WebGL history, so take that with several grains of salt.
WebGL was based on OpenGL ES 2.0 and a subset of OpenGL ES 3.0.
Google refused to support WebGL compute based on OpenGL ES 3.0 compute shaders, citing WebGPU alternative, and there are still quite a few capabilites from OpenGL ES 3.2 missing from WebGL.
Pretty impressed with WebGL2, and it is already on most platforms, so I hope it wont be pushed aside by this. I assumed WebGPU was going to be more about a Web based OpenCL implementation, is OpenCL also being pushed aside by this then? and wasnt there a plan to do some compute with WebGL? I cant keep up! I hope this myriad of new Chrome APIs being pushed slows down enough for some new browser vendors to enter the market. Or is this just a push for a new 'native' browser experience single horse race...
Vulkan doesn't run on Apple platforms. And Apple refuses to upgrade their OpenGL drivers. So no, they don't count. Then there are the consoles, which also have their own APIs. Expecting all vendors to implement a common API is a fool's errand, and we should stop trying. The vendors don't want this, and we can't force them. Any common GPU API must be an abstraction over vendor-specific ones.
MoltenVK exists and runs Vulkan on Apple devices perfectly fine. Almost certainly better than WebGPU does even. Vulkan is already a common API both in theory and in practice.
It's not a nice API, but it is common and vendor neutral all the same. A lot moreso than WebGPU is, even, since Khronos has a much more diverse set of inputs than the web does. See eg all the complaints about Chrome dominating the web discussion, and even beyond that you only really need 3 companies on board. Khronos has significantly more at the table than that.
Don't get me wrong, this is supremely cool... but I wish the W3C and browsers solved more real world problems.
Just think how many JS kBs and CPU cycles would be saved globally if browsers could do data binding and mutate the dom (eg: morhpdom, vdom, etc) natively. And the emissions that come with it.
Edit:
For example, just consider how many billions of users are downloading and executing JS implementations of a VDOM every single day.
What would that mean? And what version of "mutate the DOM via databinding" would win, because there are at least three different approaches used in various JS libraries?
Accessing the GPU in this way is something that can't be done without browser-level API support. You're describing a problem already solved in JS. Different category entirely.
Honestly no idea, but any native implementation would be more useful than a userland JS implementation.
> Accessing the GPU in this way is something that can't be done without browser-level API support.
That's true and it will open the door to many use cases. But still, mutating the DOM as efficiently as possible without a userland JS implementation is orders of magnitude more common and relevant to the web as it is today.
> Just think how many JS kBs and CPU cycles would be saved globally if browsers could do data binding and mutate the dom (eg: morhpdom, vdom, etc) natively. And the emissions that come with it.
Browsers are solving these real-world problems. With modern JS engines, frameworks are nearly as efficient as a native implementation would be.
And with web components, shadow DOM and template literals, all you need is a very thin convenience layer like Lit/lit-html[1] to build clean and modern web applications without VDOM or other legacy technology.
> frameworks are nearly as efficient as a native implementation would be
I have a very hard believing that a C++ implementation of eg a VDOM would not be significantly more efficient than a JS one. I'm just doing some benchmarks and even the fastest solutions like Inferno get seriously bottlenecked after trying to mutate about 2000 DOM elements per frame.
And even if the performance was similar, what about the downloaded kBs? How many billions of users download React, Angular, Vue, etc, every single day? Probably multiple times.
VDOMs were created precisely because the native DOM was too slow.
This is no longer the case and a tiny web component framework like Lit significantly outperforms[1] React relying entirely on the browser DOM and template literals for re-rendering... so what you're asking for, has already happened :-)
But even the big frameworks are really fast thanks to modern JIT JS engines.
Yes, I know Lit exists, and yes it's fast. But I think you're missing the point.
I'm not talking about one JS implementation vs another or if JS solutions are fast enough from the end user perspective.
The fastest JS solution (VDOM, Svelte, Lit, whatever) will still be bottlenecked by the browser. If JS could send a single function call to mutate/morph a chunk of the DOM, and that was solved natively, we'd see massive efficiency wins. When you consider the massive scale of the web with billions of users, this surely will have an impact in the emissions of the electricity needed to run all that JS.
And (again) even if there were no perf improvements possible, you're avoiding the JS size argument. Lit still can be improved. Svelte generally produces smaller bundles than Lit. Eg see:
Depends how you do it. LLM are huge, but their input and output is miniscule bits of text. If you find a way to put "narrow paths" in the hidden layers, basically to subdivide a model into smaller interconnected models, then the bandwidth will be similarly massively reduced.
This is not without precedent, look up how your brain hemispheres and the regions within are connected.
Crypto prices are still quite high, I don't see why the incentives to steal gpu cycles to mine crypto are any less now than they were during the blockchain summer.
I'm very curious about isolation. Nvidia doesn't allow virtual GPUs on their consumer card drivers so this isolation feels like it can easily be abused. Will there be more support for vGPUs in the future? I hope Nvidia and others are pushed to include better isolation and vGPU support so that WebGPU doesn't need to do all this security isolation themselves. Your browser could theoretically request a vGPU instance to work on.
In general it would be great if browser GPU processes could operate on a vGPU, so the web as a whole would be isolated from the rest of your system. Right now that's not the case, so you're relying on drivers for that browser vs apps isolation, and relying on the browser to isolate tabs from each other as well. Both have failed in the past.
AAA is by definition games that aim at the top end of what can be done in performance and graphical quality. Browsers prioritize other things. Put another way you can't be both AAA and in the browser, because if you tried, other people would come along and simply do better than you outside and you wouldn't be AAA anymore.
Specifically, browsers insist on very strong levels of sandboxing at the cost of high overhead, and they don't want you to run native code either, so you lose both performance and compatibility with most existing game libraries/engines. They also insist on everything being standardized and run through the design-by-committee meat grinder. Whilst Microsoft are polishing up the latest version of Direct3D browser makers are still trying to standardize the version from five years ago.
Browsers are optimized for lots of tiny files, but game toolchains tend to produce a small number of big files. For example browsers aren't good at resuming interrupted downloads or pinning data into the disk cache.
PC gamers have unified around Steam, which offers various advantages that raw web doesn't. Steam is intended for native apps.
Many games need to be portable to consoles because that's where the revenue is (bigger audience, less piracy). Consoles don't run web apps.
Browsers not only don't make it easy to implement anti-cheat but actively make it as difficult as possible.
Debugging tools for native code in the browser aren't as good as outside.
And so on and so on. That's not a full list, it's just off the top of my head. Other types of apps the web ignores: CLI apps, servers, anything to do with new hardware, OS extensions ... the list goes on. Really, we must ask why we'd ever think it'd make sense to ship AAA games as web pages. If you want the benefits the web brings for without the problems then you'd want a new platform that tries to learn from all of this and be a bit more generalized. I wrote up a design doc a month ago that tries to do that, see what you think:
One technical reason is that AAA games would require a complete rethinking of their asset loading strategy, since they'd basically have to use the internet as a very slow and very unreliable hard disc to stream their assets from (which basically means you can't stream the kind of high resolution assets expected of AAA games at all, so you'll have to find a simplified graphics style that looks explicitely 'non-AAA').
You don't want to wait minutes or even hours to download all assets before the game can start (and then again next time because the browser can't cache so much data).
TL;DR: the web platform is different enough from native platforms that ports of bleeding edge games (even from 10 years ago) are not feasible. You'd have to design the entire game around the web platform limitations, which are mainly asset streaming limitations.
But that doesn't happen because there's no working monetisation strategy for 'high profile games' on the web platform, the whole business side is just way too risky (outside some niches which mainly focus on casual F2P games).
The 3D API is only a very small part of the entire problem space (and by far not the most important).
In the end it's mostly about the missing 'business opportunity'. If there would be money in (non-trivial) web games, the games would come.
That is always my example why WebGL is only usefull for ecommerce stores, shader toy and little else.
We still don't have any debugger quality like Renderdoc, Instruments, PIX, and there is nothing with the quality of Infinity Blade, the game Unreal and Apple used to demo iPhone's GL ES 3.0 capabilties.
Streaming like XBox Cloud seems to be the only path for "AAA game experiences running in the browser".
I think a main technical factor is that the devs and middleware authors interested in the platform have been constantly kept waiting for the next platform features just around the corner, and making big games is a long, 5-10 year process.
But then there's the business side. There's a longish history of web games and there are game genres that don't need 3D acceleration, but there still aren't many (any?) profitable web games.
> Performance is decent with around 30-40 FPS on a modern desktop system (ranges from 20 FPS in Edge, 40 FPS in Firefox, to 50 FPS in Chrome)
Achieving 2004 levels of performance & quality with nearly 20 years of hardware improvements is hardly impressive. It's really rather pathetic if anything, although I also got better performance in the opening area than the project page claims but I didn't play very long to find out if it drops later on.
But also note that it's not actually Doom 3 proper, but includes changes from other ports as well as a completely different renderer. There's sadly no side-by-side original vs. port screenshots to compare what the differences are.
Lots of reasons. Here are a few in no particular order from someone who's shipped games on the web, on consoles, and on PC:
* Deploying large software (i.e. games with all their textures and sounds and models) to the browser is a pain in the ass. Your content will get dumped out of the cache, the user's connection may be spotty, and the browser tab might use up too much RAM and get killed. Console and PC game distribution has an install stage because you need one and that simply is not possible in the web model [1]
* Browsers provide bad latency and stability characteristics. They will drop frames frequently due to garbage collection or activity in other tabs. The amount of multiprocess communication, buffering, etc involved in running a webapp also adds input and rendering latency. This makes games just feel sluggish and janky. If your only option for releasing your game is the web, you'll pick the web, but if players could get a smoother experience on Steam or PlayStation instead, you'd be a fool not to release there. The worst scenario is mobile, where in some cases the input delay on touches is upwards of 100ms.
* Browsers have subpar support for user input, especially on phones. For native games users can pick up an input device of their choice and begin playing immediately (unless it's an ancient PC game that doesn't support hotplug - this is more common on Linux for reasons that aren't obvious to me). In the browser, gamepad input doesn't report until you press a button - moving the analog stick to move a menu cursor isn't good enough - which is a weird and jarring experience. Fullscreen is required for certain types of input as well, which means people who prefer to game in a window on their desktop are out of luck. Apple gets bonus points for just intentionally making all of this stuff worse on iOS to force you into the App Store for that sweet 30% cut.
* AAA game experiences are expensive and more importantly time-consuming to develop. There are studios that started building AAA web game experiences a long time ago, and over the course of years most or all of them flamed out. Game development is hard so these failures aren't exclusively the fault of the web platform, but the web platform certainly didn't help. See https://www.gamedeveloper.com/business/rts-studio-artillery-... for one example - they started out building an AAA web game, then pivoted to native because they couldn't get around all the problems with web games, and then eventually shut down.
* Browsers have limited access to resources. I gestured at this in the first bullet point, but if you run in a browser tab you have less address space, less compute throughput, less VRAM, and less bandwidth at your disposal than you would in a native game. For "AAA" experiences this is a big problem, but for simpler games this is not really an issue. For large scale titles this can be the difference between 30fps and 60fps, or "all the textures are blurry" and "it looks crisp".
[1]: There are some newer APIs that alleviate some of the issues I listed, but not all of them
One reason is that WebGL isn't a 2010s technology, but more of a 2005 technology (it doesn't even have compute shaders). WebGPU will finally bring the Web to the state of 2010.
For the general population, is this a bad thing?
- less friction for users
- less hassle dealing with installer bloat
- ability to just “close” a webpage instead of having to remove many files in obscure locations if you want to uninstall something
- easier syncing of state across multiple devices with browser session sync
- granular permissions per “app” such as file access, camera access, etc. - lower barrier to enter for developers wanting to ship cross-platform software without having to bundle electron / tauri / whatever
Not to say there aren’t downsides.
- no longer “owning” your software (although debatable if this were ever the case)
- potentially being tied into vendor-specific browser implementations
Emacs have a similar story to tell. Effectively, the browser is nowadays an Operating System running a VM. The computer history is having these kind of OS models for 50+ years now.
The real problem is more that bad and malicious code get more and more easier to deploy, the browser getting more complex to mitigate this.
The good news is that the level of trust in the code to run app natively is very high, and in the age of highly connected computers, if not done in the browser, it would have been needed at OS level anyway.
So maybe browser looks like a sad future as an OS replacement, but at least, it collected issues and solutions to mitigate arbitrary code loaded from the networks.
Whatever happens after, this history will be kept. (it has already started on current OSes with sandboxed software and on demand permissions).
Adding broadly useful features to the web platform is a net gain if it removes bloat from N web apps that most users run.
It might take multiple years for something like WebGPU to start paying off, and even longer for the deprecation of older APIs for 3D rendering, video compression and compute.
WebGL 2.0 is basically a PS3 / XBox 360 kind of graphics, and WebGPU would be PS4, and that is about it.
All the other cool things, Mesh Shaders, Ray Tracing, Nanite... forget about it, with luck in the next 10 years, if it follows WebGL 2.0 improvements rate.
Incredible insight, but I fear it won't work out well: when the option is there and ubiquitous, websites will rely on it, so disabling features is basically choosing to save some of my resources in exchange for a broken web experience.
Since I'm doing 3D graphics in Web Browsers, I consider not having 3D graphics APIs in Web Browsers a broken web experience. The browser is the easiest and safest way to distribute applications with virtually no entry barrier to users. I don't trust downloading just any binary and therefore I won't execute them unless they are from a trusted source. But with 3D in browsers, I can trust any web site not do damage my system and try a large amount of experiences that others create and upload.
There are applications where the point is 3d graphics. I don't object to letting them use my gpu if they ask nicely. There are other applications -- really the vast majority of them -- where 3d graphics is completely beside the point, and has nothing to do with the reason I'm on your website. That is my concern: that that second class, being the vast majority, will say, "hey, I have access to all this compute power, why don't I just throw some of it at this needless animation that I think looks cool, or at this neural network, or at this cryptominer". Past experience suggests that this is a reasonable concern.
What do you think there will be more of: a) existing functionality moves to more efficient implementation on gpu, or b) "hey, we can use gpus now, let's shove some more compute-intensive features into this app that add nothing but make it seem more innovative".
> Jevons paradox occurs when technological progress or government policy increases the efficiency with which a resource is used (reducing the amount necessary for any one use), but the falling cost of use increases its demand, increasing, rather than reducing, resource use.
If whatever compute intensive feature they come up with wasn't possible on the CPU, I highly doubt it is a bloated useless feature they are adding just for the heck of it.
Looks like no -- there appears to be no tensor core or similar support and this SGEMM (fp32 matrix multiply) benchmark gets awful results (my laptop gets 330 gflops on this when it should be capable of 13000 gflops fp32 and probably 100000 gflops fp16).
Google’s main tensor flow library for the browser runs fastest with its webgl2 backend as compared to cpu so I suspect running it on webgpu is possible and maybe preferred. That said there is a WebNN api that should eventually expose neural network accelerator chips which should be faster and more efficient than GPU at some point.
I guess you're being downvoted because of the swipe at JS, but that's pretty much what's happened yes. Desktop OS vendors dropped the ball on sandboxing and internet distribution of software so badly that we ended up evolving a document format to do it instead. The advantage being that because it never claimed to be an app platform features could be added almost arbitrarily slowly to ensure they were locked down really tight, and because of a pre-existing social expectation that documents (magazines, newspapers) contain adverts but apps don't. So ad money can fund sandboxing efforts and if it lags five years behind the unsandboxed versions, well, it's not like Microsoft or Apple are doing the work.
I think the real reason the web was built was because Google, etc... Decided they need a distribution platform that could not be locked by OS vendors, as that would be a theat to their business.
That's a cynical take. DHTML predates Google. Demand was already there before the giants appeared.
What really happened was that developers figured out that the deployment story via web was massively simpler and less burdensome. Producing good installers was hard, people in offices often couldn't install anything, and then you had to deal with DLL Hell... whereas the browser was always there already. So a series of unfortunate events was set in motion that ended up with what we have today.
Maybe I'm dumb but what does WebGPU get you in reality that WebGL doesn't currently get you? I don't think we're going to see browser games any time soon (because it's trivial to just install a game and run it, and also the browser audio API sucks). And most visualizations on the web would be perfectly fine with plain old WebGL.
Basically, an improved API that targets a more modern graphics pipeline and has better support outside of browsers. You can write a graphics engine or custom renderer in Rust and have it ship across desktop and web. See Vello for example:
This makes running larger machine learning models in the browser feasible - see e.g. https://github.com/webonnx/wonnx (I believe Microsoft's ONNXRuntime.js will also soon gain a WebGPU back-end).
Huh? WebGL uses two (incompatible to each other) versions of GLSL, not C.
The topic of the WebGPU shading language has been discussed to death already, with no new insights brought to the discussion for a very long time.
If you have a SPIRV based shader pipeline, you can simply translate SPIRV to WGSL in an offline compilation step (and to get to SPIRV in the first place, you need an offline compilation step anyway).
If you use one of the native WebGPU libraries outside the browser, you can load SPIRV directly.
AFAIK: It's a mostly-but-not-100% textual equivalent of SPIR-V that, syntax-wise, is a weird hodgepodge of all the other shader languages. At the same time, it's rare it'll be actually written by hand by humans - it'll mostly be generated from another language at application build time, and then it'll have to be compiled back into bytecode by the browser, which feels like really redundant work.
Further, it's often said that the main reason the language exists in the first place is because Apple veto'd SPIR-V due to their legal disputes with Khronos. I'm assuming that's why the comment above called it "ad-hoc".
Because it is too different from other C-like DSLs GLSL, HLSL. GLSL was doing its job perfectly and there was no reason to invent another one. With wgpu, I now need to spend time on searching through its reference pages for things that I already know how to do in shaders. Now all countless WebGL tutorials need to be migrated to completely new syntax. This could be much easier by just making a superset of GLSL.
And I had't got any sane answer on "why it should look like Rust?"
No idea what the comment about Rust means, but we already have several shader languages. There was literally no reason to invent another incompatible one
Most current shader languages are very close to C in terms of syntax and behavior and these are some of the worst aspects of C as a language. I guess they could have went with SPIR-V but generally a compilation step shouldn't be required in web standards.
Literally all of these are owned by big corporations (Apple, Microsoft, Sony) and made specifically for their proprietary gpu APIs. C++ is just a superset of C which doesn't fix syntax problems. Making an open source standard with a modern syntax is a better idea than picking a favorite corporation.
The language is called WGSL. It's not ad-hoc (I mean, all DSLs are to some extent?) and it has a pretty easy learning curve if you know GLSL. I don't get what the fuzz is about.
JavaScript is the programming language for the Web. WGSL syntax should have been based on JavaScript syntax. It would have made writing shaders easier for millions of existing Web developers. Now they have to learn a new syntax.
WebGPU would most likely have to create its own subset of SPIRV anyway to fulfill the additional safety and validation requirements of the web platform.
Apple has a Khronos allergy, and both SPIRV and GLSL are Khronos standards.
Also the original 3D API proposal from Apple was essentially a 1:1 Javascript shim for Metal, which looked quite different from WebGPU.
Apple also originally proposed a custom shading language which looked like - but wasn't quite - HLSL. Compared to that, WGSL is the saner solution (because translation from and to SPIRV is relatively straightforward).
Mostly caused by how Khronos has managed OpenCL, and ironically Vulkan is slowly adopting HLSL as the main shading language, as they don't plan to invest into improving GLSL any further.
Firefox has dom.webgpu.enabled for browser control. Chrome has flags for it right now, but those often disappear once a feature is introduced. You can probably disable hardware acceleration to get rid of webgpu though.
It will be interesting to see the performance differential. Tensorflow.js provides a benchmark tool[1]. When I ran them[2] on an M1 MacBook Pro, WebGPU (in Chrome Canary) was usually 2x as fast as WebGL on large models, sometimes 3x.
Dear god no, use CSS for that. It's already hardware accelerated in the regular browsers. You're talking of website presentation and that's the purpose of CSS.
Not really. If you want smooth and efficient rendering of basic animations/effects, you should be using CSS, because then the browser will natively rasterize, scroll and composite everything in parallel using hardware acceleration. It's far more efficient than rendering basic elements with WebGL and will probably be better than WebGPU in most cases.
You're in almost every thread about WGPU, with negative opinions about it. What would your ideal graphics API be, and why isn't it coming to fruition, do you think?
Not the person you're replying to, but a start would be to use an existing proven shader format like SPIRV or DXIL instead of making up an entirely new shader ecosystem in order to satisfy the whims of a single browser vendor and waste everyone's time.
> , and why isn't it coming to fruition, do you think?
Politics and lack of tooling.
Why the negativity?
Consider that after 10 years, there is no Web game that can match AAA releases for Android and iOS written in OpenGL ES 2.0 (already lowering the bar here to WebGL 1.0).
And SpectorJS is the best GPU debugger we ever got.
What does Mozilla use Spidermonkey instead of V8? Why did Apple create B3 JIT instead of just using Turbofan?
Competing implementations are a cornerstone of standards. Indeed in many domains it's a requirement for a spec to have multiple compliant implementations to be considered complete at all.
The older I get the more I disagree with this POV. All other things equal, a single open source implementation is superior to several ones. Several implementations lead to duplication of effort, both for those developing them, and more importantly for those developing for those implementations. Software has to be tested separately for each implementation, often with vendor-specific hacks.
This is super exciting for us as we develop interactive browser UIs. I hope future versions of WebGPU will be backwards compatible to all those with Chrome 113.
We need an iOS like pace of updating for browsers!
The difficulty is the number of people still running old versions, as soon as new iOS versions come out a super high percentage of people switch. And with browsers you have to account for the update cycles of every browser on the market.
Suppose you have a massive 3D model stored in the cloud, which weighs in at 100GB and requires most of the computation to be handled on the server side. In this scenario, would utilizing something like WebGPU be beneficial, given its primary responsibility for the final 2D projection?
Games in iOS safari are still at a big disadvantage even if they have access to WebGPU, because Apple intentionally undermines input and fullscreen APIs there to keep games in the app store.
It's good to know that Apple, MS, Firefox, and Google are all onboard. It's great that browsers are finally taking advantage of the underlying hardware.
Next up for browsers should be NPUs such as Apple's neural engine.
They count the beta channel as the release channel so if it is in beta channel it is released from here if no bugs are found would be pushed to stable.
WebGPU implementations are still pretty immature, but certainly enough to get started with. I've been implementing a Rust + WebGPU ML runtime for the past few months and have enjoyed writing WGSL.
I recently got a 250M parameter LLM running in the browser without much optimisation and it performs pretty well! (https://twitter.com/fleetwood___/status/1638469392794091520)
That said, matmuls are still pretty handicapped in the browser (especially considering the bounds checking enforced in the browser). From my benchmarking I've struggled to hit 50% of theoretical FLOPS, which is cut down to 30% when the bounds checking comes in. (Benchmarks here: https://github.com/FL33TW00D/wgpu-mm)
I look forward to accessing shader cores as they mentioned in the post.