Personal WASM anecdote: I just implemented a WASM based 3d model boolean operation for three.js where I can intersect two models together. The pure js version ran in around 10s on a relatively simple model whereas the WASM version did the same thing in a little under 1s. I've been very impressed by the technology so far! Obviously this kind of CPU intensive work is where it can shine and it's a useful tool to have in your toolkit.
From benchmarks I've seen elsewhere, native code is generally roughly 2x faster than WASM, so I doubt that's the case. I'm sure it depends on a lot of factors, though.
Single-threaded, non-SIMD x86-64 code being 1.2x to 2x faster than WASM is what I'm seeing as well. Interestingly I don't see such big differences between browsers as the PSPDFKit benchmark, my suspicion is that this mainly measures overhead when crossing the WASM-JS border.
PSPDFKit CTO here: We've been using WebAssembly since 2017 to render PDF documents directly in the browser and strongly believe in the future of the technology.
We published this benchmark to have a public indicator of the performance gains WebAssembly promises in a real-world product.
Yes, please contact our sales team to discuss your use case. We sell to indie developers, early startups up to large Fortune 100 companies with a wide range of use cases so we found custom pricing to be the best way for us to be able to sell our software to everyone and at the same time support our team size.
Hey, thanks for the interesting article, and the benchmark of course!
One bit of constructive feedback: add a date of publishing to the article. PSPDFKit already has multiple articles on WASM before, and it's nice to be able to tell which one is the most "up to date" at a glance.
It's still getting faster too, e.g. engines are starting to use trap-based bounds checking which greatly reduces overhead when running on 64-bit platforms.
Could you elaborate on “trap-based bounds checking”, or provide a reference to learn more? My searching mostly pointed to “trap” as being a WASM concept, rather than some processor feature that improves bounds checks.
It means using the processor's memory protection features to prevent programs from reading memory that they're not supposed to read, rather than inserting check instructions at every memory access.
Excellent post! Webassembly appears to be approximately half as fast as native. Do you know where that gap comes from, and how much of it can be expected to close?
Thanks for publishing this benchmark, but I've got a question about the results. The description of the charts states that 'A lower score is better', and between the two charts you say 'when we compare WebAssembly with our JavaScript fallback ... all browsers except Firefox are still slower'. But WebAssembly has a lower score than JavaScript on all browser/OS combinations except Chrome67/macOS, Safari/macOS and Edge/Win10. Could you clarify which way the results are pointing?
A big part of the score is compilation time, which improves a lot with Liftoff, our baseline compiler for WebAssembly in V8. We've been in contact with the benchmark authors and have identified other bottlenecks in our engine (e.g. instantiation time), which we are now working on optimizing. Look forward to some increasing scores soon :)
Liftoff is still substantially slower at initial compilation than Firefox. Are there any plans to close the gap? A Unity exported game takes over 40 seconds for a new model low end Chromebook to compile. The upcoming 3 fold decrease will be nice, but FF could do it in 2 seconds on the same hardware.
We still have a bottleneck at instantation time where we compile many JS<->WASM wrappers. We are working on parallelizing this compilation as well, so we'll get another bump out of that.
IIRC they've also always had the fastest Typed Arrays. I wonder if there is a connection there - it wouldn't surprise me if memory access is the biggest bottleneck for WASM in other browsers.
Ha! Not sure if you've seen it, but we offer native SDKs for iOS, Android, Windows, macOS and Linux. Our Web SDK also has a version that uses PSPDFKit Server via a Docker container to get faster performance.
PSPDFKit for Web Standalone is a great choice for an easy-to-deploy web viewer, annotation editor and form filler, and performance is great. Definitely better than pdf.js - which is typically where people start with, before they upgrade to PSPDFKit for Web.
Right? The built in safari reader does everything correctly and interacts with the rest of the OS properly. I assume that the built in chrome one also Just Works (tm).
Also why would I want to download an entire pdf engine when i already have an AOT version? (Stealing that phrasing by the way :) )
Safari's PDF viewer doesn't allow you to annotate, especially not in real time with other users: https://pspdfkit.com/instant/
It's also quite basic when it comes to forms and often loses data.
For cases where more performance is required we always recommend using PSPDFKit Server, so PDFs render in native code on a high-power machine, and very likely using cached tiles.
Turns out that it is - it's amazingly flexible and pretty much anything out there will be able to render it. There's practically no other formats that can claim that.
FYI: WebAssembly is effectively starting to get WebAssembly only APIs. Chrome is shipping the WebGL commit API which is designed to support spin loops in native apps (others are expected to ship it too)
To support that model TPTB added a worker only `gl.commit` function that does effectively the same thing as `glSwapBuffers` so that native apps can keep their spin loops and don't have to refactor to be event based.
Of course if you decide to use a `gl.commit` spin loop your worker can no longer receive events (no onmesagge, no XHR, no websockets, no fetch) so you're supposed to use SharedArrayBuffers to communicate with a spin loop worker.
SharedArrayBuffers are being re-enabled once they are isolated to same domain only processes.
AFAICT this is the first browser API that is really targeted exclusively at WebAssembly. I'm guessing more will come. Not entirely sure how I feel about that but I guess it's inevitable.
By WebAssembly only I don't mean you can't call it from JS. I mean rather it's designed for Wasm not for JS. JS is generally event oriented. Native C/C++ games are usually spinloop+polling oriented. With the current design of commit is for spinloops but once you do a spinloop you can no longer receive events.
That blog post you linked to is 2 years old and out of date with what browser are actually planning on shipping.
For JS you no longer need commit at all. canvas (both 2d and webgl) work offsrcreen just like they do without offscreen. You render, when your event exits the changes get magically propagated to the corresponding canvas. commit is solely for when your event never exits (you're in a spinloop)
> The team was especially helpful in pointing out bottlenecks in our JavaScript implementation, and we’re already incorporating these changes into our next PSPDFKit for Web release.
Any chance you can talk about those bottlenecks? Might be interesting.
We should be at the point where network computers are practical for casual users. A full install of the BeOS operating system is just 45MB. A tricked-out one was barely more than a couple of hundred MB. We should now be able to have operating systems which are mostly cached from online connections, running in clients like Web Browsers, but purpose built for running such an OS.
You can't compare small/micro-benchmarks that call into WebAssembly and then get the result with a larger full-stack application where parts are in JS and parts in WASM - both are highly interlinked. This is much more reflective of real products. We couldn't find a benchmark that shows such end-to-end usage, so we built one.
If you render to canvas, you would need to reimplement scrolling and zooming, checkboxes and input elements. Any custom reimplementation of this stuff will end up slower and clunkier than the optimised, native versions already in browsers.
Also (as far as I know) canvas content is inaccessible to screen readers. Accessibility matters.
> Browser vendors can reach out to us and obtain a more permissive license key so that the benchmark can run on different machines and even on their continuous integration servers.
Sounds like a great idea for encouraging performance improvements and avoiding regressions!
Great work, looks like a very useful PDF toolkit. I've been working on a side project implementing something similar for some time but has stalled due to life getting in the way. Can you share anything publicly in terms of pricing?
I see everything is through a sales person now and I'd like to get a better high level understanding of high level costs for passion projects (<1000 users), small-medium (<100k), and enterprise (1M+). What metrics are you charging usage on?
Metrics are required platforms (iOS, Android, Web, Windows, macOS, Linux), components (see matrix on https://pspdfkit.com/pdf-sdk/), number of users, number of server nodes (if Docker-based), number of apps/[sub]domains, internal/external access and expected support, to name a few. We're usually quite fast to figure something out that works for both parties, pricing is in the 4 (passion projects) to 6 (enterprise) figure range. Simply drop us a message via https://pspdfkit.com/sales to get things started. We try to make this process as easy as possible.
^ this. Please provide a pricing page with a few options and even a usage calculator. Without it, you'll be hard pressed to find enough folks giving this a real shot and will just assume that it's enterprise-only and out of their range.
Shoutout to the Safari team! They were able to track down the slowness in the current WebAssembly implementation and ALREADY FIXED IT. Patch is here: https://trac.webkit.org/changeset/233378/webkit
(trace points caused a significant (4x!) slowdown as our benchmark performs many calls into the VM)
Since all languages should be able to compile to wasm, that question is pretty close to "Which language is the most performant?", which is a can of worms that I'm not going to open. Maybe something like the Benchmarks Game would be useful here. Or already exists, and someone could point me to it :)
A similar question would be "What languages do you know of that are focused on performance and compiling to wasm?" For that, I'd say C++, Rust, and Go, though I'm sure there are others.
It's really hard to say, especially as the wasm interpreter stuff itself is under so much flux; different implementations act differently, etc.
There's also the question of what aspect of performance is of interest to you. Do you include the download time in performance? If so, languages that produce leaner wasm files will be better performing, even if the execution is identical. Do you only care about execution time? Then that aspect isn't relevant. etc.
That depends on (a) how you define your measure for "performance" (see other comments that talk about the difference between microbenchmarking and application benchmarking), (b) the compiler that you use, (c) the WebAssembly implementation that the code finally runs on.
So, do not expect a clear answer to that questions. Probably no one really knows because of a lack of benchmarks and deployments in the wild that have been analyzed. That's why they made the benchmark.
I'm guessing these "features" will be tied to the Microsoft Store, something they're already doing with PWAs.
Many were quick to criticize Valve and Epic about their own criticism of the Microsoft Store, which is why Microsoft is now using a "boiling the frog" strategy to tie everything to the Store over the next decade or so. And this process will likely accelerate once Windows 7 is dead and buried.
