Well, I should rephrase: I didn't know CRDTs could be fast enough to be practical. At least in the context of collaborative text editing. I've been in the field I was arguing against using CRDTs for over a decade.
I ran a bunch of optimization experiments 18 months ago[1] and realised I was wrong. I demonstrated (at least to my own satisfaction) that all the problems that CRDTs had (memory usage, disk usage, CPU usage) seemed to be finally solvable. Last year I put out a blog post[2] talking about how to get insane performance. Both of these posts were discussed in detail here on HN.
One result of this is that automerge (an excellent CRDT) has started optimizing for performance. I hear their performance on this benchmark has improved from 5 minutes to ~2 seconds. Yjs can do the same thing in 1 second. Diamond types (my own CRDT) is now about 6 times faster than it was when I wrote that post last year. I can run the same benchmark in ~10ms (0.01 second). Memory usage has dropped from automerge's 880mb, to 2mb in diamond types. And I think we can get it much lower. Disk size for the same data set has dropped from 4mb of JSON down to 100kb while still storing the full edit history. (And this is for a 100kb document!)
There's a lot of tricks involved, and plenty more in the tank. But in short, modern list / string CRDTs are now close to the speed of their equivalent native data structures. With a text CRDT in rust, more CPU gets wasted in wasm overhead than is spent maintaining the CRDT itself.
Thats not something we knew was even possible a year or two ago. So, thats whats changed.
I ran a bunch of optimization experiments 18 months ago[1] and realised I was wrong. I demonstrated (at least to my own satisfaction) that all the problems that CRDTs had (memory usage, disk usage, CPU usage) seemed to be finally solvable. Last year I put out a blog post[2] talking about how to get insane performance. Both of these posts were discussed in detail here on HN.
One result of this is that automerge (an excellent CRDT) has started optimizing for performance. I hear their performance on this benchmark has improved from 5 minutes to ~2 seconds. Yjs can do the same thing in 1 second. Diamond types (my own CRDT) is now about 6 times faster than it was when I wrote that post last year. I can run the same benchmark in ~10ms (0.01 second). Memory usage has dropped from automerge's 880mb, to 2mb in diamond types. And I think we can get it much lower. Disk size for the same data set has dropped from 4mb of JSON down to 100kb while still storing the full edit history. (And this is for a 100kb document!)
There's a lot of tricks involved, and plenty more in the tank. But in short, modern list / string CRDTs are now close to the speed of their equivalent native data structures. With a text CRDT in rust, more CPU gets wasted in wasm overhead than is spent maintaining the CRDT itself.
Thats not something we knew was even possible a year or two ago. So, thats whats changed.
[1] https://josephg.com/blog/crdts-are-the-future/
[2] https://josephg.com/blog/crdts-go-brrr/