Thymer[1] uses CRDTs for everything. It's an IDE for tasks and planning. It's a multiplayer app, end-to-end encrypted and offline first, optionally self-hosted, and an entire workspace is a single graph. So CRDTs were the logical choice.
All operations in Thymer get reduced to a handful of CRDT transformations. It doesn't matter whether you are moving or copying text, changing "frontmatter" attributes, dragging cards, uploading files, or adding tags. It's all done with the same handful of CRDT operations. Although this was a lot of extra work up front (we're not using any libraries) the benefits make it totally worth it. When your application state is a single graph you can move text between pages, link between pages (with backlinks), have transclusions, and do all sorts of cool stuff without having to worry about synchronization. CRDTs guarantee that all clients converge to the same state. And because CRDTs are by their nature append-only you get point-in-time versioning for free! We did end up having to make a couple of compromises for performance, though. Version history is not available offline (too much data) and in some cases we resort to last-writer-wins conflict resolution. On balance I think CRDTs are very much worth it, especially if you design an app with CRDTs in mind from day one. I probably wouldn't use CRDTs if I had to retrofit multiplayer in a more conventional AJAX app. Mutations in CRDTs are first applied optimistically, and then when the authoritative sequence of events is determined all clients need to revert their state to the last shared state and then re-apply all events in the correct order (thereby guaranteeing that all clients end up in the same state). Sometimes your app might need to revert and re-apply days worth of changes if you've been offline for a while. This all happens behind the scenes and the user doesn't know how many tree transformations are happening in the background but I guess my point is that CRDTs affect the design of the entire application. Most apps that are popular today were designed back when CRDT transformations were not yet well understood.
Both "everything in the cloud" and "everything local" have their obvious technical advantages, and I think they are mostly well understood. What really drives the swing of the pendulum are the business incentives.
Is the goal to sell mainframes? Then tell customers than thin clients powered by a mainframe allow for easy collaboration, centralized backups and administration, and lower total cost of ownership.
Do you want recurring SaaS revenue? Then tell customers that they don't want the hassle of maintaining a complicated server architecture, that security updates mean servers need constant maintenance, and that integrating with many 3rd party SaaS apps makes cloud hosting the logical choice.
We're currently working on an Local First (and E2EE) app that syncs with CRDTs. The server has been reduced to a single go executable that more or less broadcasts the mutation messages to the different clients when they come online. The tech is very cool and it's what we think makes the most sense for the user. But what we've also realized is that by architecting our software like this we have torpedoed our business model. Nobody is going to pay $25 per user per seat per month when it's obvious that the app runs locally and not that much is happening on the server side.
Local First, Forever is good for the user. Open data formats are good for the user. Being able to self-host is good for the user. But I suspect it will be very difficult to make software like this profitably. Adobe's stock went 20x after they adopted a per seat subscription model. This Local First trend, if it is here to stay (and I hope it will be) might destroy a lot of SaaS business models.
Thymer has real-time multiplayer with CRDT based synchronization. With org-mode you can collaborate, but you'd have to DIY with git or something similar. Thymer is also end-to-end encrypted. With org-mode you'd have to self-host something to sync with your phone.
Org-mode is text based, Thymer has a document graph. Advantages of the graph approach is you can nest as much as you want and zoom in on any part of the graph. You can transclude sections and have references and backlinks. With text based approaches you have to sprinkle location identifiers (guids or relative paths typically) into your document so you can track the location as files gets edited. But guids are awkward and paths can result in broken (back)links. We built Thymer as a document graph from the ground up. Tree mutations don't break (back)links.
With Thymer every node in the graph has metadata. Who last edited it? When? Where did it move from? Previous versions? For text paragraphs this isn't super useful, but when you have tasks this metadata is very useful.
org-mode on a phone? I'm sure there are apps for that, but I think it's designed with a keyboard in mind. Thymer uses the (subjectively) more modern command palette approach where you don't have to remember as many shortcuts, and it also translates more easily to touchscreen devices.
For the table situation we solve this by have a generic concept of a "card" that is rendered as a table cell when it's the grandchild of a table (table > tr > td) but otherwise it's just a standalone item, much like a markdown callout.
We're using a single end-to-end encrypted document tree synced with CRDTs for our collaborative task IDE[1]. All data for a team is a single tree (graph really, if you count transclusions) and its kind of magical how simple everything gets when you know all state will sync in a deterministic way between all clients. It doesn't matter whether you drag&drop some object or add a new user, or rename something. It all reduces to a handful of CRDT operations.
(We have a central server and the app works offline so the algorithm from the linked article doesn't apply in our case.)
We've looked at Yjs but ultimately decided to write our own thing from scratch.
Even scale-ups with thousands of users will have people working on different parts of the document tree in practice. The client doesn't need to receive every keystroke for people editing somewhere far away in the document tree. Those updates can be sent in batches every once in a while.
If 10.000 people decide to edit the exact same location in a document then performance will degrade. The client will start to lag behind if it can't keep up with the stream of updates (cpu or internet bottleneck) but eventually it will process everything and all clients will end up with the same state. We have two channels. One for state updates (soft real-time) and one for UI hints ("hard" real-time) and other user actions that aren't CRDT mutations.
That's not why Pascal compiles quickly. Resolving undefined symbols isn't difficult or slow, it just means that you have to keep track of what's resolved and what's unresolved and that takes up memory, which was very precious back in the old days of Pascal. Pascal is designed for very fast single-pass compilation, but symbol resolution is only a small part of it.
I suspect this might be why most open source software struggles to reach the quality of commercial products. If it takes 10 attempts to get the major design questions right a company will either get there eventually or get outcompeted. Open source projects typically grow out of somebody scratching their own itch, and that doesn’t involve the scrapping and iterating necessary to make a product that fits many users’ needs.
I know I'm biased because I'm working on an E2EE todo/planning app[1], but over the years I've become convinced that E2EE apps are the future.
All the syncing code you have to write anyway if you want your app to work offline. Despite all the buzz about big data, most individuals and most companies generate only a modest amount of data. Even when you have a long tail of archived data (e.g. emails that go back 10 years) the key benefit is in having all recent data with you, available offline, and end-to-end encrypted.
Self-hosting is a lot more difficult in practice than in theory. Somebody has to make backups, maintain a server architecture with redundancy, apply security patches, etc. For hobby purposes you can get away with just not doing any of that, but for businesses and people who don't want to tinker with their own servers outsourcing all this makes a ton of sense.
I don't agree with your claims that traffic is expensive. In any case, downloading data once and syncing subsequent deltas is still a lot more data efficient than roundtripping JSON for every user action.
Ok, I'll byte. Let's take a hypothetically example that might just happen.
Imagine a company has a Wikipedia hosted by a 3rd party. I interpret E2EE as: after I submit a post, it is encrypted, and the servicing company can't look at the unencrypted data, my coworkers can decrypt the posts.
It is simple to ask "how would you support linking to posts" - but "how would you support searching for posts"?
Would you maintain a key-value lookup list? Would you handle semantic search? Are there E2EE algorithms that facilitate enhanced searching?
So, searching through encrypted text is a really interesting problem!
It's actually possible under some ciphers / search algorithms - if the client generates encrypted inverted index entries when submitting a post, and encrypts their (vectorized) search query using the same key, the server can compare them and return a scored list of IDs.
You do need to weaken the encryption scheme somewhat - in particular, term-frequency analysis becomes possible.
It’s also a multistep process with a touch screen: 1) find the button on the screen 2) lean over to reach it 3) touch it and look at it while doing so to confirm you did it right.
In a clickwheel car the wheel moves the focus rect. You twist it blindly to approximately the right spot. Then you look at the screen and adjust one or two clicks and press to confirm. You won’t trigger the wrong action by accident and the focus rect makes the operation async: you don’t have to look while you turn the wheel, you can look at the screen when it suits you.
All operations in Thymer get reduced to a handful of CRDT transformations. It doesn't matter whether you are moving or copying text, changing "frontmatter" attributes, dragging cards, uploading files, or adding tags. It's all done with the same handful of CRDT operations. Although this was a lot of extra work up front (we're not using any libraries) the benefits make it totally worth it. When your application state is a single graph you can move text between pages, link between pages (with backlinks), have transclusions, and do all sorts of cool stuff without having to worry about synchronization. CRDTs guarantee that all clients converge to the same state. And because CRDTs are by their nature append-only you get point-in-time versioning for free! We did end up having to make a couple of compromises for performance, though. Version history is not available offline (too much data) and in some cases we resort to last-writer-wins conflict resolution. On balance I think CRDTs are very much worth it, especially if you design an app with CRDTs in mind from day one. I probably wouldn't use CRDTs if I had to retrofit multiplayer in a more conventional AJAX app. Mutations in CRDTs are first applied optimistically, and then when the authoritative sequence of events is determined all clients need to revert their state to the last shared state and then re-apply all events in the correct order (thereby guaranteeing that all clients end up in the same state). Sometimes your app might need to revert and re-apply days worth of changes if you've been offline for a while. This all happens behind the scenes and the user doesn't know how many tree transformations are happening in the background but I guess my point is that CRDTs affect the design of the entire application. Most apps that are popular today were designed back when CRDT transformations were not yet well understood.
[1] https://thymer.com (almost ready for beta)