Hacker News new | past | comments | ask | show | jobs | submit login

Hashes and append only logs are all not very good for realtime data because of the extra overhead that has to be calculated, but CRDTs are.

CRDTs naturally fit with P2P/decentralized topologies, and we've generalized them in https://github.com/amark/gun which is the most popular (8K+ stars) open source (MIT/Zlib/Apache2) realtime decentralized database.

It is running in production on P2P alternatives to Reddit and other apps, that have pushed over half a terabyte in a day.




It's not incorrect to say that hashes and signatures add some overhead, but the question is whether the overhead is significant enough to matter for the usecase. Probably not.

Dat is realtime. You are notified about updates as soon as they are distributed. In Beaker, the files-archive API has a `watch()` method to do this. If you're accessing a dat files-archive, you're participating in the syncing swarm and so you'll receive those updates automatically.

You'll want to use a UDP socket if you're streaming a high volume of data with low latency requirements, for instance for a multiplayer FPS. But Dat has been used to stream live video, so it's probably real-time enough for most Web use-cases.

Small aside: Comparing Dat to CRDTs is apples to oranges. It's like comparing Javascript to B-Trees; they're not quite the same kind of technology. In fact, the new version of Dat uses CRDTs in order to allow multiple users to write to a files-archive.


When we met just a month or so ago you didn't tell me you were adding CRDTs!!! This is very exciting news. Dominic had mentioned a specific type of CRDT he had added (but wasn't generalized).

Append only logs have overhead that would make it a poor choice for most realtime applications, like GPS tracking, yes FPS games, google Docs, website creators, and many more. Basically any use case where data mutates.

In 2010 I used and built all my own custom event source system, I was the hugest proponent of this / append only logs. It was so futuristic. But 4 years in I hit all sorts of scaling problems and had to redesign everything from scratch and that is when I found CRDTs. On all accounts they are superior, in a mathematical or logical manner, because they are a superset to DAGs, immutable/append only logs, and many other popular data structures. Not apples and oranges.


I built a little side multiuser wiki side project that’s actually using two levels of CRDTs... hyperdb (underneath hyperdrive) and then automerge on top of that. Sort of hard to explain the full design in a short entry, but you can play with it here:

https://dat-tiddlywiki.glitch.me

The UX definitely needs some improvement, but it’s just a personal project so far.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: