A comparison to PouchDB would make a great reference point to help people understand the use case. From what I can see in 2 mins of reading it looks like it's more relational which should be a plus, but then on the downside there's much less of a story on the sync side, with remote persistence ending at offering you interfaces to persist your data but not an actual data store that works. By comparison, PouchDB connected to a server side CouchDB makes an amazing combo.
I didn't have much trouble binding PouchDB into a Vue reactive data store so the reactive part didn't seem as novel to me. What does seem novel is supporting a proper relational schema for this, whereas all the other options I know in this space are schemaless / key-value type approaches.
I wonder if t might be a high-value / low effort proposition to come up with an out-of-the-box setup with something like PostgREST that would then work end to end? This was the super compelling part for me with the Pouch/Couch combo, as with a couple of configs and barely any code your browser localStorage is suddenly sync'd and saved remotely in real time which is mind blowing when you first experience it.
I actually love it when people invent things without full knowledge of prior art - a lot of innovation happens only when people aren't biased by what is already there. Perhaps if you knew all about PouchDB you might not have had enough motivation to do this, and then this gap of a relational reactive store would not have been explored. What you have done looks really great!!!
This looks really cool, love seeing more innovation in this space!
At first glance this seems to be mostly targeted towards single-user apps where each user would have their own database that can be sync'ed to a remote server, but still isolated from data for other users, similar to the CouchDB+PouchDB model?
At least it looks that way since I couldn't see anything around authorization and conflict resolution. Not that there's anything wrong with focusing on this use case, a lot of apps can function perfectly fine this way.
A few other interesting new players that use an optimistic local update + server validated mutation model to support collaboration:
By "reactive" I mean... if the data changes, the UI does (or at least, an event is fired, and then you can drive a React render, for example, from that). In contrast, you expect to have to poll an RDBMS for updates. An MIT team recently wrote an awesome essay on the same concept (https://riffle.systems/essays/prelude/), wrapping SQLite for the purpose.
With "data store", rather than "database", I'll admit I am hedging my bets. Through one lens, this library can look more like an app-level store in a classic React sort of way. Through another (most notably the table/row/cell structure), it looks like an RDBMS. I'm not ready to go all-in and declare it's a database yet. Such nomenclature comes with Assumptions™.
A store where you do a query, get a result set, but also a callback (reactivity) in the future if that query would return a different result set if you were to run it again.
Imagine a "row count" HTML element which is automatically updated (reactive) when a different part of the web app inserts a row in the data store.
Ah, I was whining in the surrealdb thread for new advances in this space over couch/pouch (which we have been using for over a decade). Would be great to see comparisons, especially in areas where couch/pouch are bad and new solutions are better. We have not moved anywhere yet because couch/pouch, when you are used to it development wise, is automatic. You do nothing and it works. Other solutions we tried require way too much domain knowledge (which we have but don't want to think about when doing things we think should just automatically work, like synching; like replicache which is just too much work for us on our existing datasets) or are very hacky, as in, lose data and require you to write code to fix that.
Interesting to hear that Couch/Pouch “just work”. I’ve been looking at using them and I find there’s a lot to wrap your head around. In particular for apps where different users should see different sets of shared records.
Yes, that’s correct, but we have been used to thinking that way for a decade. I don’t remember how painful it was in the first place, but once you think in that way, things fall in place.
Any good resources you can recommend that would show me “the beaten path” for e.g. a todo list where users can selectively share items, or join teams - just a step or two more than the usual “every user gets their own data only”.
Nice work. I love anything that makes local-first easier. I’ve been paying close attention to Dolt for this. In a perfect world I’d have the gut semantics from Dolt and EdgeQL from EdgeDB.
This focuses on the reactive feature - the ability to set listeners and get alerts. There seem to be some nice React integration, but haven’t built with React enough to immediately see what that unlocks.
Thank you. I figured that getting/setting/listening are the key primitives, and then the React layer (which is just one of many theoretical UI library bindings) can use hooks and components, or whatever, to simply wrap them. I'm not familiar with Dolt, but seems like I need to become so!
I’m developing a local-first app and a game with Electron and I am using my own “data store” by simply storing serialized JavaScript object literals in localStorage. It works, but at times it’s a bit annoying and tedious to use, so I’ll definitely check this out.
My only other frame of reference for something similar is TaffyDB, though It would not call it “reactive”.
Pretty cool. I do a lot of game dev and have searching for solutions like CastleDB but built in the NodeJS ecosystem. This looks like it could check a lot of boxes for me.
What I want is to be able to define my tables with something like JSON schema, and automatically have the typescript typings available when querying. The data would be serialised to Git friendly json, and there would also be support for different media types which get saved and organised in the local file system.
For me, a game developer who wants to be very hands on with bespoke tooling/editor functionality, I only want a simple foundational base, something along the lines of Airtable, with the workflow centred around Git.
Yes, one imminent idea I have is to codegen a .d.ts from an explicit schema (or even implied, from populated data). One might even be able to produce typed functions like `setPetsPrice(petId, price: number)` instead of `setCell('pets', petId, 'price', value: any)`.
I looked into building something like this for a React Native app. (Actually I first considered the practicality of making a reactive version of sqlite... I decided it would be too complex and not something I'd want to use without upstream support).
We ended up doing something a bit more traditional/basic/manual, but this looks nice. It would definitely have been in consideration if it'd been available at the time, and I'll keep it in mind for future projects.
Cool! Please take a look and see if it works out. I should point out that the ui-react library (that provides React bindings) does _not_ depend on ReactDOM, so you can use it just as easily in React Native.
I wonder if I should have a font-weight (or -family) toggle alongside the dark mode setting. Would be a shame for people to miss out on reading about the software because of the typography!
Meta: hacker news is known for this, but it's really not personal (modulo very few mean folks). People have an adversarial mindset and will poke holes in everything, even things that are awesome. If the biggest critique against your project is the font (superficial) you can interpret is as an absence of more fundamental criticism.
Fonts and fancy looking things in particular tend to be critizised extra hard if it interferes with accessibility or scroll (scroll jacking comes with capital punishment in the hacker news universe). If you want 0 critique the safest option is #000 Times New Roman on #fff background. Then you will please the hacker news zeitgeist (at the expense of everyone else).
Can you explain your motive for focusing on the combination of reactive data and local deployment?
My first thought is that local-first = one user = I know what’s changing because I’m the only one using it. Perhaps valuable for monitoring external data feeds?
Yeah, I see your point. I've been assuming that local-first doesn't mean local-only. Even if it's just you, you could easily be polling something else to get data updates, as you suggest. (One original use case was a personal GitHub dashboard app that polled for new activity but provided local analytics).
I feel it should be possible to build certain types of apps that can view, query, and manipulate denormalized data without even being online. (Perhaps I shouldn't over-glamorize those days when one used to jam Access databases into Windows apps, but hopefully you get the point.) Reactivity just serves as a nice binding paradigm.
Honestly, just my personal confidence currently. There's very basic locking and store-wide read-write. I really aspire to some sort of CRDT implementation that can be considered a bit more state-of-the-art and tolerant to the innumerable challenges of that problem.
I think that CRDT and OTs are a bit overkill for TinyBase. Most apps are not text editors or figma. Most apps are just forms. The general solution for forms is to track which fields on a record have been modified by the user. The user’s change is the source truth and never gets overwritten by the network. Other fields are free to be updated collaboratively from the network. Once the user has synced that field it is free to be updated by the network.
This kind of solution can be built on top of TinyBase without CRDTs.
The main limitation I see with tiny base is that it has to load all data in one batch and modifying any record results in the whole data store being persisted to disk. Surely this can’t scale well…
The data is in memory, and the persistence strategy is kind of up to you. So it would scale reasonably if you put the sync/persist on a different schedule to the UI - perhaps when the browser is idle.
BUT of course, this is not a library to be used with data sets of billions of rows! I'd recommend a proper RDBMS for that ;)
No, there's no _actual_ RDBMS involved. The query engine uses a functional style of query that has similar concepts to SQL, but all the evaluation (and reactivity) is first party, in the library itself.
Of course it would be pretty easy/fun to persist your TinyBase content into a SQLite database, or pull a dataset from it. The demos load up TSV but you could easily do something similar from an actual database somewhere.
It could be made to be. The current persistence implementations are all batch but I believe they could be row, or even cell-level, theoretically. Nice idea.
I've been looking for something like this but sadly I can't use it because I can't rely on JS. I wish there was some decent local-first event based db that was language agnostic. Very few projects are reactive/streaming.
If anyone's interested my current workaround is piping JSONL between my processes (and into files) and it works pretty well, but lacks more advanced features.
I've been working on something that may fit your criteria. The API is kafka-like and it's whole goal is to provide an easy to use local-first stream datastore/data-engine.
It currently only has support for Rust and Python clients but working on expanding to node(JS) and C/C++ in the future.
Respect to the authors of course, however I am not sure if it helps solving common data sync problems to introduce one more datastore with it´s own special query language. Maybe I am misunderstanding the use case.
I do not want to take on CRDT lightly, since I want to make sure it doesn't balloon the library - it'd probably be an optional module. I feel like that's one of the next big things to tackle, so does v3.0 sound good? :)
In the meantime try opening up the drawing demo (https://tinybase.org/demos/drawing/) in two or more (of the same) browser windows and move things around. This is just via local storage but imagine how cool it would be to have that across broader collaboration...
I’m my limited experience, a reliable CRDT implementation necessitates a CDRT-first design baked into the core of any data / transaction model. Like I’ve heard some game developers say- multiplayer needs to come first because tacking it onto a single player game is a nightmare.
I need to revisit the space soon. Hopefully I’m wrong because I think CRDTs will play an important role in the next generation of applications. A smooth on-ramp for existing apps would do wonders for adoption.
No, there are no dependencies. It has its own in-memory data structure, and ways to serialize it to other, more persistent, formats. I could imagine an option for more fully fledged RDBMSs being those persistence layers though.
I didn't have much trouble binding PouchDB into a Vue reactive data store so the reactive part didn't seem as novel to me. What does seem novel is supporting a proper relational schema for this, whereas all the other options I know in this space are schemaless / key-value type approaches.
I wonder if t might be a high-value / low effort proposition to come up with an out-of-the-box setup with something like PostgREST that would then work end to end? This was the super compelling part for me with the Pouch/Couch combo, as with a couple of configs and barely any code your browser localStorage is suddenly sync'd and saved remotely in real time which is mind blowing when you first experience it.