So, my major concern with OrbitDB is the use of in-memory indexes to hashes. It's a very naive approach for multiple reasons. It just doesn't scale well. Node.js handles these particularly poorly, and develops memory leaks in my own experiments with AOILs (append-only immutable logs) inspired by Secure Scuttlebutt.
Though, there are some really hard limits to file locks / mutexes that are needed for a decent persistent K/V store. Since, pretty much any database is going to need an index, and an index is just a K/V store. So, in essence, if your platform can't be relied on to implement a good embedded K/V store, it can't be relied on to create anything other than toy databases.
I considered adding SnapDB to OrbitDB, but it would've required a lot of refactoring to support asynchronous lookup. I eventually realized I could do a lot of what I wanted to accomplish without the elaborate CRDTs like OrbitDB employs to keep an ordered log of information.
I would also add that, although it should be obvious, just in case it isn't, AvionDB now builds on OrbitDB which itself builds on IPFS. So, that's a big consideration for those looking to work with this.
(The below is mostly from personal experience and not terribly relevant to AvionDB and OrbitDB)
I've been meaning to do a more formal post-mortem on the project, but haven't gotten anything fully organized yet. I discovered all this after creating a project a while back for what was essentially a P2P image gallery built with IPFS and Electron. I wrote it in TypeScript, React (with Hooks), Redux Toolkit (which makes Redux a lot less insufferable than it used to be, my god), and SnapDB and OrbitDB (but not together). I also had go-ipfs running in a process managed by the Electron client, and included it with the build.
As a result, the app image binary would generally be in excess of 500MB. And this was just our 0.1.0, we hadn't gotten to the really good features yet. The performance was decent in the UI, but the app itself couldn't scale very well past 1000 images. go-ipfs was the real limiting factor at the time; it would just fail in really bizarre ways, and I'd have to not only write code for restarting the go-ipfs process in case it died, but also retry requests that didn't go through. I suspected it might've been due to me having pubsub enabled, but my ability to debug that was hamstrung by the fact that OrbitDB hard fails without pubsub enabled. I haven't tried it with go-ipfs 0.5.0, I think pubsub is enabled by default with that now, and I've already moved on from that implementation.
My biggest concern in terms of performance aside from go-ipfs (which was disappointing and unexpected), was the fact that due to the design, there would be multiple copies of the same data in-memory everywhere: the in-memory indexes used by OrbitDB, the data in Redux, the data in the DOM, etc. I'm able to effectively integrate all of those using a Rust desktop binary. I'm using Iced for the UI, and I've moved on from bundling IPFS entirely. It allows me to build desktop and WASM binaries from the same codebase.
It's possible I could probably make the Electron app work better at this point, but there are still the fundamental limitations I pointed out earlier.
For me, content-addressable hashes and IPNS signatures are perfectly fine. I value performance (CPU, RAM, and disk) over anything else, primarily because I'm developing P2P apps, and without a server, the nodes have to do all the work itself, including the UI. That's why I moved off of TypeScript and into Rust. PubSub would be nice, and OrbitDB has a decent implementation of Lamport Clocks and CRDTs for decentralized ordering of events, but it's just often overkill for most things that could just be content-addressable.
---
Just to be clear, I've made a lot of claims here, and I'm hardly an expert on any one of these things, so if I've gotten any of this wrong, I'd really like to know more. Also, the code I wrote earlier isn't available to the public. If I have any say in the matter, it will be locked away within an enormous warehouse, Indiana Jones style, hopefully never to see the light of day again.
Hey cryptoquick, I remember you from a while back. I'm https://twitter.com/aphelionz, one of the maintainers. Nice to see you :)
You're not wrong about the memory usage. We decided early on that the tradeoff would be more RAM than to take the performance hit with the I/O (either to and from IPFS or to and from the filesystem in the case of SnapDB).
That being said, I have been experimenting with removing indexes like the ones you pointed out in favor of generators that emit the values as they are traversed. There are two main efforts here, one is feasible and the other is more difficult.
1. The `entryIndex` inside of ipfs-log can probably go, and the ipfs-log#traverse function can be made into an async generator function that passes the oplog values up to the store
2. The indexes you linked to that hold the calculated STATE are harder to get rid of - they could be persisted to IPFS or the file system as well
Open to ideas on #2. SnapDB might work for all I know, as I haven't attempted it yet.
I can't speak to Electron, React, Redux, Iced, etc, but my guess is there are optimizations one can do there as well.
One more thing: The code in AvionDB is really more representative of how OrbitDB could be used to create a decentralized document store, but OrbitDB offers a lot more, and IPFS offers a lot more, too.
I think this project could be considered less of a library dependency, and more of an example OrbitDB project.
I totally agree with this. A lot of our work on AvionDB involves improving OrbitDB as AvionDB is basically a project built on top of OrbitDB. Exactly speaking we have 2 new store implementations:
Store.js [1]: This store manages all the "Databases" & the "Collections" within the respective "Databases".
Collection.js [2]: This is the store that implements the MongoDB-like interface so that you can run powerful queries, and manage "Documents" within the respective "Collections".
So, my major concern with OrbitDB is the use of in-memory indexes to hashes. It's a very naive approach for multiple reasons. It just doesn't scale well. Node.js handles these particularly poorly, and develops memory leaks in my own experiments with AOILs (append-only immutable logs) inspired by Secure Scuttlebutt.
What I mean is, take a look at this code: https://github.com/orbitdb/orbit-db-kvstore/blob/master/src/...
And this: https://github.com/orbitdb/orbit-db-docstore/blob/master/src...
I'd suggest perhaps building that in SnapDB:
https://github.com/ClickSimply/snap-db
Though, there are some really hard limits to file locks / mutexes that are needed for a decent persistent K/V store. Since, pretty much any database is going to need an index, and an index is just a K/V store. So, in essence, if your platform can't be relied on to implement a good embedded K/V store, it can't be relied on to create anything other than toy databases.
I considered adding SnapDB to OrbitDB, but it would've required a lot of refactoring to support asynchronous lookup. I eventually realized I could do a lot of what I wanted to accomplish without the elaborate CRDTs like OrbitDB employs to keep an ordered log of information.
I would also add that, although it should be obvious, just in case it isn't, AvionDB now builds on OrbitDB which itself builds on IPFS. So, that's a big consideration for those looking to work with this.
(The below is mostly from personal experience and not terribly relevant to AvionDB and OrbitDB)
I've been meaning to do a more formal post-mortem on the project, but haven't gotten anything fully organized yet. I discovered all this after creating a project a while back for what was essentially a P2P image gallery built with IPFS and Electron. I wrote it in TypeScript, React (with Hooks), Redux Toolkit (which makes Redux a lot less insufferable than it used to be, my god), and SnapDB and OrbitDB (but not together). I also had go-ipfs running in a process managed by the Electron client, and included it with the build.
As a result, the app image binary would generally be in excess of 500MB. And this was just our 0.1.0, we hadn't gotten to the really good features yet. The performance was decent in the UI, but the app itself couldn't scale very well past 1000 images. go-ipfs was the real limiting factor at the time; it would just fail in really bizarre ways, and I'd have to not only write code for restarting the go-ipfs process in case it died, but also retry requests that didn't go through. I suspected it might've been due to me having pubsub enabled, but my ability to debug that was hamstrung by the fact that OrbitDB hard fails without pubsub enabled. I haven't tried it with go-ipfs 0.5.0, I think pubsub is enabled by default with that now, and I've already moved on from that implementation.
My biggest concern in terms of performance aside from go-ipfs (which was disappointing and unexpected), was the fact that due to the design, there would be multiple copies of the same data in-memory everywhere: the in-memory indexes used by OrbitDB, the data in Redux, the data in the DOM, etc. I'm able to effectively integrate all of those using a Rust desktop binary. I'm using Iced for the UI, and I've moved on from bundling IPFS entirely. It allows me to build desktop and WASM binaries from the same codebase.
It's possible I could probably make the Electron app work better at this point, but there are still the fundamental limitations I pointed out earlier.
For me, content-addressable hashes and IPNS signatures are perfectly fine. I value performance (CPU, RAM, and disk) over anything else, primarily because I'm developing P2P apps, and without a server, the nodes have to do all the work itself, including the UI. That's why I moved off of TypeScript and into Rust. PubSub would be nice, and OrbitDB has a decent implementation of Lamport Clocks and CRDTs for decentralized ordering of events, but it's just often overkill for most things that could just be content-addressable.
---
Just to be clear, I've made a lot of claims here, and I'm hardly an expert on any one of these things, so if I've gotten any of this wrong, I'd really like to know more. Also, the code I wrote earlier isn't available to the public. If I have any say in the matter, it will be locked away within an enormous warehouse, Indiana Jones style, hopefully never to see the light of day again.