Thank you! I actually did consider doing exactly what you said. Early on, I decided one my goals for the project would be to make it easy to swap back and forth between Mongita and PyMongo/MongoDB. For me, that meant getting as close as possible to their implementation and using things like BSON, ObjectIds, etc. For that, I sacrificed some performance but I think most people who would use something like Mongita would prefer a faithful reproduction over a few clock cycles.
Oh thanks for the insight, that makes a lot of sense -- pretty sure mongita is one of the first projects I've seen even attempt this, and it makes sense to target the platform that fits your use case (and obviously everyone who uses PyMongo) the tightest.
Your project looks very high quality -- benchmarks, tests, and comparisons are basically an indicator in my mind. Looking through the code you've also already left me (or someone else) space to try doing the SQLite as long as we implement it as an Engine[0] -- am I understanding that right? If I trace the code from database.py to engines/.py it looks like that. I really like the balance you've picked between pragmatism and space for expansion/modification.
A couple questions:
- How do you feel about type annotations in* the code (as opposed to just the comments, as far as I can see)
I like your idea. As I understand it, since SQLite performs better on most benchmarks anyway, why not have an engine subclass that utilizes SQLite itself?
This would be a little trickier than your envisioning. The engine class is the lowest layer but doesn't handle the slow bits like the finds, managing the indicies, etc. So to really get the benefit of SQLite you would have to pull those slow bits in.
I do really like the idea of offering a third option so that you have
- Memory (fastest)
- SQLite (almost as fast but compromises on perfect MongoDB reproduction)
- Disk (slower but is faithful to MongoDB)
Happy to discuss it more if you want to email me.
> How do you feel about type annotations in* the code (as opposed to just the comments, as far as I can see)
I used type annotations for a while when they came out but didn't find them to offer advantages over docstrings. The world might have moved over to type annotations though and I've not been aware.
> PyPy? I wonder if you'd get a ~free speedup
I'm getting a lot of suspicion right now from other commenters who are convinced I have my thumb on the benchmarks and I don't want to give them any more excuses :tears-of-joy:. But joking aside, you're probably right and it's not something I had considered.
I could be wrong on that though.