Disclaimer: I work on an open source alternative to Algolia called Typesense [1]
If I had a dollar for every time I've heard Algolia is expensive! And if I had a dollar for every time I've said that statement!
If cost is an issue, definitely checkout Typesense. You can self-host it for free, or use the hosted Cloud version [2]. There are no per-record or per-search charges, which makes it very economical even for high volume use cases. In either case, I've seen savings of up to 96% in some cases compared to Algolia's pricing.
Tested out TypeSense when we were evaluating search options for work. Performance and features were amazing, are there any plans to add support for larger-than-RAM indexes?
While you probably aren't using B+-tree indexes like databases, can you use a setup like them were you can create a "paged" data structure and use some buffer pool to bring pages in and out?
Certainly that's possible but doing fast type ahead fuzzy search requires keeping large parts of the indexing data structure in-memory.
There are, of course, many use cases where one does not need that speed. Eventually, like Redis we will probably have an option to use SSD storage smartly for these non-latency sensitive use cases. That's not our immediate focus though.
We have fairly large variation in index size, from ~100MB through to 30-50GB, and probably larger still in future. In the order of thousands of indexes as well.
Application level sharding/merging for more consistent index size is on the list of things to do, but given the complexity of that it’s a “not before we really need to” kind of task haha.
If I had a dollar for every time I've heard Algolia is expensive! And if I had a dollar for every time I've said that statement!
If cost is an issue, definitely checkout Typesense. You can self-host it for free, or use the hosted Cloud version [2]. There are no per-record or per-search charges, which makes it very economical even for high volume use cases. In either case, I've seen savings of up to 96% in some cases compared to Algolia's pricing.
[1] https://typesense.org, https://github.com/typesense/typesense
[2] https://cloud.typesense.org