When does a simple linear search become an issue? For example, a cos similarity function which runs even over a million of vectors is surprisingly fast. We can further shard vectors by user (i.e. if they can only search in their own data they uploaded), or even by document (for things like "chat with this PDF") then it's likely that most searches will happen over small ranges of vectors.
I suspect the main downsides would be increased CPU usage and having to load vectors to RAM all the time. However, for small projects with low RPS you could probably get away without a special database?
Does anyone know of any benchmarks where they compare vector databases to a plain linear search? How they scale in regards to increased vector count and RPS, etc.
Brute force search is both exact (100% accurate) and pleasantly linear – a predictable algorithm. CPUs and caches like that, so performance is much better than you might otherwise expect.
"Brute force doesn’t care about the number of neighbours, so its performance is 679ms/query regardless of “k”. When run in batch mode (issuing all 100 queries at once), average query time drops to 354ms/query."
This was 500 dimensions over a 3.7M dataset (the English Wikipedia), in 2014. So, ~700ms/search, or half that if you can batch several searches together at once. YMMV.
Do you mean using brute force (exact search) and not approximate nearest neighbor search? From my unscientific benchmarks, doing an exact search with 100k vectors takes about 1 second. Usually chatbots take much longer to generate the text, so that's still an acceptable time for doing exact search, which also means you will always find the best matches.
>Do you mean using brute force (exact search) and not approximate nearest neighbor search?
Yes.
>doing an exact search with 100k vectors takes about 1 second.
The speed can depend on the language/runtime of your choice and the number of dimensions. What language/runtime did you use, how many dimensions in a vector? (I heard too many dimensions for vector search is an overkill)
I suspect the main downsides would be increased CPU usage and having to load vectors to RAM all the time. However, for small projects with low RPS you could probably get away without a special database?
Does anyone know of any benchmarks where they compare vector databases to a plain linear search? How they scale in regards to increased vector count and RPS, etc.