It would be awesome if this could use Quickwit as a backend which is a new promising alternative to Elasticsearch, I’ve been using it internally and it’s much more lightweight and easier to run.
Wow man Quickwit looks amazing. Probably would have reached for it when I started Logflare if it was around.
I have not seen anything open source that really separates compute and storage like that in one package. You can setup Clickhouse to use an object store but there's a bunch of nuance there. People are starting to hack together something similar using DuckDB on Lambda, basically as a query engine on top of S3. I wonder if you can use DuckDb to get a SQL interface going?
Going to have to take a close look for sure. SQL is definitely a blocker. With Supabase we give people the ability to query their logs with full SQL. And with Logflare Endpoints you create an API endpoint with a SQL query.
I'm convinced that a combo supabase + quickwit can be quite powerful.
It's possible to make Quickwit support simple SQL queries like "SELECT xxx FROM yyy GROUP BY something" quite fast.
If you are eager to explore a POC with quickwit, let's have a quick chat, here is my email francois [at] quickwit [dot] io.
Supporting quickwit and its query language is definitely feasible and would not require quickwit to have SQL support. However, we've got SQL DBs on our roadmap at the moment, so it might be a while until we get to quickwit
