Have you all taken a look at collaborating/interoperating with Materialize.com ? You've built a composable query language on top of PostgreSQL; they've built an in-memory database that accepts PostgreSQL and behind-the-scenes builds composable reactive real-time materialized views on top of high-volume streaming data, based on the Timely Dataflow framework, to the extent that BI tools can run on top of it.
The one area that still holds back Materialize's developer experience, and I say this with all the love in the world, is... well, all the limitations of SQL, particularly when you're dealing with a chain of dozens of tables and views and CTEs for a complex analytical query. (I cannot stress enough that it's aeons better than writing thousands of lines of low-level stream handling code oneself!)
If only there was a sane layer on top that could make it far easier to reason about the queries you're typing...
The idea that some day, someone designing an application with real-time high-data-volume needs could build some combination of SDL schemas and EdgeQL queries that creates and awaits an optimized/tunable cascade of materialized views a query might need... and if built on Materialize's database this would mean that subsequent calls would retrieve fully real-time results for those views... is intoxicating. I'm excited to keep my eyes on this space.
Have you all taken a look at collaborating/interoperating with Materialize.com ? You've built a composable query language on top of PostgreSQL; they've built an in-memory database that accepts PostgreSQL and behind-the-scenes builds composable reactive real-time materialized views on top of high-volume streaming data, based on the Timely Dataflow framework, to the extent that BI tools can run on top of it.
They have a similar blog post on their journey here, which also started in 2014: https://materialize.com/blog-roadmap/
The one area that still holds back Materialize's developer experience, and I say this with all the love in the world, is... well, all the limitations of SQL, particularly when you're dealing with a chain of dozens of tables and views and CTEs for a complex analytical query. (I cannot stress enough that it's aeons better than writing thousands of lines of low-level stream handling code oneself!)
If only there was a sane layer on top that could make it far easier to reason about the queries you're typing...
The idea that some day, someone designing an application with real-time high-data-volume needs could build some combination of SDL schemas and EdgeQL queries that creates and awaits an optimized/tunable cascade of materialized views a query might need... and if built on Materialize's database this would mean that subsequent calls would retrieve fully real-time results for those views... is intoxicating. I'm excited to keep my eyes on this space.