On: We Built a Streaming SQL Engine

[leontrolski]

I've been playing around with writing an application-orientated Incremental View Maintenance library along these lines:

https://stepping.site

It's based off of the DBSP paper, which counts Materialize's Frank McSherry as an author. Definitely not ready for the prime time, but any feeback welcome!

[gz09 → leontrolski]

Very nice! Are you using the dbsp implementation from the paper authors for this (e.g., https://github.com/feldera/feldera/tree/main/crates/dbsp)?

[leontrolski → gz09]

I didn't - I wanted to build something I could understand, and the rust implementation comes with a load of (reasonable) complexity, some resulting from the execution model, some from performance requirements.

stepping has 3 Zset implementations - in memory, sqlite and postgres. Currently considering whether to write a small rust one.

There's some more details here: https://stepping.site/docs/internals/how-it-works/

[gz09 → leontrolski]

neat! I sent you an email

[leontrolski → gz09]

Ahh, I see you're a cofounder at feldera :)

I have a couple of dbsp questions if you're feeling generous and have 10mins free - reach out ojhrussell at gmail