Strongly consistent FoundationDB = Likely similar write performance to CockroachDB or TiDB when you avoid secondary indexes.
Secondary indexes in "distributed strongly consistent" systems is what ruins performance: because each index is +1 write to another "index table" (... +1 raft quorum check!).
I don't think FoundationDB has "secondary indexes" to begin with, so one may never run into the +1 write per index issue.. it's just a layer on top of TiKV (equivalent to RocksDB in CockroachDB).
I am not sure if I am quite getting your point here, but if we're talking about indexes, I've just implemented indexing using FoundationDB and writing to the index happens in the same transaction as the main data write (not really sure why it would ever be otherwise). Definitely not "+1 raft quorum check".
Calling FoundationDB "just a layer on top of TiKV" is… well… :-)
I can't speak to FoundationDB specifically but in general, why would secondary indexes require a second quorum check?
Can't secondary indexes just be implemented in your state machine?
I.e. your state machine handles the insert and also writes a secondary index at the same time. Every state machine could do this identically off the log.
> Can't secondary indexes just be implemented in your state machine?
Secondary indexes are implemented as an additional smaller table in most databases (Index -> Primary Key), which requires a write, too.
Don't take my word for it- run some benchmarks yourself to see the performance cliffs appear in these "distributed strongly consistent" databases when you have a handful of indexes.
It's why many require 10x the hardware to scale past the performance of a single node postgres / mysql.
> Calling FoundationDB "just a layer on top of TiKV" is… well… :-)
FoundationDB recommends running TiDB ontop to get an SQL layer...
>FoundationDB recommends running TiDB ontop to get an SQL layer...
You might be mixing up projects here - TiDB/TiKV are completely separate projects from FoundationDB. I don't think FoundationDB has a supported SQL layer.
> Secondary indexes are implemented as an additional smaller table in most databases (Index -> Primary Key), which requires a write, too.
> Don't take my word for it- run some benchmarks yourself to see the performance cliffs appear in these "distributed strongly consistent" databases when you have a handful of indexes.
I'm not saying secondary indexes aren't a performance cliff. I'm saying I can't think of a reason they must require additional quorum on top of the initial message that would require a secondary index update.
Since you said:
> Secondary indexes in "distributed strongly consistent" systems is what ruins performance: because each index is +1 write to another "index table" (... +1 raft quorum check!).
They will be partitioned by a different key. You may in fact have to touch two extra quorums - one to delete the old entry and one to insert the new entry.
This depends on the specifics of how your database works, of course. I’m not sure this would involve multiple consensus quorums in FDB for instance, given that I believe it instead relies on a centralised timestamp service.
I don’t think there is any officially recommended way to run SQL ontop of FoundationDB. I suspect the TiDB comment you saw was someone speculating about an idea maybe (I vaguely recall some GitHub issues about it).
There are some well supported document interfaces like the official record layer or Tigris though.
Secondary indexes in "distributed strongly consistent" systems is what ruins performance: because each index is +1 write to another "index table" (... +1 raft quorum check!).
I don't think FoundationDB has "secondary indexes" to begin with, so one may never run into the +1 write per index issue.. it's just a layer on top of TiKV (equivalent to RocksDB in CockroachDB).