As an operator, having fewer dependencies I have to deploy and worry about is a boon to my productivity.
If you were building a kafka like system in house for private use only, then yes I agree with your sentiment. If you are building something to be used by hundreds or thousands of organizations then the cost benefit tradeoffs shift to where it probably makes sense to pull the consensus logic into the primary application itself.
As an operator, shouldn't you worry if something battle-tested, that implements extremely critical functions that are tricky to implement correctly, is replaced with something that is unproven?
As someone who enjoys sleeping at night, I wouldn't poke something like that with a stick before it has matured for a few years, and before many brave (?) operators have smoothed out most rough edges by landing on them repeatedly with their faces.
Yeah, don't deploy it when its in beta or its first few releases. At some point it will be stable and safe at which point everyone from then on will benefit.
If you were building a kafka like system in house for private use only, then yes I agree with your sentiment. If you are building something to be used by hundreds or thousands of organizations then the cost benefit tradeoffs shift to where it probably makes sense to pull the consensus logic into the primary application itself.