What's the clustering story these days? Last I checked, Redis Cluster still had lots of issues losing writes, and the design doesn't seem to have been revised since Aphyr ran Jepsen tests on it back in 2013, making it (at least from my perspective) practically useless for anything that requires distributed consistency, which means most things that I look to use a distributed data store for.
I've also been warned by fellow devs that much of the clustering logic is actually done in the client, and there's historically been a lack of mature clients for all languages. Even if you find a mature client, the complexity of the implementation implies that not all clients may behave identically, or may have different bugs. Then there's the issue of lock-in, where you become dependent on a specific client library and its development lifecycle. I don't know if all of this is true, but I also don't hear a lot of people talking about Redis Cluster these days.
I know you can use Redis in master/slave mode via Sentinel + Twemproxy, though even this solution seems to have some issues with data consistency. Running all three also appears a lot more complex than an integrated system.
I see a lot of comments implying that Redis is mainly used in single-node setups, so that might be where it shines?
Hello! Redis Cluster has yet the same tradeoffs, because they were basically designed, and were not shortcomings. Btw Aphyr never tested Redis Cluster, and does not made any sense to test it because he tests things for strong guarantees so it should test Redis-CP instead, for instance, a consistent store for Redis implemented into a module using Raft.
I believe that for the Redis use case the cluster tradeoffs make sense because:
* Best effort consistency in the practice work quite well, even if it does not have any guarantee, but does certain things to avoid losing writes in trivial ways.
* If you want to cluster Redis, you want Redis, not a cluster that automatically becomes a lot more slower, memory hungry, and so forth. So replication should be asynchronous.
However what I may change in the future, and there are plans for that, is to add a failover strategy that does not just pick the replica that is more ahead in terms of received writes: but even stop the failover if there isn't a majority of slaves reachable. This improves certain properties, and if well orchestrated can also show strong properties if writes are acknowledged only after transferred to the majority of slaves.
Redis Cluster is used in many organizations right now. The next step is to improve it (in different ways than having strong properties mostly), and provide an official proxy for it.
Where can I find information about "Redis-CP"? Google isn't giving me anything.
I don't really understand the nature of the "tradeoffs" you mention. As Aphyr pointed out, Redis (with Sentinel) is not safe to use as a database, a queue or even as a lock service. That really narrows the possible use cases. I can absolutely see Redis being appropriate for many "lossy" applications: Caching, web sessions, rate-limiting counters, precomputed analytics data, intermediate outputs from distributed data processing pipelines, that kind of thing.
But the use cases where I'd reach for Redis seem a lot fewer than with data stores that have high consistency guarantees, such as FoundationDB, TiDB/TiKV, CockroachDB, or Cassandra/ScyllaDB. With the exception of TiDB, these are a bit easier to reason about since there's no Redis/Sentinel/Twemproxy split.
On the other hand, I certainly appreciate the specialized data structures and Lua support that Redis comes with.
Interesting approach! Reminds of an option my group considered a couple years ago, where we purposely didn't set up read replicas, but considered failure to connect/read of the master as a 'C' failure. Hashing keys to multiple masters and we had "highly available" redis with stronger consistency.
We ended up not using it, but I always wanted to revisit the pattern.
BTW, thanks for such a wonderful piece of software!
BTW, Without a critical mass of users regularly vouching, the data isn't much help.
if someone is happily using the package in production, that won't tick any download numbers even though it's arguably the thing anyone considering a package would really want to know.
Thanks for this link. Some developers see more than others and can articulate differences very well. I wish I'd been scanning his blog for years!
> "There is more: I believe that political correctness has a puritan root. As such it focuses on formalities, but actually it has a real root of prejudice against others. For instance Mark bullied me because I was not complying with his ideas, showing problems at accepting differences in the way people think. I believe that this environment is making it impossible to have important conversations."
It absolutely is. Concern trolling means disingenuously expressing concern about an issue in order to undermine or derail genuine discussion, which you've done here, and your other remarks about "SJWs" lay bare your motives.
I'm asking you to knock it off and discuss in good faith next time.
In Switzerland we had quasi slaves till the 1980s (Verdingkinder). This is horrible and the skin color of the victims was most of the time white. And many of them are now old white dudes.
There are probably more old white dudes in Europe that suffered directly from slavery than there are Americans with Grandparents who were slaves. So please don‘t judge people by their skin color.
SighThe responses to your question (systems falling over on Jepsen) are already the most depressing kind:
- that's not our use case (then why are you advertising distribution???)
- they didn't use this feature (WELL THEN, SHOW THAT FEATURE PASSING THE TESTS)
- we're not that kind of distributed (oh, you mean the one you can use?)
Jepsen has been the first step for trusting anything that involves data and distribution. Everything is going to fail under his hard stare, it just comes down to how much BS smokescreening is done by the purveyors of the software to judge how to trust it.
I've also been warned by fellow devs that much of the clustering logic is actually done in the client, and there's historically been a lack of mature clients for all languages. Even if you find a mature client, the complexity of the implementation implies that not all clients may behave identically, or may have different bugs. Then there's the issue of lock-in, where you become dependent on a specific client library and its development lifecycle. I don't know if all of this is true, but I also don't hear a lot of people talking about Redis Cluster these days.
I know you can use Redis in master/slave mode via Sentinel + Twemproxy, though even this solution seems to have some issues with data consistency. Running all three also appears a lot more complex than an integrated system.
I see a lot of comments implying that Redis is mainly used in single-node setups, so that might be where it shines?