I was guilty, for a time, of flinging 'CAP' around. Matters are, I believe, both simpler and far more complex.
Here's the simplest way I can imagine a scenario. This is very similar to what my company deals with all the time.
Given two, network distant data centers. Requests can land on either datacenter. A request can be a PUT or a GET.
We deal with phone calls, so we have to provide an exceptionally reliable and consistent service.
The problem: a PUT lands in datacenter A. All subsequent GETs have to be consistent with this PUT. Even if they land in datacenter B.
The rest of the problem: datacenters have to be able to function independently. This means that we can't block a PUT while it sync's to the other datacenter. The other datacenter might be down, or there might be some network delay, or whatever.
In my mind, it's simply impossible to have complete consistency and complete reliability when the redundant pieces are WAN connected.(1) One can only approach and approximate this, with more and more hardware, circuits and development complexity.
Is my assertion incorrect? I'm unclear if HyperDex helps me with this problem.
By the way, I'm pretty excited about this product overall. If I wasn't in catch-up mode at work, I'd be hammering out a proof of concept project on it right away. It seems pretty compelling.
And thanks for this offering!
1) I believe a 'thick client' can, almost, solve this problem. Consider some Javascript web app. It can be developed to transparently handle various network partition situations. More or less. But many of our requests allow absolutely no code on the client side. They are standard REST calls.
What CAP really says:
- if you cannot constrain your faults in any way,
- and your requests can be directed at any server,
- and you insist on serving every request,
- then you cannot possibly be consistent.
No shortage of systems that preemptively give up on C, A,
and P, especially C
End quote.
I think that's exactly correct.
Which leaves me with a nearly impossible problem. But it sure is fun to work on!
HyperDex performs best in a single-datacenter environment. You are right that synchronous replication between datacenters can (and likely will) be slow.
There's much work on this (my favorites are Walter and COPS from SOSP 11) that is on-going. We'll likely be throwing our hat into the ring soon as well.
You are right that pushing some logic to the "thick client" is one of the viable solutions.
Feel free to contact us as you work on your PoC. We'd love to hear about positive results and help with any rough patches you encounter.
Here's the simplest way I can imagine a scenario. This is very similar to what my company deals with all the time.
Given two, network distant data centers. Requests can land on either datacenter. A request can be a PUT or a GET.
We deal with phone calls, so we have to provide an exceptionally reliable and consistent service.
The problem: a PUT lands in datacenter A. All subsequent GETs have to be consistent with this PUT. Even if they land in datacenter B.
The rest of the problem: datacenters have to be able to function independently. This means that we can't block a PUT while it sync's to the other datacenter. The other datacenter might be down, or there might be some network delay, or whatever.
In my mind, it's simply impossible to have complete consistency and complete reliability when the redundant pieces are WAN connected.(1) One can only approach and approximate this, with more and more hardware, circuits and development complexity.
Is my assertion incorrect? I'm unclear if HyperDex helps me with this problem.
By the way, I'm pretty excited about this product overall. If I wasn't in catch-up mode at work, I'd be hammering out a proof of concept project on it right away. It seems pretty compelling.
And thanks for this offering!
1) I believe a 'thick client' can, almost, solve this problem. Consider some Javascript web app. It can be developed to transparently handle various network partition situations. More or less. But many of our requests allow absolutely no code on the client side. They are standard REST calls.