It's interesting that Netflix decided to write this rather than using Amazon's own DynamoDB. I wonder specifically if DynamoDB was too expensive (as I have found), or was there some other reason?
This is completely different than DynamoDB. Essentially this is a framework for sharding or horizontally scaling multiple datastores (MySQL, Memcache, Redis, etc).
Why would you use multiple datastores? Isn't it much more difficult to atomically store data this way? Assume a transaction succeeds in store 1, and fails in store 2, you'd need to roll back the transaction in store 1 after it has committed (!)
Because there is no such thing as a one size fits all datasotre. Some data may make way more sense stored in a relational database, while some data makes more sense as key:value pairs. Some data may be infrequently accessed and stored to disk while others are stored in memory for quick access. Trying to make a single datastore work for many use cases will cause more pain than it is worth.
You wouldn't likely use multiple data stores for the same data. That is, you probably wouldn't have a single system which stores data in both Redis and MySQL.
However, you can imagine different use cases for each of these stores and you can potentially increase operational efficiency by having a common layer which handles the distribution of data for each store.
This is the sort of project you do to get cross-region replication on your existing data stores, many of which look more like PostgreSQL than KV stores.
