This seems to be a pretty good implementation of FriendFeed's schema-less MySQL ( http://bret.appspot.com/entry/how-friendfeed-uses-mysql ) and this option would probably beat most of the simple key-value stores in terms of perfomance, backup and replication.
One should note thought that more modern NoSQL databases aren't only key-value stores. For example, Redis offers KEYS command which can scan the store by a pattern. Tokyo Tyrant (another "key-value" store) offers Lua scripting.
Tokyo Tyrant's Lua scripting of the db server has been great when developing http://wasitup.com - The following example would be hard to implement in an efficient manner with a traditional key/value store without scripting capabilities:
-- Return keys with a given prefix for a particular node where there are a
-- known total of nodes. Uses the reamainder of the keys' hash values to
-- segment the keys.
function slice(prefix, index_and_total)
local args = _split(index_and_total, "\t")
if #args ~= 2 then
return nil
end
local index = tonumber(args[1])
local total = tonumber(args[2])
local res = {}
local keys = _fwmkeys(prefix)
for i = 1, #keys do
local key = keys[i]
local url = string.sub(
key,
string.find(key, ":", string.find(key, ":", 1, true) + 1, true) + 1
)
local modulo = tonumber(string.sub(_hash("md5", url), 1, 10), 16) % total
if modulo == index then
table.insert(res, key)
end
end
if #res == 0 then
return nil
end
return table.concat(res, "\t")
end
> this option would probably beat most of the simple key-value stores in terms of performance, backup and replication.
I wonder why do you think so? Here's a sample test of plain and memcached mysql vs tokyo tyrant (http://www.mysqlperformanceblog.com/2009/10/19/mysql_memcach...) It was done without the additional inserts for the index table... I don't see how it can be anywhere close to the TT's performance for example. While mysql's replication is very easy, TT's replication is trivial. Backup is probably comparable - just flush and snapshot...
Unless you're comparing it to mysql+MC. Then yes - mysql's performance at 99.8% hits to MC can be ignored. We're left comparing MC with something else. Things might change dramatically if you have a write-heavy application. Or when you have more data than ram. Author's benchmarks will always show great results - let's wait for some real-world data.
My data is from a real-world application. One that gets plenty of traffic. Friendly has served almost 50 million hits since we put it in to production two weeks ago.
It actually happens to be quite a write-intensive workload. We have definitely seen a slow down on writes, but the speed up on reads has more than compensated for it.
We do have more data than RAM. But, most of the data that actually gets read is fresh, so it's hot in cache.
Of course Friendly isn't going to be right for everybody. But, I just want to make it clear that my data isn't from synthetic benchmarks. It's from production.
I didn't mean to imply that those benchmarks are synthetic. What I had in mind is that when you write some software, you will improve it to solve your problem better than other software (why would you do it otherwise ;) ). I meant real-world as in -- from various sources and applications -- just so that we can see how useful it is and in what situations it's better / worse.
Most != all :) Tokyo Tyrant and Redis are some of the fastest key-value stores and when I stated that it wasn't with them in mind. There are tons of other key-value stores that can't beat the perfomance of MySQL.
I agree that replication in Tokyo Tyrant is generally good, but MySQL's is much more mature and one can get help more easily. It should be stated that I am the author of LightCloud (a distributed db built on top of TT) and I am running big installations of both TT and MySQL, so I speak from experience.
Neat - I just tried it. I am fairly much "in like" with both activerecord and datamapper, but Friendly is cool. I wonder how much work it would take to support PostgreSQL.
Friendly looks especially good in MySQL shops - otherwise I would suggest taking a good look at MongoDB (or Redis, CouchDB, Cassandra, etc.)
The difference is that it's schema-less on the backend. It's using MySQL to store JSON (I assume), so schema changes happen at the code level, not at the database level. Like a NoSQL approach that uses MySQL as the backend.
One should note thought that more modern NoSQL databases aren't only key-value stores. For example, Redis offers KEYS command which can scan the store by a pattern. Tokyo Tyrant (another "key-value" store) offers Lua scripting.