I don't want to throw rocks, because I used to work for Larry (in fact, I wrote the original codebase for Ma.gnolia.com) and I like him.
But I distinctly remember talking about backups and how a raw sync of the innodb files wasn't really what we needed (it could sort of work in the original deploy of ma.gnolia, but would not scale as the site scaled). I'm kind of surprised that the community has been so tolerant of this failure. Personally, I'd be furious if I found out that a site I trusted lost all my data because they hadn't even tested their backup system.
> I'd be furious if I found out that a site I trusted lost all my data because they hadn't even tested their backup system
Exactly - first rule of backups - test them. For a database, it can be as simple as using your nightly backup to create your dev or test database or whatever. I guess it isn't feasible with a half terrabyte database every day, but to not have at least a several week old backup set is somewhat unforgivable.
On the other hand, I bet he will never ever make the same mistake again, and it will make a lot of people on here think long and hard about their own backup strategies ...
Well, half a gig is trivial to backup. Half a terabyte is not. Assuming you meant to say 500 gigs, then yes, there is difficulty, but it's not uncharted territory. He had two xserves -- perfect for replication.
But I distinctly remember talking about backups and how a raw sync of the innodb files wasn't really what we needed (it could sort of work in the original deploy of ma.gnolia, but would not scale as the site scaled). I'm kind of surprised that the community has been so tolerant of this failure. Personally, I'd be furious if I found out that a site I trusted lost all my data because they hadn't even tested their backup system.