We currently allow 1.1 PB in a single "namespace" (zpool).
There are non-obvious ways to make a petabyte-sized zpool non-scary ... but even with those employed we still utilize raidz3 and have contingencies for rollbacks.
edit: for obvious reasons, that 1.1 PB number will grow by 50% in the very near future...
It's easy enough to jam enough disks into a rack with these 90disk 4u super micro jbods. The thing that always scared me (ESP with rsync) is how do you get performant metadata for a few billion files? Or even tens of millions.
And raidz3 resilver must be horrible at those densities!
Again. Just curious. Email at jmancuso@expandrive if you feel like chatting. I know we offer a similar product, but we are about to leave zfs for the above concerns. Wouldn't mind sending some business your way.
http://rsync.net/