I'm not sure if that's what kondro meant but what I would like to see is a virtually unlimited size ("elastic") volume where the local disk acts only as the cache.
This would make a wide range of big-storage use-cases ridiculously trivial - those where only ~10% of the data-set is frequently accessed.
I.e. one could lazily scale the expensive local storage with throughput-demand, while the S3 backing store takes care of the long-tail (which can easily be many terabytes long when you're dealing with media files).
Let's say you expect to grow to 20TB of data. Storing that for one month on S3 costs $2,560 (standard) or $1700 (reduced redundancy). In contrast, a Dell R515 with twelve 2TB drives costs $7,000. In a year that's one-third to one-quarter the price of using S3.
Implementing a tiered storage system yourself is pretty complex. Using this S3 gateway might be simpler, but it's not trivial (e.g. you'll need VMware ESXi just to get started).
Well, I only glanced at their current offering, missed the VMware part. My request was mostly wishful thinking.
I.e. instead of VMware it'd be more useful for us to hook in with a FUSE-layer or a patched variant of a filesystem such as GlusterFS.
You're of course correct about the pricing. Their current prices cover some middle-ground but would need to be discounted to make it feasible for larger deployments. However, at the low-end (your 20T figure) the price seems already justifiable when you factor in staff and infrastructure costs (rack+power alone make up for half of the difference).
ok, when you get to the TB of storage part, things get cheaper running in house, but couple of notes:
1: the dell at $7k does not include power, and your 12 2Tb drives gives you 20Tb usuable with RAID6 (loosing 2 drives). if you loose more than 2 disks, you are screwed... so, you need to back that up somewhere...
2: you need someone to manage that machine also...
3: ESXi, for what you would need here (8gb ram or so) is free, unless you want support....
i think in all fairness, that depending on the amount of storage you need or want, its swings and round-a-bouts... i like the idea, but i would also like the idea of having a box in house with a lot of storage (like the big dell) and only select some parts for off site backup... this is what i do... most of my stuff is stored locally (RAID 1, Thecus NAS, Drobo) and only important stuff (music and videos i bought, photos i took, etc) is backed up to the "cloud"...
This would make a wide range of big-storage use-cases ridiculously trivial - those where only ~10% of the data-set is frequently accessed.
I.e. one could lazily scale the expensive local storage with throughput-demand, while the S3 backing store takes care of the long-tail (which can easily be many terabytes long when you're dealing with media files).