I'd like to see someone provide a service in a similar vein to this but just for photos.
What I'm thinking is a service that uses Glacier to store my photo library and some type of front end service (like dropbox) that keeps a low res version of that photo along with some meta data about it.
My reasoning for this is that we all build up pretty significant photo libraries (mines already over 60GB) and I'm always trying to make sure I have them backed up. I currently use a paid plan at Dropbox so I can put them all up there but it's kind of a waste since I hardly ever pull many of them down again. Every once in awhile I browse through them looking for certain pictures that I might need to get a copy of (which is why I'd need the low res copies easy to access/browse) and then be able to choose which ones to pull down from glacier. The other good thing about a service like that is the need is not typically immediate.
Maybe I'll look into building this since it's something I'd love to have for myself!
I'd be interested in seeing a service like this but for professional photographers. A friend of a friend is a wedding photographer. The work flow for something like this is to have multiple shooters each taking ~25GB worth of photos for one shoot. Then they process the worthy photos, making 3 large files per photo (RAW, .psd, and a final .png). During the turnaround time between wedding and delivery, it would be ideal to keep a copy of each of these files somewhere, but at least keeping the RAW somewhere safe is a minimum requirement. We were trying to figure out a feasible backup solution for the volume of data that such a shoot would require. The time to upload this data is longer than the delivery time for the photos, given the current available upstream bandwidth.
We came to the conclusion that putting files on BluRay disks or hard drives and keeping them physically separate was the best solution for the cost. The problem is that disks take a long time to burn, and external hard drives aren't the best archival medium (not to mention they don't always travel well).
Has anyone here solved the problem of data backups where bandwidth limits make pure online/cloud storage infeasible?
Have a look at The OpenPhoto Project. It's an OpenPhoto photo platform you can sign up to use or install yourself. Works with S3, Dropbox, Box.net, local filesystem or anything else you want to write a little code for.
Interesting, but we run into the same issues as before. We are either storing locally (backups are good, but local storage doesn't have quite as much redundancy as a data center), or taking infeasible amounts of time to upload to a remote backup.
Ideally, we would be able to ship BluRays to a data center (USPS has terrible latency, but great bandwidth!), where they would load them into their servers. If anything happend to the local copies, we could either have them ship us back a hard drive or BluRays, or download the lost files (downstream is much faster for us, and because we are theoretically only downloading a subset of what we backed up, direct downloads are feasible).
The issue I run in to in my thought experiments is that a data center doesn't necessarily want to physically handle data shipped to them or create recovery media to ship back.
Actually, all that is really needed is a secure place to archive the BluRays. A way to view and download backed up files is a nice-to-have, but not required.
Backblaze should provide an option for you to ship them a hdd or blu ray disk. They can store those however you want but if/when you need it you can restore files over the Internet or they can ship you physical media.
Probably not cost effective for them but that's what you're looking for, I think.
I had this _exact_ idea when Glacier came out, and even registered a domain with a splash page. Still thinking about building it but lacking time. I was looking at specifically storing RAW files for photographers and generating low-res versions to show in some sort of library.
Had exactly the same idea myself. Whoever makes it work first is probably onto a winner (I probably won't pursue it myself due to lack of time, but I'd love to use such a service).
I use Backblaze--$50/year--primarily for photos. My photo drive sits at around 900GB currently. It's entirely in Backblaze, in case it ever explodes and local backups fail.
I let it handle other drives, too, although I exclude my media/music/download/etc drive for simplicity.
Is it encrypted on the client side? That's pretty much my only requirement now, since I have separated my two use cases (backups and syncing). I use encfs on Dropbox for encrypted syncing, and it works great, but I need backups encrypted on the client, and I haven't found anything great and cheap yet.
Actually it is a yes, they cannot decrypt your data as they do not keep your password around. You have to enter it every time you want to restore a file, and it is not saved on their side only used to get the files then thrown away again.
Of course, if you have so sensitive stuff that you cannot accept it ever being on another server in clear text then it still a no on that question.
That does sound like a good idea! Sounds perfect for S3, can just store the low-res/thumbnails in S3 for browsing via a web page, then retrieve an occasional photo if necessary through glacier.
I have actually been toying with building out something like this as well. My wife is a photographer and building something around dropbox and glacier only made sense.
What I'm thinking is a service that uses Glacier to store my photo library and some type of front end service (like dropbox) that keeps a low res version of that photo along with some meta data about it.
My reasoning for this is that we all build up pretty significant photo libraries (mines already over 60GB) and I'm always trying to make sure I have them backed up. I currently use a paid plan at Dropbox so I can put them all up there but it's kind of a waste since I hardly ever pull many of them down again. Every once in awhile I browse through them looking for certain pictures that I might need to get a copy of (which is why I'd need the low res copies easy to access/browse) and then be able to choose which ones to pull down from glacier. The other good thing about a service like that is the need is not typically immediate.
Maybe I'll look into building this since it's something I'd love to have for myself!