Hacker News new | past | comments | ask | show | jobs | submit login
The Dropbox terabyte conundrum (sixcolors.com)
105 points by jaimebuelta on Sept 23, 2014 | hide | past | favorite | 73 comments



Shameless plug for my product, ExpanDrive

http://www.expandrive.com/expandrive

Mount Dropbox as a network drive. Smart local cache. Access the data on demand without syncing the repo in first. Also supports gdrive, s3, sftp, onedrive, box and more.

Makes a 1TB account make a lot more sense if you only have a 128GB SSD. Use selective sync with the primary client to only sync a portion of your Dropbox. Then use ExpanDrive to offload the rest and access it as needed.


On the product page, please list your price more clearly. It was non-trivial to figure out what your product actually cost. A walkthrough of my thoughts:

1. He said product, so it's probly not free. I wonder how much it costs?

2. Scrolls through the whole page I don't see a breakdown of price. Hmmmm

3. ctrl-f "$" Ahh, I see it says upgrade pricing is $24.99. Wait, does it cost that much, or is that a special "upgrade only" price?

4. looks around on the page more Ahhh, there's a store link, I hope it lists the price there!

5. Ok, so the price is $50 for a single-user license. Why was that so hard to find?

I don't mean to be rude with these remarks; your product seems perfectly relevant and it's awesome if it works as you say. But as a customer, I found that a bit confusing and I thought you as the business owner may want some feedback about one particular users experience.


Fair. I want people to download and not think immediately about price. It's not cheap. I see your point and you might be right. But it's not an entirely uncommon/bad pattern. I had a/b tested it a while back.


I'd like to note that an a/b-test saying it's more profitable doesn't mean you should automatically follow it.

A random example: The people who don't download your software because they can't find the price might think very negatively about your product.


> A random example: The people who don't download your software because they can't find the price might think very negatively about your product.

If that doesn't hurt your profit, shouldn't you still do it?


But how do you measure that it doesn't hurt your profit? Conversion rate may go up, but if those not buying because they felt lured with a lower price, through negative mouth-to-mouth advertising, make fewer new potential customers visit your site in a few months time, profit could eventually go down.


Negative mouth to mouth because you had to go to the store to see the price options? There are two, fwiw. Lots of people pick the lifetime upgrade option.


Looks like a nice product, but $50 for a single user is quite steep, IMHO.


What would you pay?


> Shameless plug for my product, ExpanDrive

Please take this as a compliment, but I didn't realize you needed to "plug" ExpanDrive. I sort of assumed that it was produced by a large company. Anyway, thank you for a wonderful piece of software, and for continually developing it.


Thank you for plugging this! Exactly what I was looking for. Agree with the other poster that your pricing should be more prominent; I thought it was $25 at first glance which (naturally) makes one disappointed to see it's actually $50, though it seems well worth it.

Is Yosemite currently supported? Would it make sense to support iCloud Drive too?


Yup. Still beta but I'm using it!


ExpanDrive is nice (and a quite appropriate plug), but it doesn't quite hit the sweet spot.

The issue is that products like ExpanDrive make all online storage drives to which you have to copy first, and upstream is still relatively slow even in countries with cheap high bandwidth downstream. It's still just FTP with a nicely integrated interface.

The ideal would be something that keeps the files locally until they have been synced in the background, and then removes the local copies whilst still allowing you to access them in the same way as if they were local copies. I.e., placing them in your cloud drive is instant, even though it actually takes a while for them to be transferred.

No, that's not an easy problem to solve, but anyone who does solve it will get my money.

(Really awesome would be if it intelligently kept local copies available based on usage and available space. The bottom line is: not having to consciously worry about where to put the files.)


That's exactly what the latest version does. Async writes. Large local cache. Stream out changes in the background.


Any arguments over https://itunes.apple.com/us/app/airfile-cloud-manager-for/id...

+ Any plans for an iOS version?


Yeah ExpanDrive is a full network filesystem. Air file is like a drag and drop ftp client.

iOS version is almost done. Tried to get it out for iOS 8 launch, but didn't quite get there.


I just want to point out that spacemonkey (spacemonkey.com) handles this very elegantly.

spacemonkey was designed from the beginning to handle storing much more data than you'd ever be able to store on your laptop hard drive. The way it does this is it actually mounts itself as a FUSE filesystem and keeps it's own LRU cache of your spacemonkey data on the drive. If you access a recently created or accessed file it will just load from the disk cache. If it detects that you are running out of hard drive space it will automatically prune the stuff that hasn't been accessed in a while (so spacemonkey never causes you to have an "Out of disk space" error). you still see it though, it tells you its size, metadata etc. and if you try to open one of those files it will grab it from the network transparently behind the scene for you it just might add a little bit of latency (as if you were accessing something on a network share).

In 2014 you should not have to be thinking about what to "selectively sync" the service you use should just figure it out for you (and let you customize it if you need to)

disclamer: I used to work for spacemonkey but do not any more, although I think they make a great product and are a super smart team.


SpaceMonkey was recently acquired by Vivint, Inc. "The Space Monkey team is excited to announce that the company has been acquired by Vivint, Inc." I don't hold out any hope that SpaceMonkey as a product will continue on, or if this was more or less an acquihire.


Co-founder of Space Monkey here. I can assure you this was not an acquihire. At the highest levels of the org from CEO and down, Vivint believes in the Space Monkey vision. We're super excited to have their backing, and think you will be excited to see where this goes from here.


I have this exact problem. The original reason I use Dropbox is now the thing that annoys me about trying to just store all my files there in the cloud.

It's like I need a ~/Dropbox and a ~/DropboxCloud where the ~/DropboxCloud is just virtual, not real disk space on my HDD.


Bitcasa's Infinite Drive is exactly what you're describing for ~/DropboxCloud.

Unfortunately it looks like they've changed focus to the enterprise.

https://bitcasa.com


I think you've articulated a great point about the current trend in the Cloud services and fast SSD personal computers. I have personally stayed away from the large storage plans at services like Dropbox because, as you point out, they would fill my entire 128GB Macbook Air instantly. But that seems to be the common theme with laptops these days – blazing fast hard drives at the cost of larger storage.

I actually think iTunes Match does a good job of solving this problem, on a media storage basis. I have my whole library "uploaded" to iTunes, but then I can selectively download and listen to songs on any device when I need them, even to my 8GB phone. Now my music library doesn't clog up my computer or phone or tablet, etc.

If dropbox found a similar model, I would sign up for that in an instant.



(Disclaimer: tooting my own horn)

It sounds like you want my PhD thesis project.

Link: https://github.com/jcnelson/syndicate


This is a very interesting problem, espeicially since consumer computing devices quickly shunted to small storage space when SSD's arrived. A few years ago, a laptop with 500GB of hard drive space was rather small! Nowadays, a 500GB SSD is top-tier. Granted, SSD capacity is quickly rising, but with the combination of small computing like phones and tablets, space is at a premium.

Funnily enough, this problem has been solved before! If you look at distributed file systems, especially something like Coda (http://en.wikipedia.org/wiki/Coda_(file_system)), they are designed to make the local computer a "thin client for storage". Basically, local storage is used as a cache for the main copy on the server, and this behavior is transparent to user applications via the FS driver.

Dropbox uses a user-space program to sync the files and cannot intercept system calls. As such, they had no choice but to synchronize all of it rather than bringing things in "on-demand" and releasing local copies that are not used. Nowadays, this can easily be done via FUSE driver in Linux. I do not know if something like FUSE exists on other platforms though.

Overall, I personally love the idea of a local machine being a "cache" for the server copy. However, the technical challenges are greater than a simple mirroring scheme, and there may be UX issues as well.


Dokan for Windows (last updated 2011) and MacFUSE for OS X do exist, but they're not battle-tested, to say the least. It wouldn't be wise for Dropbox to stake its reputation on such unreliable software.


MacFuse worked well when it was first released, but it has not been maintained in years and has no maintainers and does not really work with any recent OSX versions at the moment.

It looks like there might be something called OSXFuse which might be more recent and maintained, but I haven't looked at it, it also might not be.


OSXFuse is very well battle tested. We use it in ExpanDrive. CBFS by eldos is a commercial userspace filesystem lib for windows. We also use that.


awesome, thanks for the info! I will check out OSXFuse.


Commercial FUSE/Dokan analog for Windows costs around 15K for an all-inclusive package. I'm sure Dropbox can afford it if they are interested in pursuing this option.


your comment is spot on! see my other comment on this thread: https://news.ycombinator.com/item?id=8358527 spacemonkey does exactly that


Thanks! Spacemonkey looks terrific and is pretty much what I envision! I often feel that with the move to "cloud computing" over the last decade, we have essentially made a dichotomy between working "locally" and working "in the cloud". However, they really should be seamless! The cloud should extend your computational resources and make it more accessible, not replace it! With things like spacemonkey, Office 365 and other "seamless" apps, it seems that we are finally learning how to make things be location and device independent. Cloud computing is the vanishing mediator that is finally starting to vanish!


Ah! Same kind of user (paying customer for years) and same issues (too much space on my Dropbox, not enough on my drive).

My Macbook Air is the only computer I use, both at home and at work. Its SSD is 60 GB. Last month, Dropbox upgraded my account (for free) up to 1 TB.

Selective Sync? It's not optimal. We use several shared folders at work. Some of them become rapidly heavy without me noticing, unless I sense my laptop slowing down and find the culprit folder (with Diskwave) and unselect it from my sync. Plus: any (non-empty) shared folder you join will automatically sync on all your devices. It should be opt-in, not opt-out.

Offloading massive files? I use the (very good) web interface. It's a two-step process though: upload via the browser, and then delete your file. It's usually a one-off event, but is not what Dropbox is usually about: being seamless. Remember Doug's pitch to YC? The level of abstraction where you just hit "Save" and you're done? That's what's still missing with one-way transfers to the cloud.


Just because the author uses a machine that doesn't have that big a hard drive doesn't meant that everyone else does. I use dropbox on a computer with 15TB of HD space.... so having a 1TB drop box is great. Saying drop box should not offer the bigger size just because he doesn't have that much hard drive space is a very limited view.


I'd say that 15TB is not a common case.

Most of the hard drives installed in new computers these days are less than 2TB, and less than 1TB on laptops (which are very popular).

At the moment Dropbox has only a (free) plan of 2GB and another of 1TB. I think this is something that needs to be managed by Dropbox, if they want users to be happy.


Your Dropbox is limited to the free space on your computer with the least amount of storage. I have a Super Mega Death Rig with multi-TB of storage too... I also use an old Toughbook with a few GB of space free when I travel to the third world.


Camlistore (https://camlistore.org/) is the answer. Here's hoping it becomes just as easy to use as Dropbox someday.


I don't know anything about dropbox, but based on the types of things the author spends time thinking about, I think s/he might find http://git-annex.branchable.com/assistant/ useful.


This is exactly the problem Microsoft solved with OneDrive. You can choose what is kept only online and what is also offline. In windows at least its integration goes as far as keeping lower resolution versions of your photos offline for previewing purposes and seamlessly downloading the full-res when you try to open it in say Photoshop.

Unfortunately as nice as this sounds in practice its extremely slow (at least for us in Europe).

I am facing the same problem with hubiC which has a 10GB plan only and unless you have some crazy raid configuration on your desktop there is no way you can fill it up unless you use the web interface exclusively.


I think this limitation is actually fundamental to Dropbox's (and Google/etc) pricing model. Only a small subset of users can fully utilize the space they pay for, which means the remainder of users are more profitable. If the average user consumed 1TB, or if pricing was by usage rather than tiered, I'd expect the price per GB to go up.


I've often wished Dropbox's selective sync was managed by folder rather than by client. It would be more helpful to me to mark folders to not automatically sync account wide than to have to choose and re-choose those folders every time I setup Dropbox. I feel like that would potentially be helpful in this use case.


The selective sync is one way to manage the larger storage space, and do it effectively.


I personally like https://bitcasa.com in this regard. They have this feature that you can choose a "directory" to be a bottomless storage that is not actually on your device, it is on the cloud for you when you need it.


I just store large files I don't want synced in Google Drive and don't run the Google Drive client. Dropbox selective sync is a huge pain when setting up a new machine since it doesn't recognize yet unsynced folders and files.


I have the same problem. My MBP right now only has a 256GB SSD in it and so I can only fit about 1/8 or less of my DB on it once I load up other data/apps. I had to create a "Camera Uploads Archive" folder and move all my pictures to that then turn on selective sync for it because the "Camera Uploads" folder got way to big to keep on my MBP. I really wish I could still see the "Camera Uploads Archive" and if I tried to open or use one of the files it would download it until after I had finished with it. I'd love to be able to browser all my picture but I just don't have the space.


Yeah. I have this problem too. I want to upload a ~ 200 GB photo library so I have access to it on the go now that Everpix has shut down, but I can't do it from my main machine because the drive just isn't big enough. Selective sync seems to be the only option.

I hoped iCloud Drive would solve this: iCloud Drive supports similar capacities, but obviously iPhones can't store the whole Drive so presumably it's been designed with this in mind, but it seems like the OS X version still syncs the entire thing (at least I haven't been able to find any information to the contrary).


I'm surprised to see so many people saying they have the same "problem". You don't have to use the extra space. You had less space before and the problem didn't exist, and now suddenly it does?? It makes no sense.

1. It's a good thing for the many people who DO have more space on their computer.

2. It's a good thing because it leaves room for growth.

3. It's a good thing because it makes the service more useful for power users.

hit "Save" and you're done? Newsflash: it STILL works that way!!


The "problem" is that they WANT to use the extra remote storage, with a UI as good as DropBox's and consolidated with the other stuff they have on DropBox, but can't because they don't have enough local space or don't want to sync all their remote storage to every machine.

It is true that in some cases they may not have realized they had this desire until ExpanDrive gave them enough storage to think about it, but they do now.


Seems like Dropbox has already solved this problem on mobile devices. Using the Dropbox client on my Nexus phone or on the iPad, I can read or edit a file, and then mark it for offline use which keeps it in local storage, or just leave it alone, where I assume it's eventually deleted from the cache.

It's pretty transparent and intuitive. I would think this could be done on desktop / laptop systems, maybe via an optional client, with all files set initially to a no-sync value.


I never bought a pro account because, like the OP says, I consider Dropbox for smaller things. 100 GB was too much then, and now 1TB is ridiculous. I would have liked a smaller tier (20-25 GB at the most), but I guess they can't really make money if people actually use up their space. oh well..


You can get that much space with referrals. I maxed my referrals out with a $40 Adwords campaign a few years ago and it's been a good investment.


Wuala has supported this for as long as I can remember, it's called "WualaDrive": a network mount of your entire file archive.


The only reason I use Dropbox is because it works beyond any type of firewall (be it simple HTTP proxy or socks). Others tend not to.


Do you live in Italy or are you stuck in the U.S. where it would take few months for most people to upload a TB of data to dropbox?


Italy has an average connection speed of 5.2Mbps (47th in the world), the US is ranked 12th at 10.5 Mbps - about 2x Italy.

Just so you know.

http://www.akamai.com/dl/akamai/akamai-soti-q114.pdf


I don't understand the point of the article that 1 terabyte of storage is not worthwhile if your laptop has a small disk drive.

I use a MacBook Air with only 128GB of storage. I use "selective syncing" to not keep most of my Dropbox files on my laptop.

If I need anything not synced to my laptop, then I choose it to be synced, wait a short while, and then it is available locally.


I don't think they want you to use 1000GB of storage so the fact that you won't is a feature. Terabyte limit essentially means "unlimited but don't do anything silly."

Dropbox has a really good Linux CLI client so I'm sure there are people who will manage to bump up against that limit.


Just use Selective Sync: https://www.dropbox.com/en/help/175


I don't know if you read the post, but he says:

> Right now the only way to do that would be to put something in the box, then turn off Selective Sync everywhere for that folder. (Or upload files via Dropbox’s web interface, to a folder that’s been unchecked on all your devices via Selective Sync.)

> It’s not ideal. Selective Sync wasn’t meant to be used in that way… but for that matter, neither was Dropbox.

I think his gripe is the core premise of dropbox — file syncing — isn't the same as 1TB backup.


It really helps to read the article - this is discussed.


So, the proposed solution is basically to regress to FTP.

That's why my old startup offered FTP with no frills: it just works.


you can do this pretty easily with bittorrent sync


They basically need this feature: http://windows.microsoft.com/en-US/windows-8/onedrive-online...

Totally agree though. I'm not going to pay for a TB of storage until I can moves files I don't need on my SSD anymore (ie RAW photos I've developed) into Dropbox and have them removed from my computer seamlessly.

Sooooo easy to implement, just upload the file. When it's done, run an md5 on the local and the cloud, if same then delete local.


I've been using my Dropbox space as a way to turn my systems into a thin(ner) client. I install portable (ini and not registry based) Windows apps into d:\Dropbox that I know can be run in Wine and run them in Wine on my Linux systems from ~/Dropbox

Best part of it is that UI changes done on one system remain consistent when using other machines. Completely took the pain out of a format/reinstall, too.

Trying to constrain Home to my Dropbox folder also reawakened an old skill: Space discipline. With drive space exploding and SSDs still being relatively constrained, I had forgotten how much crap I tended to hoard and not sort through until I was feeling pinched again thanks to my Dropbox quota.


This is also what I do, but almost the inverse: I use Cygwin on Windows to get all the same applications (I do everything other than browse the web in my terminal) as I would on Linux, then use Dropbox to keep everything the same across machines.

Each of my machines is configured (almost) exactly the same, and all the media/projects/work is available on all my computers at the same time.

I also experience the forced "space discipline" and would add that I also have to have a better organizational structure in general. I'm constantly "re-balancing the tree" that is my directory structure, re-defining and re-grouping my media into ever finer groups in an effort to stop things from accumulating in digital "junk drawers".

On a different note, I disagree with the suggestion of having a "virtual filesystem" that is the entire Dropbox service. I feel that doesn't fit with what Dropbox is: a folder that syncs. I don't want to have to worry about the physical location of my files, which I most definitely would have to do based on my internet speed (which is as good as it can get right now).


I've done similar.. though I will symlink the config directories for some applications, copy others, and have a couple portable apps.

That said, don't put your irc client in there... I wind up forgetting it's connected when I leave work only to get tons of notifications at home...


Are you suggesting you wouldn't keep a local copy at all? What I'd want to do which more or less mirrors what I do--but using a backup service (BackBlaze) for cloud storage rather than a file sync service--is to backup everything from my desktop to BackBlaze. Among that "everything" are the data directories that I sync between my laptop and my desktop.

I also have Time Machine backups of both systems.


The Microsoft version doesn't hide a local copy. The file still exists in the FS, just when run/opened, it should pull down the copy in place and open the now downloaded file in the correct program. This could be done with a similar feature to selective sync, and enabling this feature for certain folders and always syncing other folders.


I was referring to the implication that the only actual copy of the file would seem to be on the cloud service in this scenario. ("I'm not going to pay for a TB of storage until I can moves files I don't need on my SSD anymore (ie RAW photos I've developed) into Dropbox and have them removed from my computer seamlessly.") Which may be OK but not necessarily something I'd be comfortable with.


Are you seriously so well-trained by Apple that you think you need to pay for software to run a local LAN Dropbox clone like rsync/btsync/ownCloud?


What does Apple have to do with anything? The same problem exists on Windows and Linux.

And, no, I don't want to run and manage a local "LAN Dropbox clone". It's all about the integration with different mobile and desktop apps and services, the remote backup, and that you don't have to manage it.


Whine whine whine, I have a mac, whine whine whine

get a computer with actual expandable storage


Like a Chromebook, right?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: