Hacker News new | past | comments | ask | show | jobs | submit login

Downloading large amount of data from tarsnap archive is very slow. Any tips/tricks to speed them up?



Scott Wheeler wrote a tool for speeding up extracts by running multiple tarsnap processes: https://github.com/directededge/redsnapper

There's also the trivial (but often unhelpful) advice of "be closer to the server" -- the problem is one of round trip latencies since the internal design of tar (and thus tarsnap) works on the principle of "read a block; do something with it; repeat". This is what I feel to be the biggest technical pain point in tarsnap, and I know in theory how to fix it, but every time I've started focusing my energy on it other things have come up requiring more urgent attention. :-/




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: