> The default way to cache things is zip, which no matter the compression isn't a great fit, you really want a format that is stream-able like tar. When its stream-able you can download the tar and unpack it at the same time, not waiting for the full download before beginning to unpack.
You absolutely can stream extract a zip file: the index at the end of a zip file is merely an optimization mechanism added to a format that is otherwise somewhat tar-like at its base. While it is theoretically possible to create files that uncompress correctly only via one or the other data structures (see http://www.saurik.com/id/17) the file wouldn't be compliant and no one normally does this.
The only real tradeoff with a zip file vs tar+gzip is that the compression it does is per file, this means you gain random access (but don't lose streaming support) at the cost of being unable to share compression statistics across multiple small files (and so the resulting archive can be a lot larger).
Yep Specially crafting a zip file can make it streamable but it’s not the general default. Can GNU unzip read from stdin? Not sure but I don’t think so?
You absolutely can stream extract a zip file: the index at the end of a zip file is merely an optimization mechanism added to a format that is otherwise somewhat tar-like at its base. While it is theoretically possible to create files that uncompress correctly only via one or the other data structures (see http://www.saurik.com/id/17) the file wouldn't be compliant and no one normally does this.
The only real tradeoff with a zip file vs tar+gzip is that the compression it does is per file, this means you gain random access (but don't lose streaming support) at the cost of being unable to share compression statistics across multiple small files (and so the resulting archive can be a lot larger).