Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
bpchaps
on Aug 20, 2016
|
parent
|
context
|
favorite
| on:
A parallel implementation of gzip for modern multi...
Why doesn't parallelized gzip get more attention? For larger files, it can be quite a pain to see a multicore machine sit mostly idle. Something like this works pretty well, but I've never seen it done outside of my own stuffs:
split -l1000000 --filter='gzip > $FILE.gz' <(zcat large_fille.txt.gz)
Consider applying for YC's Spring batch! Applications are open till Feb 11.
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search:
split -l1000000 --filter='gzip > $FILE.gz' <(zcat large_fille.txt.gz)