Hacker News new | past | comments | ask | show | jobs | submit login

The max window size for zlib is 32 KB, so I don't think the default sharding at 128 KB would change much. You can pass the -b parameter if you find out a bigger shard works better on your data.

If you are looking for details of the design of pigz, there is a very well-documented overview in the source of pigz.c:

https://github.com/madler/pigz/blob/master/pigz.c#L187




had a quick scroll thru that source code - i didnt know you could implement a try/catch in C via macros...mind blown.


This book (C Interfaces and Implementations: Techniques for Creating Reusable Software) has a whole section on it:

https://www.amazon.com/Interfaces-Implementations-Techniques...




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: