Hacker News new | past | comments | ask | show | jobs | submit login

Compress any larger piece of data on your own computer, and watch the CPU bottleneck.

Even on the fastest CPU's, writing a file that must be compressed first, will always be slower than the same file on the same hardware, but not being compressed before written to disk.

It may not be magnitudes slower - depending on the data, hardware, and algorithm, of course - but it will incur some write penalty. So, with disk compression, you get a write penalty and a read boost.




You are correct that most compression algorithms are slower than their corresponding decompression, which means the writes tend to be a different tradeoff than the reads.

I do not agree with your statement that compress + write is always slower, though. The analysis for cost/benefit of compression is the exact same logic for both reads and writes, but with different formulas based on how fast you can compress/decompress and read/write blocks. Let's imagine that your disk takes 10ms to write 64KB and 20ms to write 256KB. Then if compression of a 256KB block down to 64KB takes 5ms, writing is faster with compression done first. On the other hand, if compressing it takes 50ms, then writing it raw is quicker.


Whether it's a penalty or boost for reading or writing is entirely dependent on the performance characteristics of reading, writing, compressing, and decompressing. I can think of a reducto ad absurdum that's a write boost and a read penalty - I/O is carving and reading text from a stone tablet, compression is looking up shorthand from a dictionary.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: