Hacker News new | past | comments | ask | show | jobs | submit login

This isn't like lossless compression. Both techniques involve throwing lots of information away, with the justification that doing so does not significantly affect the end result.

The extent to which using both the techniques together will help will depend on how much overlap there is between the information each ends up discarding.




My joke was more along the lines of entropy. Entropy is information and you can't throw away all of it, otherwise you have nothing useful left.


Modern LLMs are still quite inefficient in their representation of information. We're at like the DEFLATE era and we've still yet to invent zstd where there's only marginal incremental gains; so right now there's a lot of waste to prune away.


Hence the idea to only throw away almost all of it.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: