Hacker News new | past | comments | ask | show | jobs | submit login

It appears so. I did a quick test with a huge PNG (80 megabytes), just to see what would happen. optipng took 5m9s, oxipng with "--threads 1" took 3m21s, and oxipng with the default number of threads took 2m00s. Not quite "half the time", but not bad.



I imagine most usecases where this performance would matter would be if you were compressing a ton of pngs at once. In which case wouldn't it be much faster just to run several different processes instead of one process using multiple cores?


Not necessarily. Back in the day I generated thumbnails for lots (and lots) or very large images and getting the best performance is tricky. Most of the time running multiple processes just ends up causing massive context switching and slowing down the whole thing. Even multithreading turns out to be bad, most of the time. I got the best performance by forcing GraphicsMagick to use a single thread and spinning up multiple operations. I wonder how this app would deal with something like that.


Latency could matter. Perhaps you're storing an image and want to ack that the image is written to disk before responding to the caller.


But running several different processes can be a nuisance...better performance by default is still great.


Well it must have taken them a ton of work to parallelize their algorithm also. It would have been better to just have a batch mode that lets you process multiple images at once on different cores.


> Well it must have taken them a ton of work to parallelize their algorithm also.

Not really. Both optipng and oxipng do "trials": pick a set of parameters, try to compress the image with them, repeat with the next set of parameters. Each trial is completely independent from the others, so it's trivial to run them all in parallel.

Most of the work of running the trials in parallel is done by the "rayon" crate; the oxipng author only had to do an .into_par_iter() to get a rayon parallel iterator, and then do a map/reduce on it.

The difficulty of doing that with optipng was global state. Quoting from https://www.reddit.com/r/rust/comments/48vkjy/oxipng_a_multi...:

"My initial plan as well was to fork optipng and implement multithreading on it, but optipng, like many C programs, takes pride in using global mutable shared state in every place possible, and I didn't want to try to untangle that spaghetti code to make it thread-safe."


Well, not really, because implementing a batch mode is more work than letting xargs or GNU Parallel or Make handle the multiprocessing for the "many files" case.

Parallelizing the algorithm helps in the "single file" case, which is still important.


Well parent comment was complaining that it would be more work for the end user. I was just noting it would be less work to include that convenience in the tool itself than trying to make the algorithm parallel for less benefit.


Managing lots of processes is a ton of work...


Is it? I believe it can be done in like a line or two of bash.


It takes more than a line or two of Bash just to shut them all down after Ctrl-C.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: