Hacker News new | past | comments | ask | show | jobs | submit login
Libspng – C library for reading and writing PNG files (libspng.org)
159 points by randy408 on Aug 21, 2019 | hide | past | favorite | 73 comments



I gotta say, it’s great that someone is making this attempt, however it turns out. PNG has been in a uniquely bad position re: “monoculture”, with libpng being not just the “reference implementation”, but the only implementation anyone bothers to use.

But libpng was intended as only a reference implementation—a rigid adherent to the PNG standard, for the sake of having a “runnable version” of the PNG standard—and was never particularly optimized for production use-cases, or code readability/maintainability/low attack surface, or any other criteria you might like to have. Libpng is optimized for one thing only: allowing other PNG implementations to build spec-compatibility test suites by testing against libpng behaviour.

Because of this rigid adherence, libpng doesn’t implement—and will never implement—any features that aren’t in the base PNG spec, like APNG.

And since everyone uses libpng as their PNG implementation (for some reason), nobody ends up adopting these extra features. (Sure, all the major browsers except Edge support APNG, but does your image editor? Does your IoT thermostat whose OS uses PNG image assets? Does your game console? Nah. Because these all just compile in libpng. And this is what kills support for features like APNG in the crib.)

That was never the PNG Working Group’s intent, of course. They don’t want to restrict the development of extra PNG features (yes, even though they developed MNG and might see features like APNG as “competing” with MNG, they don’t really care.) They keep libpng the way it is, not for ideological reasons, but simply because they don’t care about doing anything with libpng that moves it away from “reference implementation of the base PNG spec” territory. (And why would they? Their job as the PNG Working Group is to produce the PNG base spec. It’s supposed to be everyone else’s job, in the ecosystem, to produce conformant implementations. They even gave you libpng to make that easier!)

So, like I said, I’m glad someone is finally writing an alternative implementation (in C), such that the projects that currently use libpng could reasonably choose to switch to libspng. It’s the same feeling I get from seeing LibreSSL and BoringSSL: that an ecosystem that was essentially “dead” and stuck in a set of bad choices due to everyone sticking with one unchanging reference impl, is now coming alive again.



> the only implementation anyone bothers to use.

I don’t think that’s accurate. I use LibPNG but I also use other things too.

When I write for Linux or cross platform, my only requirement “decode bytes in RAM to texture in VRAM”, I prefer stb_image for usability.

When I write Windows software, I prefer OS provided codecs. In modern world that’s WIC for C++ code, previously OleLoadPicture API. This way I don’t have to patch image codecs in my software for security bugs, MS does that in windows updates. Also helps with binary size.


I guess what I meant was that libpng is (currently) the only implementation that attempts to fit a libpng-shaped hole. Not in the sense of being API or ABI compatible, but in the sense of being a free-floating multiplatform library solely dedicated to PNGs, rather than a higher-level multiformat library or OS API.

Because of this, libpng gets linked in by a lot of the higher-level multiformat libraries (like SDL) or graphics toolkits (like WxWidgets) when their aim is also to be “portable” free-floating multiplatform libraries, rather than libraries that have special per-OS code-paths for everything. (Which becomes especially important in things like embedded OSes/unikernels, or sandboxes like Emscripten, where there aren’t any OS image APIs.) And these portable libraries and toolkits then get used by runtimes that want portability, like Erlang or Love2D; or by software that can’t access the OS’s SDK to link to system libraries, like game-console homebrew.

In these cases, the feature-set of libpng determines the PNG features that these higher-level libraries support. SDL supports loading animated images—but not from APNG, because libpng doesn’t provide SDL with that capability. SDL isn’t going to do the work of adding their own APNG support; they’ve got enough on their plate just making sure all their components stay glued together. But if there was a “better” PNG library than libpng, that ticked all the same portability boxes? They’d switch.


Some performance tests decoding PNGs:

Platform: Windows 10, Visual Studio 2019, RelWithDebInfo config, Intel Core i7-8700K. Compiled with -DSPNG_SSE=3.

  testing loading image: D:\art\cryptovoxels\ortho overview.png (32-bit, large)
  libSPNG decode took 0.2221 s
  libPNG decode took  0.2373 s

  libSPNG decode took 0.2215 s
  libPNG decode took  0.2364 s


  testing loading image: D:\art\dof_test_output.png (32-bit)
  libSPNG decode took 0.01101 s
  libPNG decode took  0.01225 s

  libSPNG decode took 0.01096 s
  libPNG decode took  0.01205 s


  testing loading image: D:\art\skymodel2_test3\top2 CPU.png (24-bit)
  libSPNG decode took 0.008874 s
  libPNG decode took  0.009399 s

So Libspng seems slightly faster, but certainly not 35% faster, in these tests.


As I'm guessing most people who have had to use libpng will agree, the biggest strength here is a less-byzantine API. As good as the PNG format is, the library API is kind of awful.


libpng does have simplified API (png_image*) since 1.6


Wow, this is amazing. I did not know about these new API functions. It makes using libpng much more pleasant. Thanks!


Testing with Release config (full program optimisation):

  testing loading image: D:\art\cryptovoxels\ortho overview.png
  libSPNG decode took 0.1901 s
  libPNG decode took  0.2069 s

  testing loading image: D:\art\skymodel2_test3\top2 CPU.png
  libSPNG decode took 0.007862 s
  libPNG decode took  0.007801 s


This is what I use for benchmarking: https://github.com/libspng/spngt, my platform is Debian 10, GCC 8.3.0 with -O3 on an i5-4670. I use -SPNG_SSE=2 because that targets SSE2 instead of SSSE3, this matches the Debian build of libpng, both are linked against zlib-ng. The results I get for medium_rgb(a)8.png and large_palette.png are the results on the charts.


What version of LibPNG are you comparing against? I am using LibPNG 1.635. Note that recent versions of LibPNG have some SSE optimisations contributed from Intel.


I'm using 1.6.36, the most obvious speedups should come from RGBA 8-bit PNG's, that codepath doesn't rely on compiler optimizations.


To speculate on these results a little:

I'd say the performance in general with Libspng and Libpng is pretty similar. The apparent greater speed of Libspng in the RelWithDebInfo build may be due to the use of the inline keyword, which libpng doesn't use. However once full program optimisation is enabled, and functions are inlined across translation units etc.., the code comes out pretty similar and hence has performance parity.


Did you put it in a loop to eliminate startup time?


Yes, am showing minimum time from 10 runs.


I find stb_image single file format and lack of dependencies to be extremely convenient.


There are trade-offs with every library, libspng is safe to use with arbitrary data, with stb_image that isn't clear yet and lodepng is significantly slower.


Libspng seems to just have spng.c and spng.h, which is just as convenient.


But that doesn't include the zlib dependency


I wonder how these libraries compare to using something like image-rs [0] w/ a custom-exported C FFI extern? I have used that library's PNG decoder/encoder in WASM and it seems simple and small and fast. I think it'd be worthy to include in the next bench round since it's only a bit of glue code away from a "C library" too.

0 - https://github.com/image-rs/image


How does this compare to something like https://github.com/nothings/stb/blob/master/stb_image.h ?

Is it just that it has more features and can handle weird edge cases?


There's a comparison in the linked article.


Ah, my bad! I didn't read it closely enough.


Doesn't compile 'out of the box' in Visual Studio 2019.


Submit a bug report on github, appveyor doesn't report any issues with VS2017.


Just to say the obvious: by using zlib it gets future speed improvements and bug fixes "for free".


FYI, randy: `fmt` is not declared in `Decode example - libspng`.


Fixed, thanks.


Looks like it only does reading at this point - I can find docs or entry points in the code for writing png.


Thanks for including ARM optimizations and using meson. I'll try to use this with my tablet UI app.


Question: do you have to decode to a 4-channel format, e.g. with alpha? What about decoding to just RGB?


For now it's either 8- or 16-bit RGBA, adding more output formats adds complexity and requires more test cases. What's your use case for a 3-byte layout?


Can indexed images be decoded to palette indices? I've been in cases where certain indices have special semantics.


RGB uses less memory than RGBA (obviously), so if there is no alpha channel I don't want to allocate the memory for a channel completely full of 255 bytes.


A 3-byte layout can be worse for performance, I think the simplest solution would be to add a special format that always matches the PNG's format, that way you always get RGB from RGB images, grayscale from grayscale images, etc. This wouldn't require a conversion step.


Yeah, generally if the PNG file is RGBA, then I want to load as RGBA, and if it's RGB, I want to load as RGB.


I don't program C languages, but I can't help but feel that naming your incompatible library libspng you are introducing a source of potential confusion.

edit: Thank you cormacrelf for the explanation re: lib being mandatory. Thank you OP for explaining your rationale.


The 'lib' is generally mandatory so that the GCC/clang linker flag can find the binaries in its search path. '-lpng' means 'find libpng.a or libpng.so'. Hence many, many C libraries have it as part of the name of the project. In general the 'lib' prefix in a name means 'this is a C library', or at least now in the days of Rust, 'you can link to this with a C linker or normal FFI'.

Considering that, 'spng' is different enough from 'png'.


It is similar to how many JavaScript libraries are call "something.js" or Java libraries are called Something4J.

I would call this library "spng" and the original "png"; once compiled they would be named "libspng" and "libpng".


It's a very generic name and adding a single letter doesn't do much to differentiate at a cursory glance. If you're focused on what you're doing then maybe...


I suggest you not take on dependencies based on your cursory glances, then.


That’s really the original libpngs fault for picking a generic. Now everyone wants to follow suit.


maybe... libpng-ng?


Considering that, 'spng' is different enough from 'png'.

It's different, but I think it's so close as to verge on namesquatting, especially when the description and purpose is almost identical to the original libpng --- in fact, that's why I clicked, to see what exactly this is about.


Edict: all future library names must be UUIDs?

I’ve been really distracted lately thinking about what things need named handles in my data models/representations, and what they should be handles for, and how to do so, and how to change them in sane ways. So far all my thinking has just convinced me that I am a mediocre architect.



It would seem my insanity wolf idea was only low-tier insanity! Thanks for sharing.

I’m also kinda reminded of aws lambda URLs, or I guess URLs in general.


COM definitely has large levels of insanity wolf in it. It's a reference-counted object-orientated dynamically-typed self-describing interface for embedding any program in any other program ... written to target 1990s C.


Still, it enabled features that others have a hard time replicating to this day. E.g. MS OFfice exposes a ton of its functionality via COM for remote control and embedding.


COM was incredible for being able to hack bits of various programs together.

I worked on a spike once to try and get a modern web browser embedded in a proprietary business basic language who's UI toolkit only exposed some COM/OLE stuff.

It was an unholy abomination of unmaintainable software, but it worked and worked well. It made me think a lot about how things would have looked today if ActiveX managed to keep it's foothold.


There was a lot wrong with ActiveX, starting with the fact that it was proprietary and Windows/X86 only, but the basic concept of cleanly embeddable controls was extremely sound and is something the modern Javascript world is only slowly groping towards.


The biggest problem is that they start with something that's supposed to be part of application software on your PC and then try to blacklist the dangerous bits and put everything else on the web. We already knew by then that blacklisting is a terrible idea in security and won't work.

The meeting should have gone like this:

Manager: We're going to put COM on the Web!

Engineer: What about this COM function that deletes files?

Manager: We'll blacklist that.

Engineer: Or this one that reboots the PC?

Manager: Blacklist. In fact I'm assigning you the job of making the blacklist.

Engineer: I think there's an unlimited number of problematic features of COM.

Manager: Good point. OK, ActiveX is cancelled. Thanks for your time, good meeting everyone, make sure to implement PKIX correctly so we don't cause more holes there too.


Sounds like Objective-C :p


You joke, but I started with Win32 programming then later moved to MacOSX and the COM/Objective-C link is pretty clear.

My guess is that a team at Microsoft in the 90s saw the need for a cross-process runtime object system, saw Objective-C and thought to themselves "Hey, this Objective-C is pretty useful but it doesn't suck enough to be part of Win32. Lets fix that - GUIDs, GUIDs for everyone!."


I've often thought the exact same thing!

There's an amusing video on youtube where Steve Jobs is demoing remote Objects on NextStep and (I think) Openstep on Windows, and having a laugh at MS since Next managed to get remote Objects working before MS got their Remote OLE (aka DCOM) working. Steve calls it Doh'LE.

The only different thing that COM had was support for scripting the objects via IDispatch, but that was such a pain (VARIANTS !) that only those really needing it bothered.

MS were also very proud of their MTS where an on the fly proxy object was created, but even that was inspired by Remote objects.

However they really jumped the shark when they made it possible to create COM objects from Visual Basic - the endless grief caused whenever a VB developer decided to change any parameters on the interface and caused all GUIDs to be regenerated and a big fat binary with all revisions of the code. They then created a tool to strip that all down to just the last version, thereby totally removing any backward compatibility - which then caused other random components to fail.


I started in the 80's and also had the pleasure to use Win16 and OLE 1.0.

GUID were already part of DCE/RPC and Taligent.

And COM IDL is initially based on DCE IDL.


> all future library names must be UUIDs

No thanks.


Naming things is hard, I wanted to have 'png' in the name and still have a short prefix for the API that matches the name.


I don't mind "spng" myself, but perhaps it would have been more popular to spell it out, i.e. libsimplerpng, that is quite readable and harder to confuse for the original libpng.

Do you pronounce it as "sping"?


No, the letters are spelled out, the "s" stands for simple.


libsimpng?



libpng is so slow to compress. A large 2000x2000 png will actually take about 1 second to compress. This is basically forever for a user experience.


libpng compresses the image rows multiple times with different filters to optimize for size, this filtering process is not optimized. The choice of zlib also makes a big difference, stock zlib is very slow, zlib-ng is the closest to a fast zlib alternative.


Amazing info.

> libpng compresses the image rows multiple times with different filters to optimize for size, this filtering process is not optimized.

Is there a way to disable this multiple-pass approach to get a slightly larger image that is generated faster?

> The choice of zlib also makes a big difference, stock zlib is very slow, zlib-ng is the closest to a fast zlib alternative.

I'm looking for benchmarks but I haven't found any that make it look good. This one makes it look nearly equivalent to zlib: https://quixdb.github.io/squash-benchmark/?visible-plugins=z...


A sibling reply to this one tell you how to tell libpng not to bother using different filters and just always use the null filter, but this will make some input images into _huge_ files (maybe similar size to uncompressed image data) so be warned.

If you had software that you knew spat out images with specific properties you could choose the filter by reference to what you know about the images, e.g. imagine you always produce noise anyway, the null filter is fine, if you produce a horizontal gradient you want the SUB filter, if a vertical gradient, the UP filter. But you'd need to make custom PNG code not rely on a generic library.


> Is there a way to disable this multiple-pass approach to get a slightly larger image that is generated faster?

Use png_set_filter() with PNG_NO_FILTERS.

> This one makes it look nearly equivalent to zlib

I don't know how squash is set up, but decompression performance depends on which library created the DEFLATE stream (see https://github.com/zlib-ng/zlib-ng/issues/326#issuecomment-4...).


There is no multiple-pass. libpng only tries to guess the right combination of filters, without doing actual compression. It's not always a good guess, but at least it's fast.


The landing page and comparison don't explain how their solution is faster. Are they just doing fewer of those filter passes?


libspng doesn't support encoding images yet, the benchmarks show decode time. It is faster at decoding because it minimizes its per-pixel overhead, e.g. if a PNG image is has an RGBA 8-bit layout and it matches the requested output format it will just decompress and defilter the image rows, in this case there is no per-pixel code at all. Encoding is almost the same process backwards so it's more than likely it will be faster at encoding too, even if it would the same strategy for optimizing compressed size.


Why can't libpng implement this? Is there any cons to doing this?


> slow to compress

Although you're correct, this is because it spends a lot of its time optimize for decompression time: a PNG will be compressed once, but decompressed thousands or even millions of times. It makes sense to trade off compression time for faster decompression.


> This is basically forever for a user experience.

Why does your user experience force the user to wait?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: