An interesting aspect of this is that it's a bug in NPN, which is more or less a historic artifact.
When SPDY, the precursor of HTTP/2, got introduced, it needed a mechanism to signal that within TLS, a different protocol (SPDY instead of HTTP/1.1) was spoken. That mechanism was originally NPN. I don't know the exact details and motivation, but eventually people seem to have decided that NPN wasn't exactly what they wanted, and they invented a new mechanism called ALPN.
Now, that was a decade ago (the ALPN RFC is from 2014), so the question is: why do we still have NPN code in OpenSSL? I don't think anyone uses it any more. Shouldn't it have been removed long ago?
To put this in a larger context: it appears to me that OpenSSL still has a strong tendency to bloat. Heartbleed was essentially a "we added this feature, although noone knows why we need it" kind of bug, but it doesn't look to me they've changed. I still get the feeling that OpenSSL adds many features that they probably just should ignore (e.g. obscure "not invented here"-type algorithms), and they don't remove features that are obsolete.
Why it’s not removed: backward compatibility I bet.
I have to maintain a few VM’s and statically linked tools specifically for interacting with old network appliances that modern Linux hosts won’t “talk to” without crippling their SSL/TLS configuration or doing other horrendous workarounds like specific OpenSSL/OpenSSH config files for those things.
It’s actually an interesting problem when performing security assessments - some tools for scanning will have false negatives because they can’t “talk to” old shit.
You have some VMs that talk to old network appliances that require NPN? And if NPN is not supported they don't just fall back to simple HTTP/1.1? I find that sounds extremely unlikely.
I am of course well aware of backwards compatibility issues. But I don't see how they'd impact NPN.
You might have to update some code that hard-codes NPN functionality, but well, OpenSSL has done API changes in the past, quite significantly so.
Not OP, but I have some old network-connected appliances in the field that I can't upgrade and that our current OpenSSH clients have deprecated all of the ciphers and methods for. You still have to be able to talk to these things over the network, and ideally you'd want to upgrade at least some of the systems out there.
You have to understand that outside of silicon valley and the hobbyist realm there are tons and tons of systems out there where the security posture is "don't allow anyone to have physical or network access to these systems who doesn't belong here" because the companies that made them deprecated them a decade ago or they were built before the widespread practice of supporting the Linux BSP for more than a couple of months.
These systems still work and you won't convince very many users of these systems to spend a bunch of money upgrading them for some notional security risk. You may also be working for a company that is contractually obligated to support these systems in some way. So what do you do?
In many cases, you do the best you can to mitigate the risks you know are there with tools like firewalls, port knocking, non-standard ports, and so on. And you hope that the 2-3% of attackers that actually know what they're doing never try to crack these things.
I understand what you write, yet it doesn't have anything to do with NPN.
Look, I get it. Deprecating things has tradeoffs. But NPN looks like an incredibly safe thing to deprecate. It has only been used for a very short timeframe. Its only use case (SPDY) has a fallback (HTTP/1.1) making sure things still work if it's not supported. Your story about OpenSSH ciphers has nothing to do with it.
OP couldn't talk to the old devices he used to, he installed some VMs with old stuff, that solved the problem. It's an adequate solution, why spend any more time on it?
It's hard sometimes to even upgrade the software to the version that deprecates it. I think that was hard to get from my nonsensical story about tying an onion to my belt.
I have the exact same issue with OpenSSH on an absurdly regular basis, which is why I maintain a set of statically compiled versions and VM’s with old versions - just so I can actually talk to old appliances/hardware.
I noticed also that some of the security scanning tools we use “silent fail” on some old appliances because they can no longer negotiate the appropriate SSH connection due to library updates :)
I was speaking in the more general case with regards OpenSSL - not NPN specific: where changes in OpenSSL have caused stupid devices that I have no control over to become “unmanageable”, hence having to either build static tools or use an ancient VM.
I wish I could replace these old devices, and I recommend to do so, but usually replacing them won’t happen until they physically break.
Think: Industrial IoT devices at client sites and other such security horrors.
On that point: why is the previously allowed crypto settings now considered as good as nothing? I have to force SECLEVEL=0 for openvpn/openssl to allow connecting to my company's vpn. From my reading that would allow any old cipher or hash rather than the previous minimum. Why is the previous level not kept as M and the library bumps the default to N? I know that a theoretical weakness means M is vulnerable to breaking but you force people to make it even worse.
I think my boss hasn't updated his openvpn so he doesn't have new openssl so he never saw the problem. His company, his problem.
If you think of the ideal attacker as having a range of abilities and knowledge between "I found this tool on BitTorrent and am now going to try cracking your network from the outside with it" to "I spread my port scans out to multiple exit nodes over several days so you don't even notice me doing it" then having any security is better than nothing. All you have to do is make the sliver on the venn diagram of people who are trying to attack your system versus people who know how to attack your system as small as possible. It's not rocket science, and there are a lot of factors to balance here beyond the security level of a particular cipher.
I'm not sure if you've ever plugged into the internet, but there are constant probes occurring. They may not be an attack but simple information gathering that is used later (say someone finds a weakness in your configuration) and you and everything like you is attacked at once.
> Why it’s not removed: backward compatibility I bet.
I mean, in the intervening period there was OpenSSL 3 which was a large backwards-incompatible (API and ABI) release, with a huge amount of effort on the part of OpenSSL developers and users to follow along. It was the ideal opportunity to drop this sort of old stuff.
> I don't know the exact details and motivation, but eventually people seem to have decided that NPN wasn't exactly what they wanted, and they invented a new mechanism called ALPN.
Looking at the spec for NPN and knowing about ALPN from implementing just enough of if to enable TLS false start way back when, ALPN is better because it's less steps.
With ALPN, the client includes its supported protocols in the Client Hello, and the server picks one or discards the extension in the Server Hello. With NPN, the client indicates it supports NPN in the Client Hello, the server sends back a list of supported protocols in the Server Hello, and the client sends an extra message to indicate its selection between change cipher spec and finished.
There's perhaps more client profiling information on the wire with ALPN, but it's not very interesting when 99% of connections just advertise h2,http/1.1. During the development of http/2, you could probably have a pretty tight range on client versions depending on what pre-standard versions were declared.
It was and remains a stamp collecting library. "Oooh, the Half Cent South African Reverse Yellow Eagle. Only six hundred of these were ever issued and since all postage by that point already cost at least four rand the half cent stamp was entirely useless. Nine in mint condition unperforated were sold at auction in 1956 but none have been seen since"
So it's useless? Yes! So why do you want it? Because it's rare!
The fact that lots of real world code uses OpenSSL is one of those things it's going to be hard for our descendants to understand. Like explaining that people used to Smoke cigarettes on aeroplanes. But why though, that's crazy. Yes, yes it is.
Heavy use of unifdef there. Following their progress was amazing from an 'let's own this thing now' perspective, seeing the size of code changes, how they decided which things had to go.
I would have been bragging too, were I removing ifdefs by the shovelload every day for weeks.
I could read that in the opposite direction: why were the SPDY people pushing their half-finished protocols into a bunch of upstream projects? Wasn't that irresponsible? Shouldn't they have pushed those same projects to remove support for WIP protocols?
Removing old code, especially in open source, has a problem.
Unless it is obviously breaking something someone wants to do, it has roughly a zero cost (in maintenance) being left alone, especially since bugs can often be ignored, but a non-zero cost to remove it - as it generally will only garner complaints and whining from any remaining users.
And in open source in particular, it’s damn near impossible finding remaining users in advance, or even telling if there are no remaining users.
Sure, but proprietary code you can (somewhat) see who is calling it, and the group maintaining it is at least sometimes the group calling it - so has an incentive to not make it a bigger mess. Somewhat. So that part of the equation has less weight.
They have a counter balancing thing which is no one can see it to shame them.
I started writing s2n the day after Heartbleed and the first lines of code were for the stuffer interface. A stuffer is a buffer for stuff, and it's like Java buffered I/O for C. You can get a flavor from reading the header: https://github.com/aws/s2n-tls/blob/main/stuffer/s2n_stuffer...
The implementation is incredibly simple. Treat all blocks of memory as blob with a known size and then read/write into those blobs with a cursor to track progress and bounds checks on every access. Fence all serialization/deserialization through a safe low level interface. Not only do you get memory safety (which we later proved using formal reasoning) ... but when you're parsing message formats it lends itself to a declarative coding style that makes it very clear what the structure is. You can also do lifecycle things, like erasing sensitive memory with zeroes when you're done with it, making sure things don't show up in core dumps, etc. BoringSSL introduced a Crypto_bytes API that also did some of this plus bounds checking, and retrofit it into OpenSSL.
OpenSSL on the other hand is a horrific mash up of raw pointer arithmetic, ad-hoc parsers interleaved with business logic and control flow. I could never keep it straight, and it always scared me to review.
> Issue summary: Calling the OpenSSL API function SSL_select_next_proto with an empty supported client protocols buffer may cause a crash or memory contents to be sent to the peer.
Not the first time this kind of thing has happened. A rewrite in SPARK Ada uncovered a flaw in the reference C implementation of the Skein cryptographic algorithm.
Specifically this is an attempted ABI compatibility layer. That is, given rustls, which is a perfectly nice Rust TLS implementation, what if we built the OpenSSL API and then shipped the C ABI compatible library ?
In principle the result is definitely safe if your C code (which previously called OpenSSL) is safe, regardless of whether OpenSSL itself is riddled with bugs (which it likely is)
A safety focused person rewriting tricky C pointer banging code in Rust is likely to inadvertently fix bugs - what's interesting is whether you notice that as you're doing it or it's just silently not buggy any more in Rust.
A few hours ago I had some code that was doing bignum arithmetic and one call returned None where I hadn't realised it could and in a haze I thought "Oh, if it's None we can treat it as Zero" so I changed unwrap() to unwrap_or(Zero::zero()) † but nope, it was None because that function assumes we're getting integers and it's just figured out it has a proper fraction instead, so hence there is no such integer, None. Treating None as Zero meant all the related tests blew up and I realised my mistake after a couple of minutes. No, e to the power 0.5 is not 1.
† Yes that's suboptimal because it will make the zero bignum even when it doesn't need it, I should have called unwrap_or_else instead but also no I shouldn't because it's wrong.
When SPDY, the precursor of HTTP/2, got introduced, it needed a mechanism to signal that within TLS, a different protocol (SPDY instead of HTTP/1.1) was spoken. That mechanism was originally NPN. I don't know the exact details and motivation, but eventually people seem to have decided that NPN wasn't exactly what they wanted, and they invented a new mechanism called ALPN.
Now, that was a decade ago (the ALPN RFC is from 2014), so the question is: why do we still have NPN code in OpenSSL? I don't think anyone uses it any more. Shouldn't it have been removed long ago?
To put this in a larger context: it appears to me that OpenSSL still has a strong tendency to bloat. Heartbleed was essentially a "we added this feature, although noone knows why we need it" kind of bug, but it doesn't look to me they've changed. I still get the feeling that OpenSSL adds many features that they probably just should ignore (e.g. obscure "not invented here"-type algorithms), and they don't remove features that are obsolete.