> But if the code is open source, we _can_ verify it's security?
No - the code could be "open source" but unreadable. But if it's not open-source there's definitely nothing we can do.
> Heartbleed and Shellshock, the two most significant vulnerabilities found in heavily used open source software, were found by vulnerability testing and not code inspection. So while being open source is a nice-to-have attribute for a piece of software, that's as far as it goes
a) afl-fuzz and the like require access to the source code.
b) Those were the vulnerabilities that made it into production. They tell you nothing about what proportion of potential vulnerabilities were stopped by it being open source.
Fuzzers do not require source code. That is absolutely false. Afl-fuzz does, but you badly overplayed your hand by adding "and the like". Fuzzing proprietary closed source protocols by instrumenting closed-source binaries is a bread-and-butter software security project that virtually any application security consultant can do effectively.
This is very much not true. Understanding what any executable does is possible without the source code. In fact, if you are looking at the binary, you know exactly what you are dealing with, and don't have the doubt about what the build chain actually does with the source.
potential vulnerabilities were stopped by it being open source.
This presumably comes from "many eyes make all bugs shallow". But operationally this is not true, because there isn't really, except for rare circumstances, any useful review.
Thanks for the correction regarding ali-fuzz, I wasn't aware of that. Unfortunately I can't update my post to fix that anymore.
I think this doesn't fundamentally change my point though. ali-fuzz was used to find shellshock, but that vulnerability had been in bash for decades. If it had ben found and closed within months of being introduced I'd be cheering the advantages of open source with the best of them, but that's decades during which a bad actor could theoretically have found and exploited that vulnerability with impunity using code analysis. That's exactly what I mean by the balance of advantages versus disadvantages of open source being tipped the wrong way right now.
I'm not enemy of open source, far from it, I'm just arguing for an open and honest assessment of the situation. If open source is really going to be a genuine security advantage, there's an awful lot of work to be done to make it so. It's not going to happen spontaneously.
> I'm not enemy of open source, far from it, I'm just arguing for an open and honest assessment of the situation. If open source is really going to be a genuine security advantage, there's an awful lot of work to be done to make it so. It's not going to happen spontaneously.
I don't think you've shown anything about relative vulnerability rates. I agree that there are massive problems with even major open-source projects, the general state of software security is terrible, and we have a lot of work to do on that front (starting with moving to memory-safe languages post-haste). But none of those things contradicts that open-source software is much more secure than closed-source software.
You haven't yourself empirically established that open source software is more secure, so you should be less smug about the logic you're using here.
In reality, software security is a function of the amount of expert attention that has been paid to a given piece of software. Popular open source software attracts a certain, stochastic, significant amount of attention, but money buys a more reliable amount of attention. There is insecure open source and secure open source, insecure closed source and secure closed source. Open source is a red herring.
There's a reason why, for instance, Firefox had Bleichenbacher's E=3 vulnerability, while IE didn't.
No - the code could be "open source" but unreadable. But if it's not open-source there's definitely nothing we can do.
> Heartbleed and Shellshock, the two most significant vulnerabilities found in heavily used open source software, were found by vulnerability testing and not code inspection. So while being open source is a nice-to-have attribute for a piece of software, that's as far as it goes
a) afl-fuzz and the like require access to the source code.
b) Those were the vulnerabilities that made it into production. They tell you nothing about what proportion of potential vulnerabilities were stopped by it being open source.