This to me is the nightmare scenario. Well one of the two, the other one being that a developer of an obscure library I use has their password to PyPI compromised and a bad actor uploads a backdoored version of the library.
Fundamentally, the reason this is different from how thinks like Linux distos work is because Linux distros have maintainers who are in charge of making sure every new update to one of their packages is legit. I am sure you can try to sneak malicious code in, but it isn't going to be easy.
I am not advocating that PyPI (and npm) adopt the same model. That would be too restrictive. But maybe just showing the number of downloads isn't the best way to assure whether the package is legit. Perhaps some kind of built in review system would be nice.
A review system unfortunately isn't likely to be practicable with current development models. npm alone has over 500,000 packages (http://www.modulecounts.com/) so even a one time review isn't going to happen.
If people want a more trusted solution the likely outcome is that they'll need to use a smaller more static set of libraries and then either do the audits themselves, or outsource that to a 3rd party.
Ofc with current speeds of change and deployments, it doesn't seem likely that many companies will adopt that model.
> npm alone has over 500,000 packages (http://www.modulecounts.com/) so even a one time review isn't going to happen.
But at least the modules with the most downloads (webpack, react, or stuff like left-pad) could be vetted, and especially npm could implement a 2-or-more person model - basically, everyone with publish access can upload a new artifact, but to actually have it distributed to endusers, a second person would be required to sign off.
That's the thing. I worry less about popular packages. I can check that Django's GitHub repo links to PyPI and vice versa. But a random package to parse DSN's? I don't know it from Adam. I want to use it, and lots of others do too, but not everyone is going to review it. Maybe just a button on the package that says "I found insecure code!" Would be good.
> That's the thing. I worry less about popular packages. I can check that Django's GitHub repo links to PyPI and vice versa.
I worry about the most popular, and there the small and next-to-unmaintained. Just think back to the left-pad desaster that broke builds all over the world and imagine it was not a deleted package but an update containing malware. I assume there are lots of such "hidden gems" where the maintainer has gone away... the consequences of hacking just one improperly secured account are severe.
Full code review while not bad is probably not going to stop attackers. Often I can barely understand what the dev intended with non hostile code. Something more along the lines of strace on the install and run down any connects or execs. Runtime is different.
Indeed they do, and you can get some libs for Node/ruby etc there, however most companies, from what I've seen choose the option of using direct access to npm/rubygems etc.
> Fundamentally, the reason this is different from how thinks like Linux distos work is because Linux distros have maintainers who are in charge of making sure every new update to one of their packages is legit.
Because the person who pushes the code to the public repo is not the same person who makes sure it isn't malicious. You have a review process. Nothin is stopping me right now from creating a PyPI package called Django2.0 and having some poor souls download it. Or creating a tiny but useful utility, having it become popular, then introducing an update with a backdoor.
Fundamentally, the reason this is different from how thinks like Linux distos work is because Linux distros have maintainers who are in charge of making sure every new update to one of their packages is legit. I am sure you can try to sneak malicious code in, but it isn't going to be easy.
I am not advocating that PyPI (and npm) adopt the same model. That would be too restrictive. But maybe just showing the number of downloads isn't the best way to assure whether the package is legit. Perhaps some kind of built in review system would be nice.