I'm a maintainer (one of many) of an open source project, and this topic has been on my mind a lot lately as I review PRs.
I am more suspicious of PRs from new contributors by default now. Of course I keep these suspicions to myself, but besides simply reviewing code for all the regular things, I now ask myself "what sort of sneaky thing could they be doing that appears benign on the surface?"
That's great that you are considering this more now.
But the xy story taught us, that every contributor is dangerous, the most dangerous ones are probably the most helpful and most skilled contributors. If someone barely get's a PR accepted, they probably lack the skills to add a sophisticated backdoor.
Another thing that was not talked about a lot: There are many ways to compromise existing maintainers. Compromising people is the core competency of intelligence, happens all the time, and most cases probably never come to public knowledge.
> consider the route of getting "kompromat" on a developer to make them "help" them
I suppose that’s an option, but it also introduces an additional risk of exposure for your operation as it doesn’t always work and makes it much more complicated to manage even when it does work.
They might not even use blackmail, they might just "help out" in a difficult financial situation. Some people are in severe debt, have a gambling problem, are addicted to expensive drugs, or might need a lot of money for a sick relative. There are many possibilities.
The trick is finding the people that can be compromised.
I think you're going overboard on what's required. Take anybody who is simultaneously offered a substantial monetary incentive (let's say 4 years of total current/vesting comp), and also threatened with the release of something that we'll say is little more than moderately embarrassing. And this dev is being asked to do something that stands basically 0 risks of consequences/exposure for himself due to plausible deniability.
For instance, this is the heartbleed bug: "memcpy(bp, pl, payload);". You're copying (horrible naming conventions) payload bytes from pl to bp, without ensuring that the size of pl is >= payload, so an attacker can trivially get random bytes from memory. Somehow nobody caught one of the most blatant overflow vulnerabilities, even though memcpy calls are likely one of the very first places you'd check for this exact issue. Many people think it was intentional because of this, but obviously there's zero evidence, because it's basically impossible for evidence for this to exist. And so accordingly there were also 0 direct consequences, besides being in the spotlight for a few minutes and have a bunch of people ask him how it felt to be responsible for such a huge exploit. "It was a simple programming mistake" ad infinitum.
So, in this context - who's going to say no? If any group, criminal or national, wanted to corrupt people - I really don't think it'd be hard at all. Mixing the carrot and the stick really changes the dynamics vs a basic blackmail thing where it's exclusively a personal loss (and with no guarantee that the criminal won't come back in 3 months to do it again). To me, the fact we've basically never had anybody come forward claiming they were a victim of such an effort means that no agency (or criminal organization) anywhere has ever tried this, or that it works essentially 100% of the time.
Absolutely. And that's the point I'm making here. It is essentially impossible to discern between an exploit injected due to malice, and one injected due to incompetence. It reminds one of the CIA's 'simple sabotage field manual' in this regard. [1] Many of the suggestions look basically like a synopses of Dilbert sketches, written about 50 years before Dilbert, because they all happen, completely naturally, at essentially any organization. The manual itself even refers to its suggestions as "purposeful stupidity." You're basically exploiting Hanlon's Razor.
I suppose the point is that even though any given instance of an error like this is overwhelmingly likely to be an innocent mistake, there is some significant probability that one or two such instances were introduced deliberately with plausible deniability. Although this amounts to little more than the claim that "sneaky people might be doing shady things, for all we know", which is true in most walks of life.
> They might not even use blackmail, they might just "help out"
If the target knows or suspects what you’re asking them to do is nefarious then you still run the same risks that they talk before your operation is complete. It’s still far less risky to avoid tipping anyone else off and just slip a trusted asset into a project.
> “I am so and so of the Egyptian intelligence service and would like to blackmail you”
No, but practically by definition the target has to know they’re being forced to “help” and therefore know someone is targeting the project. Some percentage of the time the target comes clean about whatever compromising information was gathered about them, which then potentially alerts the project to the fact they’re being targeted. When it does work you have to keep their mouth shut long enough for your operation to succeed which might mean they have an unfortunate accident, which introduces more risks, or you have to monitor them for the duration which ties up resources. It’s way simpler just to insert a trusted asset into a project.
From what I know of today's developer culture the solution will be for one company, probably Microsoft given their ownership of GitHub, to step in and become undisputed king and single point of failure for all open source development. Developers will say this is great and will happily invite this, with security people repeating mantras about how securing things is "hard" and "Microsoft has more security personnel than we do." Then MS will own the whole ecosystem. Anyone objecting will be called old or a paranoid nut. "This is how we do things now."
As an positive counterexample, US recently reduced federal funding for the program which manages CVEs [1]. There was/is risk of CVE data becoming pay-for-play, but OSS developers have also pushed for decentralization [2]. A recent announcement is moving in the right direction, https://medium.com/@cve_program/new-cve-record-format-enable...
The CVE Board is proud to announce that the CVE Program has evolved its record format to enhance automation capabilities and data enrichment. This format, utilized by CVE Services, facilitates the reservation of CVE IDs and the inclusion of data elements like CVSS, CWE, CPE, and other data into the CVE Record at the time of issuing a security advisory. This means the authoritative source (within their CNA scope) of vulnerability information — those closest to the products themselves — can accurately report enriched data to CVE directly and contribute more substantially to the vulnerability management process.
> solution will be for one company, probably Microsoft given their ownership of GitHub, to step in and become undisputed king and single point of failure for all open source development.
A single vendor solution would be unacceptable to peer competitors who also depend on open-source software. A single-foundation (like LF) solution would also be sub-optimal, but at least it would be multi-vendor. Long term, we'll need a decentralized protocol for collaborative development, perhaps derived from social media protocols which support competing sources of moderation and annotation.
In the meantime, one way to decentralize Github's social features is to use the GH CLI to continually export community content (e.g. issue history) as text that can be committed to a git repository for replication. Supply chain security and identity metadata can be then be layered onto collaboration data.
It can be a stepping stone towards a world in which we use sandboxing and (formal) verification to safeguard against cultural degradation. There's no alternative, too many bad actors are roaming about. I hate that as much as the next guy :(
> If someone barely get's a PR accepted, they probably lack the skills to add a sophisticated backdoor.
That's true, but it's also true that a sophisticated and well formed PR is probably genuine too. Hostile PRs are the exception rather than the rule. And if only the high quality PRs are treated with suspicion, then the attackers will tailor their approach to mimic novices. General vigilance is required, but failure is likely because these attacks are so rare that maintainers will grow weary of being paranoid about a threat they've never seen in years of suspicion and let their guard down.
Early this year, I've received a hostile PR for a "maintenance only" JavaScript authentication library with less than 100 stars but which is actively used by my employer.
It added a "kinda useful but not really needed" feature and removed an unrelated line of code, thereby introducing a minor security vulnerability.
My suspicion is that these low quality PRs are similar to the intentional typos in spam emails: Identify projects/ maintainers who are sloppy/ gullible enough and start getting a foot in the door.
But, source-not-available proprietary systems are just totally hopeless from this point of view, of course an intelligence agency could slip something on. A bored developer at the company could too. Users of this sort of proprietary system have just chosen to have 100% faith for some incomprehensible reason.
I don't think that's relevant to this discussion though, as open- and closed-source subversion would seem to follow really different paths.
- Open-source subversion has the big advantage of having the code, testing and build processes in the open which allows for the attack surface to be exhaustively studied, whereas closed source requires code exfil, reverse engineering, inside intel on processes etc.
- Closed-source subversion can hide in other places -- binaries can be corrupted on a compromised server etc. Seeking to influence the code-based development seems like the hardest road IMO.
- Open-source maintenance (at least the kind under discussion here) stops at the maintainer, whereas most corporate dev is in a hierarchy with non-uniform commit authority. None of the same social techniques would apply.
One follow up to compromising existing maintainers: This makes the creators or long-term good faith maintainers maybe even more "dangerous" than new maintainers.
Who can share a threat model with specific probability estimates on this? FWIW, I’m less interested in the particular estimates (priors) and more interested in the structure.
>If someone barely get's a PR accepted, they probably lack the skills to add a sophisticated backdoor.
Unforuntately it's easy to sandbag being dumb. Just because someone submits a PR defining constants for 0-999 does not mean they're actually bad at programming.
Yes! Anytime you see a function signature like "int timeout", it's safe to assume that the unit is in femtoseconds and pass a gigantic number while you curse out the incompetence of the developer. Either name your variables correctly (timeoutZeptoseconds), or use a proper data type (like a Duration or Period in Java, TimeSpan in C#, or a user-defined literal in C++).
If someone gets stabbed in the eye, we find out about it. So our statistics on eye-stabbing are probably accurate.
We literally have no idea how many xz-style compromises are out there in the wild. We got really lucky with xz - it was only found because the backdoor was sloppy with performance and a microsoft employee got curious. But we have no data on all the times we got unlucky. How many packages in the linux ecosystem are compromised in this way? Maybe none? Maybe lots? We just don't know.
It did at least reveal the playbook, and that you have to get pretty creative to hide things in plain sight.
I'm sure any binary blobs in OSS software, no matter what the reason for having them will be viewed with suspicion, and build scripts get extra inspection after that.
Maybe I'm naive in thinking that some people are already looking into packages that are included in all base Linux builds? Including simplifying the build env, and making sure that the the build tools themselves (cmake, pkgconfig, gmake, autotools etc) are also not compromised.
The de facto standard serialization library for Rust, serde, started using binary blobs to speed up builds only a few months before the xz back door was discovered. Lots of people asked the author to include build scripts so they could (re)generate the blobs on their own and his response was basically if you want it, fork it.
You can always use the "we have no idea" argument because you can't prove something doesn't exist. Go find evidence. It's been over a month since xz and thus far we have zero additional incidents. And if you look at the specifics of xz attack: that wouldn't work for most projects because most don't have binary test files.
I'm nobody so you have no reason to believe me - but there have indeed been other, very prominent projects targeted in very similar attacks. We're still inside the responsible disclosure window.. hell, even in the blog post we're commenting on, three JS projects were targeted in failed attempts. That's 4 public projects now..
> seems to have 0% chance of succeeding for almost any project.
Its obviously more than 0% given xz was successfully taken over and backdoored. Even a 5% chance of malicious takeover per project would make the situation pretty worrying given how many well funded, motivated government agencies are out there.
I'm not talking about xz, I'm talking about that OpenJS thing: random people emailing out of the blue "plz gimme maintainer". Entirely different situation.
I did quote the "three JS projects were targeted in failed attempts" bit, which should have made that abundantly clear.
Is it a different situation? Seems similar to me, except the examples we know about (the obvious ones) are the low skill examples. If someone played the long game like xz and made some helpful improvements to the project in that time, we wouldn’t know about it.
People have also done the same thing (to great effect) on the chrome extension “store” to get all manner of malware into chrome extension updates.
“Nobody unsubtle was successful” tells us nothing about the success rate of subtle attackers. It’s like looking at all the dodgy ssh and http requests any host on the internet is connected to and concluding “yep, 0% of low effort script kiddie attacks get through. I’m 100% safe from hackers!”
Are people really looking though? Are all open source libraries being run through extensive performance profiling to look for known heuristics? Are they being looked at line by line for aberrations?
I don’t have confidence that people are looking for evidence of potential exploitation because of reasons like the ones you bring up.
With hindsight it's not the runtime behaviour of the library that you'd want to test - the weakest point in the chain is where the distributed source .tar.gz can't be regenerated from the project repository.
For how many projects is that actually checked? I bet barely any.
Its especially difficult because most projects aren't built in a reproducible way. You should be able to uncompress and compare a source tarball. But if you get a binary and the source code used to generated that binary, there's no way to tell that they match.
Luckily the source tarball is the more important one to check, because that's the difference between backdooring one distribution and backdooring them all.
It's still not trivial because there might well be legitimate processing steps that are used to create the tarball, but it should be doable.
Most commonly-used projects are watched by a bunch of people, or diffed on updates. These are not in-depth reviews, but should catch most of it. So yes, people are looking, and have been looking for a long time.
The reason Jia Tan could do their thing is because 1) the main meat was in a binary test file, 2) the code to use that seemed relatively harmless at a glance, and 3) people were encouraged to use the .tar.gz files instead of git clone. Also you need to actual get maintainer status, which is not as easy as it sounds.
I've been thinking of inserting a "// THIS LINE IS MALICIOUS, PLEASE REPORT IF YOU SEE IT" in some of my projects to see how long it would take. I bet it would be pretty fast either after commit or after tagging a release.
No. If there is strong incentive to compromise, and little to no chance a compromise is being found, it's statistically most likely to assume compromises happen on a regular basis and only rarely are found out.
Your choice of language in your comments (in this thread, not in general) isn’t bolstering your argument.
Why not be curious rather than just dismissive? This seems to be people just talking past each other at this point.
There have been a lot of changes in the last ~five years that point in the direction of supply chain security being at greater risk.
Evidence comes in many forms. The relevance of evidence depends on what part of the problem you are looking at.
Also, it is rational to talk about the probability by which different evidence is likely to be surfaced!
I think it is possible you are sensitive to people making such claims for self-interested purposes. Fair? But I don’t think it’s fair to assume that of commenters here.
> Your choice of language in your comments (in this thread, not in general) isn’t bolstering your argument.
Yeah, you're probably not wrong. I've had this argument a few times now, and it's the same dismissive "we don't know what we don't know" every time. Well, you can say that for everything and given the complexities of the xz attack that seems a bit unlikely to me, which is then again countered with "but we don't know!!11"
"Every contributor is dangerous" is spectacularly toxic type of attitude. I've already seen random people be made a target and even had their employers contacted over this before they even had a chance to explain(!!) To say nothing of "there are many ways to compromise existing maintainers. Compromising people is the core competency of intelligence, happens all the time" – so great, now I'm also potentially dangerous after spending untold hours and money over the last 20 years because I could be compromised. Great.
This was never a nuanced conversation about risk management to start with. This is not the type of community I've worked for all this time. "Let's use some common-sense tech so this isn't that easy". Sure, let's talk about that. "Let's treat every volunteer involved as potentially hostile and compromised after we've seen a single incident"? Yeah, nah.
> "Every contributor is dangerous" is spectacularly toxic type of attitude.
I view this from the lens of "How well can people reason about probabilities?" and research has shown, more or less, "not very well". In the short term, therefore, it is wise to tailor communications so as to avoid predictable irrational reactions. In the medium term, we need to _show_ people how to think about these questions rationally, meaning probabilistically.
For what it is worth, I prefer to avoid using the phrase "common sense", as it invites so many failure modes of thinking.
My current attitude is, more or less, "let's put aside generalizations and start talking about probabilities and threat models". This will give us a model that makes _probabilistic predictions_. Models, done well, serve as concrete artifacts we can critique and improve _together_.
I hope to see some responses to my other comment at https://news.ycombinator.com/item?id=40271146 but I admit it takes more effort to share a model. It is well outside the usual interaction pattern here on HN to make a comment with a testable prediction, much less a model for them! Happily, there are online fora that support such norms and expectations, such as LessWrong. But I haven't given up hope on HN, as it seems like many people have the mindset. I think the social interaction pattern here squanders a lot of that individual intelligence, unfortunately... but that pattern can change in a bottom-up fashion as people (more or less) demand, at the very least, clearer explanations.
In the end you can never fully trust anyone, including yourself. This has always been true for anything: people get drunk, have psychotic episodes or have other mental health issues, things like that. It happens. Remember that Malaysian pilot flying the passenger plane in the ocean?
Every pilot in the world will agree that we need to think about risk management to prevent that sort of thing. I think a lot of them will have issues if we start saying things like "every pilot is dangerous" and (in a follow-up) "long-term good faith pilots are maybe even more dangerous than new maintainers". Then you've gone from "risk management" to just throwing shade.
I don't disagree. But my follow-up response is "don't leave it there; factor that into the probability tree".
What should professionals in cybersecurity do? (Not my field, so I could be off-target here) My recommendation: communicate a risk model [1], encourage people to update it for their situation, and demand that people act on it [2]. Not too different from what the field of cybersecurity recommends now. (Or am I wrong?)
[1] based on a set of attack trees (right?)
[2] based on the logic that if you get pwned, you become a zombie to attack me
> This was never a nuanced conversation about risk management to start with. This is not the type of community I've worked for all this time.
I'm not quite following the second sentence. What kind of community have you worked for? Do you mean "worked for" as in e.g. "the spirit of your comments on HN"? Or something else?
I think they are using community to refer to F/OSS projects as a monolithic entity, rather than a million separate and often competing and disagreeing fiefdoms that have always had issues with toxic assholes worming their way into too much power.
There are a lot of dismissive folks who think this is some kind of one-off event because you can't prove it's not- oh wait, the other attempts we can prove aren't enough evidence either!
I understand being wary of America trying to solve this the only way we know how (PRIVATIZE IT!), but dismissing it as a non-issue makes that more likely because you're basically saying you plan on ignoring it rather than putting your own controls in place.
Yes, FOSS projects need to be welcoming to new devs. No, they don't need to pretend malicious actors aren't an issue in order to do that.
You can vet new people, and be welcoming, at the same time.
This is about "social engineering takeovers of open source projects", not "socially-engineered cybersecurity attack", which is much much broader.
I've been pretty clued up on open source for the last 20 years, and I don't really recall any other similar incidents other than the two I mentioned. I tried to find other examples a few weeks ago and came up empty-handed. It's certainly not common. So please do post specifics if you know of additional incidents, because from what I can see, it's exceedingly rare.
You seem super confident that there have been zero similar attacks that achieved their goals without detection. By definition, almost anyone who pulled off this kind of thing would try really hard not to burn that backdoor by being super obvious (for instance, using it to deface a website). We literally would not know anything about it, in all likelihood. Therefore I feel like it’s a lot more intellectually honest to say we have no idea if that has happened elsewhere, than it is to confidently proclaim that it certainly has not just because it’s been a month since xz.
What I'm argueing against is absolutist fear-mongering statements such as "every contributor is dangerous".
I'm not confident about anything, but anything could happen or have happened all the time. We need to operate on the reality that exists, not the reality that perhaps maybe possibly could perhaps maybe possibly exist. And we certainly shouldn't be treating anyone sending you a patch as a dangerous hostile actors by default.
You seem to think that vetting contributors or reviewing all code commits for malicious actions or code is some unreasonable ask. That should be standard practice.
If someone is getting angry that you actually check their code for vulns, or that you don't let them make changes to certain core areas of a large app without establishing some credibility first, you probably don't want them working on your project.
You can be welcoming AND cautious at the same time.
It has been standard practice for decades. Sometimes this goes wrong, because everything can go wrong. It happens. Casting doubt on any contributor, any maintainer, and any long-term maintainer with fantastical stories is just throwing shade. Of course no one can be trusted absolutely; that has always been true for anything from software to child care to launching nuclear bombs. Anyone and anything can become suspect if you analyse things with enough of a suspicious mindset.
And no, being cautious is never throwing shade, unless you're doing it in a discriminatory way, like assuming that Chinese or Russian contributors are more dangerous.
I don't think the "armed to the teeth" theory is correct. If you were right, people wouldn't honk at each other or otherwise involve themselves in any sort of road rage. But people rage at each other all the time, and only very rarely does someone get shot.
The reason people aren't walking around stabbing you in the eye with a needle is because there is no reason for them to do that. They gain nothing. They don't desire that it be done.
If the news articles about Instagram extortion are anything to go by, adding weapons to an extortion situation is more likely to lead to a suicide than the extortionist being dissuaded.
It's not the new contributors you have to watch, it's the sleeper contributor who has built up a solid reputation and then is "activated". At least that's how I understand XZ.
Looking at some of these cases, each PR on their own doesn’t look suspicious, but it was what they all built up to — in some cases from multiple bad actor contributors that, on the surface, weren’t connected.
Wasn't a key thing of the xz attack vector that people where encouraged to download the custom source release instead of the autogenerated Github one? I don't know if that is a pattern but it seems like best practices in the (source) supply-chain could prevent a large class of these attacks.
same with npm. i publish releases of my OSS libs to npm, but there's no guarantee that what is uploaded is what you see on github. that's a lot of trust you have to put into my opsec, etc. not good.
That is unfortunately how `the `autotools` ecosystem works; although I guess projects could guide their users to run `autoreconf -i` if working with the source code instead of the release tarballs before doing the usual `./configure && make && make install` step.
This should be the mindset for any commit. I mean, they could add something benign unknowingly. Something could hack their account, or abuse a flaw in your commit-system to appear as someone else. They could have a melt-down, or strange ideas and adding something for nonsical reasons. Shit can happen all the time from all directions for any reason.
Consider this comment: "But the xy story taught us, that every contributor is dangerous, the most dangerous ones are probably the most helpful and most skilled contributors."
What is "xy"?
It's tempting to classify this as a typo but y is a long way from z on the keyboard, while the x and z keys are adjacent.
I type xz on a regular basis because it is so often used in place of gz for compressing tarballs. I cannot imagine calling it "xy" unless I rarely used it.
It’s not the new contributors, but older ones that might have already built a rapport with you so you are less critical in your review. That what happened at Xz utility and the hacker nearly got access to every Linux machine out there.
Honestly, a good PR should have a very clear description of the idea and a sample implementation, and then a trusted core contributor re-implements the fix on his own. But Github users are entitled and spoiled by Github-marketed commercial software, so they will rage at this.
An issue usually doesn’t have the code for implementation of the solution. Yes, very often patches are attached in comments, but they are not required and usually attached by other people, not the author.
This is what I feared the most. Trust issues that lead to less progress and a community that slowly drowns in suspicions. When the XZ thing happened they were already going after one person accusing them of being part of the whole thing in one of the bug reports on github.
Why is there not a policy that any PR can be rewritten by a maintainer ? Wherever the PR looks a bit odd, rewrite it so do the same thing a different way. Enough unpredictable change to disrupt finely-tuned subterfuge.
you can wait for tree (or x) PR's passing specified unit tests for functionality and then merge a random one. But this is a luxury (effort wise) for any kind of project.
the attitude remindes me of maintaining game-servers and looking out for cheaters; once we had a handful of folks looking out for cheaters, it turned the community against itself calling everybody a cheater...
i think it is good to be cautious; but overall it's the same cat and mouse game we've seen before.
i can only say good luck on not letting it stress you out second guessing other folks actions and intent - and hope we continue writing code for humans to read vs the cryptic, obstrufcated, even "elegant" code (not to dive into the skill issue rabbit hole lol)
First I think that it is wrong to single out this issue on Open Source projects.
For example, since the first versions of app stores, when you are an app developer you would receive a lot of messages from random shady dudes ready to buy your application if it had a few users.
Also, the xz thing was kind of pretty smart, but it is also a thing in mind of most OSs developers that you can't trust any random contributor and so that you will be very careful of really knowing someone before giving some privilege.
Just look how the debian policies were designed or that things like "pull requests" were invented for Open Source projects where everyone in a project used to be allowed to push whatever in private ones before.
The scope of impact if a mobile app becomes malicious is _immensely_ smaller than if xz becomes malicious. The latter seems to be national security level
Only because most people aren't using mobile apps for real work. But if someone were using a mobile app to perform SSH and the app is backdoored, it's game over for any server you SSH into.
We've been adding features for 30+ years to open source software, they become so complex only very few people understand them anymore. Recently I looked into jfet level 2 implementation in ngspice, expecting familiar equations, but through series of small changes and maybe some DRY too, the code is almost unrecognizable. When graybeards finally retire, there will be lots of shrugs.
Your average American could quit working for 5 million dollars. They could live comfortably for the rest of their lives off that money, if well invested (read EFT for sp500)
5 million bucks is change for you average government.
"Amazon made billions on my project and if I turn a blind eye to this I can retire, fuck them..."
Sponsorship, for good or bad makes a lot of decisions simple.
Probably wouldn't even need 5 Million. According the the PBS article about the 2 US Navy sailors arrested for spying around august of last year one of them was apparently only bribed $10k-15K for the year [1]. I was pretty shocked at first but it made sense that with the financial stress many face in today's market a 10K bribe would go along way and have a high return on investment especially if the potential payoff for a backdoor or zero-day is in the millions [2]
>2 US Navy sailors arrested for spying around august of last year one of them was apparently only bribed $10k-15K for the year
This is exactly why investigations for security clearances focus mostly on a person's financial situation: someone who has a lot of debt and shows a pattern of poor financial management, i.e. someone who'd jump at a chance to make an extra measly $10k, is the kind of person they want to avoid giving a high security clearance to, because of incidents exactly like this.
It's a common misconception that clearance holders who sell secrets make a lot of money doing so, and this just isn't the case: it's comparatively small amounts like this. For someone who's deep in debt and desperate, it doesn't take much to buy them.
After Abscam in the late 1970s/early 1980s, one of the quote going expressed surprise not that politicians could be bought, but that they were so cheap.
Money isn't the motivator I worry about. Money requires people to make rational decisions about what they think they can get away with.
Threatening someone's family? Who wouldn't be willing to turn their access to a project into a hack that has some possible theoretical future harm to protect their child?
Sure, the circumstances to being able to leverage someone like this are rare, but the population which is susceptible is much larger than those willing to do somethibg similar for just money.
Friend, I read the comment that you’re replying to that if hostile governments wanted to backdoor our software supply-chains, it wouldn’t cost that much to corrupt an open source maintainer.
Whereas, if it was the norm for well used OS projects to be sponsored, it’s far less likely to be tempted by relatively small bribes.
I've been working on thanks.dev for over two years now & reading this report is disappointing to say the least. Why not spend the time to explain the value XZ Utils created for all the commercial users & what companies can do to better supporting maintainers with hundreds of issues experiencing burnout from their unpaid work? OpenSSF should instead promote FOSS programs like https://frontendmasters.com/blog/how-were-supporting-open-so... & how they help their open source community stay active. Working in open source is a social contract & corporates need to be better citizens if they want to reduce their risk profile.
On first reading your comment makes a lot of sense, and is certainly logical for maximizing the common good.
But unfortunately, companies simply don't work the way you are proposing.
The short reason is this "good citizenship is indistinguishable from corruption. Therefore good company governance leans away from both."
The somewhat longer answer is that while a "company" might have a lot of money, or might make a lot of money leveraging some common good, it is not (usually) one person's money.
The bigger the company the harder it gets to actually -spend- the money. There are procurement departments, various sign-offs and so on. First and foremost it helps if there is a tangible (defendable) reason to spend the money.
Yes, companies "give" money away. Usually under the guise of marketing. It's easy to donate money to the local cancer center. It's harder to explain the marketing value of supporting random open source projects.
For tech companies it's -somewhat- easier, but even then it's simpler to donate time rather than money.
I've said it a lot lately, but OSS development has to "commercialize" if it wants to be commercial. That means first understanding "what companies pay for" and designing products to fit that.
Or target individuals with excess cash of their own that they're willing to just "pass along".
"What companies pay for" is anything they cannot get for free. If the value of an OSS project is mostly in its code, then any license that allows it to be used commercially will mean lots of free-riding.
Thing is, they still continue using the code even as they are complaining. Which is to say, they'd very much like the warranty so long as it's also free, but if it's not, they're willing to go without.
I'm sorry but this is a huge copout. At least the big companies and governments have the money to solve all these problems. They have literally teams of lawyers on retainer and they can hire a few more people for all the other self created bureaucracy. None of the things mentioned here are laws, of nature or otherwise.
I'm not sure what you are proposing. You declare cop-out, but then fail to describe a mechanism.
Who exactly would initiate this change? Shareholders? Board members? C-suite? Employees?
What do you propose an initiator should argue to convince colleagues? Why should an initiator spend political capital on this rather than on themselves?
Of course they have the money to give to random OSS projects for no return. What your post lacks is any motivation for doing so.
Yup these three. Its been too long that we give passes to executives for their bad behavior. If a company is using a tool they should be thinking about paying back. If they are not, they are being bad members of society (and yes companies also live in a society). It is not hard if there were good people in charge. Who's only lookout isn't to make the share price go up every quarter.
And one difficulty here I believe is that those intent on social engineering think about it in more sophisticated terms than their targets, which perhaps is obvious.
And part of the process can be a kind of performative incredulity at the very suggestion that they are part of a campaign of hostile takeover, even if it's exactly accurate. I suppose you could even have unfortunate circumstances where parts of an open source community are unwitting advocates of being co-opted.
And I think you probably see a parallel in state-based information warfare, where part of the objective isn't just to spread misinformation, but to shift cultural norms so that the transmission of misinformation is inherently easier, which can involve sewing distrust in institutions or expertise, or normalizing a gish gallop argumentative style.
I'm perhaps stating the obvious here, but I suppose the upshot is that human psychology can be targeted in a programmatic way, and there might need to be something in the way of a normalized infosec-oriented doctrine relating to the stewardship of open source programs as an intentional countermeasure.
>> And I think you probably see a parallel in state-based information warfare, where part of the objective isn't just to spread misinformation, but to shift cultural norms so that the transmission of misinformation is inherently easier, which can involve sewing distrust in institutions or expertise, or normalizing a gish gallop argumentative style.
My thought immediately went to Linus Torvalds. The way he acted was tolerated in the past, but the culture was changed and it was used to force a change onto the project.
Same thing with all of those Codes of Conduct that suddenly propped up.
Are you saying codes of conduct make the transmission of misinformation is inherently easier, e.g by sewing distrust in institutions or expertise, or normalizing a gish gallop argumentative style? Are you saying Linus Torvald's behaviour prevented those problems?
It has not occurred to me before, but I don't see why the cancel culture surrounding such matters couldn't be used as an attack vector. Basically, target key maintainers who are vulnerable to this (white, male, history of questionable interactions etc) and push until you force them out one way or another. Then when project gets in trouble because of the lack of qualified manpower, pitch your own agent as replacement. For bonus points, make it someone who hits the right buttons wrt "diversity".
I think it's more that it leads to brain drain. Be it by misinformation, by discouraging anyone who doesn't fit [demographic of leadership], or by simply not giving proper code reviews in the name of politeness. All 3 ways creep up and the code base suffers from lack of care, and/or lack of talent.
We can certainly debate how effective old methods were (and yes, I have no doubt Torvald's old behavior turned off many a talent) and how we can improve on them, but in a more general viewpoint we need to remember that many open source code bases are, or started as, volunteers providing their knowledge in their free time. It doesn't take much to make them walk to the next repo. Or not contribute to OS at all.
Not going to quote the whole thing, but yes hard agree. Contrasting this factual opinion with the opinions in the “sell TikTok” hn threads is quite a delta.
The speed reading shit they do in competitive debate was in my opinion 100% caused by clandestine elements who wanted to keep the future “revolutionary” intelligentsia class obsessed with ivory tower elitism so that they don’t get too close to doing actually subversive things.
I have no other explanation for how otherwise smart people think that speed reading lacanian psychoanalysis is high school is valuable for anything.
In Europe, the most popular high school and university debate format is British Parliamentary in which spreading is not popular because you only have 15 minutes to prepare and weakly justified arguments don't require responses.
British culture certainly has plenty of ivory tower elitism, yet has passed by this. I don't think it's a special revolutionary pedagogy, just a different interpretation of how to deal with subjectivity in debate.
To play devil's advocate people seem to be quite attracted to self-sabotaging ideologies, as they offer the intoxicating justifications of "not my fault" and "no point in doing work". Hell, the typical "rebellious" things are practically hand-picked to be ineffectual at gaining any even small sort of power and demonize doing so. Just look at the dynamics of being accused of "selling out".
I agree with you that it's incredibly bizarre, but I associate it with the strange cultural norms that can only crop up in very specific academic environments that have just the right alchemy of academic strangeness, competitiveness, and idiosyncratic historical origin. I'll compare it to something I recently discovered, which is some viral video I recently saw of some sort of pig fare where kids lead pigs out on this walk to show how well the pigs are trained and I think to show off the pigs as models specimens, and the kids do this intentional intense eye contact with judges in order to get the judges to look at them. It seems so strange and abnormal, but it was explained away as just something that's part of the history of the competition and having strategic value for being effective in the competition.
I've seen videos of the college debates you're speaking of though, and I've definitely felt that they're badly in need of reforms that either impose a word count or otherwise disincentivize speed reading.
The long and short of that is just to say you can explain it without regarding it as some sort of intentional state disinformation program. I would also say I find that especially implausible just because, while I don't love the practice, I don't think it degrades our ability to follow arguments or have information literacy necessarily, and meanwhile modern social media absolutely does seem to instill habits that reinforce short-term attention spans, disjointed thinking, object permanence problems and the like, all of which would dispose people to be more receptive to bite-size arguments that don't have to fit into a comprehensive or coherent worldview.
> And I think you probably see a parallel in state-based information warfare
My own research shows it’s the opposite: they destroy trust in institutions and experts by telling the truth when those groups lie to their own citizens.
The US did this routinely during the Cold War.
More recent examples include:
- demographic facts about murder, violence, and police
- demographic facts about college enrollment, eg the racism at Harvard
- George Floyd’s autopsy report
- images of cities burning; reports of the 70+ people murdered
- facts about COVID
- facts about COVID vaccines
- facts about Ukraine’s status on the battlefield
- footage of Nazis in Ukraine
- facts about Tavistock and WPATH lacking scientific evidence for their recommendations
Because institutions and experts have normalized lying to “nudge” the public via narrative manipulation, it has become easy for adversaries to undermine the nation by showing contrary facts.
People become more radicalized by demonstrating with evidence a supposed ally has betrayed them, eg, your government lying to you. Once broke, trust in institutions and experts takes generations to repair — or a replacement of those institutions entirely.
Maybe we need a reporting system for maintainer changes of bigger projects. Some list where they get published and people can keep an eye on it.
Those changes of maintainers need to be synced to package distribution sites like npm.js or Debian packages and put in context with versions/releases.
In Europe this was introduced for banks after the banking crisis. If a bank does any organizational change, a report is sent out to all member states of the EU right away and any of the 27 national bank agencies can check if they notice something unusual. It might be possible to bribe a few people in your own country, but it’s really hard to bribe all responsible people in 26 other countries.
I'm sure some security researcher is doing this, but we could easily create a visualization of "who has contributed over time" and identify transitioning of maintainers automatically just from git.
This might be worth doing and contributing to a site like bestofjs or libraries.io (I don't really use that one though!)
It is not the idea of GPG thats bad. In fact, the idea is great! The implementation of GPG however is quite another thing. Ease of use and user experience are really not that great with GPG. It is difficult to use even for developers. Developers are users too amd so on.
> Maybe we need a reporting system for maintainer changes of bigger projects. Some list where they get published and people can keep an eye on it.
The rust project does it. There's a repo with all [active] members and their permissions on github, etc. These get synchronized and updated every time there's a change.
This is, in my eyes, one of the most important parts of "Infrastructure as Code". You should make the list of who has what permissions a critical artifact, as immutably part of the repo as any other change.
That's great, but I think that's not enough. This would need to extend to crates.io, I'm sure there are some packages, that are very commonly used and not part of rust-lang.
While I love open source, this feels to me like something companies need to pay for.
It might be that no open source contributions I've made are things people care about, but I'm not spending one second of my time for free updating databases so multi-billion dollar companies can feel safer.
Well yes, sounds great, but it doesn't really address the security problem. Now you've just got the bad guys getting two paychecks instead of one and the good guys getting one paycheck instead of zero.
One of the biggest risks for companies is securing dormant code, it's perfectly fine for a project to be no longer sexy enough to maintain. Platforms like thanks.dev have already proven how reward & recognition can help promote development in an ecosystem https://www.youtube.com/watch?v=e5FV-AnKPlo&t=1s
Did you mean: instead of trying to become a maintainer to a trusted open source project, how about bad actors simply bribe the existing maintainer to do their bidding? There would be no maintainer changes in that scenario.
Related, the motivation for trying to gain privileged access to open source projects is to leverage the existing trust associated with that project. A different long game that could be played is to create a new project with the intent on backdooring it a few years down the road, after it has gained sufficient trust.
I recently had a xz moment where the rust zip crate was taken over by a single person and the original crate was completely replaced. I'm still not sure if this was legit or not: https://github.com/zip-rs/zip-old/issues/446
Honestly, from reading this it seems like people blew it _way_ out of proportion. Someone forked the project to make updates because the original maintainer seemed to be not doing much, the original maintainer came back to say that they were correct to ask about maintenance because they didn't expect to do any more work due to health issues and then volunteered to transfer the crate to the person who forked on their own, which the person who forked it accepted.
It's kind of bizarre to me because I don't really understand what mental model could lead to not taking any action earlier than this if the was things turned out is so upsetting. If they were happy to keep using it exactly as is because no updates were needed, why not just pin the dependency to that version exactly (and republish their own fork if they were worried about old versions being "yanked" and not being able to use it for anything new or offline)? If they did expect some form of updates over time, where did they expect them to come from when the existing maintainer felt they were unable to continue given health issues? Any attempt to find some other solution to future ownership would be heavily scrutinized by the exact people who commented on this issue with strong opinions and don't seem to have much empathy for the health issues, which would defeat the entire purpose of trying to take time away from work for their health.
I'm surprised that this needs to be said, expecting people to put in extra work to project you from what happens to their projects when they literally can't keep maintaining them due to health issues will never work, and it's also just an awful way to treat people. If you have serious concerns about how situations like this could be exploited by malicious actors, you should be paying much closer attention to the status of your dependencies and taking actions to insulate yourself from potential fallout long before some like this happens. If you've gotten to this point under the assumption that you can just veto any change in ownership that you don't trust, you're already too late.
This is not a ‘xz moment’, as a sibling comment said, it is norm in open-source.
Someone with more time forked the repo, included the changes that were necessary, build up trust and then this eventually get merged. Now obviously there is no guarantee they will never act up in the future, but this is not different than for the original owner.
Trust is a necessity to open-source reliably functionning, because it in parts makes up for the lack of money, and allow to move fast.
XZ is the exception. And frankly there is not much to do against it.
> Enable two-factor authentication (2FA) or Multifactor Authentication (MFA).
Not on any third party system, where you're locked out forever if you lose your second factor. Fuck that!
Only self-hosted, where you can recover via physical access.
(That should actually be the first advice: host the stuff yourself. People lose control of projects due to hosting them on third party services. Be the guy who can pull the power cord out of the wall.)
> Not on any third party system, where you're locked out forever if you lose your second factor.
Every two-factor system I've ever seen is actually two-of-three, with an account recovery code that you save elsewhere.
I lost all my two-factor auths when my phone got wrecked, it was annoying to reestablish access to those accounts (and I now use a TOTP client which backs the tokesn up), but it was tedious rather than difficult.
> and I now use a TOTP client which backs the tokens up
What are some good options for this? I think my ideal solution would export an encrypted file, a bit like KeePass does on the desktop, but I don't know of many mobile apps for that.
It has two security levels: normal-level security codes will unlock when the phone is unlocked, high-level security codes require a separate unlock to get the code. The former back up to iCloud (E2E encrypted), the latter don't back up.
The recovery codes I've seen tend to have been 10 8-digit sequences.
If you don't have a printer and care about recovery codes, those are easy enough to transfer manually onto dead-tree material using a stylus-like handheld device that deposits graphite or ink onto the surface it touches.
When I know it's important to decipher something later, I have a 100% accurate decoding rate on my own handwriting, so that's not a problem. It's not hard to write clearly and carefully. In addition, they give you ten codes to use, so you'd need to make indecipherable errors in all of them. Also, having a safe place to store documents is table stakes for adulthood.
I would hope that when a software dev sees a widget that says "these are your recovery codes, write them down or copy them to a secure location or you may lose access to your account", they do exactly that.
Except for codes which protect my money (which go onto paper, which goes in a safe), I put them in a password vault. TOTP offers protection against getting shoulder-surfed, key logged, or phished, it isn't much protection if Mallory gets access to my entire password vault. YMMV.
> I would hope that when a software dev sees a widget that says "these are your recovery codes, write them down or copy them to a secure location or you may lose access to your account", they do exactly that.
Yes software developers are known for never making any mistake.
i hate 2FA as well, but in the end, even if i loose my access to github i only loose access to my github identity but i don't loose access to my code, so i can live with that.
of course in the light of this discussion losing access to my github identity would be part of the problem, so it's a tradeoff. is it more likely that someone will break into my account and abuse my identity if i don't have 2FA or is it more likely that i loose my second factor and have to rebuild my identity. in the latter case someone else could also pretend to be me, but since the xz debacle both of us would face more scrutiny that my hope still is that i would win.
> if i loose my access to github i only loose access to my github identity but i don't loose access to my code, so i can live with that.
That means that you need to fork your own project, and there is no way to communicate it to the users, since the new account could just be someone pretending to be you.
If there is a security vulnerability, it would remain unfixed forever.
> is it more likely that someone will break into my account and abuse my identity if i don't have 2FA or is it more likely that i loose my second factor and have to rebuild my identity
Since phones are very easy to break, and until very recently there was no way to backup google authenticator, I'd say that losing your 2nd factor was the most likely of the two.
Now if you say that you backup your 2nd factor seed in your password manager, where your password is… congratulations you're doing over-complicated 1 factor authentication!
well, yes, exactly. once i realized that, my reaction was: why thank you github, you just made my one factor auth more complicated for little gain. well, ok, i don't store the otp with the password, so cracking the password became a bit more complicated too. but for example committing code doesn't require otp and my browser has me permanently logged in, so where exactly is the added safety now?
as for the lost identity. a new user could at least share a warning. that user doesn't have to be trusted to get others to be more vigilant and scrutinize the code very carefully as eg was done with XZ once the issue was discovered. imagine an unknown user would have alerted the community that the maintainer account was compromised or locked out. they could have reached out to people who know them to verify their identity and to corroborate the claim. it would be a long and tedious process, but at least any attacker would be prevented from getting any further advantage too.
it could still mean loss off the maintainership and loss of users, but i can also host my projects in multiple places so that only part of my known and verifiable identity can get compromised at once.
in the end it's partly security theater, partly arms race, partly an improvement through raised awareness...
It's a very serious issue. I don't really know if there is any "one solution." I suspect that each project needs to set its own bar, and that any dependency that falls out of maintenance should be removed as quickly as possible (which was good practice, beforehand, but even more important, now).
[EDITED TO ADD]
I would also think about "scoring" the sensitivity of projects. Things like cryptography and low-level drivers would be highest-rated, while user-space chrome might not be as important.
> This approach bears strong resemblance to the manner in which “Jia Tan” positioned themselves in the XZ/liblzma backdoor.
No, it doesn't. I stopped after read this. Jia's "attack" was near state level actor stuff. A bunch of emails asking/begging for commit access sounds like a 16 year old sending emails from the basement of his parent's house.
+1. My team still favors batteries-included style systems, even if it means missing out on the latest thing all the cool kids on the internet are talking about. For languages, this usually means sticking with the standard library that ships with it along with one or two libraries that augment it (e.g., BOOST for C++). It's not like "batteries included" eliminates the problem - a bad actor can wander in there and cause trouble. It's just a much more controlled environment that often has a process for contributing. I'm not a fan of the move of languages away from rich standard libraries to the "Random Interconnected Pile Of Internet Stuff".
Unfortunately people like me are in the minority it seems, and the "move fast and break stuff" mentality seems to still dominate the open source world even if that phrase has fallen out of favor - the attitude still seems to exist.
IMHO, Open Source is often misunderstood by companies as free code as in cheap, when in fact it should be seen as collaborative code that aims to avoid redundancy, saving companies countless hours of development, research, fixing compatibility issues and so on, due to such collaborative union between all the developers, mainly the maintainer(s), albeit at the cost of sometimes excessively generic non-performing code, and, here we go, albeit at the cost of potential security vulnerabilities that keep requiring an investment that has been largely ignored.
Perhaps companies should take the necessary reversal for the security of their business and create roles within the organization as guys who exclusively and actively revise the code being used, at the same time as internally classifying the integrity of genuine maintainers to ensure that bad guys don't get paychecks from either the company and the community, by showing facts.
It doesn't solve the problem, but I think it should be a generalized procedure, not just something that very few big companies seem to be doing, and one can see it's not enough.
Ironically, most companies ignoring the issue appear to be targets.
Although it did cross my mind that maybe such companies are not just misunderstanding open source as free code as in cheap, but maybe they also consider the security issues and repercussions as cheap; and if this is what is happening, maybe governments across the glove should impose large fines for data leaks and vulnerabilities in their systems, proportional to the company's revenues, proportional as in making them invest in security. in addition to what the article suggest.
I don't know. I'm just thinking loudly, certainly the problem is not easy to solve.
Doesn't have to be a takeover. If I'm a state actor I'll maintain a few projects specifically so I can hide backdoors in them. When/if one gets popular and I decide to backdoor them I'll claim it was a social engineering takeover.
Well after the XZ attack, I was thinking how common this can be. Good to know that at least im not the only one and others inside the community are wondering about this. I hope someone is smart or lucky enough to find a solution to at least be able to lessen the impact of these attacks. I still wonder how many more of these are there, and my question is because of these attacks, isn’t open source more prone to these compared to closed sourced software? Usually the argument for open source is because everyone can read the code, its less vulnerable but now because everyone can write the code and have big incentives to do malicious stuff, doesn’t it make open source worse?
Many open source projects just don't get enough attention for the 'many eyes' benefit of OSS to occur. Many projects are neglected and poorly maintained, with little participation from the users.
I don't think OSS is particularly special though. If a state actor threw cash around they could find folks at many big companies to do their bidding. In my experience, commercial software reviews are susceptible to the same sorts of attacks as those listed in the article("please review my change ASAP because it needs to go into the next release before the deadline!").
I don't know what to do about this. You could subject approved submitters to better background checks. You can improve automated threat detection and code analysis. You can switch to safer-by-default languages that make backdoors and malicious behavior more obvious.
I wonder if the same issue exists in other engineering fields? Has anyone ever bribed an engineer to make a bridge or a water supply less robust?
Ah, misunderstood, you mean something like a sneaky sabotage? Hm, don't recall any, the payoff seems to be too small and unpredictable? But I think there were cases of "poisoning" the design of some weapons
The problem is, we don't know. I've seen PRs that could be curious students, or it could be a first try to see if we are paying attention. It's really easy these days to produce a halfway decent looking PR for someone in their first year of uni and my worry is that an increased volume of low to medium quality contributions will lead to maintainer fatigue. Depending on the project, that may be the point where pressure can be applied to share maintainership.
Good warning for the future... but what about the past? Any thoughts on retroactively looking at behavior for existing OS projects? Seems like an impossible amount of work.
Maybe not so impossible. Start with making a list of projects that are everywhere. Inside every Linux distribution, inside every react/angular/vue/etc project, …
Then check which companies support those projects with active development, and calculate a rating. Are the companies located inside democracies or are they mostly from china or Russia?
It’s probably not that many packages in the end. A few thousand high impact/risk projects probably.
I wanted to make a different point. If for example Google or Red Hat were deeply involved within the xz project, there might have been more people reviewing the code. The evil changes to xz were easy to overlook, but not impossible to notice.
Especially the added "accidential" semicolon made me think about probabilities. I think in a code review I would notice that with a probability of 10-20%. So if 10 people would've looked at it, there might have been quite a low chance to get away with it.
Having some high profile companies involved into an open source project the risk score would drop in my opinion, which would highlight the projects that are completely community maintained, and might be more susceptible.
Having such a list might be a security threat by itself though, because attackers would focus on the "low risk" projects first.
One possibility could be a license that requires big companies to dedicate one or more people as maintainers or at least reviewers of a project if they want to get license to use the software.
I think the only feasible option, due to the shortage of SWE talent, is a combination of automation, tooling and safer languages. We need better threat analysis systems, and we need to rely more on safe-by-default languages that make program behavior more obvious.
As a (mostly former) OSS maintainer, I don't want to be suspicious of any contributors. I want welcome contributors as co-maintainers.
I don't want to gate-keep specific tasks (such as releasing and updating the website) so that I'm the single point of failure.
It's already hard to enough to get people to contribute more than a README typo fix or maybe a single feature that they need themselves, and get them invested into the project as whole.
Somebody please create an alternative for keeping projects secure that's not based on suspicion and gatekeeping.
I've been saying for years, over and over, that we need to focus on simple architecture and improve our coding standards but I keep getting ignored. People keep making software and tools more complex... "Just use TypeScript" they say, "Just use React, with Typescript" then they end up with literally thousands of unnecessary dependencies. The bad guys are laughing at our collective ignorance and naivety.
Now probably the whole industry is compromised. Maybe our entire political system is compromised because of this.
People just had to look for projects with minimal code and dependencies. Precisely the opposite of what they did. These clean projects were starved of attention and opportunities and the overengineered projects were brought to the forefront. Now they are easy targets for bad actors. It's so easy to hide vulnerabilities amongst complexity... And we've chosen complexity.
I suspect that the bad actors aren't merely benefitting from this situation, they're probably actively encouraging it, in GitHub issue discussions, on Reddit, in the comment sections of YouTube videos, here on HN. Maybe I'm paranoid.
I think this is a reasonable assumption for most high exposure projects. There are state actors from all sides who want to inject backdoors into software and they all benefit from complexity.
It could indicate that governments are more focused on offense than defense.
It seems to be a logical consequence of our short term corporate and political mindset. Offense is the best defense, they say... Ok, but not at scale when your opponents also possess the same weapons as you do; then you both end up worse off.
Yup, that commenter did exactly what he told others not to do, bashed on something without checking if he's right. Typical bashism. Typescript is from a single big vendor with all the tooling.
React is a different story. By itself it's from a single big vendor, but all the tooling is tons of random packages and scripts...
The fact that you don't recognize how TypeScript adds a massive number of dependencies into your project is proof of how deeply entrenched the mindset of unnecessary complexity is.
Let's not even dive into the question of how `tsc` relies external dependencies written by third-parties outside of Microsoft (it definitely does). Let's not even talk about how many people are working on TypeScript and how big of a target it is for foreign state actors. Let's not discuss how easy it would be to introduce vulnerabilities into `tsc` due to how well it can hide them inside its mangled JS output. An even bigger issue is that TypeScript forces you to add a ton of dependencies into your project.
Once you add TypeScript to a project, you are essentially forced to use a bundler on the front end. Also, it forces you to add a lot of special add-ons for whatever front end framework you're using. You need special add-ons for testing, e.g. `ts-jest`, you need additional linting dependencies and you need to import a large amounts of external type definitions. You now need 2 config files instead of 1 (tsconfig.json + package.json)... 2 engines instead of 1 (tsc + node). The code you write is no longer the same as the code that's running in production; you're running a transpiled version with incorrect line numbers... Now you may need a special library to manage client-side errors with source mapping. E.g. you may need to install the `source-map-support` module or other or you might rely on third party services to help figure out the source of production errors. Your trusty older version of `mocha` now may not show the correct line numbers when tests fail; also `mocha` is now too slow for you since TypeScript takes so long to build... So you end up using the far bulkier `Jest` instead (and all of its dependencies).
Most devs these days don't even realize that it's possible (in fact, easier) to build fast front ends without a bundler. Browsers can now pre-load scripts easily and it's way more flexible than bundling. With native Web Components, you don't even need a framework these days, you can achieve similar results as reactivity using the `attributeChangedCallback` inside custom Web Components. Many people have been successful with this approach in production environments; it's far more lightweight and less error-prone than React development. It also makes it feasible to share components between different physical web pages and services since you don't need to load the entire React framework to use those components. You can follow DHH (founder of Ruby on Rails) if you need proof that you don't need a bundler.
Which of the suggested "Steps to help" would have helped prevent the xz infiltration? For a single-maintainer project, adding bureaucracy can only make things worse.
The fix is easy: license your code as AGPL3. Your project is no longer a useful target as corporations will avoid you, and on top of that you get rid of the demands for free tech support.
I wonder how the person that was running “xz” is doing. Hope he’s doing okay mentally after being thrown into the spotlight by the shit stain of a person(s) behind the “jia tan” and et al aliases
This is going to become a huge problem with state actors attacking what is basically public infrastructure. We should be publicly funding efforts to protect it.
This is only going to get worse with Large Language Models. Let's imagine a somewhat knowledgeable individual, could craft both emails, messages and even commits with a bunch of prompts. Those will relate deeply to the project.
Maybe one day it will happen, but right now LLM-generated persona would likely set off every alarm bell for a lot of people. LLMs have very recognizable style, and it usually falls right into the uncanny valley.
The "recognizable style" that people usually refer to is the default persona that most are exposed. However, the style can be changed very drastically with some fairly simple prompting.
I don't think this is going to be a big issue. Those attacks have to be high-profile attacks. If you look at the xz backdoor, there was some top notch engineering behind it.
If we ever reach a level of LLMs being able to do that, we don't need any open source contributors any more. We just tell tell the LLM to program an operating system and it will just do it.
I hope there is a better way to maintain open source projects without being overly cautious and suspicious of every PR someone makes. Maintaining open source projects is hard, and this is going to slow down development on many projects. And, rightly so, it's better to make a good code base, rather than one that is littered with backdoors.
I wonder what could make this situation better for the maintainers of open source projects?
Designing for safety helps a lot. Memory safe languages, reproducible builds, encoding safety properties in the type systems, and so on.
Sure, an attacker can subvert the types as well as the code, or use unsafe code, or try to tamper with infrastructure, but the more obvious it is that something is unsafe, the harder an attacker's job is.
The xz attacker introduced high-risk features over time and used them to justify weakening security controls and things that might have detected the problem. A culture of safety over the absolute best possible performance might help to make such attempts harder.
I am more suspicious of PRs from new contributors by default now. Of course I keep these suspicions to myself, but besides simply reviewing code for all the regular things, I now ask myself "what sort of sneaky thing could they be doing that appears benign on the surface?"