I believe it is essential for Firefox to allow installing extensions in a private way that does not force us to upload our source code to Mozilla, especially considering their renewed push for user privacy.
It could be a feature that can only be enabled from the browser UI, with appropriate warnings, but the choice must be offered for those who need it.
Until that happens, those with private extensions must resort to compiling their own browser, use unbranded or developer builds, or patch Firefox at runtime to disable signature checks.
This is how you allow unsigned extensions in Firefox on Arch Linux, the same files can be edited on Windows and macOS, restart the browser after changes:
> It could be a feature that can only be enabled from the browser UI, with appropriate warnings
That's not the issue. The problem is: where do you record the fact that the user has seen and acknowledged these warnings? Wherever it is, malware can go there and write "yes, the user has already seen the warnings, no need to ask them, you can run the extension".
You request administrative rights to store the configuration change. With root access malware can just replace Firefox.
Even with signature checks enabled malware can trick users by pointing Firefox shortcuts to a patched browser, changing shortcuts doesn't require root access.
I don't understand this logic at all, it baffles me.
Malware can do a lot of things, but I never heard of common software being designed under the assumption that malware has taken over the PC. The software I use is designed under the assumption that the user is intelligent enough to keep the computer malware free. Only the OS or explicit security software is designed to keep me from malware, but not my text editor, office software or browser.
It is a terrible excuse for ignoring user intent.
What's next? Disallowing to change the startpage from google.com because malware can change it? Disallowing downloads because users could download malware? Going that route would essentially take away the entire software in the end. Maybe at one point Firefox is only allowed to operate from within the cloud, where employees make sure it is 100% safe?
Also others have already said that one could add admin privilege to certain settings.
I mean we're not necessarily talking about malware that has taken over the computer. More like software companies whose installers 'helpfully' install their browser extension behind the users back.
The same is true for link handlers, file extension associations, context menu entries, browser plugins. None of these should be able to be changed by applications, only by the user themselves.
We're entering an age where applications running in a user session are no longer user agents but users of your computer unto themselves that need have their own permissions boundary. Applications running as the user != the user anymore.
It would be important for the Firefox team to allow a discussion with third-party developers about the exact threat model and reasoning behind forcing signature checks, and disallowing an escape hatch.
There are indications [1] that not even Mozilla employees have a good understanding of the threat model and the current solution, and that leaves one wondering how could alternative solutions be possibly explored if the topic isn't even properly understood.
Maybe no one really knows anymore why the feature was implemented that way in the first place.
I think the post-mortem has shown that the biggest problem of mozilla is fragmentation of decision-making, and the existence of probably >50 small teams that do stuff without communicating.
It is highly likely that the certificate problem isn't the only negative consequence, and that we'll see more evidence of mismanagement in the future.
I have the feeling that for some reason mozilla has established a culture where information does not flow efficiently from top to bottom and vica versa, and it even looks like the management doesn't really exist.
When a small team is formed around a task without central oversight, reporting back to someone, it will tend to justify it's existence, even if it means doing unnecessary work.
I heard the last CEO who wanted to streamline the mozilla hierarchy back to efficiency was Brendan Eich and many people got uncomfortable when he started to demand that people actually work productively.
I think a honest post-mortem would have come to a painful conclusion: That the Mozilla of today is in no way able to compete anymore, and many employees have stopped doing real work. The company only lives on because it lives off it's massive market-share of the past.
I am convinced that, within 2 years, mozilla will be confronted with massive lay-offs, threatening the gecko engine. This incidence shows me that they haven't done anything to address their structural problems, probably because most people in the company are content with the place they have, and comfortable living off the massive google revenue.
Once there's malware on the system you've completely lost, there's nothing Firefox can do ever, so there's no point in trying to provide any protection in that case.
That depends. If the malware (and to be clear, "malware" here includes various anti-virus vendors and other gray-hat software that is nominally providing the user a service but _also_ attempting to insert itself into Firefox) is only running with the privileges of the user, it can't modify the Firefox install itself, if that install requires administrator access (sudo on Linux, etc) to modify.
That gives you a trusted place to bootstrap from. The addon database is stored in the user's profile, so _can_ be edited by such malware, but that's where the the trusted code could try to verify integrity in some way. Which is how we got where we are now.
Disclaimer: I work on Gecko, not extensions and the "things injecting into the browser" situation, but have overheard a bunch of discussions about it.
Couldn't the malware replace your Firefox icon with a version that points to a malicious patched Firefox? Or whenever it sees Firefox start, it kills it it and starts the malicious Firefox instead.
Mozilla's reasoning is that these programs like to think of themselves as not malware. Configuring the user's computer is much easier to claim as "honest behavior" than patching or replacing executables.
Okay, you’re not running as Administrator on your corporate Windows desktop. How does that malware replace Firefox.exe — not to mention setting a valid signature?
The answer is why this matters: we’re not in the Windows 95 era any more and a large fraction of users are running without admin privileges, often on a system which does code signing checks. Being able to get a trusted binary to run an add-on is common because the easier direct techniques aren’t as they were a couple decades ago.
First of all: user separation is not a robust mechanism. There are countless privilege escalation exploits. It's just one layer of protection.
If you insist that exploit can't escalate its privileges, then set this addon running privilege from Administrator. That would be a good thing actually.
Exactly, and it’s important in cases where users are being convinced to install dodgy extensions because each additional step increases the chances of someone realizing that it’s a scam.
I for one am OK with Firefox's mandatory code signing. In fact I prefer it over Chrome's. At least with Firefox you can then publish the extension using your own servers, whereas with Chrome you must use their store, even though it will be hidden from the general public.
Thus, I don't really understand your statement:
> those with private extensions must resort to compiling their own browser
I certainly would never recommend such a thing, myself. If I were paranoid about 3rd party access to my extension to the point I felt I needed to compile by own browser, then one thing I'd certainly do is also disable addons and build mine right into the core product...
I'd certainly trust Mozilla far more than I can trust every addon out there that might coexist with my private addon, which is where most of the attack surface would come from (if such were my threat model).
Chrome allows installing unsigned extensions by enabling developer mode in chrome://extensions. On browser startup you'll see a warning [1] that encourages you to disable unsigned extensions, but users are offered a choice.
I do trust Mozilla, though I'd prefer if they'd also trust me to be capable of making an informed decision and give me a choice while using their product.
It's not an optimal choice, because Firefox Developer Edition is not based on the latest stable release of Firefox, and because users shouldn't be forced to switch to browsers with an alternative focus just to enjoy Firefox with a local extension.
Permanently installing unpacked or unsigned extensions is a feature that all major browsers offer for their users, except Firefox.
> I certainly would never recommend such a thing, myself. If I were paranoid about 3rd party access to my extension to the point I felt I needed to compile by own browser
That's exactly the point, you shouldn't need to compile your own browser to use a private extension!
Auto-update would be convenient indeed. That said, Developer Edition is really very stable, so while I see why you would not want that, it's not that bad a stop-gap measure until unbranded builds auto-update.
It's perplexing that Mozilla insists on forcing signature checks at any cost in release builds, when there is no consensus on the fundamental reason for doing so.
> Most fundamentally, the full Firefox team does not have a common understanding of the role, function, and operation of cryptographic signatures for Firefox add-ons. For instance, although there are several good reasons for signing add-ons (monitoring add-ons not hosted on AMO, blocklisting malicious add-ons, providing cryptographic assurance by chaining add-ons to the Mozilla root), there is no shared consensus on the fundamental rationale for doing so.
The thing I found somewhat baffling is that they didn't renew the certificate because they were certain that Firefox wasn't checking the certificate expiration. I'm wondering why they didn't renew the certificate and then implement a certificate expiration check.
I don't mean to be too harsh, but that part of the post-mortem reads like, "Oh, we don't need to renew our intermediate signing certificate because there's a known security vulnerability in Firefox that allows us to get around it."
No, it's not baffling. It's white-washing and blame-shifting.
"it turns out to be more complicated" = It's even worse than you probably imagined. We knew the certificate was going to expire and willingly let it happen because we don't know how our own product works. Also look at this new update mechanism and the other problems we caused! Forget about the fact that none of this makes any sense and please don't fire me for gross incompetence.
This was always a no-win situation for the people responsible. Allowing a certificate to expire has no reasonable justifications possible. It was always just going to be a question of if anyone would actually be held accountable for negligence or if they would excuse and blame-shift their way out of it.
I don't mean to be so negative about it. The silver lining could have been setting an example of doing a real investigation, finding that people were negligent, and holding those negligent people accountable, but I don't see that happening here because the people doing the investigating are the ones responsible.
The "it turns out to be more complicated" quote reminds me of Russ Roberts from Econ Talk. He has a number of interesting mannerisms that clearly signals his personal opinion on things. He doesn't come out and say "That's a terrible idea and I disagree!". Instead he whines "Well I'm not so sure." And when some study or evidence goes against his preconceived ideas or biases he says "Well, it's complicated".
But the communication is just the "why they thought it wouldn't be a big deal". The "why would you let a certificate expire that is actively used in code signing" is a different issue.
Looking at the Technical Report [1], I am confused...
First bullet under Root Causes:
"The server-side teams whose systems (e.g., Autograph) generate signatures knew the certificate was expiring, but did not see a problem because they had reason to believe that signing certificate dates were not checked by the client."
So, if I understand correctly, the assumed process was: Root signs Intermediate --> Intermediate signs Add-on (whether Intermediate is valid or not) --> FF only validates the End-entity is within date range.
Is this the correct procedure? You're telling me that the intended behavior is that FF allows add-ons signed by an expired cert? If that is the case, then what is the point of the signing system in the first place?
Additionally, third paragraph under Recommendations:
"Second, if we are committed to the current PKI and add-on signing approach, [...]"
'_If_ we are committed' is non-absolute. Is the FF team actually set on this approach?
> You're telling me that the intended behavior is that FF allows add-ons signed by an expired cert?
The leaf certificate is specific to a particular instance of an addon. Firefox has a separate, more flexible capability called the blocklist that can disable individual addons. Since the blocklist exists, enforcing expiration dates on the leaf certificates wouldn't help in any way, but it would require re-signing and re-distributing all addons periodically. So, yes it was a deliberate decision to ignore expiration dates on leaf certificates.
> If that is the case, then what is the point of the signing system in the first place?
"FF only validates the End-entity is within date range"
No, this was disabled back in 2016. The technical report briefly had this worded wrong, sorry if you read it before it was corrected.
"what is the point of the signing system in the first place?"
Like the technical report describes, things like "monitoring add-ons not hosted on AMO, blocklisting malicious add-ons, providing cryptographic assurance by chaining add-ons to the Mozilla root."
So, the intended behavior is that FF does not check any Add-on related cert for a valid date range? In that case, what prevents an insider threat of a Mozilla employee signing malicious add-ons with an out-of-date cert?
When you sign software with certificates that can expire, you run into the problem that software can stop working with no change on the user's end whatsoever. And of course, the shorter the expiry on code-signing certificates, the worse this problem becomes.
The solution for systems like Java and Authenticode is for Certificate Authorities to also offer a 'Trusted Timestamping' service, which certifies that the software existed at a time when the code-signing certificate was valid.
In Java's case, as the CA providing the timestamp is already trusted issue code-signing certificates with arbitrary dates, this doesn't add any new trusted parties.
> Now that we understand the importance of add-on support for mobile (ratings plummeted during the incident), we might want to reconsider that strategy/priority for Fenix.
Well that's promising at least. The listed Next Steps is still full of uncertainty and there isn't a firm decision (presumably the people involved in this post-mortem are not the relevant decision makers), but at least somebody is bringing it up.
Given the Mozilla track history, though, I'm not optimistic.
I wonder why there is no timestamping used on the binaries.
If I sign a binary with Authenticode on Windows and I use a timestamp server, Windows still considers the binary signature valid, even if the certificate expired because it can verify that at the time of signing the certificate was valid.
If you don't use a timestamp server when signing the binary, after certificate expiration the binary is considered invalid.
I'm very disappointed that although using Studies (Normandy) is briefly mentioned, it is treated as just an unusual option rather than a breach of policy. I was really hoping for an explanation of why they felt they could break their promise to users that Studies would only ever be used for, well, studies.
* All Shield Studies must be designed to answer a specific question - Shield is a tool for evaluating ideas and features in the product. If you are not trying to evaluate a proposed feature or idea there are other means of shipping your code.
Shield studies will always respect user privacy in accordance with our data collection policies
* All Shield studies will adhere to the scientific method for answering complex questions - Generate a hypothesis, test, collect and analyze data, validate or refute hypothesis, refine, repeat..
* All Shield studies require a PHD (Product Hypothesis Doc) - A PHD or Product Hypothesis Doc outlines the overarching research question the study is trying to answer. It requires the study author to think critically about the problem and the outcomes long before the study ships.
These guidelines (https://wiki.mozilla.org/Firefox/Shield/Shield_Studies#Guidi...) were created after the beach of trust represented by the Mr Robot cross-promotion as a means to restore confidence. Yet now they have been ignored, and again "Studies" has been used as a convenient backdoor.
Imagine you run a very popular website, and you have a policy that says "we will never modify any site data by accessing the database directly", to try to ensure that everything goes through site functionality and can be properly logged/etc.
One day, some weird data gets into the database somehow, the site's completely unable to handle it, and it crashes and goes totally down. You're faced with two options:
1. Rewrite a major section of the site code to be able to handle the bad data. This is expected to take at least two days to complete.
2. Break the policy, fix the data in the database, and have the site back up in 5 minutes.
Do you truly believe that Option #2 needs to be treated as completely out of the question?
Not nearly analogous. First, the policy was not about accountability, it was about privacy and specifically restoring a trust already broken.
Second, it would only have taken a worst case of 24 hours vs the 6 hours of using Studies to follow that policy. I think the integrity of a company that trades on that very thing is worth 18 hours.
The implied distinction is that adherence to principle is what distinguishes Mozilla from Google. If Mozilla is willing to sell out its principles on this instance, then when else will it be willing to sell out its principles?
> I was really hoping for an explanation of why they felt they could break their promise to users that Studies would only ever be used for, well, studies.
It's literally there:
> Because we had a lot of users who were affected, getting them fixed was a very high priority, which made Studies the best technical choice.
Which makes perfect sense to me. When you have a worldwide critical degradation of service, you fix that as soon as humanly possible, ideological pearl clutching be damned.
"Honoring the promise you depended upon to get users to install the feature in the first place" isn't ideological, or pearl-clutching.
Imagine your point in other contexts:
"If your company is about to go bankrupt unless it gets some serious revenue, then sure, you start selling the user data you promised not to, ideological pearl-clutching be damned."
"If a priest receives the confession of the killer in an unsolved mystery, he should report it to the police, religious pearl-clutching be damned."
And "critical degradation of service" is overstated; users always had the option to use another browser (like Mozilla seems to want these days). When they shut off everyone's unsigned, whitelisted addons in 2016, that wasn't regarded as a critical degradation.
> "If a priest receives the confession of the killer in an unsolved mystery, he should report it to the police, religious pearl-clutching be damned."
Confidentiality in such areas (also med/law) is usually restricted to past actions. If you confess you're going to kill someone, that does indeed have to be reported to the police. I doubt your local laws are different here.
Thus to construct a better analogy we need ongoing harm that was initiated in the past but can only be shortened by taking action. Maybe a confession like "Just had an argument with my kid and imprisoned it in the basement without food and water. Will release it after two days". Now should this have to be reported to CPS? I'm not sure, but think I would favor yes.
I mean, really? Mozilla's whole thing is integrity and principals. The Mozilla Manifesto is literally an ideology they probably don't want to be damned.
I'd have very much preferred them wait just 18 more hours and not have violated their privacy policies.
What exactly is Firefox's selling point if their principals can be traded for speed and convenience? That's just another tech company.
> What exactly is Firefox's selling point if their principals can be traded for speed and convenience? That's just another tech company.
If everyone cared as much as you do about this issue, then they'd have likely waited. The fact that they were jumping through hoops tells me that the users don't care, they want their crap fixed _right now_.
> The fact that they were jumping through hoops tells me that the users don't care, they want their crap fixed _right now_.
Surely what that tells you is that Mozilla thinks that users don't care, and want their crap fixed; in which case the policy might as well say "These policies will be applied unless their abrogation is necessary to fix your crap", or some appropriately dignified re-wording.
They didn't remotely enable telemetry/studies though, right? If you were fine with waiting, you could have just not enabled it.
I don't see how anything they did violated their privacy policies.
The policies that using Studies in this way violated are listed in my initial post. Those policies were created to prevent Studies from being used as a convenience.
You might think violating them was justified, but that they were in fact violated is inarguable.
> The policies that using Studies in this way violated are listed in my initial post.
You just claimed they had violated their privacy policy, not the shield studies guidelines. Your initial post doesn't say anything about their privacy policy. And as I said, your privacy wasn't affected at all if you didn't enable the studies yourself, so your point about rather waiting doesn't make sense either.
> You might think violating them was justified, but that they were in fact violated is inarguable.
You could argue that testing the fix meets the first two conditions you listed. I don't know if they actually provided that "Product Hypothesis Doc", but at that point you're really just looking for something to complain about.
There's no reason for local software to stop working when a remote server is unavailable, reconfigured, neglected, or your clock is wrong. They are doomed if they think this is sustainable.
It was a time bomb to disable functionality after a certain date and should never have been there in the first place. The decision to silently disable the override in about:config is just more evidence of how sick this it. This is not software empowering the user. This is software assuming the user is a moron who may break it.
> I'm stuck running an old version of Firefox because I have too much data in Tab Groups
Yikes! You're running a years-old version of Firefox without security updates? And you're browsing public websites?
I think there are some supported add-ons for tab groups that work with current Firefox. Please, please don't run an insecure browser on the public internet.
There are no working tab group addons post XUL, I've tried them all and they are all completely broken. While I agree that you shouldn't run without updates, please don't tell people that there is a working substitute for this functionality.
Before posting that comment, I searched not only for such addons but for the last updates to those addons and for reviews of those addons, which seemed to suggest that they work as Web Extensions and function as at least some people want. That said, I can't vouch for them personally, and I appreciate you providing some firsthand experience.
> Posts that assume everyone considers certain changes to Firefox "anti-features" aren't going to win everyone over, either.
This seems like a bad faith characterisation. The discussion was about ars's activities, and I think it is reasonable to assume silversconfused was guessing at ars's opinions on these changes, not at "everyone's" opinions.
the Firefox QA plan didn’t incorporate
testing for certificate expiration (or
generalized testing of how the browser
will behave at future dates) and
therefore the problem wasn’t detected
A few years ago I tried to find a VM or something similar that would let me test software with clock offsets applied. At the time, most of the VMs I looked at had prioritized the much more common requirement of not having the guest OS clock become inaccurate.
Does anyone know the state of the art in integration testing software for future dates?
This doesn't work with everything (for example doesn't work with Go because Go doesn't use libc for such things) but if it does then it's a very simple to use:
% LD_PRELOAD=src/libfaketime.so.1 FAKETIME='1999-12-31 23:59:59' date
Fri Dec 31 23:59:59 PST 1999
% LD_PRELOAD=src/libfaketime.so.1 FAKETIME='1999-12-31 23:59:59' python -c 'import time;print(time.ctime())'
Fri Dec 31 23:59:59 1999
or to have the fake clock ticking starting from the specified time:
% LD_PRELOAD=src/libfaketime.so.1 FAKETIME='@1999-12-31 23:59:59' python -c 'import time;print(time.ctime());time.sleep(1);print(time.ctime())'
Fri Dec 31 23:59:59 1999
Sat Jan 1 00:00:00 2000
It could be a feature that can only be enabled from the browser UI, with appropriate warnings, but the choice must be offered for those who need it.
Until that happens, those with private extensions must resort to compiling their own browser, use unbranded or developer builds, or patch Firefox at runtime to disable signature checks.
This is how you allow unsigned extensions in Firefox on Arch Linux, the same files can be edited on Windows and macOS, restart the browser after changes: