What is the correct action when a TRO says a company cannot revoke the cert? Is it that the company will delay revocation, but will push the judicial system to resolve the issue as fast as possible?
DigiCert probably should have revoked every cert they could within 24 hours. Instead they just pushed the revocation of all 80,000+ certs out to five days.
It's quite likely that many of their other clients pushed back on the 24-hour timeline (similar to what happened in their previous incident); I believe the delayed revocation issue (https://bugzilla.mozilla.org/show_bug.cgi?id=1910322) hints at this. The TRO gave them a convenient excuse to delay all revocations without having to explain all over again why they made exemptions for their special clients.
"Any issue with domain validation is considered a serious issue by CABF and requires immediate action. Failure to comply can result in a distrust of the Certificate Authority. As such, we must revoke all impacted certificates within 24 hours of discovery. No extensions or delays are permitted. We apologize if this causes a business disruption to you and are standing by to assist you with validating your domain and issuing replacement certificates immediately."
I think the eventual correct option is pretty clear.
A TRO like that is based on the company loudly declaring that revoking will cause real damage. That means their use of certificates is incompatible with the web PKI rules and ecosystem. That means they need to be migrated out ASAP, with every certificate authority refusing to take their business.
Make that series of consequences clear, and companies will stop trying that trick.
DigiCert and other CA's need to write in their terms that failure to follow the policies and revocation timelines can/will result in termination of the contract. Alegeus should have been dropped as a customer as soon as the incident was resolved and refused further products/renewals.
Use of a TRO protects you short term, but results in having to migrate to a new CA medium term. You can't stop them from using TRO's but you can make it not worth it.
Is the CA allowed to stipulate that the customer (before being dropped) needs to pay a significant sum for, say, "expenses" if they resort to this kind of TRO?
Possibly, I'm not sure if there is precedent for charging a company for taking valid legal actions, only vexatious in contract law. Probably depends on the legal jurisdiction and courts.
I think that's pretty unrealistic, considering that X.509 is a de-facto and de-jure standard in a lot of places that also ignore that requirement. It's not always up to that company to make it possible to replace certificates easily, it's an entire chain of vendors (I know at least Salesforce's process would fail here). Unless you want those people to run self-signed/private CA certs.
The other option would be building in a way to revoke other CA's individual certificates if there's some consensus on them being compelled to not revoke them. Not sure if the status quo or this would be more dangerous, but if a TRO can compel a CA to not sign a revocation, can it also compel to sign a certificate?
A company chooses its vendors. If its vendors turn out to be incompetent, they should choose again once the damage has been done. Just because Salesforce can't get their stuff together doesn't mean the CA industry needs to bend the knee. Let Salesforce figure out how to automate certificates, they've been in the business for long enough.
If running a private CA or self-signed certificates are even viable, then there are plenty of other workarounds that can be put in place (i.e. not updating the CRL so the software doesn't know about revocations).
If a CA's terms and legal documentation aren't tight enough to prevent a court from compel them to sign a certificate, that CA clearly cannot be trusted. Hopefully that issue can be fixed by writing clearer terms and better agreements, like people have been telling Digicert to do, but perhaps that's not possible at all. In that case, either the company should move to a jurisdiction where that kind of nonsense isn't possible, or it should be removed from global trust stores all together.
> A company chooses its vendors. If its vendors turn out to be incompetent, they should choose again once the damage has been done.
You say that and yet Teams has 320 million people using it, and I bet almost none of them enjoy the experience. Sometimes you just have to work with what you're given. Given "incompetence", we might as well throw in all of Azure.
> Let Salesforce figure out how to automate certificates, they've been in the business for long enough.
That might be true for DV, but there's classes of certificates that take at least weeks to obtain, think BIMI VMC or codesigning.
> You say that and yet Teams has 320 million people using it
Yeah, there are tons of companies who chose Teams as a vendor (because it's bundled with other stuff), and inflict it on their employees. It was still absolutely their choice.
The parent was correct - it's not about the company not using x509 certificates, but not using publicly trusted certificates. There are myriad private/internal PKI solutions available from OpenSSL & bash to millions of dollars of solutions from Big Vendor.
If you can't replace the publicly-trusted certificate quickly, you probably don't need it to be publicly-trusted in the first place.
The better question is, how do you prevent this tactic from working?
For example, suppose there were required to be multiple parties who could issue a revocation, each in a different jurisdiction, and if any of them was ordered not to do it then the others would be required to do it, and would have the technical capacity to do it but not be subject to the jurisdiction of that court.
Well, you can contest the motion for a TRO, for a start. Digicert failed to do so.
You can stick with your policies and revoke the certificate within 24 hours, instead of delaying revocation until a case is open and a motion for a TRO is filed. Digicert failed to do so.
You can stick with your policies and revoke the cert in face of the legal consequences, and deal with them accordingly. Again, Digicert failed to do so.
Correction, the petition for the TRO was filed ex parte. Digicert did not have any opportunity to respond before it was granted.
They certainly could have filed a response contesting the TRO. Then their customer could have filed another motion, and eventually (7 days later in this case) the judge would have ruled on the substance of it. Their judgement was that it would be preferable to work with the customer to resolve the technical issues with revocation, and submit a joint request to dismiss the TRO. The stated reasoning behind this was that it would be significantly faster than contesting the TRO. This is true: the certs were revoked and the TRO dropped within 3 days.
I think the communication on that point was severely lacking, as they only clarified it three months later and after significant hectoring in two different bug threads: https://bugzilla.mozilla.org/show_bug.cgi?id=1910805#c43
I also think it's reasonable not to take Digicert's statements at face value, given their history. But I think both of the points you made here are wrong:
> You can stick with your policies and revoke the certificate within 24 hours, instead of delaying revocation until a case is open and a motion for a TRO is filed. Digicert failed to do so.
Let's be clear about the timeline: Digicert notified their customers that the certs would be revoked. In between the time they notified the customer and the time of revocation (less than 24 hours), the customer got the ex parte restraining order. Are you suggesting that issuers should revoke certificates without notifying their users, so that the users don't have time to get an emergency TRO? I believe that would be in violation of the BRs.
> You can stick with your policies and revoke the cert in face of the legal consequences, and deal with them accordingly. Again, Digicert failed to do so.
By "revoke the cert in face of the legal consequences" do you mean "openly defy a valid and legal court order"? Because that would also violate the BRs.
Just to be clear, the whole incident covered over 80,000 certificates.
The TRO was applicable to only those of one subscriber - just over 70 certificates, yet caused the revocations of all 80k+ to be delayed.
To add to this, 3 days after the TRO was filed both parties moved to vacate the TRO.
DOCKET TEXT ORDER. 9 Joint Motion to Vacate 3 Order Granting Ex Parte Motion for TRO is GRANTED
I'm not sure DigiCert could have done anything about the TRO or the impacted certs, but it should have been able to move forward with the revocation of all other certificates. That IMO is the real issue/failure, alongside the concern/impact of TRO's on security processes in the future.
> By "revoke the cert in face of the legal consequences" do you mean "openly defy a valid and legal court order"? Because that would also violate the BRs.
Yes, I think this would have been appropriate action. If the contractual language is extremely clear between the CA and the subscriber, there is no legal basis on which the customer can prevent revocation. The fact they found a court that doesn't understand technology is frankly irrelevant. This detail is exactly why Tim and other parties are requesting the exact language of the agreement between Digicert and the subscriber that filed the TRO. A customer acting in bad faith and abusing the legal system does not compel you to violate your own contract terms, your terms under the CAB/BR, or to take actions which are detrimental to the entire Internet. This is exactly the type of circumstance where you do what you are required to do, and then sort it out afterwards. Any appeals court would have easily overturned the TRO as it has no legal basis.
> A customer acting in bad faith and abusing the legal system does not compel you to violate your own contract terms, your terms under the CAB/BR
Yes, it absolutely does. "I think the court will agree with my view of what the contract says once the case is heard in full" is not a valid reason to disregard a TRO.
> or to take actions which are detrimental to the entire Internet
That would be harder. But a delayed revocation stemming from a flawed validation process, when the CA is responsible for the flaw and knows that the result of the validation was in fact correct, simply does not cause any detrimental effects to the entire Internet.
You could just require publishing the revocation immediately with an effective date in the future.
Of course, if that system had been in place, DigiCert would probably be facing hundreds of lawsuits from businesses disrupted through no fault of their own rather than inside baseball PKI drama.
> The better question is, how do you prevent this tactic from working?
Make it clear that if it works, it will only work once.
The CABF should adopt policies that any such legal action or any request for extension will be considered a public declaration that the customer's application is incompatible with the requirements of the Web PKI and that not only will the current CA refuse to renew the certificate but it will be publicly documented in the Bugzilla so that no other CA will issue certificates covering any of those names nor any new names for the same company without an affidavit explicitly stating that the issues preventing compliance have been resolved and the company acknowledges this and commits to never doing so again.
Existing names that were successfully revoked in time can be renewed but neither the problematic one(s) nor any new ones will be allowed.
If they then file for another TRO in the future they may still get a short-lived order but the existence of such an agreement would at least to my non-lawyer brain cause any judge who may have granted a TRO to become very displeased when the CA presented it in their response.
It’s Saturday and the courts are closed. You park in a car park I own.
I have put up a sign saying I can instantly crush any crossover vehicles I like, as I consider them ugly and lacking in character.
As I load your car into my crusher, you dispute the legality of my sign. But the court is closed until Monday, and the sign says I can crush your car instantly, no waiting.
Should I be allowed to crush your car today? Or should I have to wait until Monday, so the disputed legality can be resolved?
That’s actually a question for you. You won’t truly know if you’re allowed to until a court decides. You can choose to proceed and crush it, and then deal with the consequences of doing so if it turns out you were wrong. Likely you’ll end up owing damages to the owner of the car, perhaps even punitive damages on top.
The TRO actually means you aren't allowed, that's the point. It's an ordered injunction that legally obliges you from not acting until the courts can review the facts.
In this case, yes. That’s pretty cut and dry for the time being. However regarding the analogy I was replying to, I was pointing out that it’s less a situation of what you’re allowed to do and more one of what you believe you’re allowed to do, and weighing the consequences of being wrong against upholding your stated terms. In other words, something you should probably discuss with a lawyer.
> How can a court inhibit revocation when every CA declares their right to do so when you purchase a certificate?
A court can rule that a term of a contract is void because it contradicts public policy, and it certainly can issue a TRO pausing an action which would otherwise be allowed by a contract while resolving a dispute related to it.
Because the judge doesn't know the details of how PKI works, and either the ToS for digicert doesn't spell out that they can revoke certs at any time (which would be problematic for a CA), or the judge didn't read, or didn't understand the ToS.
The order would normally be a matter of public record, and the ecosystem should follow these events carefully and ensure never to issue a certificate to such a company again as it undermines the entire system.
