Part of me was hoping to see some sign of hacktivisim, not for any particular political stance, but because most of us probably agree the federal government has been slacking off as far as security goes. An eye-opening event might be beneficial.
"it appeared the domain controller credentials had expired on Monday when offices were closed for the federal Presidents Day holiday"
Oh, well that's not nearly as exciting. Still a dismal sign of the state of "cyber security" though.
One would think in 2017 there would be some kind of solution that allows to track all certs in the org and raise alert well before the cert actually expired (of course, nothing can prevent admins from ignoring the alerts). But given how many cert screwups happen around, looks like it's not happening. Maybe somebody should create a startup doing this? ;)
Maybe you can clear up something for me: I am in CC for an email discussion between 2 parties exchanging sensitive personal information.
Party A uses LetsEncrypt to protect the https site storing the information (And needs a username/password or something to grant access to Party B)
Party B does certificate pinning, and complains that Party A's cert changes every three months without warning and without being sure the new cert still points to Party A.
Party A says Party B should stop the pinning and just accept everything signed by LetsEncrypt.
Party B says they want a way to be 100% sure they are talking with Party A and need a better (EV?) cert.
The discussion is going back and forth for a few months now (and re-ignites every three months when communication goes down again and pissing everyone off). Who do you think is right here?
Pinning is based on the key fingerprint, though, so reissued certs shouldn't matter as long as the private key used for the CSR is the same. I use HPKP with Let's Encrypt certs and haven't had any issues.
I'm pretty sure when you configure certificate pinning you can also specify an expiry date. Party A would just adjust the expiry time to match their cert expiry and both parties are happy.
Sounds like there should be a setup that works something like this:
Some time before expiry, A gets a new cert and advertises that cert, while still using the old cert to secure the connection. When A makes the switch, B already has the new cert, obtained over a secured connection.
I'm not sure how the "hey, I'll switch to this cert soon" bit should be done, though.
It's enough to just sign the new public key with the old secret key, and send that at some place in the new certificate.
(It's also kind of worthless. Since people normally change their secret because they have some reason to believe it's compromised, whoever has the old key can also create the same certificate. In other words, it's only useful on the use cases where pinning already fails...)
Party A currently uses certificate X, which Party B trusts. Party A then creates a new certificate Y. Party A signs certificate Y with certificate X and Party B can then be sure that it is legit.
A similar thing happens sometimes when new GPG keys are generated. Google for "GPG transition statement" for how that works (it's effectively the same process).
From my understanding, LE does not revoke previously generated certificates when a new certificate is generated.
So you could have a workflow like:
1.) Party A generates the initial certificate and passes it offline securely to Party B
2.) Party B stores Party A's certificate in their java trust store
3.) Party A creates an HTTPS endpoint where they host a copy of their current certificate and their certificate for the next time period: https://www.PartyA.com/nextcert.crt
4.) Party B upon connecting for the first time validates that the certificate on the HTTPS site is the one provided in step 1
5.) Party B retrieves nextcert.crt from the URI in step 3 and places it in their java trust store
6.) 30 days before expiration Party A generates a new LE certificate and places it at the URI in step 3
7.) 7 days before expiration Party A sets their certificate to be the one at /nextcert.crt
Day 1 - Party A generates Cert1 and provides it to Party B, secures the domain with it (https://www.PartyA.com/) and places it at /nextcert.crt
Day 2 - Party B retrieves /nextcert.crt (it's still Cert1)
...
Day 60 - Party A generates Cert2, and places it at /nextcert.crt
Day 61 - Party B retrieves /nextcert.crt (it's Cert2), and places it in their store. Now they have two valid certs for Party A (Cert1 and Cert2).
...
Day 83 - Party A secures their domain (https://www.PartyA.com/) with Cert2 (/nextcert.crt has already been pointing to Cert2 since Day 60 and continues to do so)
Day 90 - Cert1 expires. Party B only has 1 valid cert in their store now.
...
Day 120 - Party A generates Cert3, and places it at /nextcert.crt
Day 122 - Party B retrieves /nextcert.crt (it's Cert3), and places it in their store. Now they have two valid certs for Party A (Cert2 and Cert3)
...
Day 143 - Party A secures their domain (https://www.PartyA.com/) with Cert3 (/nextcert.crt has already been pointing to Cert3 since Day 120 and continues to do so)
Day 150 - Cert 2 expires. Party B only has 1 valid cert in their store now.
Then just continue this process in a loop. This allows for the two properties requested. Party B is able to validate Party A (because an initial cert was provided offline), and that the next cert is actually from Party A (as the transfer of the new certificate is done via a secure channel using existing certificates)
1. Certificate pinning is not a solution to the problem. Certificates change without warning anyhow, because they may be compromised and revoked at any time.
2. Let's Encrypt certificates are indeed the lowest grade of SSL certificate. That's not a criticism of Let's Encrypt, it's just the nature of the situation. There are legitimate reasons to consider it too low a level of assurance. I have no idea whether your parties have that level of need or not. But I can point out that if you want a better grade of cert than Let's Encrypt, you do need to go for the fairly expensive ones that require a lot of real, human validation. There's not much point in just getting another cheap-but-not-free cert that has roughly the same level of validation.
3. There's another solution to this problem entirely, if you're communicating between two parties and that's all this cert is doing (i.e., you're not speaking to the general public with this cert). Make your own certificate authority, sign your own certs, have the public key of that CA inserted into the Java cert authority.
Creating your own certificate authority is very simple. To give you a sense of how simple, here's some scripts I bashed together to set up nodes that speak to each other over SSL: Set up the CA [1] Build a new cert signed by that CA for the node [2].
I am NOT saying you can just use those or that you should even crib from them. These are generally intended for quick reference and testing only even in the context of this project ("real" users are expected to do their own cert management with real certs), and the needs of this project are significantly different from an HTTPS website. My intent in linking those is merely to demonstrate the rough complexity of setting up a CA is only a few lines in bash, not a monstrous impossibly-large undertaking, and you can Google this stuff up in a day or two, tops. The hard part is all around how careful you are with the private keys and revocation, and the fact you must set up and implement your own security policies around the certs. (My suggested revocation policy would be "we'll call you", and that if a cert is ever compromised, just re-roll the whole process from start to finish and create a new CA. "Real" revocation is somewhat complicated, and somewhat unsatisfactory anyhow. Take advantage of the smallness of the problem and the pre-existing personal connections. A lot of the at-scale SSL issues arise because we don't have those.)
I take slight issue with "Let's Encrypt certificates are indeed the lowest grade of SSL certificates".
Other issuers perform domain validation in the same ways (specific content at a specific URL, DNS records, e-mail validation, etc.) as LetsEncrypt. LE just makes it easier.
And, from a technical perspective, certificate X from issuer A is no different than certificate Y from issuer B (assuming both A and B are trusted root CAs, same key length, etc.).
If your issue is with DV vs. EV, yes, I agree that LE certs aren't on the same level as EV certs, but that's not the problem LE is trying to solve.
The existence of other vendors of the lowest grade of certificate does not mean that LE doesn't produce the lowest grade of certificate.
As I said, it is not a criticism of LE. It is simply what they are. It doesn't matter how we "feel" about that. It's not like the LE project set out to produce the highest grade of cert but failed; they accomplished exactly what they intended, and it is good.
Except DHS's issues relate to certificates in a Windows environment, where Let's Encrypt does not have very good support. On top of that, I'd imagine their Active Directory domain controllers are not in the public DNS and they don't want them to be.
in a Windows environment, Let's Encrypt does not have very good support.
There are very good ACME clients (ACMESharp, etc) for windows that allow doing http-01 and dns-01 certificate issuance. Additionally, there is IIS integration (letsencrypt-win-simple) with the stuff you would expect (cert generation for all or specific sites, automatic renewal, etc).
Basically the only thing Windows doesn't have is official support in the Certbot client.
Assuming Microsoft Windows (which is safe, I think, since "domain controller"), you can easily set up automatic renewal of certificates. I assume they probably have a Windows box running (Enterprise) Certificate Services handling all the "Microsoft PKI" stuff that chains up (eventually) to the DoD roots, but automatic renewal is likely disabled for security reasons -- that security versus convenience trade-off coming into play yet again.
We're actually more focused on issuing certs to authenticate SSH and RDP clients, similar to what Google, Facebook, etc do. We keep track of the host keys for those services, and use them for key pinning, but we don't look for other X.509 certs on servers where our agent is installed.
Are they slacking off, or are they simply unable to hire competent people, or perhaps is the bureaucracy such that competent people are unable to do their jobs effectively?
After contracting in the DoD space for about 6 years I'd say it's, for sure, all 3 plus a mixture of contractors who only make money if they fill all the seats on a contract (so even if you're useless, slowing down the process as long as you're a butt in the seat you're making them money).
They're using Windows. One of their Systems Administrators forgot to update the CRLs for the Domain Controllers, thereby locking out anyone from logging in with their smart card.
This is not really a new thing and has happened dozens of times over the past decade and a half to various government agencies.
"it appeared the domain controller credentials had expired on Monday when offices were closed for the federal Presidents Day holiday"
Oh, well that's not nearly as exciting. Still a dismal sign of the state of "cyber security" though.