Hacker News new | past | comments | ask | show | jobs | submit login
Final Removal of Trust in WoSign and StartCom Certificates (groups.google.com)
162 points by selectnull on July 7, 2017 | hide | past | favorite | 76 comments



Speaking of removing trust bits; the ever-delayed Symantec saga continues with no clear decision.

What concerns me is:

- the lack of public communication, as shown here (tl;dr: private meetings): https://groups.google.com/forum/#!topic/mozilla.dev.security...

- the contrast between symantec and the mozilla security groups path forward [as well as how its communicated] (https://groups.google.com/d/msg/mozilla.dev.security.policy/..., https://www.symantec.com/connect/blogs/symantec-s-response-g...)


I don't think you're being fair toward Mozilla. They had a meeting with Symantec and reported on it; there were no details because Symantec is going to soon publish everything. They probably had the meeting only because Gerv was in the area. Mozilla have been transparent the whole time.

Google's approach is clearly different to that of Mozilla, but IIRC they have never pledged transparency. They're a commercial entity and they're doing what they think is right and/or is in their interests.


I can understand how my comment might have come off as somewhat judgmental (to either side) but that was not my intention. The idea was to [without derailing the thread too much] give people interested in certificate issues a relatively quick summary on what has happened and my concern about it.


I wonder if "senior executives" at Google are really involved.


There is discussion on the Chromium blink-dev list: https://groups.google.com/a/chromium.org/d/topic/blink-dev/e...


And meanwhile, Let's Encrypt is on track to offer wildcards at the start of next year (https://letsencrypt.org/2017/07/06/wildcard-certificates-com...), so one of the last reasons to need a non-LE cert is going away.

I used StartCom for years, but LE made it completely obsolete for me. And StartCom seems to have been willing to throw it all away with the WoSign acquisition.


Lets Encrypt seems like a good solution for the https cert side of things.

Wish there was an equivalent thing for signing executables. eg for the binaries of our software releases.

We'd been using StartCom signing certs (for Win binaries) up until this clusterfk. ;)


If you are looking for an alternative I've been using KSoftware for years, they are cheap and good.


When I looked into this originally, before choosing StartCom, the KSoftware process was really non-optimal.

Just to get a signing certificate I'd need to get my personal id docs signed by a public notary, get those docs to the KSoftware people, and THEN they'd be ok to issue the cert.

Whilst not impossible, that's like the process was back in the Stone Age ;) for getting normal https website certs.

Obviously went with the StartCom approach instead, which may need to be looked at again if we need to sign packages in the near future. :/


I second this recommendation. I've been using them for years too.


CERTUM provides AuthentiCode-compliant certificates from about 30 euros (for open-source usage).


Interesting. I looked at the Certum website a while ago, before choosing StartCom.

The page on the Certum website about certs-for-open-source-signing didn't mention AuthentiCode at all. The AuthentiCode compliant certs seem like a different product.

If their certs for Open Source projects really are Authenticode compliant, I'd expect that to be front and center on the page about it. It's the one main feature of importance. :)


Seriously, this has been a problem for the longest time. At one point you could get free ones for open source projects I think, from comodo perhaps? But no longer the case, which is tremendously frustrating.


StartCom is even part of the KMCS cross-cert list.


There's always GPG signing.


Does that provide the same integration into Windows that Authenticode does? Plus timestamp, dualsign, and cross-sign?


>Does that provide the same integration into Windows that Authenticode does?

Nope


Doesn't solve the Windows scary warnings issue.


Technically correct, but it solves a different problem. The StartCom signing cert is a solution solving the "big scary warning" problem on Windows for untrusted binaries.

GPG signing is suitable for a different group of users. :)


There are still a few other reasons of using non-LE certs: extended validation TLS certs (the ones with greenbar + company/entity name), SAN TLS certs, etc.


For me the last big one is longer-lasting certificates. I'd like a three-year cert like letsencrypt.org uses for their site, instead of the 90-day ones they offer.

I much prefer the CA industry practice of, put a meta tag on your frontpage, or add a string to a DNS TXT record, and then download a certificate, then you're done for three years. I understand their security claims (which apparently don't apply to letsencrypt.org), but all CAs offer 2-3 year certs, so it's a feature they have that LE lacks.

certbot raises complexities if you're using a managed hosting solution that doesn't have shell access, or the owner of your machine doesn't want you running code on their box, or you have a weird custom web server setup or something.

But all the same, I'm not sure that avoiding certbot is worth continuing to pay $40 a year for my wildcard certificate from AlphaSSL. I may switch when mine expires.

What I'm even more hopeful for is that prices of wildcard certs from other vendors will drop now that there's a free alternative. I'm super excited that LE are finally offering these for free!


I don't recommend using certbot directly; I use and recommend acme-tiny (https://github.com/diafygi/acme-tiny), which is quite simple, and doesn't require as many permissions. I keep it isolated under its own user, run it to renew a certificate, and don't give it access to my TLS private key (just my LE account key and a directory mapped to .well-known/acme-challenge in my web server).

If you have a host that doesn't have shell access, it wouldn't be that hard to modify a script like that to copy files up to .well-known/acme-challenge, and copy the TLS key in place when done.


> I'd like a three-year cert like letsencrypt.org uses for their site, instead of the 90-day ones they offer.

Why? The whole point of LE is to be able to automate renewals and if you automate it then the length of validity makes zero difference.


Then why doesn't letsencrypt.org eat their own dogfood, rather than have a 3 year certificate?


At a guess, I'd say the reason is because Let's Encrypt (the service) didn't exist when they got their certificate.

From Wikipedia:

> Let's Encrypt is a certificate authority that launched on April 12, 2016

From Let's Encrypt's certificate

> Period of Validity > Begins On: 2015/2/4

I agree they should eat their own dogfood, but in the meantime, I'm eating their dogfood and it tastes pretty good.


Just in case you are not aware, Let's Encrypt does support the DNS TXT method as well.

I use the acme.sh client (https://github.com/Neilpang/acme.sh) for updating certs for a site on a managed platform, using the DNS verification method.


+1 on this. An until nginx gets LE baked into it, building docker images is kind of tricky since there is a circular dependency. I'm actually willing to pay letsencrypt 60$ for a wildcard 3 year certificate.


There is a Lua module which will do LetsEncrypt automatically for any domain in an nginx instance.


>For me the last big one is longer-lasting certificates. I'd like a three-year cert like

Long term I would be concerned that browsers, such as Chrome, may stop supporting certs that last too long. We are already seeing this behavior with Symantec after some security issues with issuing certs. I expect in the long run certs that last over a year will be done away with.


I agree that free LE certs disrupt the TLS certificate market pricing and a lot of CAs are probably going to be out of business for that matter.


I think the market for TLS CA will segment into basically two things:

a) free letsencrypt certificates for everything that only needs to be domain-validated (DV SSL).

b) $95/year EV certificates for companies that want the big friendly green banner on their ecommerce/credit card checkout pages, which is reassuring to the average non techincal user. In the last 4 years I have seen the price for EV SSL certs fall from $400/year to $95/year from several vendors. My main concern is that EV SSL issuers need to be held to a very high standard of actually verifying corporate identity. If they start getting lax about it and known EV certs go out to not-fully-vetted organizations, that would be a bad thing for the whole idea behind EV.


And this is a great thing.


Actually, it's not necessarily a great thing. Issuing certificates cost money and has to be paid one way or another. Having one very dominant CA -- even if it's free -- is not healthy long term. We need competition in this space.


> Issuing certificates cost money and has to be paid one way or another.

It's not clear that the prices being charged by commercial CAs are anywhere close or even related to the underlying costs.

I think LE is beginning to demonstrate that the costs of operating a CA are not actually that high, at the same time that the lax behavior by certain commercial CAs has demonstrated that the "validation" they provide is of very little value.


Most of the cost tied up in being a CA is being audited and reaudited by a third party that charges hundreds of thousands of dollars (and given the Symantec/Startcom shenanigans, is completely worthless).

The actual technical infrastructure required, while intricate, isn't that hard to set up.


I believe that it would be better if all CAs disappeared and a new system such as public-keys-as-urls/hashes-as-urls (like tor/i2p/gnunet/etc) or a web-of-trust based solution was adopted.

> Issuing certificates cost money

Making a post on HN probably costs more money than issuing certificates.


Even if you're not using a proper PKI, you still have to trust someone at the end of the day to not be lying to you. That's ostensibly the point behind CAs and auditing and trust roots.

LE's probably the best we're going to get for a long time. Mozilla's one of the few groups I'd go so far as to say is completely benevolent, and LE is being operated as a public service.


The cost of issuing domain-validated certificates is negligible - there's no reason they couldn't be done for free and paid through advertising or other long-tail revenue sources (or in LetsEncrypt's case: donations and endowment).


If you want to avoid certbot you can just self-sign.


I assume letsencrypt don't use their own certs to avoid a chicken and egg problem if there's ever an issue.

e.g. a software as a service database provider most likely does not use their own software for their database.


Let's Encrypt has supported SAN certificates with up to 100 hostnames ever since they launched.


You are right on the SAN cert, my bad.


I'm still really bummed about StartCom's suicide/implosion. They had by far the most sane and reasonable pricing model I've ever seen, and one I really wish someone else would do. They basically only charged for non-automatic authentication, not for what you did with that authentication, and then let each level of authentication feed into higher levels. So Level 1 machine auth (similar in spirit to LE) was entirely free, you could verify an email, domain, or whatever and then produce certs for that. If you wished you could pay money (~$60 for 2 years IIRC) to have an actual human look at your documentation and give you a call, then give you a Level 2 with your legal name. You could then use that for an unlimited number of S/MIME certs for email accounts, domains, software signing and so forth. Having gotten personal verification to level 2, you could then pay to have an organization certified. Etc.

Their UI wasn't the best, but it was functional and the overall system made a lot of sense. They didn't put an artificial price on some bits with a marginal cost of nada. It's a real shame all that got flushed, because that model of business filled a hole that has no general world solution right now (in principle authentication of their citizens should be a core role and offering of modern government, but I'm not holding my breath there).


You're forgetting that they had a charge for revocation, which was also automated. Even after Heartbleed, they refused to revoke the potentially compromised certificates without being paid. They also advertised their certificates as 100% free, even though they clearly weren't (because of the revocation charges). The certificates were not free for commercial purposes either.

IMO, just like one of those schemes where you don't pay to get in, but you have to pay to get out.

It's a shame, because they could have achieved a market share similar to Let's Encrypt, if only they had played their cards slightly differently.


>You're forgetting that they had a charge for revocation, which was also automated.

I did not forget that, I was just talking about the authentication and certificate issuance side. But though it didn't fall under the umbrella of authentication it is something that actually cost real money at that point. You misunderstood the situation and how revocation functioned in a way that's extremely important (since it's finally getting fixed but the improvements could use more widespread deployment). The original certificate revocation mechanisms, CRL and OCSP, are and always have been completely broken in terms of fundamental scaling and failure modes. Scott Helme just did an excellent summary on the situation and upcoming developments [1] which I strongly encourage you to read, but in essence due to the way they work seriously offering CRL/OCSP (as in, in a way that won't instantly die when someone breaths on it hard) services and adding certs to them requires significant distributed infrastructure, which has not always been cheap. They were bad approaches even with that though due to other practical failure modes that were beyond a CA's control anyway. Both stink to the point that many (most? all?) operating systems/applications/browsers just soft fail by default or even ignore them entirely. "Revocation" for the wider web has largely been a mirage.

StartCom did in fact have an explainer about this at one point, and their position was reasonable. Other CAs hid the cost in large part by charging a ridiculous amount for every cert issuance, despite many people never actually needing revocation services. StartCom let you have as many certs as you wanted after authentication for free, but if you lost one in such a way that you needed to make use of CRL/OCSP then you were charged. Users could pay that price, or alternatively invest in other ways to mitigate at their discretion rather then just getting charged no matter what.

If StartCom had survived until now, or if someone else adopted their model, then I'd expect at this point they'd make use of modern techniques like OCSP Must-Staple/Expect-Staple, CAA, CT, etc. by default, and in turn that problem would be solved at no cost.

>IMO, just like one of those schemes where you don't pay to get in, but you have to pay to get out.

This is completely ridiculous. Nothing inherently made anyone lose private keys. Even Heartbleed didn't defeat someone using an HSM for example. Nor would everyone who lost a private key need try to publicly revoke (to the extent that would even work). Given the cost tradeoffs this was one particular standards design failure that has been and remains wrong to make into some general insurance vs letting individuals have the freedom to deal with it as needed and/or having the standard itself fixed.

----

1: https://scotthelme.co.uk/revocation-is-broken/


Whether revocation works has nothing to with my point about StartCom, which is that they weren't honest about what was free and what wasn't. I know this because I was annoyed with it at the time and actively looked at how they presented themselves.

On the topic of revocation, you're missing a couple of aspects:

- First, both end users and CAs are required to revoke certificates that are compromised; it's not an option. It's a fundamental part of a CA's operation.

- Second, CAs provide revocation services like OCSP for revoked and non-revoked certificates alike. So, certainly the costs for running OCSP are comparable in both cases. You could argue that CRLs cost a bit more, but largely only because of the bandwidth cost (as CRL is just a file).


>which is that they weren't honest about what was free and what wasn't.

You have repeatedly stated this, and it is objectively false. The StartSSL site did not forbid archiving, and thus is available [1] going back to 2004 on the Wayback Machine. This includes both the HTML FAQ page and the policy PDFs of the time. This isn't some EULA boilerplate, actual clear policies and procedures for a CA you're interested in using for your infrastructure is something that is absolutely your responsibility to read.

When discussing Revocation (clearly titled) in the FAQ and Policy & Practice statements, even very old versions stated that a fee could be charged, and that revocation was considered a separate issue for free usage due to growth of the CRL. Later (since at least mid-2010, which is 2 years before Heartbleed was even introduced and 4 years before public disclosure) made this even more explicit, put it right in the FAQ, and put a clear price on it. From the FAQ, 2010 [2], the first sentence:

>>"72) Revocations carry a handling fee of currently US$ 24.90. [...] Extended Validation certificates are exempt from the revocation handling fee."

Please explain how they "weren't honest about what was free and what wasn't." Did you "actively look at how they presented themselves" but fail to so much as glance at the FAQ or Policies documents in doing so? None of this was remotely hidden, it was literally listed twice on the front page, both in the major navigation on the top and as one of just 10 sidebar items.

>First, both end users and CAs are required to revoke certificates that are compromised

Please back up your arguments on this. WRT to end users, under what authority at all are they "required" to do jack shit? Failing to do so in SOME industries and jurisdictions MIGHT involve negligence, but even that would depend on what other measures are taken. Please link any specific legislative or regulatory authority requirements to the contrary.

As far as CAs themselves, the authority would be the CA/B (enforced by virtue of browser/OS inclusion requirements, so the real ultimate "authority" there would be Apple/Google/Microsoft/Mozilla/OSS major distros & projects). At the time when StartCom established its initial policy, v1 of the CA/B baseline requirements [3] had not even been adopted yet (BR v1 adoption was Nov 2011, effective date was July 2012). Even then, 13.1.1 merely stated that:

>>"13.1.1 REVOCATION REQUEST: The CA SHALL provide a process for Subscribers to request revocation of their own Certificates. The process MUST be described in the CA’s Certificate Policy or Certification Practice Statement. The CA SHALL maintain a continuous 24x7 ability to accept and respond to revocation requests and related inquiries."

There were reporting requirements, and a 13.1.5 covers circumstances under which a CA must revoke within 24 hours. The most recent version, BR 1.4.5 (1.4.6 & 1.4.7 are in review), moves revocation to section 4.9 and includes additional details and sections (including, note, that there is no maximum latency on CRLs stipulated so a dishonest CA could make them "free" but run it off a toaster that takes 20 minutes per request).

However in none of this is it required that the CA make the process available for arbitrary usage at no cost. If a subscriber refuses to pay and the CA discovers a cert falls under one or more SHALL REVOKE parts of "Reasons for Revoking a Subscriber Certificate", then they have to revoke anyway and the cost may be unrecoverable, but they are free to treat it as a debt and go after the purchaser on their own for that money (or ban them from further use of their services). StartCom had a written laid out policy and procedure, and it was available continuously on their site. What requirement did they fail at?

>Second, CAs provide revocation services like OCSP for revoked and non-revoked certificates alike. So, certainly the costs for running OCSP are comparable in both cases. You could argue that CRLs cost a bit more, but largely only because of the bandwidth cost (as CRL is just a file).

Note again (and I pointed at this in my original reply): even now, BR contains no requirements on specific availability or latency beyond the original requirement (13.2.3 v1.0, 4.10.2 v1.4.5) that:

>>"The CA SHALL operate and maintain its CRL and OCSP capability with resources sufficient to provide a response time of ten seconds or less under normal operating conditions."

Ten seconds! Under normal operating conditions (DDOS survivability, ha!). Meeting requirements on the cheap is not the same as doing it right (to the extent that's even possible). And you are far to blasé about the growth of CRL in a situation where an unlimited number of 1 or 2 year length certs can be issued free of charge. One of the most principle rules of service scaling is that if you make something available that has a minor cost available at no cost, there is always a risk that users will fully take advantage of that.

One of the best parts about the new BRs is that they've started the process of allowing CAs to replace base OCSP requirements with use of Stapling (although oddly only for "high traffic" sites, whatever that means). That should be generalized/accelerated and both CRLs and base OCSP done away with entirely ASAP. But even given previous and existing rules I find your assertions that StartCom was dishonest or failed to meet requirements unconvincing right now.

----

1: http://web.archive.org/web/*/www.startssl.com

2: http://web.archive.org/web/20100704102024/http://www.startss...

3: https://cabforum.org/baseline-requirements-documents/


I've already explained in one of my earlier posts in this very thread: In my opinion, it was dishonest to advertise certificates as "100% free" when they charged for revocation.

Just because the revocation charges were documented somewhere on the web site doesn't make it right. A large percentage of certificates is revoked. Revocation is a normal and expected part of certificate lifecycle. Thus they should have been crystal clear upfront saying "we charge for revocation" in the same sentence where they said "we don't charge for issuance". They could have mentioned it on the same page, but they didn't do that either (I just checked their "free" certificate page from a random point in time.) So ask yourself why they didn't put such an important fact where most people would see it.


they were a pain in the ass

I had paid them for more than 5 years to provide a cert for my hobbyist website, then one day on a random subdomain decided that it was an 'organisation' and started demanding more money to verify this organisation, when there was no real organisation at all


Thinking about it, we are lucky Debian moved to Gandi instead of StartCom when they ditched the SPI CA.


Gandi's been a long-time supporter of Debian[1], providing preferred pricing (their best "bulk rates") for years. I assume the CA migration was also donated.

I'm personally quite glad the SPI CA is gone, and that we removed CACert[2][3]; stopped a lot of "works on my machine" confusion, and made the project more welcoming to people not already running a Debian derivative.

[1]: https://www.gandi.net/supports/debian/ [2]: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=718434 [3]: https://bugs.launchpad.net/ubuntu/+source/ca-certificates/+b...



Is there a tool that we can use to tell if we have these certs?


It'd be pretty easy to check if the certificate is invalid using something like a bash script [1]. Or you could script ssllabs to check for you (they give the site a failing grade if the Wosign cert is found/invalid) [2].

Note, I haven't been able to test that script on an actually broken site with a Wosign certificate

[1]: https://pastebin.com/ZVMM3PJZ [2]: https://www.ssllabs.com/ssltest/analyze.html?d=google.com&hi...


For what it's worth, SSL Labs also has a command-line tool that enables you to scan in bulk https://github.com/ssllabs/ssllabs-scan

However, although they don't trust WoSign and StartCom any more, it's possible that there is a cross-signed certificate to another CA, in which case the problem is not going to be detected. I seem to recollect that such cross-signed certs exist, but I haven't looked for them recently. Will do.


    echo | openssl s_client -connect somewhere.net:443 -servername somewhere.net | grep -qE "(WoSign)|(StartCom)"


That's a bit simplistic. For example, you at the very least want to have a "-servername" there to account for SNI-only hosts. You'll then need to worry about prefixed and non-prefixed hosts (e.g., www.example.com and example.com). Then, what do you do if your web site relies on a third-party sites that uses WoSign? Yes, now you have to parse the HTML response, extract all links, and check those too.

Then there's also a question of servers other than HTTP, so you need to throw in a port scanner in there. And a tool to actively follow protocols, to discover MX hosts, maybe even look at SRV records, and so on.


For more accurate browser-like results, you might want to add "-servername somewhere" so that you get the response associated with the "somewhere" virtual host rather than the default response.


No need to prefix the command with `echo | '. If you want OpenSSL to have no standard input (so it closes its connection), just redirect standard input from /dev/null.

$ openssl ... </dev/null | grep ...

EDIT: This also avoids OpenSSL sending its only input character (the newline that echo produces) to the server, which will throw it away anyway.


Are you asking because you have a large number of hosts and can't check them all manually? (Edit: Because the obvious answer is to use your browser or a tool such as SSL Labs or Hardenize to see who issued the certificate.) Out of curiosity, how many hosts are you looking after?


Something to be aware of: on Windows, certain third-party executables are signed with a WoSign cert. One popular example being the npcap library (part of nmap). If you distrust WoSign roots, side effects may occur upon using nmap.



A script running openssl s_client would do the trick.


I fully encourage you folks to prune your trusted CAs. I personally whitelist only some CAs. I keep a separate root for visiting Chinese websites like Baidu Wenku.


it is frankly astonishing how many browsers/OS platforms ship by default trusting all of the major mainland chinese root CAs.


It’s astonishing because somehow the Chinese deserve to be treated differently from everyone else?


Because the Chinese government has a well documented track record of attempting to MITM SSL, and if your system trust the root CA... game over.

https://www.google.com/search?q=china+ssl+mitm&ie=utf-8&oe=u...


Should have happened eight months ago. I can understand the careful and deliberative process behind the "CA death penalty" from a major browser, but this was such an egregious case that I think faster action was warranted.


And yet, on my brand new Android Nougat phone, I have these StartCom and WoSign CAs. I just disabled them, but I doubt many users will.


Thanks, just did this on my Pixel.


I had hopped that they would do that much earlier. WoSign and StartCom acting in a shady way has been known for quite a long time.


Where can one learn what CAs are acting in a shady way? Are there other CAs on a similar course?


I would say that Comodo is quite shady, sadly it seems that cloudflare sites use it so getting rid of it would be quite difficult.

They make multiple low quality and mostly useless applications - one could even call them scamware: https://en.wikipedia.org/wiki/Comodo_Group#Consumer_Security.... They also forked Chromium and Firefox and made their own rebranded browsers which offer nothing of value.

They have multiple clearly fake comments on their certificate pages: https://ssl.comodo.com/comodo-ssl-certificate.php and https://ssl.comodo.com/wildcard-ssl-certificates.php

And finally they tried to trademark the name of LE: https://en.wikipedia.org/wiki/Comodo_Group#Let.27s_Encrypt_t...


question for the knowledgeable: I've used AWS's ACM for my certs so far, are there any big reasons why I should be using LE instead?


No. Quite the opposite, AWS automates certificate issuance for you. You'd have to automate the process yourself if you switched to LE.

Of course, AWS only issues certificates when you're using their stuff. For everything else, use LE :)


Nitpick: You can only use ACM with ELBs/ALBs/Cloudfront Distributions.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: