This will still break DKIM (no signing for the fake second message, and even the first message's signature is going to be corrupt?), so a restrictive DMARC policy (p=reject; adkim=s) should mitigate this even if the server software is vulnerable, right?
And if someone tries to force a server to send email for an unauthorized source (e.g. make mail.example.org send a bogus mail for ycombinator.com) - well, this sucks in terms of potentially having to get the vulnerable host out of DNSBLs, but email is still not gonna get through (because of SPF) and I hope RBLs are gonna have a bit relaxed attitude towards this in the next few weeks?
(I don't seriously believe someone's not checking SPF, DKIM or DMARC those days - they're basically three cornerstones of modern mail server auth)
> so a restrictive DMARC policy (p=reject; adkim=s)
This isn't how DMARC works. Valid SPF OR DKIM will validate a message. So unless you drop your SPF policy DMARC will still pass. And unfortunately if you don't have a good SPF policy many mail servers will send your messages to the spam folder, so there isn't a good way to have only DKIM set up.
There are some discussions about requiring DKIM independent of SPF, but nothing published (nevermind adopted) yet.
Oops! Thank you for this, for some reason I always thought the policy is "AND", not "OR". My mistake.
Well, guess I have something to do for Christmas... (I run a non-Postfix MTA, gotta try to test it and see if I can implement a patch if it's vulnerable.)
I currently don't have DMARC setup on my mail server (which runs for a couple of domains plus subdomains for me and a couple of friends/family), but have SPF records set appropriately and DKIM configured. We've not noticed any deliverability issues.
Though I'd be interested to work out (or, be told!) how this might affect this vulnerability. I always assumed that most mail servers both check against SPF records and verify DKIM signatures if both are present, rather than it being a this-or-that thing, so DKIM offering some mitigation is not undone by the presence of SPF.
I doubt it. Some servers may reject based on SPF when there is no DMARC policy but there is no way to know that DKIM is enabled unless you see a signature. So a spoofer could just not include a signature and the receiver would have no way of knowing if the message should be signed.
As others have mentioned, this is not how DMARC works.
But I'd like to add to this that this is also not how 'adkim' works. It won't protect you from this particular attack.
adkim=s adds a strict alignment requirement for DKIM, meaning that the DKIM signature is not to be considered aligned unless the domain matches up to the subdomain level. The default 'relaxed' (adkim=r) alignment requirement allows you to use subdomains with a DKIM key placed under the administrative domain (aka 'root' or 'apex' domain) and vise-versa.
Unless you know exactly what you are doing and you have a situation where you do not control subdomains of your own domain, you probably don't want to use adkim=s.
TLDR: In the SMTP protocol, the end of the payload (email message) is indicated by a line consisting of a single dot. The line endings normally have to be CRLF, but some MTAs also accept just LF before and/or after the dot. This allows SMTP commands that follow an LF-delimited dot line to be “tunneled” through a first MTA (which requires CRLF and thus considers the commands to be part of the email message) to a second MTA (which accepts LF and thus processes the commands as real commands). For the second MTA, the commands appear to come from the first MTA, hence this allows sending any email that the first MTA is authorized to send. That is, emails from arbitrary senders under the domains associated with the first MTA can be spoofed.
Looks like Postfix developers are quite pissed at this release with months-long advance disclosure to commercial vendors, while learning about it right before the holiday break:
SEC Consult added an update to their page [1]. They are basically hiding behind Cisco, CERT-Bund and CERT.at.
I mean, c'mon... They did all that research and they expect me to believe they didn't understand the potential impact so they deferred to someone else's judgement? That's the excuse, seriously?
Besides, what would have been the harm in reaching out to projects like Sendmail and Postfix and ask them for their opinion? I'm more inclined to trust the judgement of the Postfix project then of Cisco.
I don't understand the hate against Timo. It's pretty obvious from the article that he only thought about the recipient side and not about the sender (postfix). Postfix does behave wrong (according to e.g. RFC 2822 section 2.3), but it still was not in scope of the article.
The disclosure did go to the parties who he (or SEC Consult) seemed vulnerable.
If anything it just shows that Wietse have a way better understanding of SMTP than most of us normal humans.
> This might not seem bad at first, but looking at affected SMTP software on the Internet is a different story. After testing some popular e-mail software in their default configuration, it turned out that Postfix and Sendmail fulfil the requirements, are affected and can be smuggled to. Speaking globally, this is a lot (figure 31)!
It seems like Postfix was properly identified as a party to this issue.
"Mitigation" suggests that the config change makes the problem less severe. This config change is a complete fix for the problem, although for a high-volume server that handles a lot of lists, it might impact performance.
Add me to the list of people who are annoyed that they didn't give due notice to the Postfix maintainers.
> This shows part of the downside of consolidating email into a handful of massive providers.
Kinda yes, kinda no. It seems the more relevant quantity is the software that is running (e.g. Cisco, Postfix, etc), rather than who hosts it. Even in a world where email was far more decentralized, this would still be a problem, perhaps even more so as there'd be a much longer tail for mail systems to be updated.
There’d be less domains to spoof per server with SPF spread across more IPs. However you’re right about the long tail and since the best value is likely internal spoofing (ie: ceo scam), there’s upside to the huge providers just dealing with it for everyone.
> This shows part of the downside of consolidating email into a handful of massive providers.
I don't agree.
The great thing about consolidating email services is that a fix for a vulnerability will typically be rolled out quickly, thus protecting the vast majority of email users from this particular attack.
Whereas with self-hosted/on-premise SMTP services we'd be chasing this for years and years. There are so many ancient, unmanaged and otherwise badly configured SMTP servers out there...
The difference here is that for the engineers that work for the large email service providers, it is their full-time responsibility to deliver email service. Whereas for the self-hosted solution, it is often a burden to the IT admin who doesn't get time/budget or have the knowledge to tend to the SMTP service. SMTP is often viewed as a set-and-forget type of thing, that can be set up in less than an hour, but it definitely is not.
I'm all for open-source, self-hosting and keeping control over your own resources, but for email (other than your private email address maybe) just outsource that to one of the well-known providers and focus on your actual job.
However not giving a large well established OSS project such as Postfix a notice upfront is not really a classy move.
> As part of a non-responsible disclosure process, SEC Consult has
published an email spoofing attack that involves a composition of
email services with specific differences in the way they handle
line endings other than <CR><LF>.
You should be able to set "smtpd_data_restrictions = reject_unauth_pipelining" in your main.cf
This option is available in "older" postfix versions and even works with postfix 2.10.
Don't know if it is as good of a measure as the 'smtpd_forbid_unauth_pipelining' that is recommended for newer versions.
Ok wow, anyone else thinks Cisco is being really stupid here? Their sanitizer might make sense for inbound smtp connections from authenticated mail clients, as any malicious or accidental "weirdness" would be turned into proper EOD sequences with no ambiguity for the actual SMTP server.
But for inbound connections from other smtp servers this actually makes sure the exploit works 100%, the smtp server behind the gateway has zero chance to detect the attack since it now gets a proper EOD sequence with another mail after. Just wow that they couldn't understand that this is a bad default.
> any malicious or accidental "weirdness" would be turned into proper EOD sequences
Yeah, that's the result in this case. In other cases, not allowing malicious weirdness to pass may prevent a whole host of vulnerabilities from being able to reach your system at all.
Trade-offs. In this case, it happened to make you vulnerable. Which is better may only be clear with hindsight and experience, but I can see their reasoning.
However, their job being to provide additional security, by not fixing the issue (they can leave other "cleaning" rules in place, just this one needs to be forwarded either literally or rejected) they're imo defeating the purpose of their product insofar as I understand
Strange they didn’t wait with this announcement a week or two, to allow proper holiday to fellow developers.
From postfix -
“Days before a 10+ day holiday break and associated production change freeze, SEC Consult has published an email spoofing attack that involves a composition of email services with specific differences in the way they handle line endings other than <CR><LF>.
Unfortunately, criticial information provided by the researcher was not passed on to Postfix maintainers before publication of the attack, otherwise we would certainly have convinced SEC Consult to change their time schedule until after people had a chance to update their Postfix systems.
”
- https://www.postfix.org/smtp-smuggling.html
For not properly disclosing this to Postfix maintainers, they earned themselves a bonk on the head.
I'm sick of this trend of successful attacks being basically free marketing, while successful defences and maintainers working long hours for free to keep open source software secure never receive any praise.
Except for the communication and the lack of responsible disclosure. Publishing the exploit without contacting first the security teams of tools as big as postfix... Is quite bad.
> Unfortunately, criticial information provided by the researcher was not passed on to Postfix maintainers before publication of the attack, otherwise we would certainly have convinced SEC Consult to change their time schedule until after people had a chance to update their Postfix systems.
Responsible Disclosure traditionally has two forms: Coordinated & Full. They mixed the two for different projects, but it's still responsible to the end users. This is in opposition to irresponsible disclosure (selling exploits to attack groups).
They started disclosing in July and it looks like everything was very professional and responsible only to publish the week before Christmas.
From the timeline, perhaps CERT/CC could have done more or could have been quicker in their review. 5th December is probably the absolute earliest reporting date, but published on 18 Dec after what I can only guess was nearly 2 weeks for internal review
So, the decision is to publish the week before Christmas or wait until early/mid Jan when staff return and they took the less than ideal option to publish early, presumably in case they were scooped
Also, they only disclose to GMX, Microsoft and Cisco, when they clearly write in the article that there's quite a big quantity of Postfix servers impacted (more than a million by the mentioned figure):
This might not seem bad at first, but looking at affected SMTP software on the Internet is a different story. After testing some popular e-mail software in their default configuration, it turned out that Postfix and Sendmail fulfil the requirements, are affected and can be smuggled to. Speaking globally, this is a lot (figure 31)!
I know I'm just repeating what others have said here, but just wow. Publishing something like this less than a week out from Christmas, with apparently no responsible disclosure.
Maybe it will come out that somehow they messed up and the postfix crowd didn't get the memo when they were supposed to, but otherwise this is almost malicious in its negligent attitude.
Seems like they got too excited by having such an increduble exploit and got ahead of themselves? Either way, good way to create unnecessary grudges and resentment.
only talking to corporates for responsible disclosure is not something that's gonna go down well with that crowd.
I hope they get some critical questions in the Q&A.
Critical questions? Was hoping for them to get booed off stage for talking to vendors of proprietary solutions and not including open source projects in their disclosure comms.
It allows someone to fake the sender, and have the fake bypass the signature schemes that are supposed to prevent fakery. But it's not that incredible; those signature schemes are addons for SMTP, which has always allowed this kind of fakery.
That is: I'm not exactly bowled-over to learn that it's possible to fake the sender of an email.
Yeah I did. As I said, maybe they thought this was going to reach the postfix crowd, but they name them in the article, indicate they're effected, and yet made no attempt to directpy inform the postfix guys?
From the wording of postfix's announcement, it's possible they did disclose the vulnerability itself, but witheld some kind of important information, whatever that means. Or else we have to take for granted that the postfix crew are lying about it.
Either way, why would you publish literally right before Christmas? People need time to check if they're affected and update or patch systems. At best it's highly inconsiderate, and it seems more negligent than inconsiderate.
I wonder what "critical information" was missing. The gist of the exploit is a very short sentence.
Not pointing fingers, but without looking at the actual email, the fault could lie on either side. "Missing critical information" is the excuse anyone would use after ignoring an important disclosure.
The drama is about notice so I did a quick search of SMTP server software marketshare (number of servers, not users). Postfix apparently runs 37% of them.
I'm finding it interesting that Exim isn't mentioned once in the article or here, even though it's marketshare is much higher than Postfix's at 55%. Would be nice to have confirmation that Exim isn't affected at least.
There's a few posts here about non-responsible disclosure. Having read the post they certainly disclosed this to a number of vendors, or at least seem to have. Is this a case of them not disclosing it to every vendor?
Not disclosing it to postfix is the biggest issue I'm seeing people talk about. From the article they reached out to three of the biggest individual vendors, but there's no indication of them disclosing it to postfix or any of the other providers of mail server software.
In the article they even mention that postfix is affected, and show it as being the most used mail server online, by a large margin.
Could it be that they disclosed it to anyone who was willing to buy their services/sign their contract? They seem to be a business, not some students from academy.
That seems very unlikely. As someone in the industry, I haven't heard of commercial businesses doing that, and in fact know of one court case in the Netherlands a handful of years back where someone tried that ("would be a shame if it became public, I could fix if you pay me x") and got sued for extortion (not sure if the phrasing was exceedingly poor but not intended maliciously and the court acquitted, or if they got convicted; there may have been two such cases, one malicious and one not, and I'm mixing them up)
My employer (me included) does similar disclosures to vendors and open source projects. Being paid is never a condition for doing responsible/coordinated disclosure. I can't speak for others but, given the extortion allegation risk, I have trouble imagining this is how it went down
Much more common is being afraid it leaks due to patches landing too early or people talking, and so limiting it to the biggest vendors out there. In this case, I'd say they could definitely have done more coordinated disclosures without risking that...
In summary¹: instead of sending CRLF.CRLF to end the email data part, send something like LF.CRLF and append more SMTP commands. The server accepting your email may interpret this as part of the body and forward it to a system which interprets it as the end of the body, with the rest being interpreted as more SMTP commands coming from a trusted system.
This allows you to relay arbitrary commands via this SMTP server and submit another email. An outbound relay may pass SPF (unlike your IP). A vulnerable inbound server may let you bypass certain checks (such as SPF, but perhaps also a malware filter) because the server you're talking to is trusted by the internal receiving server.
As an outbound example, Microsoft wouldn't allow you to pretend to be admin@hotmail or admin@another_customer_of_hotmail's when logging in with your regular user account and submitting an email, but this attack lets you submit two emails appearing as one to their initial system, so only the first one is validated.
¹ the article starts with a TL;DR but that doesn't actually mention what the vulnerability is, only how cool and widespread it is (I agree that it is cool and widespread, but that's not a TL;DR of a technical article, that's a dumbed-down executive impact overview). It then goes on a 1700-word lecture on how email and http works before telling you more about the issue.
> It then goes on a 1700-word lecture on how email and http works before telling you more about the issue.
...with every other sentence ending in an exclamation mark.
It's a very badly targeted article. It reads like celebrity gossip, but the subject matter isn't for the kind of people that read that stuff. The people who are interested in this subject don't need their bottoms wiped for them.
I bailed out halfway through. I still don't know what servers are vulnerable.
I also clicked through to an earlier article of theirs, about a dns issue. That was even harder to get through: I still don't understand the vulnerability exactly after reading the whole thing. It tries to explain every detail and somehow loses me in the weeds; I think a simple Python PoC would have been more clear than the long post. Compared to that, they're improving! This article is pretty clear about the mechanism once you get to it.
Thats pathetic. I wonder why all those parsers do not do simple thing.
Look for \r, found it? slap 0.. ignore ANY other \r or \n ocurences until next char. Same for \n. Once you have all lines in buffer.. reconstruct message
with proper \r\n like RFC says. I just saw what postfix does in smtp_get(), its gross..
I also think that we were bitten by maxima: Be gentle what you receive and strict what you send. I actually myself liked it, but now I think its not a good idea after all. Internet is hostile environment these days, its better to be strict and not trust anything coming from outside.
Tools like SPF and DKIM exist now for mail servers to double-check that the email originates from a machine which is authorized to generate email for that domain.
If a message arrived and such settings aren't properly configured, many email providers will reject the message on the assumption that it's malicious.
That's like saying "haven't computers always been able to run malware?" below an article describing a vulnerability that allowed bypassing code/apk signing. Just because a different vulnerability with the same impact existed, and was fixed, 30 years ago doesn't mean this one isn't new and also valid?
It's still easy to fake what shows up in a user's mail client. I see tons of spam and phishing sent with invalid information in the short headers, which is enough to fool most people. The progress has been in mail servers/filtering that catch when validation checks fail so that ideally spammers aren't able to abuse a mail server in the first place, and when they do those messages don't reach people's mailbox (or if they do they hit the spam folder).
This will still break DKIM (no signing for the fake second message, and even the first message's signature is going to be corrupt?), so a restrictive DMARC policy (p=reject; adkim=s) should mitigate this even if the server software is vulnerable, right?
And if someone tries to force a server to send email for an unauthorized source (e.g. make mail.example.org send a bogus mail for ycombinator.com) - well, this sucks in terms of potentially having to get the vulnerable host out of DNSBLs, but email is still not gonna get through (because of SPF) and I hope RBLs are gonna have a bit relaxed attitude towards this in the next few weeks?
(I don't seriously believe someone's not checking SPF, DKIM or DMARC those days - they're basically three cornerstones of modern mail server auth)