Mail is hard to set up properly, but it's not ridiculously hard to set up. Anyone with average Unix administration skills can do it. I might add that OpenSMTPd is a lot easier than postfix and makes a dangerous open relay misconfiguration hard to do, so it's my recommendation to the uninitiated.
It requires maintenance attention, but not a ridiculous level like some suggest.
What is ridiculous is the effort you will go to to stop your connections being bounced, or (worse) your emails going to spam folders. It's this third reason why I use Fastmail to run my email. Technical accumen is of some but limited assistance here. In essence you will you will be struggling to build an IP reputation with low traffic flow. This article has some interesting insights on this topic.
Saying it’s not ridiculously hard to set up your own mail server, unless you want recipients to actually get your emails when you send them, feels a bit irrelevant?
Most things are easy if you’re willing to accept an end result that doesn’t do one of the two (3? Maybe? Send, receive, search?) primary functions.
The frustrating part is that you can have the most impeccable Postfix configuration for your smtpd, and your system has never once emitted a single spam outbound to anyone, but your mail to office365 or other places might disappear into a black hole.
I know probably 4 or 5 people who are far more than capable of running their own mail server and have given up on it these days, bowing to the inexorable market pressure of gsuite/google workspace/office365 for their MX, because they just can't be bothered by having outbound mail deliver-ability issues. These are people who were running smtpd for regional ISPs in 1997. It almost seems like the google and microsoft spam filters are a bit of a protection racket, forcing people to buy their services.
The author of the article covers this:
>
If you self-host email, you will run into an elevated number of delivery problems. That is a guarantee. Fully implementing trust and authentication measures will help, but it will not eliminate the problem because providers weight their IP reputation information more than your ability to configure DKIM correctly. Whether or not it becomes a noticeable problem for you depends on a few factors, and it's hard to say in advance without just trying it.
Federated systems like email tend to rely on a high degree of informal social infrastructure. Unfortunately, as email has become centralized into a small number of major providers, that infrastructure has mostly decayed. It was not that long ago that you could often resolve a deliverability problem with a politely worded note to postmaster @ the problematic destination server. Today, many email providers have some kind of method of contacting them, but I have never once received a response or even evidence of action due to one of these messages... both for complaints of abuse on their end and deliverability problems [3].
I pretty much agree with you, although I think it's more organic than planned. But both Google and Microsoft offer various configuration options on their enterprise SaaS offerings that are intended to protect you from abuse of their own products (Google's setting to disable submission to external Google Forms while logged into a Google Workspace account is a prime example, reactive to the very heavy use of Google Forms for phishing). That shows a degree of, I'd call it, self-awareness that they are part of the problem, and are offering tools to solve it to their customers.
I guess what I'm saying is that I'm not conspiratorial enough to think that they created this situation on purpose, but now that it exists they sure are benefiting from it... and that undoubtedly reduces incentives to allocate more resources to abuse prevention/response at the source.
I mentioned in a footnote in the article my anecdote about a university switching to G-suite, more or less because of a serial abuse problem originating with gmail. That's not the only reason of course and there were institutional politics, cost optimization, etc involved... but to be quite honest I personally feel that Google has various ways to strong-arm universities into G-suite that are reprehensible and an abuse of taxpayer/tuitionpayer dollars. In my opinion, and at that time, even constraining ourselves to the inevitability of going SaaS the Microsoft 365 offering was both superior in quality and more cost-effective (given specific requirements of the institution like desiring interop with on-site AD and US data custody agreements... YMMV). Microsoft was also amazingly easier to work with from a pre-sales perspective, with a very accessible sales rep who got quotes and answers to detailed engineering questions very quickly, including setting up some meetings with customer engineers to plan architecture. Google was, well, more or less a brick wall... we spent more than 5 months waiting for an updated quote at one point (we were looking for some features that Google said they offered but not on their published pricing, although it later became questionable whether they even offered them).
I don't want this whole thing to turn into a rant, but, well, the university chose Google, and it really felt to me like that was largely a result of some fairly dirty tactics by Google that involved leveraging their partnerships with other academic software vendors to basically box the university out of features of other products it used if it did not select Google. Google and the vendors presented this to us in a way that made it feel remarkably intentional (i.e. the vendors were surprisingly candid that they offered integration only with Google due to commercial incentives). I also felt that Google was outright deceptive about their pricing and capabilities at points, although it felt like it came mostly from the incompetence of their sales staff (promising before they found out if they could deliver, which did not happen until post-contracting). At the time I wondered if any of this rose to a possibly actionable violation of state purchasing regulations but from my perspective now I doubt it... nonetheless it was an extremely frustrating experience that left me with a very negative impression of Google's enterprise offerings and business practices, despite their products generally being more polished and user-friendly than Microsoft's (although frankly I feel that for the last two years or so Outlook Online has surpassed gmail in terms of UX, Microsot has visibly put a lot of work into it and Google has done little except for somehow make it slower).
I suppose Google's success in the enterprise has not changed the fundamental observation that Microsoft gets enterprise sales and Google does not. Google just has the bluster and name recognition to sell anyway.
> It almost seems like the google and microsoft spam filters are a bit of a protection racket
That is expressed in such understated terms that one might be forgiven for suspecting the OP might be blowing smoke (he wasn't, of course; he just seems to be the sort of guy that doesn't care to rant).
I thought it was a very good, well-written article.
I used a VM at Linode and Vultr to spoof emails to O365 and Proof-Point as my manager at the former company to remind them to fix the settings in O365 and Proof-Point. Their default settings are very relaxed and I was never flagged so if your friends are getting spam-boxed or bounced, it's based on the admin settings for their org and SPF ~ vs -. They did fix the settings and they now properly label emails as spoofed but not spam-boxed. O365 spam settings can get sticky and confusing really fast especially if you add many work-arounds for broken things.
Perhaps I should have been upfront with the third point. I was trying to convey that, in my view, getting condemned to spam is the most compelling reason to not self host mail - and it's reason enough. The other common criticisms, by comparison, aren't that strong. If you're relying on your email to work, apply for jobs, or even just send things to friends using consumer services, you either have to sacrifice self hosting, or go through a lot of effort to obtain and build IP reputation.
I think it's a shame you can't reliably self host mail without jumping through hoops around IP reputation. But it's just the reality. You can be an idealist or you can be pragmatic.
In a similar vein, I now only ever use transactional email services (SES, Sendgrid) for automated emails. The sacrifice you make with self hosting is also significant in this context.
The issue of transactional email services is a good one that I might write about sometime as well. I send out the email version of the blog using SES because I know that I would have huge deliverability problems if I sent it myself. Even so, I apologize to my Apple Person readers that Apple Mail routinely hard rejects my emails from SES and I have not been able to figure out why... it comes and goes from month to month and I haven't gotten any useful info.
In general I've found Apple Mail to be probably the most aggressive major provider in terms of rejecting at the SMTP level. Microsoft seems to virtually never do it unless there is a major problem (i.e. SPF exists and prohibits the sending mail server). Google is somewhere in between, but seems to stop SMTP hard rejecting at all once it's seen the IP in use for a couple months.
The unfortunate thing, of course, is that hard rejections at least give you feedback. When they accept the mail you still don't know if it's actually made it to an inbox.
I realized the truth of this this year, when I launched my own self-promotional website (just a portfolio site) and assiduously configured its email, which passes every single test on the range of spam-rating testers available.
Almost none of the mail gets through.
Now I am researching outsourcing the site's mail to one of the usual suspects: mxroute, FastMail, ProtonMail, Exchange, etc. Perfectly configured mail that no-one gets to read is worthless.
It takes a couple of years, at least, of sending spam-free mail at a rate of at least a dozen a day to establish a reputation. Well, I don't know - it might be worse than that. My domain started sending mail at tha kind of rate back in 2004; deliverability to gmail and especially hotmail was never something I'd bet my house on. Hotmail, in particular, would swallow mail and silently fail to deliver.
Yeah, domain reputation works more or less just like IP reputation for a lot of email providers. Holding onto the domain for a long time and sending some amount of email from it will help it get "known good" scores with the big providers. I would also recommend setting up DMARC reports because I've heard anecdotes (no evidence here!) that some providers give a positive bump to email from domains they can successfully return DMARC results to... and it'll at least make sure you know if you have an SPF/DKIM setup problem, especially if it's somehow a transient or subtle one (I've had this happen with config mistakes where not all outbound email was being handed to opendkim).
The worst for me has been Office 365; I have an e-mail with an organization where I'm not allowed to configure the spam settings, and my spam folder is about 90% ham typically. I finally found out that you can whitelist wildcards (though it won't let you whitelist *@*.tld) so every time I get ham in my spam folder, I just whitelist the entire domain, and I run the spam filtering client-side.
I'm now down to the point where I only get one or two ham messages in my spam folder a month. I don't get very much actual spam to that address at all; presumably they have an even higher level of filtering where it doesn't get delivered at all.
Mail is hard to set up properly, but it's not ridiculously hard to set up. Anyone with average Unix administration skills can do it. I might add that OpenSMTPd is a lot easier than postfix and makes a dangerous open relay misconfiguration hard to do, so it's my recommendation to the uninitiated.
It requires maintenance attention, but not a ridiculous level like some suggest.
What is ridiculous is the effort you will go to to stop your connections being bounced, or (worse) your emails going to spam folders. It's this third reason why I use Fastmail to run my email. Technical accumen is of some but limited assistance here. In essence you will you will be struggling to build an IP reputation with low traffic flow. This article has some interesting insights on this topic.