I used to run mail server for a large corporation in the late 90s. I always get really excited and nostalgic about running my own mail server when I see these articles. I have a spare domain I use for testing stuff on so occasionally I'll futz with postfix and dovecot for a bit etc and get something working on Vultr or some other cheap VPS. Then I'll spend an hour getting Thunderbird and K-9 working and I'll send and receive a few emails.
Then I'll remember I just spent 6 hours of my Saturday fucking around with it and now have to live with it and maintain it. So I go crawling back to outlook.com like the pathetic corporate whore I am and go and do something less painful the day after.
I'd like to be a purist about email, but I just can't bring myself to care, because as long as my correspondent is a corporate whore, why shouldn't I be too? A copy of the email is going to end up on their servers anyway.
In other words, I can only protect email from my provider as long as everyone else on the email protects it from theirs. And it only takes one gmail user on the thread to ruin it for everybody.
If I'm corresponding with people who also care about this kind of thing, we aren't using email. If we absolutely have to, then we can encrypt it with PGP.
My email use cases are: (1) b2b professional emails, which you expect to be retained for compliance, etc. and (2) transactional emails, which my provider unavoidably has access to and which I protect with strong passwords and 2FA. (That said, I would like to obscure the list of my internet accounts from google, so maybe one day I'll migrate my transactional emails somewhere else.)
For me, it’s control over the domain. I can change providers at a whim. In my preteens I stupidly set up a handful of alternate google accounts for various things. They all link back to my personal main account which I have used for much of my life. If Google’s abstract machine learning algorithm ever deems my crimes punishable by extreme pain (seriously, imagine being locked out of paypal, banking, credit cards, loans, contact with businesses, etc. for me it would be my main business point of contact as well), this would be disastrous. I don’t mind about the privacy (I consider email broken anyway), I care about control.
Control over the domain is 100% my biggest anxiety with mail.
I use Outlook with a family account, and they used to allow custom domain aliases. All of my email is addressed to addresses on my domain. It's actually a pretty good value for 4 people.
If they ever decide suddenly I'm out or they're out, I'll spin up a mail server somewhere else while I figure out my next move
Effective today (November 30), the ability to create new addresses has been disabled.
They’ve been working hard to remove features from that capability. I assume it was either more than they could justify based on adoption rates, a nightmare to maintain, or seeing more abuse than they want to manage.
My anxieties over email match yours. I used to run my own email server but the entire IP range I was in got blacklisted by Proofpoint, and they were thoroughly uninterested in helping unless I owned the entire IP range and could prove it. The ISP didn’t seem to care, and at the very same time Microsoft added the same range to their lists, which meant I wasn’t able to correspond with an attorney. That was in June and by August I’d moved completely to Fastmail. I haven’t looked back, though I do sometimes miss managing my own email server.
A solid intermediate solution is to have your own domain and host your own email server for incoming email. Then google/microsoft/yahoo/et.al. will never be able to lock you out of receiving important email.
You can still use a third party to relay outgoing email if you're worried about deliverability like in your example. But you'll never get locked out of receiving.
You can do that without self hosting email though. I use a custom domain with ProtonMail. I could potentially lose access to my emails stored on their servers, but getting my email address back up and running is just a matter of changing a DNS record.
> I'd like to be a purist about email, but I just can't bring myself to care, because as long as my correspondent is a corporate whore, why shouldn't I be too? A copy of the email is going to end up on their servers anyway.
No immediate confidentiality is achievable by using a trusted mail server while interacting with untrusted ones, but doing what seems right often works as an improvement. I find it analogous to, among other things, not throwing garbage on the ground: by carrying it to a trash bin, you don't make all streets clean at once, and it may be an inconvenience, but once more people do it, streets do become cleaner. And even if others keep just dumping trash on the ground, it may be nice to know that you don't contribute to the unpleasantness. Likewise with a multitude of other areas, including good practices in programming and administration, good manners: it is unfortunate that not everyone follows them, offsetting the effort put by those who do, but it is not necessarily a reason to give up.
This certainly seems to be the consensus about self hosting email, but but I'm now convinced otherwise.
I send a couple newsletters (one tech related, other for marketing for a coffee shop/bakery I own), and it was fairly easy to land on inboxes. Now, I don't know with a 100% certainty that they were delivered, but if DMARC records and bounces are anything to go by, self hosted emails aren't all that hard.
When you buy a VPS, it helps to check if the IP you receive is already blacklisted. After that, it's easy to use some outside service to monitor DMARC responses and blacklist. There are about 40 black list databases. My domains and the IP address are on none of them.
You can self host and reach inboxes. You just have to have all of the optional headers/dns stuff set up, and have a domain and IP address in good reputation.
Probably also having the whole block in good reputation.
Realistically an absolute fuckload of spam and not a lot of genuine email comes out of residential and cheap VPS hosts IPs so they already have a negative rep.
I do self host. You have to have all DNS markers set right and DKIM working right, but that isn't complicated really.
The good reputation IP range is the biggest problem. I have a more expensive ISP at home that takes its network seriously, so I look more like a commercial interest in that regard. There are decent services that will act as relay for you and probably cost less than such an ISP (or a similarly more expensive VPS provider that takes priority care of related things) if outgoing SMTP is your main reason for considering one.
That sort of work but not all the time, your messages still end up in spam a lot when trying to reach somebody you haven't exchanged email with before.
One pragmatic solution to this is to use an email relay service to handle the outbound mail. Now, the hair-shirt self hoster's response to this idea might be, "But that means trusting a third party!" And indeed it does. But as the parent comment points out, in the vast general case we're committed to trusting whichever third parties our correspondent uses (or finding alternive means of communication.)
Well last time I had to fuck around was because the entire IP ranged ended up on an RBL and I couldn't deliver to half the people I needed to. Oh and then there was that time that Yahoo did its thing and asks you to fill in a web form to continue delivering to them, the URL of which is neatly hidden in an SMTP response message. If you get that form wrong or they don't like the answers they will just ignore you for 6 months.
It's not just about keeping your private communication private, it's about relying on your e-mail address being there. As long as you use someone elses domain you're vulnerable.
The big providers can shut it down whenever they want.
One solution is to use a custom domain that guarantees some sort of ownership. For example certain ccTLDs have more protection for private owners of their domains. And then use providers who support custom domains.
This is exactly what I do. Debian + Postfix + rspamd.
It basically runs itself, I login to the rspamd web gui every now and then and add a few hosts to the "trusted DMARC" whitelist so they get a better score through rspamd, but I don't really need to do that.
I've been running it for ~5 years this way and it hardly ever lets a spam through (Very rarely the spams that are just images get through) and I've rejected 2 mails that I'm fairly sure were legit (just bulk marketing ones though)
People always say "Use Mailcow!" and I say "But why?". Adding the additional complexity of docker with the random firewall rules it adds, individual containers that need to be fed and watered. It's a mailserver, all the processes are doing the same mail related function.
Don't get me wrong, there's a very real place for Mailcow etc, but for a small end-user mailserver I think it adds overhead that makes running a mailserver more complex, not less.
I've done both vanilla postfix, dovecot, rspamd, etc and mailcow.
For me the reason I use mailcow is it's a few commands and I have a fully functional mail, calendar, and contacts server with a web UI. It can then be updated with a single update.sh.
Using mailcow I don't need to mess around with virtual domains, connecting it to mysql, DKIM, etc which is far more than a few commands.
I also get a nice admin UI, web mail and CalDAV and CardDAV which you don't get with the above.
Also I'd consider the separate containers a good thing so that if rspamd, or clamav, for example, has a CVE they can't pivot to other parts of your mailserver.
You make a good point, especially around isolation. While I use docker and I know the basics of it, I'd hate to try and debug it if something went wrong (i.e. postfix container wouldn't start). For that reason I've stayed clear of using it for "critical" life things, like our email. For a number of years there my wife's business email I was also sending through my mailserver, so it needed to be super reliable.
I'm not saying docker/mailcow isn't reliable, only that if it did break in funky ways, I don't have the knowledge domain to fix it. But I'm familiar with all the individual programs and running them on a "flat" linux system, so that's what I did.
But yes, you certainly make some good points that are worth considering if someone else is reading these comments and thinking about doing it themselves!
As one of the maintainers of Mailu, I'd say use Mailu!
Why? three main reasons: (a) security (as you have identified isolation matters, but that is not the only thing), (b) get the benefits of "battle-tested" setups and (c) features
On security: in its default config, Mailu scans emails for malicious macros via oletools (and optionally viruses via clamav). It also uses a hardened-malloc, Snuffleupagus (a security module for PHP), gates all PHP code behind an authentication wall (webmails), ... and does both DANE and MTA-STS validation to ensure your emails are delivered to the right place. The authentication stack handles "smart" rate-limiting: you get to limit the number of authentications with distinct credentials over a time-period (a misconfigured thick client won't trigger it), you have plenty of ways to avoid running into it (application tokens for thick clients, per-device cookies that give you a way out, whitelisting of "used" addresses, ...) and you also get to rate limit the number of sent emails (useful if a spammer gets their hands on the credentials of one of your users)
On the importance of "battle-testing" setups: well, there are plenty of non-subtle ways of breaking an email setup. Experience has shown that all the layers in the stack can be problematic... I can give you a bunch of examples of what we ran into recently if you want.
On features: your setup might be simpler but your users are missing out. Whether it's enhanced filtering (like with oletools), better indexing (full text search), indexing of attachments (with OCR! via Apache Tika), configuring server-side rules with managesieve or just "having an interface" to configure ooo, change their passwords, configure aliases or delegate permissions.
I have started spending time on Mailu because I don't like the bloat that comes with Mailcow. Give Mailu a shot; it is reasonably easy to debug when things go wrong (and not written in PHP :p).
You could install a basic setup with a "apt install x x x" one-liner in ubuntu, and it would not take more than a few minutes to configure everything with sensible defaults.
The key here probably is, most people don't want to spend time learning enough about Postfix, Dovecot. Etc. To do that.
Spoiler: the "services" being "explored" are postfix + dovecot. Nothing against it, but that's not a plural comparison, that's one setup
Fixed title: "Self-Hosted Email Setup Guide"
To be fair, there's this bit (1935-ish words into the article) where there's some exploration going on
> A number of free and open source MTAs are available for system operators including Postfix, Exim, QMail and OpenSMTPD. Each comes with their own limitations, quirks and security history.
> I ended up selecting Postfix
Too bad the exploration wasn't shared in this post: that's what I had clicked for
I'm very aware that I'm in a bubble with my peers, so these are certainly opinionated.
I would pick Postfix because it's more maintained, and what most of my peers use it too. It's a PitA to configure, but there seems to be many resources online. Most of the configuration are quite common, so that one-time configuration is manageable.
Exim is pretty robust too. I don't have a strong reason to not use it, except that I don't have many close peers who use it. Really nice configuration. If you have a multi-tenant setup, Exim would serve better.
Qmail, despite being unmaintained, is praised for being quite secure. I suppose it's the low number of CVEs.
There is also msmtp that provides sendmail compatibility, but uses an external smtp server (like AWS SES) to send the actual email. This is what I use when the server I'm sending from has low IP reputation but I don't want to change my app to use a REST API to send email.
> I would pick Postfix because it's more maintained, and what most of my peers use it too. It's a PitA to configure
sendmail would like a word...
I find postfix is super easy and intuitive to configure. Still have nightmares of maintaining sendmail systems in the 90s.
Exim seemed ok from the times I've used it but I use postfix now everywhere. Feels like it has the best combination of wide support and excellent documentation, good maintenance and being widely used.
So many people are effectively cargo-culting software around at scale by just following along walkthroughs, examples, and tutorials. I’ve seen people who are moderately good developers panic when they have to troubleshoot while following a “golden path” playbook because they’ve seemingly lost the ability to troubleshoot.
No one RTFM’s before doing stuff anymore. I have learned over the past many years that this has been key to my own journey and something others are willing to pay for.
I have used simple-nixos-mailserver (not reviewed here) for three years now, for myself and several organizations. NO issues. The biggest issue with e-mail is (in my view at least) your IP reputation. In order to ameliorate this, I simply buy colocation space, which has higher quality IPs than AWS and cloud services.
I've never had deliverability issues. Because it's nix, upgrades are atomic. Installation is dead simple. Also, the entire setup was like two minutes. I eventually added LDAP authentication and roundcube which took some work, but because it's nix, I did this development on a separate VM before I simply replaced my config on my main server (actually just copied the closures). Adding domains is simple: just modify the nix config file. As is adding new users. Very easy. Highly recommend to run all your critical infrastructure on Nix. Any mistakes are easy to fix... just go back to the old generation.
I did consider solutions like mail-in-a-box but adding the complexity and statefulness of such a system just didn't seem worth it. Also, remote control panels scare me. My e-mail server is heavily SSH protected (Only my private key plus wireguard) and hardened against malicious IPs (fail2ban) among other things. Again, NixOS makes this easy. For storage, simple-nixos-mailserver keeps all data on a separate ZFS partition, which gets zfs 'send't to a remote backup (several).
At the beginning and at the end, the author mentions deliverability to major providers such as Google as the core outcome, but in between they don't substantively touch upon it. It seems unlikely that the goal was accomplished in a durable and intentional way.
Deliverability is my only remaining issue, even after years being hosted on Vultr and doing “all the correct things” and staying off any and all blacklists. I frequently have eMails never reach their recipients because their providers just silently drop the incoming eMail. It doesn’t even show up in their junk folder.
"In order to deliver email to other mail providers without being marked as spam, I ideally needed to implement all three policy frameworks with the mail server."
Then a few lines later:
"Inbound and outbound message delivery was working."
I switched to mailcow recently to test, but I have fond memories of building it myself thanks to workaround.org
He goes into great detail, and if anything edge-case arises, you can almost always find a comment relating to a fix before the main article gets updated.
Why haven't we seen a Terraform setup of containerized email hosting on VPS yet? I feel like it would be really good because it could do the DNS setup with IaC, and therefore also support moving between providers if necessary.
I know the risks of taking over someone's old IP to host email on of course, but with TF you could quickly re-deploy somewhere else until you find an IP that is good.
This was a really nice and detailed writeup, what a wonderful read!
For my own needs, I essentially let others do the heavy lifting and used docker-mailserver, which includes Postfix, Dovecot and some other software in an easy to deploy package: https://docker-mailserver.github.io/docker-mailserver/latest... Actually, that documentation of theirs is a nice read as well!
I've been using this for a low volume of mails for a few years and haven't had much in the way of issues, deliverability, or otherwise. I even got lazy to the point of only configuring SPF, not DKIM or DMARC and have no problems there either (though that might change in the future). Throwing something like a Roundcube container into the mix when I need a web UI is exceedingly easy as well, as is setting up Apache (with mod_md), Nginx (with certbot) or Caddy as a reverse proxy.
I've never used ansible but the article makes me want to consider trying it on my self hosted stuff. I wish I had that article years and years ago, the inclusion of how to configure virtual accounts for email is especially useful to anyone starting out.
I run quite a lot of email services - self hosted. UK.
It's not beyond the wit of man. I cannot believe that my IPs are blessed or whatever. My mail systems are behind "business" IPs but nothing more fancy.
At work I have six WANs - a 1GBs-1 leased line, a 1Gbs-1 FTTP and four 80/20 FTTC. Our MXs are on two of the FTTC connections. We only have SPF set up, I toyed with DKIM and DMARC but disabled that and left as is.
I run several "vanity" email domains and I never have problems with them. My own is an IP on the leased line, mentioned above.
In the UK, it is possible to run an independent email system. I have been doing it for over two decades.
I run my own mail server in the UK on an FTTC line too, run off a more expensive ISP who take there network two seriously so I'm reasonably safe from being blacklisted due to other users on nearby ranges if I don't cock up myself (no significant issues in the last decade and a half).
A friend who also has their domains going through my server did have Gmail give some recipients "are you sure this is the person you think you are taking to?" warnings a while back due to us but having DKIM setup for that domain, so I would say that is becoming necessary these days.
> I cannot believe that my IPs are blessed or whatever. My mail systems are behind "business" IPs but nothing more fancy.
The biggest issue is mail volume to establish reputation. You need a fairly high number of emails before Google or Microsoft's email Postmasters tooling even registers you exist. This leaves new servers stuck in a chicken and egg conundrum. You need to send X number of good emails before the emails are excepted, BUT you can't get emails through without having X number of good emails go through. Old servers like yourself have the advantage of already being "trusted" to an extent.
I ran through the HE IPv6 certification thing back in 2017. It involves setting up email SMTP over IPv6. I used my home connection.
You really don't need volume. You need to do it right by following the rules and also not sending crap out. The likes of mail pig and co will never work as I do ... Americans use the phrase "Mom and Pop" for a small family business - that is what I do.
I have a VM at work that acts as a SMTP/IMAP post office for a few vanity domains. I registered another one a few months ago and added it in for a friend and they have no problems conversing with the hyper scalers etc and nor do I and the rest of my small holders.
This may be an artifact of being in the UK but I doubt it. The big sods are run by people like me - professionals, who try to get the messages through, despite a sea of hate, viruses and the rest.
If you really do SMTP seriously, you will discover a world of people who really care about delivery.
I am not an "old server" per se. I refuse to deliver crap and my peers seem to appreciate that. I own my IT company and I keep a pretty tight rein on my S&M department - they do not get to randomly spam people. Strangely enough we still manage to function and profit without being arseholes that blatt the world with spam.
It doesn't change even over time, three years self hosting and I still can't get any of those tools to show anything beyond an error that there's no data available because my personal volume is so low.
Multiple sources and over 10 years of email server shenanigans. IP reputation partially due to low mail volume is a very well known issue for mail servers and DMARC and DKIM are definitely not enough. Here is one article that covers it that I quickly pulled up: https://mailtrap.io/blog/email-ip-reputation/
Mail deliver-ability has been discussed multiple times here on hackernews with the poor state of the postmaster tools being explicitly mentioned in the articles. For example Google is eating our mail: https://news.ycombinator.com/item?id=19756125
> Most of the Postmaster Tools dashboards will only display data when there’s a sizable daily volume of email traffic (up to the order of hundreds) coming from your Authentication Domains and/or certain other conditions, in place to prevent abuse.
I can totally understand the pain involved in setup. I have done the same with postfix+dovecot for our outgoing email servers and we now have 8 outgoing and 1 + 2 backups incoming allowing us to reboot for updates without losing too much capacity.
I agree that the setup is a complete PITA but to be fair, postfix, at least, is very old and didn't have the benefit of what we've learned about best-practice configuration like having a single conf.d directory so things sit alongside each other nicely. The documentation is also very terse but Wietse did tell me that he's busy enough maintaining the software and porting all the docs is a mammoth task. I did do something myself converting the text files to markdown and using a SSG but there is too much stuff that I didn't understand well enough to re-word. There are also a tonne of compatability which has been added over the years which is no longer needed but still makes the docs feel a little overwhelming.
My top-tip though, is to invest time in the Ansible setup script to avoid the scourge of following out-of-date blog posts line by line. We got to the point where we can provision a new server (manually) run the script and have everything setup ready to go, which is super helpful.
The other important part is to monitor what is happening. Blacklisting can come upon you quickly and not always for an obvious reason but if you don't notice, it might be harder to resolve later and you might have lost a tonne of email. Instead, I semi-regularly spot an error in a log file and have to email someone or login to some ancient web page to request removal from a blocklist. We are generally always successful but it is also worrying that we have no direct control over it.
I remember some articles on HN a while back about running your own email servers. I think they basically said that they would end up in spam for just about every contact. It seems like most people today use email that runs through Gmail. Who get to more or less decide how the worlds email should be processed. SMTP was meant to be an open protocol. It's really messed up what happened to it.
If you have DMARC and DKIM configured correctly, use a dedicated IP (so you can control its reputation) which you don't just start sending millions of emails from and have your SMTP server configured to communicate via TLS/SSL, you'll probably be fine -- assuming your IP hasn't been blacklisted in the past.
I've never had any issues with deliverability. However, unlike most people, I purchase colocation space. This comes with more rights and better IPs than the cloud providers. It's also significantly cheaper when you factor in all costs.
I've ran MTAs and MDAs both personally and professionally at scale. I've hosted my own postfix and dovecot instances for about 13 years now for one reason, sieve. I cannot live without it.
Honestly, the original setup of everything is a big lift if you've never done it but once you have ldap+krb5/PKI for auth{n,z}, DNS working properly, and a working POSIX setup (what used to be sysadmin 101) it basically runs itself in perpetuity. The underlying tech is old, feature-complete, and rock solid. Not to mention you're in full control of the entire stack. I honestly haven't had an issue after the first year and I use email HEAVILY.
Though to be fair, I do have a bunch of ldap entries and routing rules in place so the power users can choose sieve+dovecot and all the plebs can use the Gmail webui.
For me it's not worth the effort to self-host, plus figuring out SPF, DKIM etc. Not being included in a blacklist because your server's IP address is on the range of a spammer... I use Mailfence for a couple of emails with my own domains, works well
This is a good guide to set up an email server, yet it lacks describing PTR Records (aka reverse DNS) and that some cloud providers block outgoing email unless you ask support to turn it on.
yes, running a mail-server is a complex task - but nothing to write home about!!
personally i prefer exim over postfix - because of its configuration: exim has a really flexible configuration-language which enabled me for example to operate qmail-ldap and exim-ldap from the same LDAP-tree - for migration purposes a few years ago -, using the qmail-ldap.schema vs postfix which insists heavily on the posix.schema ...
additionally i prefer courier-imap over dovecot, for no apparent reason, just because i use it since ... ever. and i don't care about peoples performance who keep something like 10 k emails in their inbox/IMAP-folder ... yes, for this use-case dovecot is significantly faster / because of caching ;))
just my 0.02€
ps. i used qmail-ldap and qmail for over 20 years, and it was a really great piece of software - kudos to djb, but even brilliant code ages over the decades - for example, i didn't want to implement IPv6 support for outgoing mails myself / the available patch didn't cooperate well with the ldap-patch itself if i remember this correctly etc.
pps. i would recommend: use an LDAP-directory for email ... openldap is an extremely lightweight and easy to operate piece of software. and its easy to replicate the db around - if you have multiple nodes in your mail-system - etc. ;)
I went up this road. I ended up settling on Mailcow because it does all this same work and more, with pretty good integration and sensible choices, together with some valuable tools for things like migration and backup.
If and when the Mailcow project goes away, it’s all still standard components, so moving to i.e. vanilla postfix and dovecot should be straightforward.
I'm running small hosting service for me and my friends powered by ISPconfig (postfix/dovecot/rspamd) for more than 10 years, no issues at all and it's really easy to set it up these days. Just recently I installed new Debian 12 server, two-three hours work tops to setup 10 domains, mails, etc....
Long time miab user here, recently switched to MailCow to make my host a run docker stack only (no more vms). Also, mail search on non email body finally is fast. Much love for MIAB still though.
Self-Hosting email isn't that hard as it used to be (as long as you don't have to deliver to MS)
because in the end most of the work you have with your selfhosted server is to figure out why MS (or, in my experience also google) rejects your mails and how you can fix it because you need to reach one person there.
i gave up and started to use a secondary domain hosted with a paid email provider just so i can reach people that my selfhosted server can't reach.
aside from that my selfhosted server is maintenance free.
Then I'll remember I just spent 6 hours of my Saturday fucking around with it and now have to live with it and maintain it. So I go crawling back to outlook.com like the pathetic corporate whore I am and go and do something less painful the day after.
I'll just keep my private comms off email.