I run my own mail infrastructure. To say the least I wouldn't recommend it even to my worst enemies. It's horrible.
Actually it's fine until it's not. Then your email doesn't work and you could be missing out on important communications. And then you're scrambling to figure out how the spammers managed to exploit your setup this time. And you have to learn a tonne of crap in order to manage it... and the text files! Configuration... configuration everywhere. Obscure configuration. Configuration that has real consequences and causes spooky action at a distance. Configuration that will soon be exploited in strange ways.
I was so frustrated the last time my mail server went down that I started writing an SMTP protocol handler in Haskell with the intent of writing a MTA with the goal of minimizing configuration and being secure and resistant to attacks by default. So that hopefully more people can run their own infrastructure without prematurely aging. I dunno how useful it will be to others but at least it will keep my gray hairs at bay, I hope, when it's ready for use.
Until then though we need more guides like this for us poor souls who do go down this route. There are way too many out-dated guides awash in the sea of information.
"And then you're scrambling to figure out how the spammers managed to exploit your setup this time."
...
" ... writing a MTA with the goal of minimizing configuration and being secure and resistant to attacks by default. "
As a 20+ year UNIX sysadmin and fellow owner of my own email infrastructure for 18 of those, I am surprised to read this and am not even sure what you are talking about.
Can you explain what you mean by attacks and exploits from spammers ?
Other than accidentally running as a relay or actually having a remote exploit in your server(s), what are the attacks that you have in mind ?
"Configuration that will soon be exploited in strange ways."
Again, am genuinely interested in an explanation ... other than running as a relay, and maybe handling backscatter (but that's not really config, it's just a blacklist) what are you referring to ?
I'm in the same boat - been running my own mail since the 90s. I could understand complaints about lots of weird hoops to jump through to get mail from your server accepted elsewhere - that has become harder over time[1] - but there's a very short list of things that need to be done to avoid being hijacked.
Mail server setup for the uninitiated does look a little daunting, especially if you're more accustomed to "all-in-one"-ish software. If you want to do this, I recommend starting with making sure you have a clear mental model of mail server architecture, especially where it touches other things (DNS, DKIM, spam filters, local delivery, probably IMAP, maybe LDAP, maybe databases, etc.) Without a clear idea of the dataflow and reasons for different decision-points, you're going to have a very bad time troubleshooting things.
[1] SBCGlobal.net has been rejecting me for over a decade despite there being zero spam ever having been emitted by my domain. Eh, my userbase communicates with exactly one SBC user; ATT can bite me.
I've been running my mail server for 6 years now. Mail to Microsoft's servers (live.com, etc.) keeps ending up in the spam folder, even though spf, dkim and dmarc are set up and my IP has been clean for the entire period. The bright side is that I only notice this in the rare case of "mail to all contacts", since noone is on outlook.com these days.
Yes, live.com is a problem for us too. It seems they silently blacklist ip addresses and make it very difficult to get off that blacklist. There's a thread here about it with some of us trying to figure out how to solve this problem:
But other than that, running my own mail server hasn't been much of an issue. Set up sendmail, use public blacklists for spam control, and it pretty much runs without any intervention.
Outlook.com/Microsoft blacklists entire subnets because there are spammers in the same IP range. It's ridiculously lazy practice. They should just block IP addresses that are actually sending SPAM. It's not like that's a problem technically. You can fit all IPv4 addresses into a 512MB database.
I'm thinking of solving it by blacklisting outlook.com domain, so that senders at least know that I can't respond to them. I can put a message in the error response, that will be reliably relayed to the sender by the sending system.
Google's slightly better. Recently I did an experiment and created a few gmail accounts and sent some riduculously spammy messages full of typical keywords in between those gmail accounts, and they were all successfully delivered. Always.
Then I sent e-mail from new gmail account to my email server and simply responded and it went to spam. It's ridiculous that such a simple heuristic like someone responding to a message gmail user sends doesn't get the message through spam filters, even though the system can clearly determine taht the message is legit based on many variables (References field referencing message-id of the original message (noone else than the recipient should know this), reply being from a correct source (DKIM/SPF), message having normal looking business content, etc.).
There's way too heavy a weight on sending server IP range reputation.
With gmail you can at least request that they unblock you, and they will do that. With live.com and icloud.com you have to spend inordinate amounts of time bouncing between useless support people before you get anywhere. gmail in general seems to have the best spam filter (lowest false positives and negatives).
I had a problem with ipv6 myself with gmail, but the problem was that I just didn't have ipv6 fully set up on my server, so either SPF or reverse DNS wasn't working or something like that. I think I just configured sendmail to only use ipv4 and that solved the gmail issue.
My issue was that I wasn't very familiar with ipv6, and my ISP (OVH) apparently gave my server a range of about 256 ipv6 addresses, and I didn't really know how to properly set up reverse DNS and SPF. After spending a day or two getting nowhere, I just decided to turn off ipv6 completely for the server.
To be fair I've been running my infrastructure for years and it's usually fine.
The problem I have is that it's a pain in the ass to setup correctly and when things do go wrong it's really hard to figure out what's going on.
> Can you explain what you mean by attacks and exploits from spammers?
I haven't encountered a remote exploit (yet) but yes: accidentally running an open relay, backscattering, etc. You also have to worry about the system security of the box you're running it all on but let's treat that as a separate concern.
I'd been running my setup for years and as recently as August of last year I was dealing with an outage of my email server. I must've missed something somewhere because someone had figured it out and started using my server as a relay. It took me a few days to figure it out and I'm still not sure what the problem was or how I fixed it. Though my server stopped bouncing emails and started forwarding things again so... win?
As I was fixing this issue, not the first time, it occurred to me that it was just criminally easy to not know if your system is being exploited as a relay or not. There are a bunch of different configuration files in all different formats and guides that require working through dozens of steps. It's way too easy to get something wrong.
For someone like me who works at deploying software running on hundreds of nodes it seems manageable but I don't think it's ready for my cousin who's good with computers.
That's where the idea for a secure-by-default MTA that couldn't possibly be configured to be an open relay came from. Minimize the configuration so it was hard to get wrong even at the expense of flexibility.
I dunno, maybe it's not a great idea. But it's fun to hack on in the evenings when I've got nothing better to do and hopefully I'll have a fewer text files to manage in the future.
> That's where the idea for a secure-by-default MTA that couldn't possibly be configured to be an open relay came from.
You're close to describing the motivation behind Postfix. Between the design and the documentation, Postfix is hard to screw up, from a security perspective. And it is really easy to configure, at least in contrast to What Came Before - believe me, if you think this is complex, buy a crusty sysadmin a beer sometime and mention 'sendmail.cf'.
The problem here is that almost all mail servers need to selectively relay, and I don't see how the server is going to guess appropriate policy. For instance, trusted IP ranges (mynetworks, in Postfix). I suppose you could demand authentication unconditionally but that tends to break down when not all of your senders are made of meat[1]. Maybe that's acceptable to you, but it won't be for many.
In my other comment in this thread, I recommended gaining an clear understanding the architecture if you're going to do this. That includes things like knowing what (for postfix) mynetworks does - you can get mad at software for not intuiting local policy preferences, but I've never found that to get me very far.
[1] Getting better, but I still depend on a fair bit of software and hardware that doesn't speak SMTP auth.
I've run my own mail server since probably 1998 or so. I have had a few problems here and there, but nothing major. The worst is when the power goes out, but smtp has a wonderful resend feature when the destination server isn't responding.
I know my emails aren't being farmed from the NSA (at least from my side). I know Google isn't scanning all my emails to sell me garbage. I know it won't be shut down because of some errant algorithm. I have a really short email address. I can setup family and friends easily. I can send newsletters without having to deal with someone else's policies (on my side anyway). I love it.
Backup MX is usually not necessary, especially for home setups. Anything RFC-compliant will retry sending its email to you for days after being rejected.
It's also generally not a wise idea unless you also have your anti-spam/anti-virus setup running on it as well, since things like graylisting can only be done BEFORE the MTA accepts the message.
Spammers WILL know how to search your mx records for things like mailbagging servers and try to exploit them to bypass your security, just don't do it - the sending party will retry.
You can chain SMTP like this: SMTP (backup) -> SMTP (spam filter) -> MTA.
Checking white-lists can take a while and some SMTP servers will give up if they're stuck for over a second, sometimes less (eg when your server checks the spam-lists). Some SMTP servers or DNS server (cought Google caugh) will not try secondary though, so if you can afford it - have just one powerful front server with good uptime and network, or load balance / proxy to internal farm.
My backup MX is set up exactly that way, with an almost identical configuration to my main MX, to avoid this problem. Fortunately the config hasn't changed in over five years.
Running since 1998, haven't been on any blocklists so far.
In the age of password leaks it is really common that email accounts are easy targets and get hijacked – most users are probably still using the same password for their email as they used for LinkedIn in 2012. So it might happen from time to time that an otherwise normal account suddenly sends 20 000 emails to hotmail.com unless you have somehow predicted that and configured the mail server to not allow it.
Oh yes it is, we have around 200k users and fighting with old leaked passwords is like a full time job. Very easy to abuse as well, the attackers can make low frequency requests from scattered IPs and thus are almost impoasible to prevent
You added a predicate not present in the statement you replied to, "where mailprovider like LinkedIn_dump.mailprovider". Anecdotally, in authorized enterprise password audits I am frequently forced to remind people who were part of that breach that they need to change all instances of their compromised password, not just the one for the email they used in the breach. Maltego is a thing.
Also, with regard to your claim to have data about which passwords work. You should be aware that accessing someone's account with their username:password without permission (even a breached password) is likely illegal in the United States under the CFAA.
Identity aware proxy. User has to auth over http, then mail client can connect to IAP and ultimately to mail. And or two factor. Giving the user 10% easier usage gives the sysadmins 200% harder job. Take away the 10%.
Most users that reuse passwords statistically did not rotate their passwords across all websites because most users that reuse passwords statistically do not rotate their passwords.
We aren't talking about the overall Linkedin userbase, just that specific subset that reuses passwords for secure services.
I run an E-mail server with over 250,000 users. I started by following some generic "dovecot+postfix+mysql" tutorial on howtoforge and I'm still using mostly the same setup over 4 years later.
>Then your email doesn't work and you could be missing out on important communications
Pretty much every E-mail server will retry sending your E-mail for a long time (like 2 days is default on postfix). Once your mail server comes back up all of your E-mail you missed during the downtime will come in slowly as messages are retried
>you're scrambling to figure out how the spammers managed to exploit your setup this time
Any tutorial should point you in the right direction restricting open relay on your mail server, just basically requiring authentication to send E-mail outside of your server.
>I started writing an SMTP protocol handler in Haskell
Do you have any link to your progress? Postfix's configuration definitely shows age, but all of the options do important things that you could actually want to change. It seems other MTAs either have just as complicated configuration (to do the same things), or have stunted functionality.
>being secure and resistant to attacks by default
I agree about sensible and more secure defaults in configuration. But the application security of postfix and dovecot are both pretty robust[0][1]. Considering they are 19 and 15 years old, both applications have seen several developer-lifetimes of effort.
>we need more guides like this for us poor souls who do go down this route
I agree, though mediocre howtoforge tutorials seem to have worked fine for this poor soul.
Would you be able to share some stats, like how much disk space these accounts take up and such?
For larger scale e-mail sending I built my own MTA (https://github.com/zone-eu/zone-mta), our main instance sends about 750k emails a day (mostly normal ISP traffic, maybe 25% marketing emails) and its most valuable feature is juggling with IP addresses and blacklist detection, so if some mailbox gets hacked, starts sending out spam and the IP ends up in Spamhaus et al then this IP is removed from the list for other users automatically. Has saved us a ton of time.
Sure, the mail storage currently takes up 1.01TB, using dovecot's mdbox. This mail store started in January 2016, after the service got raided twice by german authorities[0] (at request of u.s. gov), otherwise it would be larger.
I'm not going to run like per-user stats but I know historically there's been about 11kb per E-mail on average, so that's about 90 million mails stored. The MTA itself sends and receives a bit over 20 million E-mails annually, so the extra from that is probably from the mailing list I run on the same server (where each message only counts as 1 for statistical purposes)
How I prevent outgoing abuse is a black box, but I do it well enough that mail from my server almost never ends up in spam. But given that it's a free service I definitely don't have the budget to warm up and dynamically scatter mail across IP space to maximize deliverability. Cool technique, though!
Oh, btw, the easiest way to test ZoneMTA would probably be to install the WildDuck mail server to some empty VPS. The install script, amongst other things, installs and configures ZoneMTA https://github.com/nodemailer/wildduck/tree/master/setup
I ran my own mail server (a few VMs in the cloud) for about five years. It's fun as you have control over accounts, aliases, forwarding, sieve tricks as well as backups. It's good to know that your broker and doctor's mail is delivered directly to your server and not Google.
It sucks ass when your mail does not get through as GMail/Hotmail rejects a connection from the IP address your cloud company allocated for you, in spite of DKIM etc. Or they decide to mark your mail is "suspicious" and it goes into the recipient's Spam folder. Fighting these huge hosting companies is impossible as they don't want to talk to you.
Also constant tweaking Spamassassin is not fun. My setup worked well for for the first four years but then started letting more crafty junk through... So, I had to compromise by moving the mail to Fastmail.
> It sucks ass when your mail does not get through as GMail/Hotmail rejects a connection from the IP address your cloud company allocated for you, in spite of DKIM etc.
God, yes. Gmail, in particular, really enjoyed just dropping connections from my IPv6 addresses every few months. Sometimes it would return an error, something obtuse and wrong like maybe, "your forward and reverse DNS records do not match" even though they absolutely did and hadn't changed in years.
Outlook/Hotmail was far better and would always return a sane, yet sometimes inscrutable, error that bore some resemblance to reality. It still took them a week to get me off the hit list that I got dropped on for no discernible reason.
> Or they decide to mark your mail is "suspicious" and it goes into the recipient's Spam folder.
Gmail was the ABSOLUTE FUCKING WORST about this. My spouse's e-mail to Etsy customers who used Gmail--not spam, not newsletters, person-to-person e-mails INITIATED BY THE CUSTOMER--would work for the first couple of e-mails in a thread and then start going into spam. Or, even more fun, friends with whom either of us had e-mailed would send us a mail and then we'd reply and nope, into spam.
Gmail would do this for WEEKS and no amount of poking or begging or pleading would make it stop. And then it spontaneously would. So maybe the begging did work but was just delayed. I dunno. I tried signing up for Gmail's "Postmaster Tools" but no data was ever reported because "your mail volume is too low."
The ultimate ignominy? The final humiliation that sent me to Fastmail? I finally, through a couple of professional contacts, got to actually e-mail (via Gmail, of course, because they were blocking my personal domain again) with a real, live Gmail SMTP admin to ask why my latest round of e-mails were going into the bit bucket. Know what he said, even though my domain has been hosted on a server in the same colo facility with the same IP addresses for a decade? Something along the lines of "your domain reputation is coming back as too new."
> no data was ever reported because "your mail volume is too low."
That is fucking shameful indeed. Domains with tiny volume need postmaster tools the most…
But it seems like having a little conversation from your domain with a gmail address helps. Just send a mail to your google account, click not spam, reply, do a couple more back and forth replies.
Yes, dude, I feel you. I was there. I got the same shit. A few more examples of their wonderful software:
- GMail sometimes accepted mail from IPv4 but not IPv6.
- I rebuilt the VM running postfix, got a new IP address
and then mail was blocked due to "IP address is known
as dynamic end-user"
Worst of all, the mail does not always bounce... and when it does, this is no one to email and ask. What are mail admins supposed to do?
We chatted about it for a little more but he wasn't able to figure it out either, so I gave up and moved the three active users I had over to FastMail that evening. A handful of semi-used mailboxes are still on the other setup that I still keep trying to make work but not nearly with the same force and gusto as I used before.
Actually one fun way of doing it is to run your own email setup using one of the big ESPs as a "proxy". You can do it for both inbound and outbound.
You can have a domain registered with Gmail or Rackspace Mail or whatever, and use the SMTP credentials they give you for outbound, while keeping MX records point to your own SMTP for inbound. Or you can let them process the inbound as well (fighting spam for you) and you just forward everything (or sweep via IMAP) and put it into your own Dovecot.
This way you control what's important to you (email storage/backups/etc) but outsource delivery and spam filtering.
But then they can already read everything you send and receive. At that point it's better to just use IMAP if you want to have all the e-mails local and backupped...
> It's good to know that your broker and doctor's mail is delivered directly to your server and not Google.
That's only if your broker and doctor are not using Gmail or G Suite, otherwise your email (with them) is likely to be read by Alphabet regardless of your server setup.
My experience was quite the opposite (granted I'm an ops guy these days anyhow). I've been self-hosting mail for ages with a postfix + dovecot + cyrus + spamassassin + opendkim + dkimproxy + lets encrypt stack. Everything just works, although I'm going to move it into a jail sooner than later so I can rebuild it more easily.
The biggest pain is no longer having a static IP or reverse DNS (thanks Sonic!). Having reverse DNS setup and an IP address that doesn't map to a pool of dynamic IPs are the big things that will keep you off of spam lists.
Coming from sendmail I think postfix and dovecot have been a breath of fresh air. The defaults generally just work. Getting inbound (so spamassassin can check) and outbound dkim to work were the trickiest part. Aside from regular OS updates it's largely hands off.
> So you call it easy to configure together 7 different systems just to get a working mail server?
Yes, with the qualification that I brain farted on cyrus (not using that for mail) and you can just as easily skip the two dkim packages (which were a bit tedious to figure out).
Postfix works out of the box. Diffing the stock main.cf with mine the differences are: new additions that never made it into my main.cf, a few lines to enforce stronger TLS ciphers (optional), a manual blacklist (optional), virtual domain stuff (also optional), milter lines for dkim and spf (optional), a line to deliver to procmail instead of directly to a mailbox (optional), and a line to enable mail relay access to authenticated users (also optional). You could install postfix and have a good experience out of the box.
Dovecot: a line to enable verbose process names (optional), a stanza to enable the SASL backend (optional), a line to specify which keys to use for TLS (mandatory), a few lines to up per-user limits (optional). Setting up SSL is going to be the hardest part, and even that shouldn't be too bad.
Procmail: A couple lines to pipe everything to spamassassin and deliver to a mailbox or a spam folder. This may be considered tricky by some but procmail is ancient and, in my experience, examples abound.
Spamassassin: No changes. In fact I think I've ripped out most of my custom config except for some whitelist stuff.
Four packages and you'll have a working SMTP and IMAP server. No, I don't think that's difficult. Wrangling an OpenLDAP install, sure that's difficult. This? No. Scaling it out to hundreds or thousands of users is trickier, but not something the typical home user would need to bother with.
For everything above there's actually quite good documentation. What's difficult is figuring out which packages to install in the first place and deviating from the stock configurations for the little extras you may want, but even then the big challenge is in taking the time to read the documentation and figuring out which features you want. Maintenance is trivial, although I probably should update my postfix config to get it in line with the current config syntax.
Not Cyrus IMAP, but their SASL library. Actually, I just looked. Cyrus-sasl was pulled in as a dependency for mutt. I'm using Dovecot for postfix's SASL backend these days. This allows relaying mail for authenticated users only.
Dovecot uses PAM for authentication, and so any user with an account can log in and access their own mailbox and send mail out. The biggest downside is that because I'm deviating from the default FreeBSD configuration for postfix I'm on my own in terms of updates.
I should also clarify the above statement about moving it all into a jail -- I mean a jail managed by (in this case) ansible. Yes, if you just slap something together it will be brittle no matter what stack you choose. Ansible + Jenkins + Jails make it much easier to manage changes.
Edit: Oh yeah, and I've thrown procmail in there for good measure.
No. Let's Encrypt was a fairly new addition. Previously I was using self-signed certs that I'd install manually in the client.
But I've been self-hosting mail services for over a decade at this point. It's seen a variety of permutations including some with Sendmail, some with postgres backed lookup tables for virtual hosting, and so-on.
I recently moved off of Gmail and now run my own personal/family E-mail server, and I have to say so far it has been pretty pain free. I probably just haven't been doing it for long enough to run into trouble yet. Bog standard Debian install, Exim4 + dovecot + SpamAssassin. TLS, SPF, DKIM, and other acronyms all set up and working. The initial setup was a few days config file tweaking (why don't these things come working and secured out of the box?), but since then no issues--knock on wood. My only complaint is the spam filtering so far is not as good as Gmail's was, but I probably just need to better configure things. Bookmarking this guide for later!
Postfix has a very clear separation of concerns and least privledge in the use of lots of helper daemons. Exim4 doesn't not have this architecture and is less secure because of it.
Heh, I have had the exact opposite experience. My mail setup has been stable and reliable for years. (I'm a little bit afraid to report this because one possible explanation is that I simply have not been on any serious hacker's radar screen.) I'm not doing anything fancy: just postfix+dovecot, gray listing using a custom milter I wrote in Python, and Spam Sieve running client-side on my Mac.
I also am using opensmtpd, and have been for years. Even setting up starttls encryption and a DKIM proxy were simple. I have never had an issue with accidentally setting it up as an open relay or anything else that would place me on a blacklist.
Absolutely horrible – I tried to setup Postfix for my friends to hide Gmail behind their organization's domain.
Configurations were incredibly unfriendly and tedious. Despite all anti-spam measures, Microsoft was blocking their IP according to some private spam blacklists. Then they couldn't send emails to some ancient government server in Guatemala because of encoding incompatibilities.
I ended up moving them to the free MailGun plan. It took me 2 hours instead of 2 days.
I have to say, I've been running my own mail server without a single problem for years. OK, the way I do it is a stretch of the definition of "running my own web server" but I have full control.
What do I do?
Get a nice VPS from a company like GoDaddy. Setup whatever domains you need. Setup as many email accounts as needed. And off you go. No problem. I don't even have to think about email.
I thought about rolling my own on one of our Linode servers but every time I compare the no-brainer of doing the above to what it would take to run this ourselves on Linodes I can't justify the pain and aggravation.
What I don't like about the Gmail approach (other than it is Gmail and I do not trust Google to not shut down all of our accounts for some stupid reason) is the cost. I can spend a few bucks a month on a VPS and have a hundred email addresses. The same on Gmail would cost significantly more and you would be under their irrational thumb.
A few years ago I looked into running Zimbra on Linode. Back then it was so resource hungry it just didn't make any sense. I wonder if it has gotten any better over time? I really like the concept.
My use case is different, as I'm a Windows guy, but this is what I've got:
1. Custom MTA that handles incoming SMTP.
2. SpamAssassin that is called by the above MTA which flags email as spam/not-spam.
3. Custom MTA forwards not-spam to the users inbox.
4. Custom MTA forwards SPAM to postmaster (this bit needs significant re-working)
5. Main Mailserver is Microsoft Exchange run On-Premises.
6. All Mail clients are Outlook 2016, or Android Phones.
The reason I had to write my own MTA was so that I could leverage a 3rd party anti-spam solution, as Exchange doesn't come with one, and add-ons for Exchange are quite expensive.
Running your own mailserver is a complete nightmare. If I knew it would end up this complex 20+ years on, I'd have never done it in the first place.
Your custom MTA is impressive, but if you want to reduce the amount of custom code you need to maintain (and don't mind one third party service in the mix), you could use something like SpamHero. It's reasonably effective and fairly cheap. Postini was even better, before Google bought them and bastardized the product.
See, for me running my own mailserver removes one of the biggest weaknesses with third party mail servers: the helpdesk. How many mail exploits are simple social engineering attacks against the service provider? Probably more than half.
Of course I still have to worry about attacks at my domain provider level, but at least it removes one weak point. I'm waiting for the day that someone pretended to be me calls me to get me to reset my mail account password.
There's no reason it _has_ to be like this. I feel like it became common wisdom that you just shouldn't run a mail server too early and the cool kids all gave up working on mail servers. The author is right though about how terrifying it is that we've ceded running email to an advertising company
All the things that make running your own server hard are software problems, the protocol itself lends itself very well to running your own. We just need someone to build the [Caddy](https://caddyserver.com/) of mail servers. You have the right idea with your Haskell server, though I think in this day a Go project might be more successful.
The "Caddy of mail servers" should be a plugin to Caddy, since server types are pluggable. Caddy's HTTP, DNS, and TCP+UDP servers are all just plugins.
I typically don't recommend someone running their own mail server, but if they do, I wish there'd be a good one they can use, and that one would definitely be a Caddy plugin.
I'm entertaining the idea of running my own SMTP server for Spamnesty (https://spa.mnesty.com/) so I can at least cut down on some of the Mailgun invoice items. Would you still not recommend this if I were just sending mail to spammers?
Same experience here. Previously I maintained a Merak/IceWarp mail server. The "idea" was great, they provided a nice interface for a small organization (15ish people). But, it became an uphill battle very quickly.
I've been running postfix+dovecot for years on a virtual server and have never not once had to do anything to administer it once I got it up and running.
Edit: it's been so long that i forgot it was in fact postfix.
I wonder if cryptocurrencies could be used as an anti-spam prevention measure. In order to send you an email, maybe I have to send you 2 cents worth of cryptocurrency.
Everything old is new again, the bitcoin proof of work mining began life as a way to require people sending email to burn a second or two of cpu time see: http://www.hashcash.org/
Postfix, dovecot, fail2ban, spamassassin. I don't have a secondary; I just keep up to date image snapshots and regularly backup the disks. It's only for my personal communications so I try to keep things as simple as I can.
I run my own mail server, for several years now. A Mac Mini on Mac OS Server on a static IP address. It's been great, literally zero maintenance, but it completely hides everything about the underlying architecture.
> Before we dive in, an important caveat: You will become a sysadmin of your own mail server.
This for me is the deal breaker. I say this as a sysadmin.
It's all well and good running a single server but if that shits the bed then you have to deal with it immediately.
So to run something that doesn't require 24/7 support I now have to run a cluster of servers.
I'll also need something to manage those servers like salt/ansible/puppet
I also need to deal with being my IP blacklisted because of a previous owner, or just entire domains now delivering my email because they don't like the fact i'm not using $email provider
Then there is spam filtering and the constant battle that is.
Or I could just pay someone a couple of bucks a month to worry about all that shit and not worry about it.
I've been running my own mail servers since the mid-90s, and always only a single server. SMTP keeps retrying, so you can be offline. The transport layer was designed to deliver mail to servers even if those servers only dialled in to one of the "internets" once per day, so it's fine to have some downtime.
It's totally up to you how you prioritise getting a server back up again, and frankly if the message can't handle at least a few hours delay then email probably isn't the right medium for it.
> frankly if the message can't handle at least a few hours delay then email probably isn't the right medium for it.
This may have been the case in the mid-90s, but it is certainly not the case today. I frequently receive emails that require immediate attention, whether from my employer, my bank, or any number of other notifications. Normal email users expect emails to be delivered in seconds, not hours.
Of course not, like most things email is best-effort. But it's hard to blame users when for the last 10 years we've been able to send/receive email in <10s.
I feel sorry for you if you base everything on user expectations... I sure hope you do not work in professional IT if that is the case because if you do you must be one stress out person...
Most email on the Internet is delivered all the way from the original sender to the end recipient in seconds. Many user experiences require this, such as password reset interactions or email address verification upon new account creation.
When email isn't nearly instantaneous, it's definitely upsetting to users and a violation of the norm. Email is only not instantaneous in well-managed systems when one of the parties is experiencing an operational problem of some kind.
There are exceptions to this, such as large scale marketing systems that deliberately send at a slow rate to avoid overwhelming recipients. Once deciding to actually send a message, though -- if not rate-limited by the receiver -- messages tend to arrive in the inbox in moments.
Email isn't that complicated of a protocol, when you get down to the mechanics of email delivery. Loading a web page is orders of magnitude more complicated than sending an email; there's no reason why sending an email would need to take longer.
This is actually frustrating, e.g. with a large power outage when you expect an important message. I run my mailhost (single-node) since 2005 and I thought that I need HA more than once or twice.
Sadly, I haven't figured out a proper fully-redundant solution. Syncing mailboxes with e.g. Syncthing is way too fragile. Recently, I wanted to experiment with DBMail + CockroachDB cluster but haven't yet found time for that.
Never found any MRA that uses document-oriented database like RethinkDB; and GlusterFS is way too sensitive to high latencies - so DBMail+CRDB looks like the only readily accessible option sans of trying more "raw" programmable solutions like Salmon or Haraka and writing own storage backend.
Haven't seen this project before - and I've tried to search for mail servers using document-oriented DBs, although not sure if I've searched specifically for MongoDB. Will definitely take a look.
I guess the project is too new and it is hard to stand out amongst other players that have been around for 20 years by now. Even though not having been around so long has advantages as well, for example it allows skipping non-needed features and adding new stuff like full unicode support – I host андрис@уайлддак.орг on my WildDuck test instance (even though I can only communicate with Gmail using this address, other servers either refuse my messages or do not send to such address).
Dovecot has an excellent implementation of 2-way synchronization of mailboxes: run Dovecot on 2 different servers in different physical locations, set up MX records and a SMTP software on both servers and you have your HA solution.
keep your DNS up always - so have a backup if this is running DNS - many mailing list programs and other things will drop a mail immediately if the destination cant be resolved..
You need to know your DNS well, you need to know your OS basics well (when's the last time you understood what S.M.A.R.T statistics really mean?), you need to know your monitoring well, you need to know your backups are up to date and functioning. Do you have some sort of alerting set up so you will even know if anything went wrong? Oh your are the collateral damage of a DDOS to the datacenter and the datacenter has disconnected your server, and they sent you a mail about it ? To your mailserver ? You have a few other users, and now they would like to use a webmail system? Let's get to it! Whoa, viruses, let's install amavisd to filter those, let's see where is the documentation? Ah. Just the sourcecode ? Very well!
It's really more of a second job.
But it's ever so much fun once you have a decent grasp on all these things
I run my own mail server, mostly for fun and to learn how it all works, but so far, after a few teething problems, it works pretty well. I don't have any critical services registered with addresses from this server; I use it for random websites (like Hacker News newsletters), cron job outputs on my server, Stack Exchange, etc., and as a way to back up my Gmail account remotely. The great thing is that I can dish out unique email address to each service, and have them all end up at the same place, so I know who sold my data when I get spam.
If the server falls over during the night, then meh... I'll probably fix it reasonably quickly if I'm not on a beach somewhere, and the sending server is supposed to try again anyway.
«The great thing is that I can dish out unique email address»
That's the number one reason I would like to run my own server. (And, no, the Gmail trick of adding "+foo" to the username doesn't satisfy me because spammers know this trick and can strip it.)
Couldn't you just setup a domain with an email hosting provider and then turn on catch-all for that domain? For me I can just use any email @my-domain and it will be forwarded to my actual email address.
Yes, you can do that. But if you run your own server you can configure more complex filters to handle situations where mail is flagged - discard loudly (telling sender), discard silently, move to special folder if unsure of spamminess, let through.
Then you start getting an amazing number of emails to alice@yourdomain.com, bob@yourdomain.com, chris@yourdomain.com, dan@yourdomain.com, ceo@yourdomain.com, sales@yourdomain.com, etc.
Most decent ISPs let you create email forwarders. This is exactly what I use when signing up for a new service.
Quickly add a new forwarder and feed the service that email address instead of my normal address. When they turn out to be spammy, the forwarder gets removed.
Do you mean ESP or ISP? Relying on an ISP email forwarder doesn't work because if I move and cancel my service, I lose all these forwarders.
What ESP do you recommend for unlimited (or large numbers of) aliases indistinguishable from individual email addresses (such as ycombinator4853@foo.com facebook843983@foo.com but not john+ycombinator@gmail.com john+facebook@gmail.com)?
Edit: fastmail's standard $50/yr plan offers 600 aliases. This might be good enough. I've created online accounts at 371 different services over the last 20 years.
It's all well and good running a single server but if that shits the bed then you have to deal with it immediately.
Well, that depends on your needs. SMTP will keep retrying, so you won't lose any emails, you just might not be able to access them immediately. For my personal email, this is perfectly OK. Not that it ever happened, in my six years of running Exim + Dovecot, but that may have just been luck.
24-72 hours is common from my experience, but that likely varies widely and depends on volume and resources.
If you're running this in the cloud and properly automated everything, and its not a billing issue, responding and fixing things could be as simple as a few hours.
If you're running your own physical machine, repair times for hardware issues can be days.
If you're on vacation for a week or longer, and are making a concerted effort to step away from technology and lines of communication, or worse hit by a small bus and in the hospital for any length of time, you may not be aware of issues for some time.
To be fair, the post is about running your own email server. But yes, if you're running one for a business or other group of people, things become more complicated and demanding.
I started to go down the running my own mail server road and started to worry about every one of these issues. I decided that my time is worth more than the cost of something like FastMail.
Yeah - that's a very important point. Part of what you're paying for is expertise in a fairly complex and changing field, and with that in mind the price FastMail charges seems to be a bargain.
Off the top of my head, DKIM and SPF are somewhat new, using TLS for server-to-server communication is somewhat new and not fully deployed, etc. Mail server operators need to keep track of new developments and update their software and infrastructure to account for them.
There is no possible way that the current state of email is how it will be forever.
According to Wikipedia DKIM was conceived in 2004 and SPF was standardized in 2005 (although designs date a few years before that). I wouldn't call them particularly new (it's almost 14 years now). Even DMARC was here in 2010, that's 8 years.
One recent thing is MTA STS but the progress is slow on this front (from my POV).
Yeah, it's still underway at the IETF. Last session we came close to finialising most of the documents in the UTA working group, which includes MTA STS.
Meanwhile, there's ARC which is still in early stages of deployment, and hopes to fix the indirect mailflow problems of DKIM.
(and of course there's JMAP on the client side, which is super exciting for those of us working on it)
I've seen some providers like Gmail already using MTA STS records and subdomains, but they don't read the records of my domains yet. Still it's an interesting solution.
I'm disappointed in the mailing lists software and configuration "in the wild". Refusing to just change From address and put the original From into Reply To and breaking DMARC (that I've seen some operators just claim "is badly designed") is a very sad state of the world.
I've read JMAP spec but was not pleased with the RPC style design. I've seen "why isn't it REST" and the FAQ says REST is not mobile friendly. Still, reading all these Foo/get methods looks bad to my eyes (sorry, personal preference) an I don't see how it could improve caching (as the FAQ claims). I like the idea of replacing IMAP with something modern though.
Having used a JMAP-style API for years, it works really nicely with batching in a way that other APIs I've worked with don't. Of course, as one of the original designers of JMAP, I'm bound to like it.
It would be quite easy to write a REST mapping for people who like more chatty protocols of course.
I'm not a giant fan of Foo/get either - it's a very recent change from the previous getFoos|setFoos|getFooUpdates naming, and there are pros and cons of the change. I try not to interfere too much with stylistic details like that though.
I'm literally RIGHT NOW writing up an email to the IETF EXTRA mailing list, following up from the email I sent as co-chair to the IETF JMAP mailing list about conflicts for our session in London at IETF101 in March.
We've been working inside the IETF on JMAP since Chicago (IETF98) in March last year.
We're keeping jmap.io up-to-date with the latest spec (including the calendar and contacts bits - calendar is partially underway through ietf calext - contacts are not yet on standards track)
Honestly, when Windows 9x would crash, it was because something was doing something wrong. Drivers and legacy mode programs written and running under the no-longer-true assumption that they're the only program running on the computer will absolutely wreck your day when they poke in places they don't belong and do things they shouldn't. And if you got rid of those misbehaving programs, Win9x got a lot more stable.
I used to have crashes all the time. Then one day I got the bright idea to actually read and understand the error code when the computer crashed. I did a few web searches to demystify the error code and its cause, and ripped out the software firewall I was running. And that was the end of my bluescreen troubles.
Now, if you want to argue about poorly designed tools and environments that make it all too easy to do the wrong thing, then I think that would be a valid and productive argument.
I was commenting more on dismissing the issues of others because you have not personally experienced them. There could be numerous examples used in place of the BSOD.
> if that shits the bed then you have to deal with it immediately
If it's just for your own use, no you don't. If a mail server is down, other mail servers trying to send mail to it will re-try for some time.
Just this week I was remotely logged in and screwed the iptables making the machine catatonic to the outside world. Came home in the evening and didn't even bother fixing it until after dinner and various other activities. Like, "oh yeah, before I sleep, I should fix that mail server ...".
Or, set up a backup MX. For a number of years a friend of mine and I acted as peers for each other. When I was offline for several weeks, every bit of mail I missed got redelivered. Eventually he got tired of running a mail server of his own and I just set up a clone instance on a cheap VM slice, and things Just Work.
Yes, servers are supposed to resend when they can't connect, but in practice for those periods where the backup MX was down, I'd still lose some messages I know I should have received.
As a sysadmin I completely agree. There are a few things I don't run personally because I want them to "always" work, and I can't drop my professional responsibilities to fix my personal email setup during working hours, but at the same time do not want my personal email down for long periods of time or at all.
Reliability and delivery consistency is more valuable then pretending your personal email server actually protects you from dragnet surveillance (in my opinion). The truth is every other smtp server you communicate with could be compromised, and if what you're sending is that sensitive, encrypt it at rest, not just in transmission.
About 2 years ago I had a customer that required their own private mail server. I set up Mail-in-a-Box on a $5 Digital Ocean droplet, and they've been happy with it ever since.
The integration with Let's Encrypt and a relatively smooth upgrade process has made it one of the more enjoyable services I manage. I would highly recommend it.
Came here to suggest MiaB or co-sign. I've been curious about running my own mail server from scratch since you can't run MiaB on the latest Ubuntu flavor.
Another happy mailinabox user here. I love have total control over where my data goes. If you use third-party no-installation-required server monitoring (with Pingdom or something), you can get mobile notifications if your server ever goes down, which for me has only ever happened due to kernel upgrades.
I also have no trouble sending email to Gmail users after carefully following all the steps in the manual! The only sticky point I had was that it required fancy DNS records that Namecheap doesn't support -- I think if I were doing it all over again I'd transfer domains over to a better nameserver.
> Please note that the goal of this project is to provide a simple, turn-key solution. There are basically no configuration options and you can’t tweak the machine’s configuration files after installation.
Can't configure post-install? At all? This seems like it would be a showstopper for some.
I'm surprised by how many people think mail is an impossibly hard problem. There some things about mail that are stupid; the core protocols are old and it shows. But, once things are working, you can leave it alone for years (aside from regular software updates) without trouble. I have. I've never not run my own mail servers (and often for several other people/companies/projects, as well) in the past 20 years. I used to consider myself an expert on the topic, but it's required so little of my time/thought in the past several years that I've forgotten most of that expertise.
I think what I'm trying to say is that it's not harder than it used to be (though some problems, like spam and security requirements for safety, have gotten worse in absolute terms). With modern tools and packaging on modern Linux distributions, you can be up and running pretty quickly. My company ships a turn-key solution as part of Virtualmin, but you can build something similar without that in an afternoon or two if you're reasonably Linux-savvy and have some notion of how all the pieces fit together (maybe a couple extra afternoons if you don't know the basics; DKIM and SPF can be tricky, since you also have to know or learn you some DNS).
It's harder than a web server or DNS server, but not something you should flee in terror from. Admittedly, it's gotten cheaper in recent years to outsource it...and with microservice-based architectures, maybe it makes more sense to have some other API than SMTP (though SMTP is very easy to use from every language I've ever worked in). But, there are problems and complexities with outsourcing, as well.
I seriously always failed getting over that first hurdle every time I tried. I didn't try this one yet, but most guides deviate from whatever my setup will be by a little, and that then increases to a lot of investigation work with pretty unclear config values and not a lot of debugging tools.
Have I invested enough energy? Apparently not. But I'm over the weekend-sized amount by a multipler bigger than three.
I've done this before. It's a pain in the ass. You have to run several different programs each with hundreds of configuration options, make sure the different programs can communicate with each other properly, make sure your email doesn't get flagged as spam, and be your own sysadmin. All for marginal benefit.
It would be great if someone wrote a program handling all of this that could be deployed as a single binary with secure defaults and limited configurability, but I don't see that happening any time soon. Email providers are good enough for almost everyone and the people who are good enough programmers to make sense of all the different protocols they'll have to deal with and get everything to interoperate nicely probably have other things to work on that people will actually pay for.
Edit: That said, this guide does look like a great resource for someone who is interested in doing this. It's interesting to learn how email works and if I had this guide when I started out I'd have saved a ton of time.
I started working on this a while back and got as far as writing a collection of scripts that spins up a mail server from scratch and does all the configuration. It's not complete (doesn't include spam filtering or search) and it uses Common Lisp to drive the process, which is the main reason I haven't published it. I didn't think there would be much interest in something that obscure. But if there is interest I'd be happy to clean up the code and put it on github.
There seem to be at least a couple of projects on GitHub on this topic. Eg. https://github.com/aimxhaisse/docker-mailz Now how many of them actually work is a different question altogether.
I have been running a Zimbra mail server for years and it's been great. Everything is included, configuration is quick and easy and there's lots of documentation online. The only negatives are that upgrading is a pain (I have to spin up a whole new server and migrate everything with Zextras Migration Suite) and the whole thing takes up quite some resources.
I've run my own mail server ever since I got something resembling broadband internet in 1996. Back then spam was non-existent, Sendmail was the emperor without clothes about to be dethroned and I hacked sendmail.cf without needing to look at the the bible [1].
I've never regretted running my own server, nor have I ever contemplated moving to a hosted solution. Spam is not a problem either, Spamassassin in combination with a greylist make for a nearly spam-free experience. The whole setup has been migrated from the original Pentium-66 via an aBit-BP6 (SMP for the masses [2], retired in 2009) to the current Intel SS-4200 (upgraded to a dual-core Pentium but still limited to 2GB). In practice a Raspberry Pi would be enough to run a viable mail server so even this rather anaemic setup does its job without breaking a sweat.
The whole setup consists of Debian (Sid) running Exim through a smarthost, feeding through Spamassassin + greylistd into Dovecot. Apart from some auto-manual intervention to cope with Microsoft/Google/... not coping with the greylisting and thus needing whitelisting it more or less just works. In other words, just go ahead and run your own server.
Initial setup took a few days of experimenting, that was back in the days of Sendmail. After that is ran fine for a while until spam starting becoming a problem. I eventually implemented a spam filter with Spamassasim with a nightly cron job to train it with spam which made it past the filter. After a few years I added greylisting to the mix which ended up complicating matters a bit due to some common mail hosts (Google and Microsoft being the biggest culprits) breaking the concept of greylisting by using different mail hosts for resend attempts. I made a script to add all SPF-allowed IP ranges for a domain to the whitelist which solved that problem.
Total time spent on the mail infrastructure over these 22 years is negligible, probably ~8 hours per year.
Not the OP, but I've been running my own email server since about the same time-frame (and it started life on a dual CPU Pentium 100Mhz box). Total time investment since circa 1998: estimated maybe 100-200 hours total (note, over an approximately 20 year span), and most of that was spent during the inevitable of reinstall new OS on current or newer box, then move config over portions of the process.
I've also always been on Postfix, which has always been way more secure than sendmail ever was at the time. And Postfix is also much easier to configure for the basic case of "send/receive email for a personal domain".
This is an interesting write-up, but doing it like this files like way too much work.
I have been using mailcow[0] for years now and it does all of this for me and works great. The UI is beautiful and intuitive. And setting up mailcow literally takes a few of minutes, since the project was ported to docker. Highly recommend it.
How to run your own mail server (for experts): Don't.
I say this as someone who has run my own mail server for 20+ years.
Now, if you absolutely have to get off Google, and the other available hosted options don't work for you, then this article looks like a good start. I'd add roundcube for web access and letsencrypt for SSL.
One thing you'll never really get, in my experience, is good spam handling. The big providers just have so much more data to work with to prevent spam.
> One thing you'll never really get, in my experience, is good spam handling. The big providers just have so much more data to work with to prevent spam.
I've found the exact opposite. Big providers have to run more lenient rules than I'm willing to use. I can use SORBS and Spamhaus to bounce mail. Sure, both SORBS and SH suggest never ever doing this, but when you self-host you can check your logs, and over the course of a few years I've never had a false positive.
Now that I'm reliant on my ISP for inbound mail I don't have that low pass filter and the amount of spam I get is staggering.
Google is the worst offender in my experience. They don't take spam reports and they don't do a good job of filtering users' inboxes. Sure, there's a SORBS list to bounce mail from hosts that have bogus WHOIS info... but then you block Google... and everyone uses Google... so you just get to put up with all the spam they send out. From the other end, my gmail inbox is nothing but spam these days (the report spam link is generally useless).
> One thing you'll never really get, in my experience, is good spam handling.
Not in my experience. I'm in general very happy with the results of SpamAssassin. Occasionally sth. slips through but when I compare it to my gmail account it's not worse, maybe even better. The only thing I use in addition is postscreen [1] (pregreet test only) to stop the dumbest category of spam bots.
I used SpamAssassin, greylisting, SPF, a variety of RBLs, and hashcash, but never got anywhere near what my gmail experience has been. I literally almost never get spam to my main mailbox in gmail, I would get a dozen or more a day on my own server.
I have my opensmtpd setup relay incoming mail to a Citadel IMAP/webmail server. It works a treat. I'm paranoid, so I don't allow external IMAP, force SSL on the webmail, GeoIP block it to only 3 countries and have a simple auth prompt on it as well just in case the landing page has an exploit.
> One thing you'll never really get, in my experience, is good spam handling.
Not quite the same calibre, but really, SpamAssassin+RBLs tuned a smidge too high plus a user whitelist is pretty darn good. Scripting something that auto-adds to whitelist based on sent mail would do even better (but is exactly the kind of 'fun' that people don't have to deal with using 3rd party service). I've heard you can do better with dspam.
But the spam handling you can actually buy. A spamexperts (to name just one alternative) inbound filter account for 1 domain costs about 1.5 dollar per month for an unlimited number of email boxes.
> on FreeBSD using Postfix, Dovecot, Rspamd, and LDAP.
That seems like an awful amount of work / overkill. This is not a good introduction for a normal geek, and die-hard open source fans will figure it out without too much trouble anyway.
Postfix is not the best choice for novices, FreeBSD is not the most well-supported/documented system just because it's not as popular as some others, and a directory service shouldn't be necessary.
We had to setup Exim, Postfix or Sendmail for school. Sendmail was universally hated the most, Postfix came in second, and Exim was... well, not exactly logical or easy, but the best of the three mainstream MTAs.
I'm running hMailServer at home. Windows-only, unfortunately, but until I find a proper replacement, I'll just keep running it in a VM. Nothing else even comes close in admin-friendliness. It's just install and run, with either a local admin interface or a web interface (using PHP, so it runs anywhere).
I'll give you that in general, mail servers are not for novices, and they have a huge array of configuration options, but Postfix is by a wide margin the best piece of software that I've come across the last couple of years, and I've been using it for a long time because of that.
It's reliable, it works exactly as documented, it's up to date, it's documentation is excellent, and it's community helpful, if overly precise - but then that's exactly what makes it so good.
I've literally never come across any problem with it, that turned out to be problem of postfix.
I can't say that for any other piece of software that I know.
I invite you to look at hMailServer. I set this up when I had a reasonable clue about SMTP and only a vague understanding of POP3 or IMAP. Let alone how spam filters worked ("they look at words like viagra" was my understanding) or what mailing lists are.
It's really a Windows-like utility in the sense of next-next-next-finish. By now I despise most Windows Server wizards for not being flexible enough and prefer command line tools on GNU/Linux, but this mail server is one thing I just cannot find a good replacement for, and the interface provides everything I need (no need for command line or config file magic).
Postfix is popular, secure, flexible, well documented, and likely to be around in 10 years. The defaults are quite sane and it's battle tested on numerous large mail servers.
For a simple domain you only have to edit a few lines that numerous tutorials cover. Combining it with dovecot (similarly well designed, secure, popular, battle tested etc) makes for a robust setup.
Sure don't make the mistakes the OP made and allow relaying for the the entire internet. I can't imagine any sane tutorial has relaying enabled by default.
I've never heard of hMailServer, but hard to imagine it would be worth while and I wouldn't want to bet on it being around in 5-10 years. If editing a few config files and changing a few lines scares you then don't run a mail server. Otherwise pick something popular, well tested, and robust. Sure postfix+dovecot isn't the only solution, but I wouldn't pick something small/obscure/new either.
FreeBSD isn't that bad, but it is different than Linux and the community is smaller. There are some things that the Linux world does better (installing updates), but there are also some things that FreeBSD does better (ZFS, I prefer the split between base and 3rd-party software, etc).
The documentation for the base system, however, is very good. The man pages are complete and well-written, and the handbook[0] is a great source of documentation as well.
> FreeBSD isn't that bad, but it is different than Linux and the community is smaller.
Which was my point. I know it's better integrated and has better documentation (some coworkers were fans, plus what I read online about it), but I still think going with mainstream is the better choice when writing a general post like "how to run your own mail server".
Not that guides for "how to run a mail server on FreeBSD" shouldn't exist. Just that then it should be called that, and not be general advice to anyone who might be googling the general title. There's so much bad press for running your own mail server, while really it's not that hard with the right tools and explanations.
"just install and run" is not admin friendly though. At some point you want or need to tweek your stuff. I'd argue the most logically tweakable is the most admin friendly.
I started running my own email server around 2014. I am using every mitigation I can find, but I still get an unending torrent of spam that slips through.
If tweaking spam filters and deleting spam is your idea of fun, then run your own email server.
I've been running since 2001. There were some rough years in there with spam and big providers rejecting for no good reason. Now it has been years since I've had any delivery issues. As for spam, I get 2-3 a week that make it through, but I don't make any effort to tweak it.
Thanks for bringing up the painful memories of adding new filters each night. So glad those days are over for me.
For me, rspamd with corresponding exim-acls practically solved that issue since 02/2016. Spam is mostly rejected, otherwise tagged (and filtered with dovecot-sieve downstream into a review-box) or - if borderline - greylisted. Have had no false positives I'm aware of and only 1-2 false negatives per week slip through that cascade.
edit: Although some training (i.e. manually moving spam to IMAP-Junk) was necessary during the first few months.
I run exim on Debian stable. I use greylistd for greylisting and dovecot for imap. I backup emails daily by taking a btrfs snapshot and rsyncing it to my home server, which I have a regular backup cycle for. If you were really worried you could setup exim to forward every email to a backup server.
Failover is manual. I would create a new vm, install debian stable and run a script I've prepared a script that installs packages, copies config files and creates users. Downtime with email is not a big deal, servers will retrying sending for a few days. If you have users complaining to you it's obviously a bigger deal.
It could be worse : the spam filter you have no control over and no insight into what it is up to can vaporize that important new customer lead email that was going to make you $$$
I've run my own mail for about 15 years. Just recently stopped.There was nothing wrong with the process, it worked. But I thought about how much time I've spent on running my own domain and email over the last couple decades, and added up what it got me over those years... and the value just wasn't there.
I know the article is concerned with owning your own data, and I appreciate the point. But finding a mail provider that meets your needs is, IMO, a better way to spend your time than just saying "Gmail isn't good for me, so I'll do my own."
I use iRedMail and set up my own mail server for my small business. I am beginning to wish I hadn't, even though I've had literally 0 problems outside of a single reboot to fix an issue.
I dunno. I feel like I'm sitting on a timebomb. It's hosted on DigitalOcean and while it works great with RoundCube and gmail... eh. I dunno why I even did it in the first place.
I use iRedMail and have been for 3 years now and I love it. It's really just a collection of scripts and updates are just updates to the underlying packages. Everything is transparent if you want to know what's going on. It takes awhile to get used to all the moving pieces (if you care to even know them as it isn't necessary) but I've found their setup to work very smoothly and be very stable.
I'm also a developer, btw. This mail server is only for me so it's not a big deal but if I had to I'd probably be comfortable setting it up for a small business. I wouldn't recommend it for that still, though, since you are the support in that case.
Other than the feeling of 'sitting on a timebomb' are there any other specifics you can cite? Have you looked into contracting someone for a few hours a month to check/update things? I have considered doing what you have done, and your feedback has piqured my curiosity.
Not really. Just everyone smart I talk to says the same thing as everyone else in this thread: Don't do it. I'm not a sysadmin, I'm a developer who runs a company and doesn't even work in IT in my own company anymore. Bleh. So far so good, though....
> Getting off GMail is one of the best ways to take back your data in the face of dragnet surveillance.
This just isn't true. You can host your own mail server and GMail will probably still end up hosting a large fraction of the email you read and write, because the people you correspond with are still using GMail.
(In the same vein, you can refuse to have a Facebook account but Facebook probably has a dossier on you anyway. Enough people you know have dumped their contacts into Facebook that they already know your place in the social graph.)
You can and should work to reduce your footprint if that concerns you, but there are still systematic issues that make it hard to stay completely outside of there services. Mainly what OP mentioned with contact uploads.
Icing on the cake: Sprinkle a little Let's Encrypt in there to cover SSL. It doesn't take much to have a cron make a new cert and restart dovecot and postfix.
My setup is similar, but it uses MySQL instead of LDAP.
I love being able to make aliases and even better - deleting them when I'm done with them.
> Sprinkle a little Let's Encrypt in there to cover SSL.
Unfortunately most MTAs aren't configured to check the certificate chain, so they'll happily take any SSL cert they're handed and start chatting. MITM or downgrading is trivial.
There is an IETF draft ( MTA-STS ) from 2017 that should address this.
While I share your ECC preference, for today I see no reason to refuse free, reasonably secure LE support by default. ECDSA signing with LE's RSA intermediates is supported from Feb 2016, and full ECDSA cert chain will be added on July 2018[1].
Any reasonably modern x86_64 CPU can do more than 1000 RSA2048 signs (~ TLS handshakes) a second, per core. Performance considerations really aren't a good reason to not use RSA for TLS KEX.
There's no reason to not at least run mail-in-a-box if you consider yourself a technical person but have limited time. It's absurdly easy to set up and the maintainers / contributors do an awesome job.
Running your primary email on these big hosting companies is taking your privacy and pissing it right down the drain.
I believe OS X uses many of the same components under the hood. I have toyed with OS X Server.app (mostly just to get the necessary certificates to get IMAP push notifications working with iOS/Dovecot[1]) and it seems like a really solid choice.
Not sure how you go about colocating a Mac Mini in a datacenter though.
When you ask your ISP for a static-IP address, your service goes from Residential to a more expensive Business account, and they open up outgoing mail ports. (At least for Verizon FIOS that I'm on, pretty sure Comcast is same as well.)
Cloud IP addresses are probably more problematic for spam blocklists.
Glad it works for you. My comcast connection is far too unreliable to ever rely on for email!
Curious though, can you also set your own reverse DNS entry for your static ip? I’ve found many mail servers will reject your mail if rDNS records don’t match up.
I'm also running a server on a Comcast business service with a static IP. My service has been really stable. This must vary on area. Also it seems if you're on business class, the techs take you more seriously. When I did have some stability problems the tech they sent out did a few tests, then went to the entire first floor (5 units) of the condo building with big sticky notes telling them to schedule a service appointment or have their service blocked.
Getting reverse DNS was easy. Following the directions I found via Google I went into the Comcast business service forums, read their pinned posts and sent a PM to the current service guy. He set the reverse DNS for me. It took about a day.
I've never had to worry about reverse-DNS, and I email thousands of fashion/media/tech companies around the world, big and small. Are there really servers that block based on reverse-DNS? I've never seen it.
It seems to be a flaw to have reverse-DNS block mail servers, since a single mail server might serve multiple domains. (My server serves email for multiple domains)
The services deprecated include davmail, dovecot and postfix. They say the changes are "to focus more on management of computers, devices, and storage."
For remote access I use two things: RoundCube webmail, and K-9 Mail on Android. For sending mail from K-9, I connect home, via authenticated SMTP which is on port 587, rather than 25.
I have developed a little web app called Tamarind for generating throw-away mail aliases.
I run some mailing lists which use GNU Mailman. For archiving them, I don't use that horrible pipermail, but rather a hacked version of Lurker. I patched Lurker to pass through HTML so that HTML mails end up rendered as HTML in the archive. The HTML has to be scrubbed, so I wrote a little scrubber for that:
First, I'd start with "Don't run your own email server unless you absolutely have to."
Second, if you must... I'd recommend Mail-in-a-Box. MUCH easier to setup / maintain than this one, at least from a cursory read. https://mailinabox.email
Running your own personal, non-commercial server may be just fine. But if you’re business is pumping out volumes of email, no amount of DMARC, DKIM, SPF, CFL participation and fighting to stay off IP blacklists will prevent you from eventually succumbing to the user that clicks the Junk button to delete their emails. This will tank your domain’s reputation and get your emails routed to Junk (or silently discarded) for all recipients at the big ESPs like Gmail, etc. SPAM scoring is outsourced to companies like Symantec, CloudMark and others. They are the mail-zapping, score-keeping monoculture of the email ecosphere. You basically need a large company who will take your word for it that you’re not SPAMming and interface with the filtering industry on your behalf to de-nerf your domain once the Junkie McJunkbuttons of the Internet screw your reputation over.
Yep. Basically the big email companies have outsourced their anti-spam filtering problem to the email service companies.
I used to run my own email server for around 15 years with minimal issues, but I gave up when my mail started disappearing into the hotmail/gmail/yahoo blackhole.
Customer Feedback Loop. When Junkie taps the Junk button you get notified by the ESP to delete them any mailing lists and never email that address again. They typically track this as a metric against your domain. And some ESPs, like Comcast won’t even sign you up to participate unless you’re big.
Yeah, it's learnable, it's doable, it really doesn't need to be all that hard.
And then ... after you've done every damned thing exactly by the book, and DKIM'ed the dickens out of your headers, killed the spam, policed yourself off the blacklists, etc. etc. - turns out you might as well not have bothered. The googles and the microsofts (the microsofts especially!) will one day drop your outgoing mail without the slightest notification, because the ip range, or because the full Moon, or just because they can.
I ran that show for nearly fifteen years, but threw in the towel last year, and handed over to Fastmail. With regret, although their service is first rate.
Email is not a succesful federated protocol these days. The monoliths effectively killed it off.
If you run your own mail server you can save a ton of work and time and get great defaults and features by using Sovereign from https://github.com/sovereign/sovereign
Love this, I spend many nights in the past fiddling with email.. and giving up. But I manged to get it to work one day and I learned a lot! I started with [0], but eventually found the mail-stack-delivery package in the Ubuntu repos, it sets up almost everything correctly out of the box and I ran with that. While Googling that package name I found this: [1], may also be nice.
I also learned that having it in your basement means trouble: Someone pulls the plug, your IP changes or ends up on a spam list. For some years I ran my own server on a DO droplet. It is very cost effective when you can make as much mailboxes as you want for family (+ unlimited aliases, addresses that deliver to both you and your wife, being able to email 500 mb to familie, etc). I still don't know down what sinkhole emails to my brother-in-law's outlook.com address went down. The literal response of MS at the time: We don't manage our own spam filter, try adding more text, make it look more real...
But man, the pain, the complexity, the reverse DNS, the startTLS, the SPF record, the DKIM records. It took me a long time to understand the difference between mail servers and MTAs and why there are different ports for them. Also, few providers in the Netherlands even allow you to use port 25, luckily mine did. Email is truly an old protocol that has been hacked up-to-date (more or less) and setting up your own mail server will make this very clear to you :)
I'd recommend it though, you'll learn a lot! But to be honest, I now pay 3 euros a months to a dutch email provider because email is too important and I didn't want to go through the pain again when 16.04 came out. I might still have a go at it in the future, there is something beautiful about running your own email server :)
One nice thing about the programs he chose is that their config options are fairly stable (can't vouch for Solr). That many moving pieces would be absolutely unmanageable if the options changed frequently. Been using a similar setup for years without difficulties.
Adding something like fail2ban into the mix wouldn't hurt.
If you're going to do this, first check that your VPS / ISP allow inbound traffic to port 25/tcp. AWS allows it upon special request. GCE doesn't. Don't know about the others. In the US, most residential ISPs block 25/tcp inbound.
The configuration of mail servers is so unbelievably grueling, it can almost reduce one to tears! I love having my email and data on my own server, but it is horrendously time consuming to get it up and running, and if something goes wrong, you could be looking at a weekend of work (often involving trial and error with poorly documented config formats). Sometimes I wish email could be replaced with something simpler but just as ubiquitous. Sigh
If youre running your own email services and wonder why outlook.com / hotmail.com are blocking you you can signup for this.
https://postmaster.live.com/snds
Its useful as quite often youll get blocked just for being in the same IP block as a spammy server. My experience to date is that once you notify them your IP isnt the culprit the block gets removed pretty quick.
Usually you are supposed to add a year to older articles :)
Curious if you looked into other mail server options? I mean Postfix (not Postgres) easily handles the load of a single user, but it is still rather hard to configure and modify.
I recently started to work with Haraka and even thought its ment for high traffic use cases i wont ever look back to Postfix (not Postgres).
Given my first real Postfix experience was a scaleable forwarding service that is supposed to support a variation of configs without breaking the config file.
Postfix is super easy, and well documented for normal use cases.
>perlgod: I've spent years tweaking my mail server setup (Postfix, Dovecot, RSPAMD, LDAP...) and did a full writeup a few months ago. I've used other guides online but found most of the rest lacking on details
Fastmail will let you do it for $50 per year if you pay upfront. Just not sure why anyone would run an email server other than extreme paranoia. It’s hardly a set it and forget it affair.
I ran my own mail server for years because I didn't trust Google with _all_ my data. Started colocated server for a while, then VPS, then cloud. Setting up an maintaining SPAM mechanisms on a low volume mail server is a nightmare.
Switched to Fastmail and haven't looked back. Their Android and Web client are great and they've opened their JMAP[0] interface.
Looking at that postfix main.cf gave me flashbacks and not in a good way. Google can have it. Or ProtonMail. Running my own mail server didn't pay enough ($0) for the complaints I got from its users (family). I'll focus on VoIP PBX and home automation until someone gobbles that up too.
Instead of running each component individually, I would recommend looking at something like Zimbra or another OSS mail package that handles a lot of this.
I hosted my own email for over 10 years and maintaining the bits are as painful if you don't have a plan in place.
A decent comparable for do-it-yourself hosting is the kind of luck a product like MDaemon provides - it decent job on windows of rolling all the features into a reasonably manageable server, as well as being quite affordable. I don't work for MDaemon, but tools like this make hosting email relatively trivial.
I've hosted my own mail server for a while. I've gone the Postfix/Dovecot route, I've used qmail, etc. My current setup is WebMin/VirtualMin. It works really well, and will run on the cheapest DO droplet.
During if you want to to from 1-2 spam a day to 1-2 a month you might want to block the garbage domains like click, link, party, top, webcam, xyz, stream etc. Probably worth enabling a DNS based block list.
So an apt get or two, 2-3 lines in a config file. Seems trivial to me, most every mail server HOWTO mentions them, should be just a cut/paste.
Sure 9-12 months from now it won't work as well, thus updating SA periodically, just like anything else internet facing.
Existing mail server solutions seemed so complicated and just plain wrong that I went and wrote my own mail server software from scratch https://wildduck.email/
This isn't bad. I ended up creating an Ansible role to do it for me, and it works great.
I thought about Fastmail, but its too expensive when you want to do more than one email address or domain. I run mine on a $5 server from DigitalOcean.
Every time a thread about running your own email server comes up I think "Oh yeah, I was going to set up FastMail for my domain"
The current pricing[0] seems to let you have 100 domains plus 600 aliases[1] for $5/mo.
I've got this thought that I will create aliases for every different thing I sign up for and use them to track who's selling my email address to who. I think that a domain plus aliases will do the trick and I think that what I want to do will fit in their limits.
My guess is that by "more than one email address" they meant more than one person/address. That's why I haven't switched, adding in accounts for my wife and kids makes it to much. It's to bad they discontinued the family account.
I run my own outbound mail server for my infra. I couldn't justify paying a 3rd party just to make my mail more deliverable. I understand the cat/mouse game of spam but that should not prevent the rest of the world from running thier own email services.
I also accept inbound mx for some of my personal domains, but I don't currently manage my own mailboxes, the mail gets aliased to my Gmail account.
If at some point I get fed up with this arrangement I can transparently change where the email ends up.
It's basically a single binary that has an SMTP server and webmail server. It works absolutely fine for me without much stress. I do hit a couple of bugs here and there, mainly on the mail parsing, but it's not big deal.
Regarding web mail: I am much happier with RainLoop [1] than I was with roundcube (better usability). If you are running a Nextcloud anyway, their mail app might be enough[2].
I appreciate the write-up! I always wanted to do a similar write-up for my own setup, but I never got around to it. (My setup uses slightly with different components: exim+dovecot+spamassassin+openldap.)
However, after reading this, the software developer in me feels like "These tutorials should not be necessary."
This should be a GitHub repo, trivial to fork and trivial to test out locally for anyone.
I've been running a mailcow install for the past year and it has worked remarkably well. Not sure if I'll stick with it when I move my server, but I've got no complaints.
I think when you setup that whole stuff yourself you are better prepared to fix any problems that might arise. I mean those are 'tutorials', sound like teaching ;-)
Honestly if you're lazy or want the support it might be nice to get the VPS cPanel license. It sets it up for you with full customization allowed. It also has spamassassin with RBL support in the interface. I think it's $17~ a month excluding VPS cost. You'll also get so many other features that can be disabled.
> Before we dive in, an important caveat: You will become a sysadmin of your own mail server.
I don't want to become a sysadmin of my own mail server, however I'd appreciate not having my personal data harvested by webmail providers (e.g. gmail, outlook.com). Using a Pi or dedicated server would make me a sysadmin.
Using AWS email seems like a decent middle ground which isn't too expensive ($0.10 per 1000 sent and $0.10 per 1000 emails received).
I have an old server that I need to migrate. It uses exim4 and pipes email to programs for certain addresses, and needs to send out automated email for forgotten passwords, etc. Is it worth moving to Postfix or something better? This is the part of the migration I look forward to the least.
Is there any modern webmail client that's secure and easy to set up? I know about Squirrel Mail and there was another one using the Horde framework, but I'd rather use a uwSGI/PSGI app that preferably doesn't need a mySQL database.
I run my own email server as well and the problem I see with it is that Google has all my email anyway -- probably 90% of the people I communicate with use gmail, so even if I don't, most of my sent mail is already in their system anyway.
Thanks for posting this. I'm currently running a similar setup and to me it's a horrible technical debt. Any ideas how to migrate multiple mailboxes for multiple domains to a managed solution?
I've recently moved to Fastmail and I'm blown away by the service. Everything feel well thought, well integrated, their web interface is fast, their mobile app is great, they respect standards... I could continue all day.
From reading their blog[0] you can tell that they are passionate about email.
It is a bit pricey for a personal email with a few accounts, but I'm happy to give them their well-earned money.
Isn't it possible to make all these changes in a server and make it available as a docker container or VM (with minimal customization required for end users?)
For sure! After years of running a personal mail server with a setup similar to what's described in the article, I moved to using a pre-built docker setup and haven't looked back: https://github.com/tomav/docker-mailserver
The easiest way to get this type of VM setup going is to start up the container on your mail host with all of the fun features (filters mostly) turned off, verify that the new mail container works as expected, then slowly start turning on features one by one so that if you happen to break something with a bad configuration you know how to roll back to a configuration that is functional.
I run my own mail server. Have done since Evolution mail was in BETA circa 2003 ish (time flies). Started with a Linux Mandrake based Postfix, Procmail, Dovecot, Fetchmail setup with SpamAssassin then moved onto a proper Debian setup with Exim, Courier, Procmail, SpamAssassin and then finally after all that faff I found Zimbra.
If you like email Zimbra's great - it's a fully baked mail server which you install on your Linux distro of choice and it goes off and installs all of the above for you. Everything is managed via a GUI and you have a great web interface and standards based IMAP, CalDAV support.
If you buy the paid version you can even get ActiveSync and Exchange Web Services for it.
Despite all that though, I would give anything to have all those countless hours I put into running my own mail server back. It is a colossal time sink. I can't even stress how much work it is, especially if you have anyone relying on the box for their primary mail account. It's no fun at all.
You're gonna be debugging Fetchmail for when you or your user's want POP3 accounts downloading mail locally. Procmail for filtering. SpamAssassin (gawd if ever there was something which consumed my life it's that software and its myriad libs and helpers), not to mention familiarize yourself with DNS MX records, SPF, DKIM etc. etc.
All of the above works surprisingly well and is fairly solid - until it isn't. When Google added DKIM/SPF protection and blacklisted servers which didn't was a fun weekend that I'd rather have spent with my family. When customer emails started bouncing because their IPs had hit an over zealous RBL list which Zimbra was using was a fun afternoon of debugging. When Zimbra decides to randomly let in 10-20 spam emails a day into my mailbox is another weekend project which I've yet to get round to. Thank god I'm self employed is all I can say because no employer would tolerate an employee putting the care and feeding required to maintain a personal email server!
Unless you want to nurture a career as a mail sysadmin seriously, don't host your own mail server.
Bottom line I'd recommend to anyone thinking about hosting one to either:
It took about three days to set up postfix and dovecot for multiple domains from a cold start, including certs and spam filtering.
Postfix supports a selection of block lists for spam, and I get no spam at all - although unfortunately my gf sends me email from a Yahoo account, and certain Y! servers get blocked, so that mail bounces.
She doesn't do it often enough for me to spend more time getting whitelisting to work. (Currently it doesn't - I don't know why.)
The hard part is getting a working config file for postfix, but there are tuts and examples online.
Like a lot of older FOSS code, postfix is basically an insane collection of every possible switch for every possible feature, dumped with no particular thought or care into a single config file, and written up - ditto - in a single help file.
You only need about 10% of it, but you won't know which 10% until you try.
The server gets regular hack/relay attacks from all over, but those get killed by Fail2Ban. I'm not exactly a high profile target, but unlike an old WP site I used to run - it was hacked in weeks - the servers seem to have survived for more than five years now.
Yahoo servers don't seem very well managed. Obviously they have lost a billion passwords or so. They also serve malware from their home page often (I've gotten chrome, opendns, and firefox warnings). The ads shown are often pretty ghetto (payday loans and the like).
In the interest of security I'd help your gf to a different provider.
One thing I'd like to see covered is sender dependent outgoing mail routing.
For example, suppose I have things set up so all outgoing mail from my home goes through my SMTP server. If I send an email with a from address of tzs@mydomain, then the setup in the article is perfect.
Suppose, though, I send an email from home with my from address set to tzs@employer, where "employer" is my employer's domain? Assume this email is not to an @employer address [1]. With the setup in the article (and in almost every other similar setup I've seen covered in similar articles) this might run into spam filter issues unless I've convinced my employer to add my SMTP server to their SPF record.
The way I want this to be handled is for my SMTP server to see that the mail is from an @employer address, and instead of trying to deliver it directly, relay it through employer's SMTP server.
This is similar to the common "smart host" configuration often used when you run an SMTP server at home, but want it to send all outgoing mail through your ISP's SMTP server instead of trying direct delivery. Essentially what I want is a conditional smart host based on the from address.
Postfix supports this. In fact, it seems to support it in a couple different ways. I played with it a bit but could not quite get it working.
What I'm doing for now, until I find out how to do it right, is only send work email outside of work from my desktop Mac. I took tzs@employer off the list of mail aliases for my mydomain mail account, and created a second account in Apple Mail for @employer. I set the incoming mail server to POP3 on 127.0.0.1 so that it would fail, and set the outgoing server to smtp.employer. It complained for a while that it could not contact the POP3 server, but eventually stopped complaining, and the address in the configuration dialog changed to 0.0.0.0.
With that setup Apple Mail sends mail from @employer directly to my employer's SMTP.
Sometime recently, after an OS update, that stopped working. It would no longer let me enable an account unless it could successfully talk to the incoming mail server for that account.
I did find an ugly workaround for that. I gave it the correct address for employer's POP3 server, and the correct password. Once it was happy, I went to Keychain Access, found the saved password for the POP3 server, and changed it in Keychain Access to something incorrect. Mail then complains that it cannot login to the POP3 server, but that does not cause it to disable the account. Net effect: a send only account in Apple Mail. (It is important to do the password change in Keychain Access, not in Mail, because Mail won't save the change until it sees the new password work).
(If that had not worked, I probably would have written a dummy POP3 server that always reports no mail and used that).
[1] This happens reasonably often for me, because I have my mail server set up to use fetchmail to fetch my incoming work email and deliver it via procmail. Same for any other SMTP accounts I have. That way I only have to configure mail clients to work with mail server and I get access to all my mail from all of my non-web email accounts.
Handling this server side seems weird. Why not just have a profile in your mail client that lets you send personal email through the personal server and work email through the work server. You end up with a much more robust setup that works without extra hops through a 2nd mail server. It also provides immediate feedback if sending an email fails for some reason.
If Apple mail is broken and keeps dropping things I'd suggest thunderbird.
> Handling this server side seems weird. Why not just have a profile in your mail client that lets you send personal email through the personal server and work email through the work server.
Then I have to have my desktop email client set up for both personal and work email sending, and my iPhone email client set of for both, and my Surface Pro 4 email client set up for both.
If everything, both sending and receiving, goes through my server, then all my client devices just have to use that one server.
In general, it just seems cleaner to keep the email clients as simple as possible and put as much of the decision making as possible on my server.
Email is always a nightmare. My ex runs a BSD mail server that seems to work alright (and he has a cloud service that will handle mail in the event of a server failure). Me? I pay Fastmail like $50/year to do that for me, and they’re wonderful. I tried to sign up for a trial with my own domain, and when I got a strange error that didn’t let me log in, I put in a ticket (as a non-customer) and the issue was rectified quickly.
I swear, I don’t work for Fastmail, but I’d much rather use them (or any email provider that is halfway decent) than fiddle with my own mail server.
I've done this, more than once, both for ISPs and personally.
I even put together a self-contained single-DVD installer that would install and set everything up securely and solidly. (I don't think I have a copy any longer and if I did it's very out of date)
I still don't recommend doing it. Even for the security-concious. It's just not worth it.
I don't know why there's so many downvotes and patronising "fly swatter" remarks. There is a use case for this.
For example, sharing a photo album with many pictures. Or a mixtape. Or some song lyrics. I can't upload those to the cloud for copyright reasons.
Or if I want to make the transfer while away from Internet e.g. skiing, hiking, waterfall trip, etc. - there's no access to Internet services, so iCloud/Dropbox/Google Drive just won't do.
I can use lighttpd to serve the folder, but then the user has to hold down and "Save to Camera Roll" for each picture separately.
If both phones have Airdrop, then that's fine. But many phones are not iPhones. We could both install a special app for Bluetooth transfers, but we don't have Internet access, remember? So no app store.
There's a Mail client on every phone, however locked-down it is. If I can serve a collection of files over a local network and send those directly, not one at a time, then that would be very useful to me.
I think a mail server is one solution. If you can propose another flyswatter that meets my requirements, go ahead. Otherwise I still want to find a way to code a mail server, and the parent article is helpful for that.
Actually it's fine until it's not. Then your email doesn't work and you could be missing out on important communications. And then you're scrambling to figure out how the spammers managed to exploit your setup this time. And you have to learn a tonne of crap in order to manage it... and the text files! Configuration... configuration everywhere. Obscure configuration. Configuration that has real consequences and causes spooky action at a distance. Configuration that will soon be exploited in strange ways.
I was so frustrated the last time my mail server went down that I started writing an SMTP protocol handler in Haskell with the intent of writing a MTA with the goal of minimizing configuration and being secure and resistant to attacks by default. So that hopefully more people can run their own infrastructure without prematurely aging. I dunno how useful it will be to others but at least it will keep my gray hairs at bay, I hope, when it's ready for use.
Until then though we need more guides like this for us poor souls who do go down this route. There are way too many out-dated guides awash in the sea of information.