Setting up an email server is a non-trivial task. I am in a rush right now so I can write a more detailed response. But I hope others will chime in and provide some links to support my argument.
Curiously, this topic comes up every few days and we don't seem to have made any progress towards a good self-hosted email solution in the last 10-15 years. This is a start-up idea waiting to be implemented well; and for substantial profit, I might add.
For many years I ran my own mail server, for a while I ran it on my home broadband using a miniserver then I moved it to dedicated hosting (incidentally on Bytemark who are fantastic). I expanded to support my fathers company, my brothers' domains and ended up with probably 10 different domains being run from my server? I had that for a few years running postgres, mailscanner, dovecot, mysql, etc.
Then fairly recently I gave up, I am not an ISP, I am just a nerd with a job and a family. The system was generally hands off and ran itself, but I had to be careful: I had thousands of hack attempts every day (I even had a fun script that added firewall filters dynamically based on security probes), I had users who complained about mail deliver, I had logs to monitor and tidy and the worst part was a failure while I was on holiday requiring me to SSH in via my phone.
In the end the final straw was when the dedicated host had a disk failure, Bytemark were fantastic in helping me migrate, oddly the RAID did corrupt some users and I spent ages fixing things and everyone accepted the disruption because they were getting something for nothing (I never charged my family, even if I should have).
But at the end of the day I don't care enough that GCHQ or the NSA want to read my emails, yes it is a violation of my privacy and I am not that bothered. I know that if they really do escalate into a dystopia that I have the skills and knowledge to hide anything I want, but as a reason to go through the grief of running a host? No thanks.
I've now moved most of my services to dreamhost, got my family to make their own provisions and I just have to worry about passwords (mine are really complex). A 10 year experiment with hosting resulted in me realising that it probably isn't worth the hassle, even if things are made easier, when they go wrong this is Linux land and you still have to dive in deep to fix the problems.
Glad I did it, might do it again one day, but my life is easier now that I don't.
Setting up the software is the easy part, making sure that you have good deliverability by keeping your IPs and domains off blacklists and not being filtered is the hard part.
See, that part can be outsourced: just configure your outbound SMTP to relay via deliverability companies like Mailgun or Mandril, for personal use the cost is $0.
The reason one might want to use his own email server is - I suspect because of Snowden story - security: you control your own email archive and possibly encryption. The delivery will have to happen over the open Internet anyway, so why not just relay to a 3rd party from the start.
That fixes half of the story; the other half is inbound spam filtering (with all the RBL, greylist, spamassassin dance). I then prefer an unique solution such as http://mailroute.net that handles both inbound and outbound.
Using an SMTP relay sounds like a great idea as it would cut down on the number of moving pieces the home sysadmin needs to worry about when running a mail server.
Does Mailgun support TLS between my outbound SMTP and the Mailgun relay? And beyond?
I'm not so sure. I send 1 million+ emails a month from my server, and have had no issues at all in terms of being on blacklists. Deliverability has been comparable or better than 3rd party providers.
I don't spam, and I don't have very many people sending email from the IP address. Maybe that helps.
Curious how you imagine a startup going about tackling this? It seems a lot of the motivation around running your own mail server is that you have full control. For example, you don't want any third party to be compelled to grant the government access to your communication.
I've been pointing all my MX records to an ancient Dreamhost account (I know, the shame) instead of figuring out how to do this without Gmail or paying for a mail server on each domain.
Curiously, this topic comes up every few days and we don't seem to have made any progress towards a good self-hosted email solution in the last 10-15 years. This is a start-up idea waiting to be implemented well; and for substantial profit, I might add.