I've been running MiaB for about 6-7 months now. It's one of those things that 'just works', and with 'just works' I mean:
* Well configured, secure email server
* Ease of adding/removing domains, aliases, catch-alls and user accounts
* Ease of using email, either with the ready-to-go Roundcube webmail or any IMAP/POP client
* Batteries included: DNS is included, DKIM, SPF, you name it. DNSSec
* Want to host a static HTML site? MiaB's got you covered.
* Yes, that included Let's Encrypt SSL certs
* Awesome status page to check: version updates, misconfiguration, system health, and for each domain: DNSSEC, nameservers, DNS, TLS/SSL, blacklisting, etc.
* Open source! Huzzah!
If you're serious about moving away from Google or any other corporate mailprovider is feasible, take a look at Mail in a Box.
If you're comfortable with command-line Linux, and have a basic understanding of DNS, setting this up from scratch is actually not all that hard. After researching the hell out of this a few weekends ago, the best guide I found on the subject is: https://workaround.org/ispmail/jessie
Sure, it's technically for Debian, but if you're comfortable with Linux it's easily adaptable to other distributions. In order to get Gmail to accept your email without it going to spam, you need SPF/DKIM and a SSL cert. Let's Encrypt works, and is a cinch to set up.
I thought let's encrypt was only for domain names and not for email encryption. I'm a beginner in self hosted email.
I'm using spf and dkim, ip is not blacklisted, and the domain I'm sending from has a let's encrypt cert, but my emails still go to gmail spam. (Outlook also, but yahoo goes to the inbox)
Now that I'm thinking, the emails are sent using postfix through the fqdn (subdomain.example.com) which is not served by http and hasn't hot a cert.
If this is the case should sending the emails through example.com or add a cert to host.example.com work?
Or is there another mechanism to use let's encrypt certs for email?
One of the things that tripped me up at first was the postfix settings. smtpd_* options are for receiving mail, and smtp_* (no d) are for sending. So you need
If you don't and are rather a windows guy, I strongly recommand Smartermail. They have a free edition. Very easy to setup, all in one solution with a good webmail. Supports ActiveSync (not free though), SPF, DKIM. But not let's encrypt.
One thing to know, you need to activate the option to use TLS for outgoing email. A bit of a stupid default in my opinion.
> If you're comfortable with command-line Linux, and have a basic understanding of DNS, setting this up from scratch is actually not all that hard. After researching the h... out of this a few weekends ago, ...
You make it sound hard, just there, or do I misunderstand?
Ah that wasn't my intention :) Personally, I had just kept hearing all these nightmare stories about self-hosted email and never really looked into it further. Until I went from zero familiarity with the options to having a working setup in a weekend. But it probably took me longer than it needed to, because I've been learning BASH at the same time and wanted a fully scriptable install process. And some time spent wrestling with nginx (for https webmail), which I was also new to.
I poked a bit and it appears that the setup script is interactive which complicates playing nicely with Docker somewhat. It installs nsd + postfix, rewrites config files in /etc and mucks with ufw. Could probably mimic some of this by mounting locally customized config files into the Docker container.
There was some effort towards it and I'm still running a dockerized version without any problms whatsoever, but the maintainer considered Docker too hard to setup in the end (e.g. https://github.com/mail-in-a-box/mailinabox/issues/16)
This includes Roundcube which, the last time I checked (admittedly, 2-3 years ago) was basically unusable on a mobile browser. Has this changed in the meantime? (Not that it matters much, one could always install another webmail client elsewere).
Having said that I recently migrated to fastmail as it costs me <$5 a month and just works for all of my domains. I found I was spending more time making my email work than actually building things in my limited spare time.
This is good for those without existing infrastructure looking for a non-customizable mail system. I already run my own DNS, backups, firewalls, monitoring etc and want something I can hack on, so I run Citadel at home. Cit is the same idea (complete mail solution including webmail), but has many mechanisms for maintainably extending its functionality. Runs wonderfully on Debian with a single apt-get
This is the exact reason I built Fodor: https://fodor.xyz - it lets you easily setup GitHub projects on DigitalOcean so it truly is one~ click (usually about 4, but hey pretty good?)
You add a fodor.json file, add a link to Fodor and it makes it super easy to get it setup for yourself
I run a pretty similar setup, but could never get z-push to work properly with iOS exchange sync. I spent a few days tinkering with it and trying to fix php bugs, but to this day I can't get search to work properly from my iPhone. Are there any alternatives to z-push out there, or has someone had success with this setup?
I'd like a mail server for jailbroken iPhones. I tried to install one from Cydia, but it was designed for Apple TV and sent my iPhone into a boot loop and I had to restore.
Having a local mail server would be useful for making local P2P WiFi links, and emailing photos/music/etc directly between phones without Internet access.
Perhaps using an email server that's written in a scripting language would work? For example, https://github.com/zedshaw/lamson is written in python. Perhaps you could get it running on your jailbroken device if you could get python and pip installed.
Been using this on a Xen VPS. http://poste.io/. Seems to work relatively well and easy as pie to install. Even used it for a client and had no issues from them since.
Well email is a complicated beast, so much so that most it's security related features are not included right out of the box. Some of these features are used to help determine spam (i.e. Spammer doesn't usually setup things like SPF or DKIM records). Mail-in-a-box provides a solution to a lot of these other components, thus not requiring the spammer to set much up at all. Making them more reliable to spam engines than before. I'm not saying Mail-in-a-box is a one stop shop spammer solution, but it's pretty close.
Gmail and other really big providers are in a unique position for spam filtering. They have so many users that they get a continuously trained filter pretty much for free. They can instantly classify never-seen-before kind of spam in minutes because someone gets push notification on their phone and clicks "this is spam".
Nobody with purely-technical solution to this problem can be better in practice. (although we're pretty close) This means you're more likely to get actual spam in your custom deployment.
I run my own mail server, and I get very little spam. Sometimes I go for days without a single spam message even getting through to my spam folder, much less my inbox. The last spam mail that got through to my spam folder was on November 25th. And it's not because spammers don't try, my server has rejected 206 mails in the last ~24 hours (and that's not counting the attempted open relay abuse).
My setup consists of Postfix with postscreen and SpamAssassin. Postscreen blocks clients on a certain type of protocol error that spammers are prone to (speaking out of turn) and based on DNSBLs[1], notably zen.spamhaus.org, which blocks most spammers.
SpamAssassin, in addition to the standard rules and bayes filtering, is configured with Pyzor, Razor2, DCC, and iXhash, and I have some custom rules as well.
I actually get far less spam with my self-hosted setup than I did when I used a paid e-mail service (although it was not Google).
Public IP reputation (DNSBLs) and spam fingerprint databases (Pyzor, Razor2, DCC, iXhash) make self-hosted spam filtering very feasible.
I intend to switch to rspamd soon, which has built-in greylisting. However, in my test run with rspamd I did not get good enough results because my current SpamAssassin setup relies heavily on rejecting spam that hits two or more content filters (bayes, pyzor, etc.). I will need to implement at least a pyzor plugin for rspamd before I can switch.
rspamd, in addition to bayes filtering, has it's own fuzzy fingerprinting system. You can use your own fingerprint db and/or a public one. The public one that is configured in rspamd by default didn't seem to catch any of the spam I get though.
Don't know about rspamd, but SpamAssassin used to be way too aggressive in its stock config. E.g., it had a default rule for some "MS-Outlook" headers that just happen to be in every email sent from Outlook via a non-Exchnage mail server. That created a ton of false positives - wasn't hard to fix, but still it was a hassle.
I ended up moving away from Postgrey because the delays (especially coming from Gmail) were a huge problem. Since the mail would come from some random IP in Google's massive server farm, each time it tried to get delivered, postgrey would delay it again.
What types of spam do other people see in their personal mail servers?
The majority of spam I receive is from addresses that have been leaked ( Santander bank is particularly bad for those for some reason ) or bought in an acquisition; perhaps two or three per year. I just blacklist those addresses.
What really annoys me, though, is that eBay passes my unique-to-them email address to vendors from whom I buy items and several of them have added me to their mailing lists over the years, but thankfully de-subscribe has always worked.
I receive tons of spam (on a very old email address).
I am progressively transitioning to giving a unique email alias for each service.
1. If the address gets leaked or sold I can identify the culprit and stop the spam by deleting the alias
2. It is a useful security measure too, if a website gets hacked, it becomes harder to correlate my account across websites since it is a unique, non predictable address.
Run a honeypot. Just an usual Postfix with catch-all virtual mailbox and a learn-spam for the destinatin. Let spammers happily fill it (they will, in no time) and train your very own rspamd/spamassasin/whatever.
Sure, this works as long as they don't try to poison your filters, but given its a small and rare target, it's unlikely someone would bother.
A proper honeypot takes a lot of time and effort. You actually have to inject that email into the right environments. For example nobody will steal your honeypot address from someone's mail history, some service's stolen user database, or others.
This is a definite trade off. I have my personal mail server + a gmail account and gmail is better at spam prevention. It isn't that much better though as local Bayesian filters do very well.
I setup my own mail server last year and shortly gave up on the idea. Setting the server up isn't the problem. The problem is keeping your IP off of all the major blacklists.
I wasn't even sending out mass emails and 30%+ of my email would never be delivered. I had to constantly check to see if my IP addresses were on the various spam lists (and fight to get my IPs off) and I just got tired of it.
Companies like Google have entrenched themselves in many things like email and are slowly becoming the only option out there. A large amount of email addresses are @gmail.com or run through one of their servers and they ultimately control whether the recipient receives/sees your email.
The 'promotions' tab in gmail also made things worse for many small businesses. Google doesn't want you competing for their advertising space and pushes any emails it deems a 'promotion' off to the side, so users don't actually see it. I'm not even talking about actual spam emails here, but emails users knowingly signup for and are expecting.
Many people don't realize just how much a handful of companies controls the Internet and your ability to make a living online.
Once you have SPF/DKIM in place and make sure your IP isn't already on a blacklist for some past (previous users) infractions you should be good to go. I've run my own mail server for years and have only had to remove it from a blacklist once. So once you get past some initial blacklist monitoring work, you are good to go.
That does bring up the point of how to do blacklist monitoring. There are various commercial services out there that will allow you to check for free and monitor 1 host or something (eg. https://mxtoolbox.com/). I'd prefer to run my own though, does anyone know of a good setup for this?
Yep. I work on software that runs on thousands of mail servers (I work on Virtualmin), and it is entirely possible to run your own mail server with good delivery rates. It's honestly not even that hard, if you get SPF, DKIM, PTR records in DNS right, and you correctly handle bounces and unsubscribes promptly. We've been sending out a few thousand emails a day for over a decade without incident, and we have several people who run bigger mailing lists than we do.
Though there are a few minor caveats to this. Microsoft (Hotmail, Outlook, Live, etc. addresses) mail servers are ornery, in that they hold grudges against IP addresses for a long time (seemingly forever, as the server we moved to recently had been in our possession for non-email use for a couple of years, and it was still on a Microsoft blacklist from a prior owner's abuse), and they make you jump through a few hoops to get it removed. Even with SPF and DKIM, they rejected 100% of our mail until we got off of their blacklist. Our previous server never had that problem...but we'd been on the same IP for like five or six years.
You need to be on an IP that is dedicated and that you're going to own for a long time, and not part of consumer IP blocks; you can't effectively run a mail server on a cable or DSL line, even business class, without jumping through a lot of hoops. But, if you're in a colo, you'll be fine. This also applies to AWS and other cloud server IP addresses; as I understand it, huge swaths of them have been burned by spammers who spin up and spam until they get shut down, and then move to another.
So, I guess it's relatively tricky to get things working at the beginning and you may have to fight a little with some of the big email vendors, but it's not really an ongoing thing, in my experience. Get it right, and then don't spam or let your users spam, respond appropriately when abuse does happen, and you can run your own mail server relatively painlessly.
Not something I can run, but http://www.mailradar.com/ seems like a better free service. They let you monitor 5 IPs for inclusion on blacklists which works better for a personal setup where you might have more than 1 domain but wouldn't have more than a couple VMs as the MX servers.
"Once you have SPF/DKIM in place and make sure your IP isn't already on a blacklist for some past "
you'd think that it would be this simple, but it's not. I have had SPF/DKIM set up from day one, a totally clean IP, doesn't show up on any block lists at all, yet i'm still having some problems delivering to certain ISPs. Verizon is the biggest problem right now.
I had problems delivering to Gmail because my server didn't have a good enough reputation. Everything from my IP was going right into the Gmail spam folder and there was nothing that I could do about it except sit back and wait. It took weeks for Gmail to finally decide that my reputation was good enough.
It's scary just how much power Google has over many things email these days.
Now I have to convince Verizon that my IP is not a dynamically assigned one.
Question: where's your IP address come from? I'd have guessed that common cloud VM IP pools are likely all trashed permanently already? I somehow doubt AWS or DO or Linode or Rackspace et al are worthwhile places to host an outbound mail server? I'd also guess ISP pools of home IP addresses are probably just as poisoned. Is proper SPF/DKIM setup "enough" to overcome that? (Or are my suspicions about pools of IP addresses unfounded?)
Cloud providers are not as bad as you'd think. I've tested several DO IPs using various checkers, as I've been thinking about moving my server there (currently at prgmr.com), and they have all been clean. SPF and not being an open relay seem to be the 2 important things to keep you off the blacklists. I still don't have DKIM (been on my TODO for a while... but lazy) and haven't been put on a blacklist in many years.
I used Digital Ocean for a year. No problems, never sent spam. Then my mail started being spamfiltered, apparently because a neighbor was spamming. Seriously don't recommend using cloud hosts if you care about people receiving your email.
Hmmm, thanks for that. I just checked my CloudAtCost VM, which I'd never have considered running outbound mail from, and it's not on _any_ of the 90+ blacklists mxtools checks. This astounds me!
FWIW, I stopped using my home connection because it was listed on Spamhaus' PBL, you might want to check that out if you plan on using yours: https://www.spamhaus.org/pbl/
I'm using DO currently and it's been working fine, though it's just for personal email.
I'm surprised to hear there are home ISPs that still allow outgoing traffic on port 25... I used to run an email server at home, and both ISPs I used required you to route all outgoing email through their SMTP server (which presumably had an outgoing spam filter on it). This worked fine for me because it meant my outgoing mail had a good reputation.
Why? I expect from ISPs to deliver IP packets to/from my address, without filtering on basis what is in payload.
Though once when device connected to my wifi got infected and started sending spam, I got angry (not e)mail from ISP, so I drop tcp/25 on my router firewall.
So you, someone technical enough to set up their own email server, was spamming people, and it presumably took days or weeks for someone to report you, and you to check your mailbox and get around configuring your firewall.
Now imagine the typical user who has no idea what the letter means or how to configure their router and just ignores it...
I'm surprised your whole ISPs dynamic IP pool isn't already on every spam block list.
edit: just realized you aren't the poster I was replying to, so presumably you're not running your own email server
I used DO for a year without a problem, and then my IP was blacklisted (apparently a neighbor was spamming) and I couldn't do anything about it. Be warned, and frequently check if Google accounts receive mail frequently.
I use DO without any problems. You need to make sure your IP isn't blacklisted before you start (and if it is, trash the instance and try again), and keep an eye on blacklists in case your range gets caught, but you should really be doing that anyway.
Don't forget DMARC. Setting it up strictly and monitoring it via something like dmarcian or other tools should keep your domain clean as long as the IP is safe.
While I hear you on Gmail's overbearing influence, I'm not a sysadmin (I'm a designer and I spend most of my time talking to people about "user journeys" and fonts), but I run my own (and my family's) email using Ubuntu on a VM. There are lots of easy HOWTOS on setting up Postfix with DKIM and bind with SPF. I've got an alert on one of those blacklist monitors and not been in any blocking lists at all in several years.
So if I can do it, you can too.
BTW I use Gmail as the front end though - it picks up the mail up from my POP3 server and sends it out via my server over TLS/SSL. That last step was a bit complicated to set up since Gmail doesn't accept self-signed certificates for SMTP sending. But I managed to work out how to set up and renew a free cert from StartCom.
LetsEncrypt actually has multiple options available for validation. Only one challenge type, http-01, requires port 80 to be open. Another, tls-sni-01, requires port 443. dns-01 requires configuration of your DNS provider. I personally make use of tls-sni-01 and dns-01 in different situations.
> The problem is keeping your IP off of all the major blacklists.
I run several email servers, and this is NOT the problem. You just need to configure things correctly. It's not 1996 anymore, you need to set up things like DKIM, SPF, etc. Also make sure your hosting provider has not sold you an IP that was previously used by a spammer. [0]
That's all there is to your "problem".
Besides that: Your solution to Google's insane domination is to put even more of your life on their servers? When we know thanks to Snowden that Google is part of the NSA's PRISM mass surveillance program? Surrendering is not what winners do.
I completely agree, especially with your "defeatism is doing your opponent's work for them" viewpoint, but if your goal is keeping information off the big provider's systems, then it's not gonna leave you with many people to email...
I think you're mischaracterizing the promotions tab. When users interact with a web merchant, they're exchanging money for goods/services -- they're not signing up for spam. Hiding marketing email is a feature, not an evil plot.
It's not just G that differentiates between transactional email (a receipt) and bulk/marketing mail. Trans/bulk are so different that email companies consider these separate products (e.g. mailchimp vs mandrill).
Generalizing based on myself, I can guarantee that people really hate receiving crap. At least I can visually distinguish the G ads from my email and ignore them.
I found the Promotions tab to be a really handy "middle ground" between spam and actual mail. It usually contains mail from companies I've done business with, who I might be interested in receiving offers from, but I don't want those offers mixed in with my day-to-day stuff. Having them separately allows me to read them on my own time, and also lets me see at a glance if anyone is sending me too much. They're also usually the kind of companies who will honour an unsubscribe request.
That's why I've left gmail years ago. I do not want to support this kind of "business model".
I also do not want to invest the time to maintain my own email server. Imho there are good alternative solutions provided by companies such as protonmail.com or mailbox.org just to name two I have personally experience with.
As with everything it is mostly about overcoming your personal comfort zone and start to act. Good luck.
What are you doing to constantly have to fight the blacklists?!
I've been operating my own mailserver for over a decade and I haven't ever had to remove my IP addresses from any blacklists, and I haven't ever heard such a thing from other people I know who are operating their own mailservers either.
There's still a middle ground available. You can move your emails to a smaller but serious commercial email provider you actually pay for. I moved to Zoho for example (I pay them directly and use own domain), but there's still loads of others available.
I don't get the complaint about the promotion tab though. I had the emails auto-labelled before more or less as private / adverts / notifications / mailing lists. This maps pretty well to Primary / Promotions / Updates / Forums. Anything that ends up in Social I just unsubscribe from. I'm pretty happy with their autoclassifiers. They don't make me not see emails.
> The 'promotions' tab in gmail also made things worse for many small businesses. Google doesn't want you competing for their advertising space and pushes any emails it deems a 'promotion' off to the side, so users don't actually see it. I'm not even talking about actual spam emails here, but emails users knowingly signup for and are expecting.
The difference between what a business thinks a user has signed up for and is expecting and what an actual user is actually expecting in their actual mind tends to resemble night and day. For most users the "Promotions" tab was a godsend that rescued them from significant amounts of email that was swamping the stuff in their inboxes they expected and wanted to read. It allows users to engage with promotional material at their own choice and in their own time.
Google's smart filters have been a fantastic win for the user experience.
Agreed, for this particular point it's less about small business vs. Google and more about small business vs. users. Google stepped in and saved a lot of users a lot of time and effort. I'm not sad that it's tougher for businesses to consume my time while I'm cleaning my inbox out, I can go over the promotional stuff when I actually want to.
My company's mail is on Gmail, and because of some missetting on google's side ('webmaster@' and 'abuse@' addresses not mailable), we ended up on the RFC-Clueless blacklist.
I got that fixed, and have submitted half-a-dozen requests to have us removed from rfc-clueless, and they've all been ignored (and do their best to hide how to submit and what for). Fuck rfc-clueless - it's a blacklist for 'people who don't follow the rules', but they're bad netizens themselves and don't follow their own rules. Just... fuck them.
I can't imagine what it's like to end up on one of these ignored blacklists when you don't have the might of a professional email service behind you.
I've been running my own mail server since 2001. Only once had a problem with blacklists when I moved the server to a new IP at a provider with bad spam reputation. Otherwise, this has never been a problem.
Don't let Google and Facebook monopolize our communications.
> Don't let Google and Facebook monopolize our communications.
I do the same but the reality is that 99% of my correspondants use gmail, yahoo or hotmail. So these companies get a copy of all of my communications...
I set up my own mail server and was sending bulk email (opt-in) with relatively little experience. Set up dkim and spf, and you should be fine. A bit arcane, but it's not the maintenance nightmare you imply. Once you get into automated bounce tracking, etc, it's a bit trickier, but no need to do that for personal email.
The promotions tab is the only reason I even look at marketing email. Otherwise, I'd just ignore it completely. Every once in a while I skim through it and clear it out.
I've been running a postfix for years (since 2008ish) and never once checked or worried about blacklists. My 3 users haven't reported any problems at all.
> never once checked or worried about blacklists. My 3 users haven't reported any problems at all.
What if they missed emails they never knew about in the first place? What if they tried to contact someone and just assumed they've been ignored? Unless you know you're not on blacklists, you should be worried at least a bit.
If you're only concerned about delivery and can live with a third-party SMTP relay, I'd hope you could set up this system to relay through sendgrid or mailgun. If it's only for personal use, you'd probably be fine on their free plan indefinitely.
* Well configured, secure email server * Ease of adding/removing domains, aliases, catch-alls and user accounts * Ease of using email, either with the ready-to-go Roundcube webmail or any IMAP/POP client * Batteries included: DNS is included, DKIM, SPF, you name it. DNSSec * Want to host a static HTML site? MiaB's got you covered. * Yes, that included Let's Encrypt SSL certs * Awesome status page to check: version updates, misconfiguration, system health, and for each domain: DNSSEC, nameservers, DNS, TLS/SSL, blacklisting, etc. * Open source! Huzzah!
If you're serious about moving away from Google or any other corporate mailprovider is feasible, take a look at Mail in a Box.