Ha, okay. I'm going to include actual emails in "the email protocol", because if you just count the network protocols you're not doing anything useful. Emails are anything but simple. They were born in the early days of the Internet, and no doubt time and the impossibility of predicting the future is responsible for how things turned out — but they are anything but simple.
Email messages are ASCII only. You can put non-ASCII characters into a header however. It looks like this:
Subject: =?UTF-8?B?SMOpbGzDtOKApgo=?=
Writing a parser for this is an unnecessary challenge in this day. (If you ever need to do email, for parsing or outputting, get a library.) Watching a coworker unwilling to take the time to the read the RFC try to write a parser for this… is a very special hell.¹
The body also has to be ASCII, so just about every email client out there has to do:
Content-Transfer-Encoding: base64
And then encode the body in base64, wasting space and network bandwidth. Technically, you can send full 8-bit message bodies, but the sender and receiver have to agree ahead of time. Either GMail doesn't support it, or it just doesn't bother even when the email destinations are all GMail. (Or, it could be because some IMAP client might get the message, and not support it.)
There's also that every message has a text version and an HTML version — hopefully semantically similar — but HTML support in email clients (at least the big ones) is terrible. (see for example http://www.email-standards.org/ ) You'll think support is great on the web after you're done.
I've not covered un-wrapping headers. Or the things people think aren't valid email addresses. (Your average Joe thinks [a-zA-Z0-9]+@<a domain name> matches all valid emails. Oh, and that they're case insensitive too.) Or IMAP. Or encryption. Or even whether email gets encrypted while on the wire from Hotmail to Comcast.
> Slack pretends to be “an email killer”
I'm not sure they're out to kill all emails… just the ones that would be better as a real-time communication.
¹Don't test your library's output email on Gmail, either. Gmail is creepy good at correcting bad input. "It works on Gmail" != it's correct.
I do wish I could send restructured text emails and have the client just render them. In theory, email has everything I need. Clients just need to support it, that's all. I'll admit, that is kinda cool.
> Email messages are ASCII only. You can put non-ASCII characters into a header however. It looks like this:
To me that is a good example of how email is flexible, and also simple.
It's flexible because it could support non-ascii text strings, without changing a thing.
It's simple because the protocol doesn't care what the subject says. It only cares that it's an ASCII string.
That sounds like good engineering to me, as well.
You could argue that a new version of the email protocol could be improved to do away with those hacks, but that doesn't take anything away from email's simplicity and flexibility.
> To me that is a good example of how email is flexible, and also simple.
I fail to see how a complex encoding is "simple". (Maybe I should mention that there are complex rules about where that encoding can appear, and there's two variants of it)
> It's flexible because it could support non-ascii text strings, without changing a thing.
No, clients had to be upgraded to support it.
> It's simple because the protocol doesn't care what the subject says. It only cares that it's an ASCII string.
Well, sure; why would the protocol care about it? But at some point you're encoding the email for delivery (say, you're a web service) or you're an email client trying to display it. At that point, its a deceptively simple not-ASCII string.
> but that doesn't take anything away from email's simplicity
I'm arguing that you have a twisted definition of "simple".
Email is simple the way a plane is simple. As long as you don't see all the moving parts, and get to go along for the ride, it just seems like a magical flying tube!
I spent about 18 months on an email client project (was already 3 years in when I joined the team). I can tell you email is insane... so much more broken than you would expect (on all sides).
There is a reason many clients do "gmail" instead of "email". "Email" is a nasty barely functional ball of duct tape greased with WD-40 in the places it needs to go fast -- the fact that it works at all is breathtaking and only explained by the fact that it is one of the "killer features" of the internet.
The number of broken clients and servers is shocking and breathtaking. So you have to accept and expect bad data from everywhere. From the server directly, violating the imap spec, and in the body of the message violating all sanity. Yet, your users will see any failure as "this email client sucks" because the OTHER older email clients have already seen this idiocy and custom coded around it. Email clients end up being giant laundry lists of "if <stupid server> (act way 1)", "if created with <stupid client> (render 'right' way 2)"...
The reason new email clients are relatively slow to be created, and even fewer take off, is they are forced to deal with an insane clusterfuck of standards and corner cases to get to that 80% good enough mark -- or simply build out to one service at a time (we support gmail, hotmail and yahoo... etc).
There are 20+ RFCs you need to deal with the cover the spectrum of "basic" email with calendaring across a number of servers. That is just the "good" standard parts. On top of that you have the hundreds of server oddities that you need to handle else your email client will fail to get email off broken server X. Add to that the hundreds of rendering special cases you need to be aware of (ohh, this game from old outlook, so X means Y).
After you do all that, you have to deal with spam (both how servers tell you about it, how you show it, how you maintain it, etc) and an entire new set of oddities (very similar but simpler) for sending mail.
FastMail has an idea floating around that appears less terrible: http://jmap.io/ -- but I am blissfully out of that industry and simply don't care anymore.
EDIT: For the record, I agree with the article, email will be around for a long time, it is the "point of record" -- many of the "email killers" require your email address to sign up... so...
Looks interesting. IMAP is a pain! We spent months trying to get really simple stuff working for a mail client we're building (http://www.sortd.com). Actually, to be accurate it runs inside Gmail because it's more of a way to manage email rather than a standalone client, but we still need to connect to the mailbox itself via IMAP. Started using context.io, which was useful, but they went kind of wayward from their original vision.
What about performance though? Things like search are almost just not viable with IMAP - WAY too slow - we find ourselves having to hack the Gmail DOM to do things we should be able to do directly via the mailbox.
Seems to me that IMAP needs to be updated or replaced. What do you think of the new Gmail API?
Wow, how long have you guys existed? I am shocked I didn't know about this product. That is IMHO, EXACTLY the product that has needed to be built for awhile. Get all those eccentricities into one library that provides a clean interface.
We announced this a couple of months ago, though the company has been around for a while. We're also just quieter than most startups. Less blogging and talk, more code. :)
> Watching a coworker unwilling to take the time to the read the RFC try to write a parser for this… is a very special hell.
I had to do this once (some fifteen years ago, when finding a bulletproof library wasn't necessarily an easier task) and after reading the RFC very carefully, it still wasn't exactly a cakewalk.
Ha, okay. I'm going to include actual emails in "the email protocol", because if you just count the network protocols you're not doing anything useful. Emails are anything but simple. They were born in the early days of the Internet, and no doubt time and the impossibility of predicting the future is responsible for how things turned out — but they are anything but simple.
Email messages are ASCII only. You can put non-ASCII characters into a header however. It looks like this:
Writing a parser for this is an unnecessary challenge in this day. (If you ever need to do email, for parsing or outputting, get a library.) Watching a coworker unwilling to take the time to the read the RFC try to write a parser for this… is a very special hell.¹The body also has to be ASCII, so just about every email client out there has to do:
And then encode the body in base64, wasting space and network bandwidth. Technically, you can send full 8-bit message bodies, but the sender and receiver have to agree ahead of time. Either GMail doesn't support it, or it just doesn't bother even when the email destinations are all GMail. (Or, it could be because some IMAP client might get the message, and not support it.)There's also that every message has a text version and an HTML version — hopefully semantically similar — but HTML support in email clients (at least the big ones) is terrible. (see for example http://www.email-standards.org/ ) You'll think support is great on the web after you're done.
I've not covered un-wrapping headers. Or the things people think aren't valid email addresses. (Your average Joe thinks [a-zA-Z0-9]+@<a domain name> matches all valid emails. Oh, and that they're case insensitive too.) Or IMAP. Or encryption. Or even whether email gets encrypted while on the wire from Hotmail to Comcast.
> Slack pretends to be “an email killer”
I'm not sure they're out to kill all emails… just the ones that would be better as a real-time communication.
¹Don't test your library's output email on Gmail, either. Gmail is creepy good at correcting bad input. "It works on Gmail" != it's correct.
I do wish I could send restructured text emails and have the client just render them. In theory, email has everything I need. Clients just need to support it, that's all. I'll admit, that is kinda cool.