First, I love reMail. Gabor breathes email and search. If you own an iPhone and use Gmail or Google Apps, get reMail--you'll thank him.
That said, it's easy to skim this article and nod, "Yeah! Yeah!", but when we're discussing these, laymen may need to know a few of these seem over-simplified.
> "TCP connections are great, but for transferring large amounts of email securely, HTTP is the way to go."
2. I generally have at least 5 devices using IMAP (on Google Apps) at once, and interleave my interactions with these devices arbitrarily, yet expect each to show me the exact same state when I look at it. Keeping them all in the apparently same state is the sort of intractable software problem bedeviled with implementation details, and I agree throwing out IMAP and starting over with lessons learned could be easier than, say, the RFC linked below.
3. http://ajaxian.com/archives/json-vs-xml-the-debate - Seems this point is something like GIF vs JPG vs PNG, with a bit of "Markdown vs HTML" thrown in. Meanwhile, both XML and JSON are missing an important concept given labels/folders and conversations: multiple references to single objects.
4. Conversations or threads exist today as In-Reply-To: GUID and References: GUID headers, among others, but see the JSON problem above. We don't realize these headers are there because clients generally ignore these and try to group on Subject instead. Gmail grouping respecting existing headers is much better, but from usability point of view, Gmail's approach makes it challenging to bulk delete individual matching messages across a broad set of conversations. For example, to bulk move or mark all messages on a particular date results in the full conversations being moved or marked. So I like this idea as long as I can still optionally operate on sets of messages without affecting the rest of the conversation.
5. http://googlesystem.blogspot.com/2007/08/organizing-chaos-fo... - It's not yet clear that people prefer tags to folders. After all, monkeys don't expect a banana to be in two boxes at once. I don't want to label anything with keywords. I want semantic search.
6. Excellent point. Changing GUIDs out from under us is just lame. Having had to resort to http://github.com/rgrove/larch recently to move a decade of email onto Google Apps because Thunderbird was putting new IDs on each moved message, I fully agree this problem is evil.
7. Seems we're stuck with MIME until we get rid of every legacy email server in the world, or control what servers our recipients use. Between the new server and new clients, sure. But the server will have to know it to send it. Granted, IMAP is for receiving, and SMTP is for sending, but something in the chain has to know MIME.
8. "Call" an HTTP endpoint that "returns" when messages arrive? Just nomenclature, since IDLE is a "call" that "returns" when state changes. Either way, the socket is held open, so neither of these is "push". Maybe Gabor meant "posts back" instead of "returns"? In any case, under the hood, iPhone push and Microsoft ActiveSync are exploiting out-of-band channel such as SMS to notify the phone it should poll (or in case of ActiveSync, maybe doing a "long pull"). See http://msexchangeteam.com/archive/2004/04/26/120520.aspx versus http://tools.ietf.org/internet-drafts/draft-maes-lemonade-p-...
9. Will the RFC specify a search results algorithm, or will the same search return different results depending on the implementation of mail server the client is connected to? We manipulate a "client" tool to perform the search and access the email, so our natural mental model is of the client at hand, not of the remote server. This lets the brain wrap itself around Gmail's web search versus Apple Mail's search box vs OS X Spotlight vs Gabor's reMail all returning different results. Practially speaking, search at the server does make far more sense from both a data retention and security standpoint. I have 509 MB of index in Gabor's "reMail" app on my iPhone, and would rather not.
Thanks for brainstorming this, and thanks for a kickass iPhone app.
1.HTTP uses a TCP connection. For that matter so does SMTP and IMAP and the proposed REMAP.
Yes, of course. He is not claiming that TCP is bad, but rather that it is too low-level. i.e. One wants an application protocol (e.g. HTTP, SMTP, IMAP) rather than a transmission protocol (e.g. TCP).
The HTTP verbs should be sufficient for email, so why not leverage them?
It's not yet clear that people prefer tags to folders. After all, monkeys don't expect a banana to be in two boxes at once.
Heh, dare I reply? Tags can be used as pigeonholes, and this can be enforced if desired. But why? Tags are so much more useful because they are not pigeonholes.
Here is a classic example:
Does my app config go in a system directory or an app directory or a config directory? the executables? the daemon? the log files?
The classic approach is to put system-level config files under /etc, log files under /var/log, executables and daemons in various locations, etc. Some newer approaches (HomeBrew, GoboLinux, etc) keep everything organized per-application.
Instead, keep every file in the global folder / namespace. Tag the executables as exe, the log files as log, config as cfg, and also tag everything with the app's name. Possibly the version, etc.
A file is uniquely identified by its tags and its name. Any combo can be used, so long as it doesn't already exist.
You don't have to make painful decisions about the unique place in the tree that a piece of information should exist. Just tag it appropriately and move on. The tag system could even create an optimal concrete hierarchy based on tag and tag association metrics if needed.
But the server will have to know it to send it. Granted, IMAP is for receiving, and SMTP is for sending, but something in the chain has to know MIME.
I'm not sure if this is GoF, but this sounds like the classic gateway pattern. You put MIME gateways at the edges of the new email network. MIME-only servers must go through the gateways.
Your intended point about pigeonholes can sometimes backfire within the context of email. Consider a Gmail user who sends a query to a number of different recipients, expecting unique replies from each. Gmail, because of the like headers, will group all of the responses into the parent conversation. Let's say the users tags the first response with a label corresponding to that sender, and then the second response comes in. At this point, the user is at an "oh shit" moment, finally comprehending the implications of Google's labeling of conversations rather than individual messages.
You need to be able to specify both parent and child tags to be able to facilitate search and give users something that isn't too confusing re: organization.
I understand the pigeonhole filing example, but I quit "filing" years ago. It's tedious. Tagging also takes too much thought, particularly at recall time: did I tag that "development" or "programming"? I just use search.
I have a script that monitors my desktop. Any file not modified within the past 8 hours is moved to a folder dated for the end of this week. That gives me 52 folders a year, granular enough to find a file manually if I know roughly when I dealt with it.
My desktop stays clean except for what I'm actively working on, and I don't look for files hierarchically or by category/tag. To find something, I use Spotlight search. That's faster than navigating a folder hierarchy would be even if I knew exactly where the file was.
That's also why I love Gabor's reMail on the iPhone.
> and then the second response comes in. At this point, the user is at an "oh shit" moment, finally comprehending the implications of Google's labeling
I am having trouble understanding what the problem is, here. I also suspect this may be a Gmail problem rather than a tagging problem.
It's tagging a conversation rather than tagging an email.
I frequently put different emails from the same conversation in different folders, because they contain different types of information that I need to do different things with.
Having had to resort to http://github.com/rgrove/larch recently to move a decade of email onto Google Apps because Thunderbird was putting new IDs on each moved message, I fully agree this problem is evil.
What version? This doesn't happen to me on 2.0.0.23
That said, it's easy to skim this article and nod, "Yeah! Yeah!", but when we're discussing these, laymen may need to know a few of these seem over-simplified.
> "TCP connections are great, but for transferring large amounts of email securely, HTTP is the way to go."
1. http://wiki.answers.com/Q/What_is_the_difference_between_tcp... - HTTP uses a TCP connection. For that matter so does SMTP and IMAP and the proposed REMAP.
2. I generally have at least 5 devices using IMAP (on Google Apps) at once, and interleave my interactions with these devices arbitrarily, yet expect each to show me the exact same state when I look at it. Keeping them all in the apparently same state is the sort of intractable software problem bedeviled with implementation details, and I agree throwing out IMAP and starting over with lessons learned could be easier than, say, the RFC linked below.
3. http://ajaxian.com/archives/json-vs-xml-the-debate - Seems this point is something like GIF vs JPG vs PNG, with a bit of "Markdown vs HTML" thrown in. Meanwhile, both XML and JSON are missing an important concept given labels/folders and conversations: multiple references to single objects.
4. Conversations or threads exist today as In-Reply-To: GUID and References: GUID headers, among others, but see the JSON problem above. We don't realize these headers are there because clients generally ignore these and try to group on Subject instead. Gmail grouping respecting existing headers is much better, but from usability point of view, Gmail's approach makes it challenging to bulk delete individual matching messages across a broad set of conversations. For example, to bulk move or mark all messages on a particular date results in the full conversations being moved or marked. So I like this idea as long as I can still optionally operate on sets of messages without affecting the rest of the conversation.
5. http://googlesystem.blogspot.com/2007/08/organizing-chaos-fo... - It's not yet clear that people prefer tags to folders. After all, monkeys don't expect a banana to be in two boxes at once. I don't want to label anything with keywords. I want semantic search.
6. Excellent point. Changing GUIDs out from under us is just lame. Having had to resort to http://github.com/rgrove/larch recently to move a decade of email onto Google Apps because Thunderbird was putting new IDs on each moved message, I fully agree this problem is evil.
7. Seems we're stuck with MIME until we get rid of every legacy email server in the world, or control what servers our recipients use. Between the new server and new clients, sure. But the server will have to know it to send it. Granted, IMAP is for receiving, and SMTP is for sending, but something in the chain has to know MIME.
8. "Call" an HTTP endpoint that "returns" when messages arrive? Just nomenclature, since IDLE is a "call" that "returns" when state changes. Either way, the socket is held open, so neither of these is "push". Maybe Gabor meant "posts back" instead of "returns"? In any case, under the hood, iPhone push and Microsoft ActiveSync are exploiting out-of-band channel such as SMS to notify the phone it should poll (or in case of ActiveSync, maybe doing a "long pull"). See http://msexchangeteam.com/archive/2004/04/26/120520.aspx versus http://tools.ietf.org/internet-drafts/draft-maes-lemonade-p-...
9. Will the RFC specify a search results algorithm, or will the same search return different results depending on the implementation of mail server the client is connected to? We manipulate a "client" tool to perform the search and access the email, so our natural mental model is of the client at hand, not of the remote server. This lets the brain wrap itself around Gmail's web search versus Apple Mail's search box vs OS X Spotlight vs Gabor's reMail all returning different results. Practially speaking, search at the server does make far more sense from both a data retention and security standpoint. I have 509 MB of index in Gabor's "reMail" app on my iPhone, and would rather not.
Thanks for brainstorming this, and thanks for a kickass iPhone app.