From a user point of view Microsoft butchered Messenger a long time ago. That's why Facebook killed it very easily.
First the official client (which I never stoud btw) became shittier with each release. Then they made that (very very) bad Facebook copy and forced users to display their real name instead of nicknames.
Moreover Messenger suffers from protocol version fragmentation. Features in alternative clients are random. Some people see nicknames, others real names, some are unaware that their real name is displayed. Sometimes you get old avatars, sometimes new ones, sometimes you don't even know that your avatar does not work. Same for offline messaging, and multi-connection.
And sadly Pidgin MSN's implementation SUCKS and it was a bad day when their survey indicated that people wanted up-to-date support and they just dismissed it.
I've never seen an alternative implementation of MSN that didn't suck. I've worked with the MSN protocol quite a bit, and the older versions were very nice in their simplicity. All the new XML junk they've added ruined it for me.
It's amazing to think how ubiquitous MSN Messenger was as a teen growing up, and how its use seemed to evaporate overnight with the coming of social networks and cheap text messaging.
Funny to me how asynchronous messaging seems to be more useful to people - would not have predicted that.
I was involved with the effort [to avoid] standardizing instant messaging back in 1999-2000. One of the products the company I was with at the time was offering was an instant messenger (with a twist that is not important for this story), so we tried to support all the prevailing protocols of the time (AIM, ICQ, MSN Messenger, Yahoo Messenger, PowWow, IRC, ... I don't even remember). And I was eagerly following the instant messaging standardisation effort at the IETF.
What became painfully clear to me after a while was that the big guys (AOL=AIM+ICQ, MS, YAHOO) who were running the show were all actively making sure that no standard would be decided upon. It was all dressed up in technical arguments, so it didn't look that way, but it was. Perhaps the engineers participating in the discussion weren't even aware -- it's easy to miss the big picture -- but as an outsider it was painfully obvious that the effective mandate that was given by the respective companies to their representative was to NOT standardize anything.
What ended up being the IETF standard was Jabber -- which was way too complicated and wasteful. And the reason it ended up that way was that while the big guys were avoiding standardisation, someone just put out an open source / open spec product which -- with all its deficiencies -- eventually worked well enough to gain non-trivial adoption, including by the IETF. And all of that happened awhile after the instant messaging working group was disbanded because of "irreconcilable differences" -- those differences being that every participant would only accept a standard that forced others to concede defeat and implement that participant's existing system (always in a way that limited everyone except said participant, of course)
Watching those discussions was an eye opener about how things supposedly work (and even look that way on the surface), and how they really work.
EDIT: REMOVED claim that Jabber/XMPP is still too complicated. I'm sure it still has 90% bandwidth and parsing overhead compared to comparable protocols (unless it's become backward incompatible), but standards have changed, and I guess that's no longer considered wasteful or bad engineering practice.
EDIT: ADDED: Just to be clear, this is an anecdote about the distant past (as in, 12 years ago now!). I have not been following Jabber/XMPP closely since 2004 and not at all since 2006 or so. Back in the day it was ill-and-under specified, ridiculously chatty (as in >90% overhead before you count TCP headers, both in bandwidth and parsing). What is acceptable has changed since then (no one cares about 95% overhead anymore), and for all I know all the kinks have been worked out.
And for the good parts - it is out there, there are multiple server and client implementation, and it works. That's a thousand times better than the product I was working on at the time, which was proprietary closed source, and though it was super efficient in every way, was discontinued and is no longer available.
I haven't been following it since 2006 or so, but unless it got simpler since then, yes it very much is still too complicated. It does get the job done, and is in widespread use, and has multiple server and client implementations, which makes it a gazillion times better than almost anything else out there. Hats off.
It was unbearably chatty - to the point of being unusable as a server-to-server protocol in a high-load environment in 2003. internal protocols (and even external protocols such as OSCAR, ICQ, and MSN Messenger) required 10% of the bandwidth and 10% of the CPU power. Jabber was usable but very expensive as a client-to-server protocol.
Nowadays, CPU is much cheaper and so is bandwidth, so these problems are gone. And perhaps clients are better behaved and standardised now (I have no idea how video, file transfer, etc are handled these days; the answer back in the day was "not at all, or in a complex and non standardized way").
As someone deeply concerned with real-world operation of systems, I find your removal of the "claim" distressing.
It is true that developers routinely create very poorly thought out protocols that have massive overhead, but it is far from good practice, and anyone who tries to foist such a disaster on me for any reason other than business-critical interoperability will be SOL. I simply will not deploy it, they will have to go back and create something that is at least moderately sane.
XMPP does retain massive overhead, in excess of 50% even before considering the XML bloat (I'm just referring to redundant or otherwise unnecessary messages here), and that's the main reason I have refused to use it in my projects.
Premature optimisation is the root of all evil. If you're deeply concerned with real-world operations, you should be painfully aware of this.
XMPP is primarily an instant messaging protocol. An overhead of 50% is nothing if the payload itself is often just a few bytes, and a few kilobytes at most. Of course it makes XMPP unsuitable for a few scenarios, but in almost all cases, XMPP message traffic will be dwarfed by other transfers. I think you have to think pretty hard to come up with a situation where you'd really want to use XMPP if only it wasn't for the overhead. Huge amounts of small messages exchanged between embedded devices, maybe?
Incidently, your message had 722 characters, the whole page had 4.83 kb. Pretty bad overhead, and of course HN is extremely frugal. I'm sure Twitter is much much worse. Apples and oranges, I know.
Argument by cliché, and a badly misunderstood one, at that. It's not a pass to engage in poor design practices. Would you deploy bubblesort on the grounds that quicksort is a premature optimization?
> An overhead of 50% is nothing if the payload itself is often just a few bytes, and a few kilobytes at most
You've just given me an estimated range of three orders of magnitude. As it happens, that is also the range I commonly get from people who have just "designed" a protocol with zero thought.
> I think you have to think pretty hard to come up with a situation where you'd really want to use XMPP if only it wasn't for the overhead. Huge amounts of small messages exchanged between embedded devices, maybe?
When I'm trying to process 10,000 messages per second on a CPU, and most of the overhead is already in ingesting, parsing, and routing each message, a 50% increase in message quantity means I need 50% more CPUs.
> Incidently, your message had 722 characters, the whole page had 4.83 kb.
I don't have to design, implement, or deploy websites, and consider the entire model quite broken for communications, so the comparison changes my opinion not at all.
> Would you deploy bubblesort on the grounds that quicksort is a premature optimization?
Unless it turns out the sorting routine has a significant impact on the overall performance, I wouldn't care a bit if it used Bubblesort, Quicksort or Bogosort.
> You've just given me an estimated range of three orders of magnitude. As it happens, that is also the range I commonly get from people who have just "designed" a protocol with zero thought.
Argument by snark. The assumption is that a chat protocol is developed for certain payload sizes because there is a trade-off involved: you'd do things differently if you had to create a protocol that works for 5 byte messages exactly as well as it does for 5 GB messages. Under that assumption, it's perfectly reasonable to give a range of expected message sizes. And since Jabber was developed in the first place as an IM protocol, 5 to 5000 bytes seems about right, with the distribution heavily skewed towards smaller messages.
> When I'm trying to process 10,000 messages per second on a CPU, and most of the overhead is already in ingesting, parsing, and routing each message, a 50% increase in message quantity means I need 50% more CPUs.
Yes. So? How many people are running large enough Jabber servers that this is a problem for them? And why can't they simply buy another CPU?
> Unless it turns out the sorting routine has a significant impact on the overall performance, I wouldn't care a bit if it used Bubblesort, Quicksort or Bogosort.
By your evident logic, then, Bogosort is the right default choice, because anything else is premature optimization.
> Yes. So? How many people are running large enough Jabber servers that this is a problem for them?
Well, the person I originally replied to appears to have been one of them. As are Google, Microsoft, AOL, Facebook, and Yahoo. I know one much smaller company I can't name who did field trials of XMPP on their embedded consumer device and promptly dropped the project.
Had I made the mistake of trying the same thing, I would now be in such a position.
> And why can't they simply buy another CPU?
And the server to put it in, and the space, and the power, and the cooling. I might be able to get a chip delivered for $200, but it's going to cost me another $400-800 plus people time to put it into service, and another $500+/year to run it.
Far cheaper to make the engineer spend a day or two to come up with something reasonable.
> By your evident logic, then, Bogosort is the right default choice, because anything else is premature optimization.
How does that follow from my logic? What I said was: I don't care about implementation details unless it turns out to be relevant. If I spent 99% of my time waiting for IO, and 1% on sorting, then yeah, the sort routine could be Bogosort for all I care, since the potential gains are tiny.
The right default choice for sorting is your standard library's sort routine. If you use that and sorting still takes a long time, then you can worry about what goes on in there and if it's worth replacing it.
> As are Google, Microsoft, AOL, Facebook, and Yahoo.
> I don't care about implementation details unless it turns out to be relevant.
Then you are reckless. It is both possible and desirable to anticipate obvious problems in a design and avoid them before they present themselves operationally.
> The right default choice for sorting is your standard library's sort routine.
Smarter people than you or I decided what the standard library's sort routine would be, and did so with careful consideration of the advantages and disadvantages of their choice.
Here we are speaking of the design of a protocol from scratch, and/or the explicit choice of a protocol for a given set of requirements. It is your duty to do as the standard library author did, and pay careful consideration to the implications of your design, as it will impact your business for years to come.
> All of which now use XMPP despite the overhead.
They have an interoperability requirement. Did you miss that detail of my earlier comment?
Okay. I agree. But carefully considering your options and paying mind to your requirements is a far fetch from flat out disregaring one option simply because of a detail -- overhead -- that may or may not be relevant in your situation. I still think you could refer to this as a kind of premature optimisation, although I see now that this was unhelpful, because it brought up the whole sorting thing which was kind of a tangent.
> They have an interoperability requirement. Did you miss that detail of my earlier comment?
Some of them do. Some of them explicitly turn off interoperability and still go with XMPP; Whatsapp (no affil) is another one of those. This just stresses the fact that efficiency/overhead is not the only consideration when it comes to protocol choice, and pretty far down the line when it comes to IM protocols.
Considering a single TCP segment will probably be used with or without XML bloat, is that really important nowadays? Not that it isn't bad practice (something in e.g. Protocol Buffers should probably be used), but to refuse to use it seems harsh.
XML bloat would still add to bandwidth costs (or add more CPU/RAM overhead if you compress it), TCP segments are not fixed size.
I know there's this tendency to dismiss bandwidth these days, but it's quite misguided. You literally wouldn't believe the quotes we got in Taiwan -- I thought there was some major misunderstanding until I got there and actually sat down with the network team at the telecom provider.
Have any third party XMPP clients implemented the proprietary X_MESSENGER_OAUTH2 authentication mechanism yet? Or is there a way to get the authentication to work with existing clients?
Is it on the roadmap for Pidgin (or libpidgin) or Adium?
I tried doing some quick Googling, but I couldn't find any answers...
It still looks a bit rough, e.g "Roster management is not supported".
I wish it would not end up being just a client concern but also impact server-side at some point thanks to XMPP federation. Google Talk does federation but last time I checked Facebook Chat did not. Such a shame.
Apparently Messenger does link with Facebook and Yahoo, but I don't see anything indicating a general federation.
It occurs to me that IM services have established a twisted, mutant, incompletely-routed version of an IRC network amongst themselves. And as far as I can tell, they still haven't gotten group chat right...
I work at MS and we use Lync Communicator a lot. Group chat seems to work well (as far as I can tell). Then again, who knows what server they have us connected to - it's probably v.latest.
I've read that there is an XMPP federation service [1], although having never touched it I don't know if there are any caveats.
What's wrong with XMPP multi-user chat? And I don't understand your argument of "incompletely-routed", especially if compared to IRC. imho XMPP federation is much more powerful concept than IRC networking.
His argument seems to be that MS ties into Facebook Chat, but not using XMPP federation (since that seems to be disabled on FB's side).
So - it's behind the scene. It's (by a stretch) comparable to IRC links, maybe.
The complain about group chat was, as far as I understand the GP, not about XMPP multi-user chat, but about the inability to _use_ that if you're not using real XMPP federation. In other words: How do you start a group chat from GTalk with your MS Messenger / Facebook friends? The technology would support it, but 'they still don't get it right'. I agree.
I was speaking more generally than XMPP. I don't know the fundamental causes, but I haven't seen group/multi-user chat work reliably in years -- I think the last time I did, everybody was on ICQ using official clients.
The problems I have encountered are:
1) Group chats just don't start.
2) Not everyone can see/"hear" everyone else.
3) Rarely, #2 starting and stopping randomly for people.
I've seen these behaviors on chats with as few as 3 people, up to 8ish (don't think I've ever gone higher than that).
I doubt the protocols are fundamentally broken in this regard, I assume I'm encountering implementation bugs, but they seem to be pervasive.
> And I don't understand your argument of "incompletely-routed", especially if compared to IRC.
"incompletely-routed" may not be a good phrase to use here, least of all because not many people "get" IRC anymore.
IRC networks eventually stabilized into a model whereby there was network-wide coordination of routing. Every server on a network can talk to every other server, and there is no leakage outside of the network. You know what you're getting when you connect to one of the major networks. With IM, links may be established between A and B, and B and C, but not A and C.
> imho XMPP federation is much more powerful concept than IRC networking.
I would absolutely agree in so far as the intent of XMPP federation goes. In practice, there are two problems:
1) Some IM providers intentionally cripple it (e.g. Facebook).
2) Not all IM providers use XMPP in the first place. Even some of the ones that make available a limited XMPP interface don't actually base their services off of it (Facebook, Microsoft).
It's important to realize that IM is not the same as XMPP, so when I say "IM", I mean "IM", not the XMPP subclass. :)
Now, I hope the other protocols I use such as Skype, Steam (And I guess Origin too) would follow suit. I would love to know when my friends are playing games, without having to be tied to either wine, virtualbox, or my gaming PC or yet another client.
They use a proprietary authentication mechanism, so you can't just use any XMPP client...
The client-to-server authentication used is SASL, with a specific and proprietary mechanism called X_MESSENGER_OAUTH2. ... Which means that all current XMPP clients are NOT able to connect to messenger.live.com.
I wonder if this is because Microsoft is becoming better and better at embracing open source when it sees a business advantage to do so, or if it's simply a last-ditch effort to keep Messenger alive as more and more people use it less frequently or not at all because the likes of smartphones, apps, and Facebook (Chat) are becoming more widely adopted.
Doesn't this basically summarize Microsoft's entire business strategy these days? Wait until some piece of tech is firmly entrenched, then move to implement your own instance, more often than not with an enterprise angle. More often than not infuriatingly incompatible unless your entire stack is Microsoft.
This happened with iPhone, cloud hosting, touch/tablet OS's; the list is endless. The only surprising thing is it took them this long to realize that Live Messenger needed to get onboard with Jabber if it's going to survive.
Wasn't that Microsoft's playbook from the very beginning? Industry writers were commenting on that behaviour in the 80s and 90s. Microsoft was always better copying other people's ideas and outcompeting them in the late game.
I'm guessing it's the former. In the past 2 weeks I've heard MS submit a patch to Redis, they've helped bring Node.js to Windows, Phil Haack joined GitHub primarily to bring C# into that open source community's limelight,...
and now this. Seems to me like someone's been pushing the open source agenda within Microsoft.
First the official client (which I never stoud btw) became shittier with each release. Then they made that (very very) bad Facebook copy and forced users to display their real name instead of nicknames.
Moreover Messenger suffers from protocol version fragmentation. Features in alternative clients are random. Some people see nicknames, others real names, some are unaware that their real name is displayed. Sometimes you get old avatars, sometimes new ones, sometimes you don't even know that your avatar does not work. Same for offline messaging, and multi-connection.
And sadly Pidgin MSN's implementation SUCKS and it was a bad day when their survey indicated that people wanted up-to-date support and they just dismissed it.