"What other alternatives are there? (Twisted, EventMachine, ...?)"
For every major high-level language, there is at least one Node-like library, and sometimes more than one (Perl has POE, Event::Lib, based on my experience the raw glib wrapper isn't half bad albeit perhaps not the fastest, but you get good access to anything else based on glib, in fact Perl has so many that there's an Any::Event wrapper to remove your dependence on the underlying event library!). My point isn't that Node.js is bad. I actually don't think it is.
My point is that the hype is bad. It's wrong to think it's bringing anything unique to the table, because it simply isn't doing anything that has not be done literally dozens of times, except it's doing it in Javascript. If you want it done in Javascript instead of Python, more power to you. I'm particularly incensed by the idea that Node.js' approach to asynchronous is the only way to do it and the number of people it has produced who it has anti-educated into thinking Haskell and Erlang and all kinds of other languages can't possibly be asynchronous because you can't see the manually-chopped-up event handlers in the code. I'm not guessing. I've met these people online. You may know better, but a lot of people don't; whether or not it was intended the hype is actually lying to people about the state of the programming world, comparing itself to the world of 1995.
I am also trying to speed up the education cycle that all of those other dozens of attempts have been through in which manually-compiled event-based programming inevitably explodes into unmaintainable complexity, and none of the dynamic languages, including Javascript, have the necessary constructs to truly contain it. Some of the dynamic languages are even more powerful than today's Javascript, such as Python with its generators (though ECMAScript is supposed to be getting those, I don't know if any browser has them yet) and it's still not enough. The structure of event-based programming demands such an explosion. Been here, done this.
You can see it already starting to poke out from under the hype, if you're watching carefully. This is going to get worse, not better (because there isn't a solution, just a variety of hacks long since tried and found to only slightly improve things at significant complexity cost themselves), and I'm actually trying to do the community a favor by deflating the balloon so it doesn't pop so hard.
(If you know Haskell, and you look at the implicit type signatures being put on things like callbacks, it becomes easy to see the problem. The clearest place to see the problem is a function that takes a callback for something, until one day you need to pass in a callback that itself has to go do something that requires a callback and suddenly you've got a big problem. The usual callback in Node.js is actually just a relatively-pure function, they are not in IO, which is done behind the scene for you. Then when you need to do something else, you've got some real problems. Solvable, yes, but at a fairly significant complexity cost, partially because any given issue can be addressed but you can't really address all of them simultaneously (simplicity, exception correctness, dealing with control flow across callbacks, etc.). Every time you write a callback or choose where to break the function up into a callback you're actually laying down far more restrictions on the code than you can easily see, but I don't go to this explanation very often because by the time you can understand it it is also borderline obvious.)
What other alternatives are there? One, use Node.js with awareness of the issues. There are places where it is fine. I just would incredibly strongly anti-recommend it if you know you're going to be continuously developing whatever you're building in it, especially your core product, rather than writing "a proxy socket server for web sockets to conventional sockets" and being done at some point. The other alternative is to actually work in a language/runtime where you don't have to manually perform all this tedious work. There's a number of them coming out and one of them is probably going to go mainstream at some point; of the current lot Go would be my best guess. Google isn't pushing it, but it's still got Google's name on it, and I don't know of anything else right now with the equivalent name power. History suggests name power is necessary for a language to crack the mainstream in anything less than 15 years. It probably requires the least adaptation to a new style of the bunch, the other advantage it has from the mainstream point of view.
I friendlily (!) request less anti-hype and more explanation of the technical issues, preferably with illustrations in code. For example, in the above post there is one point at which you come close to being specific and then back off, saying it's borderline obvious. It wasn't obvious to me.
There is one point at which I somewhat follow you. You say that the complexity introduced by callback management grows nonlinearly with program complexity. I understand this to mean that logic organized into async callbacks isn't composable (you have to write new logic to implement the composition, as opposed to just applying some operator to combine them) and isn't orthogonal (if you want to call some code that's written this way, your code also has to be written this way, and each new layer gets harder to add). It's easy to see how this could rapidly get out of control. But I'm not convinced that it must. There may be designs that nip this complexity in the bud. For example, the work done by callbacks themselves could be kept to a minimum (and preferably be standardized, i.e. when i/o is received, store the result in some standard place). In this way the callback chains always return as quickly as possible. Of course then you need some parallel strategy for managing the control flow of the program itself - some sort of state machine, perhaps.
Perhaps this is greenspunning Erlang but if so I'd like to know how.
From my experience with Node.js, your interpretation is correct. I also share your optimism for finding a general solution to this problem, but that's exactly what jerf said isn't worth the complexity once you implement it.
Even if you do manage the callback complexity issue, there is still the issue of exception handling, which jerf also explains here: http://news.ycombinator.com/item?id=2150800
That said, jerf hasn't proven that trying is not a rite of passage. Sorry for the double negative.
Not a general solution. An app-specific solution. That is, I don't want a framework; I just want a consistent simple design for an individual app written in this style. That's a big difference.
For every major high-level language, there is at least one Node-like library, and sometimes more than one (Perl has POE, Event::Lib, based on my experience the raw glib wrapper isn't half bad albeit perhaps not the fastest, but you get good access to anything else based on glib, in fact Perl has so many that there's an Any::Event wrapper to remove your dependence on the underlying event library!). My point isn't that Node.js is bad. I actually don't think it is.
My point is that the hype is bad. It's wrong to think it's bringing anything unique to the table, because it simply isn't doing anything that has not be done literally dozens of times, except it's doing it in Javascript. If you want it done in Javascript instead of Python, more power to you. I'm particularly incensed by the idea that Node.js' approach to asynchronous is the only way to do it and the number of people it has produced who it has anti-educated into thinking Haskell and Erlang and all kinds of other languages can't possibly be asynchronous because you can't see the manually-chopped-up event handlers in the code. I'm not guessing. I've met these people online. You may know better, but a lot of people don't; whether or not it was intended the hype is actually lying to people about the state of the programming world, comparing itself to the world of 1995.
I am also trying to speed up the education cycle that all of those other dozens of attempts have been through in which manually-compiled event-based programming inevitably explodes into unmaintainable complexity, and none of the dynamic languages, including Javascript, have the necessary constructs to truly contain it. Some of the dynamic languages are even more powerful than today's Javascript, such as Python with its generators (though ECMAScript is supposed to be getting those, I don't know if any browser has them yet) and it's still not enough. The structure of event-based programming demands such an explosion. Been here, done this.
You can see it already starting to poke out from under the hype, if you're watching carefully. This is going to get worse, not better (because there isn't a solution, just a variety of hacks long since tried and found to only slightly improve things at significant complexity cost themselves), and I'm actually trying to do the community a favor by deflating the balloon so it doesn't pop so hard.
(If you know Haskell, and you look at the implicit type signatures being put on things like callbacks, it becomes easy to see the problem. The clearest place to see the problem is a function that takes a callback for something, until one day you need to pass in a callback that itself has to go do something that requires a callback and suddenly you've got a big problem. The usual callback in Node.js is actually just a relatively-pure function, they are not in IO, which is done behind the scene for you. Then when you need to do something else, you've got some real problems. Solvable, yes, but at a fairly significant complexity cost, partially because any given issue can be addressed but you can't really address all of them simultaneously (simplicity, exception correctness, dealing with control flow across callbacks, etc.). Every time you write a callback or choose where to break the function up into a callback you're actually laying down far more restrictions on the code than you can easily see, but I don't go to this explanation very often because by the time you can understand it it is also borderline obvious.)
What other alternatives are there? One, use Node.js with awareness of the issues. There are places where it is fine. I just would incredibly strongly anti-recommend it if you know you're going to be continuously developing whatever you're building in it, especially your core product, rather than writing "a proxy socket server for web sockets to conventional sockets" and being done at some point. The other alternative is to actually work in a language/runtime where you don't have to manually perform all this tedious work. There's a number of them coming out and one of them is probably going to go mainstream at some point; of the current lot Go would be my best guess. Google isn't pushing it, but it's still got Google's name on it, and I don't know of anything else right now with the equivalent name power. History suggests name power is necessary for a language to crack the mainstream in anything less than 15 years. It probably requires the least adaptation to a new style of the bunch, the other advantage it has from the mainstream point of view.