No performance comparison with their working C code. Hrmph. Erlang's a better choice for this kind of thing because you don't have to worry about a lot of the things that could go wrong in the C version, and it's a bit higher level than C, but still... C is fast and working code is working code.
I would presume the Erlang version is much smaller than the C version. Nobody argues that Erlang is faster than C, but it might be fast enough for the task at hand.
Functional programs tend to be slower than imperative programs. They also tend to be much smaller. And they tend to be easier to understand and work with once the programmer gets the right mindset.
This just shows another area where C is not the only competitor anymore.
Sure, all that's true, and are good reasons to reach for Erlang before C when writing something like this. It seems a bit weird to rewrite working code though, and it's unfortunate not to measure it. I would bet on Erlang not being that much slower than C, and like you say, much smaller, nicer, etc... so it would have been interesting to have some data to think about.
Yes, you could theoretically struggle to write C code to work like Erlang, but processes are very cheap in Erlang, and distributed, parallel code is extremely easy to write. C performs well, but can be unstable; Yaws performs as well as Nginx.
Erlang's greatest advantage is not in its vertical but its horizontal scalability, which does have consequences for performance.
you could theoretically struggle to write C code to work like Erlang
Lots of people write non-blocking network servers in C (epoll, kqueue, etc -- libevent); that does not constitute "struggling to write C code that works like Erlang", but it still delivers a very high-performance end result.
C performs well, but can be unstable
C is perfectly "stable", it can just be hard to get right.
Where Erlang is really strong is that you can handle fairly long running processes that won't block other stuff that's happening. Of course you can find ways to do that in C, but it's just easier and way cleaner in Erlang. This works because of Erlang's internal scheduler.
The original motivation was exactly because it was interesting and fun. When we got something working, I benchmarked it on my MacBook Pro and saw 1500 reqs/sec being handled on a single core. Those benchmarks made it much more real and so I decided to put the extra effort into making it ready for production.
It may or may not be as fast as the FastCGI based C polling service that we had been using, but it's certainly more maintainable. I'd much rather manage three Erlang nodes than the 240 FastCGI processes that we were running.
In fairness, we could have written a threaded poller in C and ended up with the same ease of management, but it would have been a heck of a lot more code.
"In fairness, we could have written a threaded poller in C and ended up with the same ease of management, but it would have been a heck of a lot more code."
What's the equivalent of Greenspunning when you find yourself trying to duplicate the features of Erlang in some other language? Do you just say "Greenspunning Erlang," or is it only proper to use Greenspunning when the other language is Common Lisp?
I don't think that writing a threaded poller in C would be trying to duplicate the features of Erlang. Epoll / kqueue (and historically, poll/select) are in c, made to be used with C. That being said, the 283 lines of 37S application code is impressive to me. It would be fun if they opened up this bit of infrastructure (or at least its functional spec) so we could add it to the programming language shootout.
To save you the googling: "GreenSpunning is the ad hoc implementation of domain-independent language features in a language which lacks the feature; it can range from a simple one-line function or macro to the canonical description of GreenSpunning: a "half-assed implementation of half of CommonLisp''. "
The poller works great. Polling definitely not optimal, but Comet isn't without issues of its own. The Erlang poller was a fun project that Jamis and I worked on in our spare time. Transitioning Campfire to use Comet or some other push-based room updating is a much larger undertaking.
If I understand well, the client browser polls the server, via JS timers. There is no practical, scalable way for the server to update the client browser, so the client has to poll this way. Their code just answers these polls.
Of course, performance wise, this sucks badly, because most of the time polling returns no results.
There are a number of practical, scalable (albeit tricky to set up) ways of pushing events from the server to the browser, gathered under the umbrella term Comet. Google have been using Comet for chat within Gmail since around 2005.
Even with keep alive long polling, you have to reconnect frequently. With alive long polling, you also have the trouble of managing open, persistent connections.
I'm surprised to read that the Campfire servers does a mysql connect on each poll. I built an internal tool for my company similar to Campfire but allowed tabbed conversations in the same window (so that you can have a visible thread running with a few people without bothering others).
To store messages I used a simple PHP script that wrote out a json object per line to a file named with room+date. To poll for messages when you opened the client it would query the poller script with the room name and it would return all the json objects for the current day as an array plus the length of the file. On each subsequent poll the client would send the room name + the file offset - the server script would then open the file, jump to the offset and read in the rest of the file and return it as an array (empty if no new content) plus the new length of the file. It scaled really well (not that we had Campfire sized loads).
