Libevent is rock solid -- it handles the event loop for memcache, among other things. This guide looks like a great improvement/accompaniment to libevent's docs.
One problem with using an event dispatcher like libevent is that libraries that make their own blocking calls won't cooperate. That means you'll often be stuck with limited capabilties, unless you write your own libraries or hack up existing ones.
There's a Python project called gevent (http://www.gevent.org/) that works atop libevent. It monkey-patches the python socket module with a version that automatically cooperates with libevent's loop. It uses greenlets to make code written sychronously run asynchronously behind the scenes. If, for example, you want to use an Amazon S3 library that makes use of python's httplib (which would normally block and choke your event loop), it'll automatically work with gevent/libevent. If you need to handle lots of concurrent connections and don't need to be working in C, check it out.
Note that gevent is a fork of Eventlet (http://www.eventlet.org/), and there have been talks of reincorporating changes that went to gevent back into eventlet.
It's too bad that no standard method for a asynchronous events has been developed for Unix in general. Where I work, there's one single library for concurrency, and every program or toolkit uses it. We do a lot of network programming, so it would be unthinkable for any performance intensive application to do blocking calls exclusively.
libevent tries to provide a common interface to select, poll, kselect (on BSD), epoll (on Linux), and comparable calls on Windows. It's not a standard part of Unix, of course, but it seems reasonably portable.
Sure, but libraries underneath you have to cooperate for you to use it to the fullest potential. That no common means have been invented for that is what I'm sad about.
Also for python is pyevent, which works pretty well. I have used it to much success -- some people don't like the minimalist style. Also, the maintainer only updates sporadically, but I've never had real problems with it. http://code.google.com/p/pyevent/
OK, I gotta ask: what's with the sudden interest in event-based I/O paradigms? This is hardly a new idea, in fact it's very well-traveled ground. It was used heavily in the 90's when threaded multiplexing was still new (and threads were expensive), and it works well enough. But it's difficult and error-prone. It splits sequential algorithms up into multiple callbacks with manually managed shared state. It makes "what happens next" a really hard problem.
And it leads to some nasty state bugs, because conditions that are obvious in sequential code ("parse failed? Return ERROR_PARSE_FAIL to the top level request handler") become subtle (e.g.: parse failed, stash a failure result somewhere to be returned when the socket-is-closed callback happens, but oops! forget to actually close the socket so it lives as a zombie.)
Really, I'm curious. What exactly is it that made this thing the snake oil of the week?
It isn't snake oil - it works and works well in the right situations.
Sequential code is very readable, but not efficient at scale. If you run a single thread (without the event based approach), each request must wait for preceding requests to finish entirely before running. That can be extremely bad for latency. On the other end, say you run a thread per socket - that's not effective as the memory usage for reserving thread stack space will limit your maximum connection count.
Event programming is useful in that it effectively performs cooperative multitasking without the overhead of thread stack space. Multiple requests can be running in parallel without fully blocking each other. Within a single thread they do serialize, but only by the length of your callbacks. If a request has to do I/O, it will queue up the operation and then yield so that other callbacks can run. Ultimately, it is reasonably fair scheduling-wise, eliminates time wasted on blocking I/O, and is more memory efficient than threading when handling large numbers of simultaneous requests/connections.
- epoll, which was added to the linux kernel ~2002, makes it possible to hang on to potentially hundreds of thousands of concurrent connections on one machine. Just can't do that with threads/processes.
- the desire for real time web applications, which have to hold large numbers of concurrent requests (e.g. friendfeed)
- a focus on lowering web application latency, which can necessitate making data requests in parallel
- advances in software (such as gevent, mentioned below) that reduce the complexity of writing async software
- increasing use of remote services / slow requests over the internet (if you're working in a threading/process model, you'll need a process open for each concurrent request, which can become prohibitive)
- web applications are simply serving more requests than they used to be, and probably have more to gain from being asynchronous
Without async I/O operating a large modern web application would be a lot harder. Lots of huge sites rely on memcache, for example, which would suck if it didn't use event-based IO.
I don't think all of those items are really arguments about I/O paradigms, though. Again, event stuff has been here forever, and clearly is appropriate for some tasks. But it's a tradeoff: you have performance needs you can't get from a simple implementation so you resort to the fancy tricks. So if you're writing the client handler for a database, or something like memcached, you need this technique (whether you need an abstraction library is another argument...).
But most apps don't fall into those categories, and I'm not getting any of that vibe from this. It seems that a lot of people sincerely believe that this stuff is "easier" or "better" just because it has the fancy new Spicy Event Sauce. And the truth isn't like that at all, and in fact these are truths that we've all known for years now...
Hey ajross, do you know any good sources for reading up on different I/O paradigms?
It's one of those things that I've read about different methods here and there, but I've never come across one good source that compared and contrasted the various techniques.
The short answer may very well be no, since it somewhat depends on the programming language one chooses to use. As you'd likely handle I/O differently in something like Haskell, versus python. Anyway, I'm just curious Thanks.
FWIW, the C10K Problem page (http://www.kegel.com/c10k.html) compares various approaches to handling tens of thousands of simultaneous clients. It's from 2006, but is a good overview of well-understood techniques.
Argh... was just googling to find that link when you posted it. But yeah: this is the first attempt in the modern world to sit down and come up with a real guide to high performance I/O. Most of it is still relevant, though it was written before scary parallel SMP became common (8-way is routine for a server these days).
Yeah. If you have any other links, please share. It wasn't too long ago when people were excited about fork-for-each-connection servers, though at least half of it was "OMG Unix system calls from Ruby" from people who probably don't know C. (That "* is Unix" meme.)
Simpler approaches are good enough sometimes, though. There are plenty of services that won't ever need to handle more than a couple simultaneous connections, and that way you don't need to bother with asynchronous IO.
I submitted the link. I've been reading about multiplexing asynchronous IO because I'm writing a distributed filesystem of sorts. It seems like a good fit for both the indexing hubs and the storage nodes, and it pairs well with Lua's coroutines. Also, I'm not convinced that the "nasty state bugs" from reorganizing around an event loop are necessarily worse than the ones that come from using threads with shared state.
I know what you mean about an event loop complicating error handling from callbacks, though. It feels sort of like writing in continuation-passing-style.
Fork-per-connection (sequential) is simple and pretty fast, but it does have its limits (it works fine for a simple webapp, but writing a high-performance IRC server that way will be rather painful.)
Once you need more than fork-per-connection can easily provide, you're not doing sequential code, you're doing events and/or threads. "Threads suck" is a common theme lately, but both can be pretty complex. (Event-based code obscures control flow, as you note; threaded code is subject to very subtle and hard-to-reproduce bugs.)
One problem with using an event dispatcher like libevent is that libraries that make their own blocking calls won't cooperate. That means you'll often be stuck with limited capabilties, unless you write your own libraries or hack up existing ones.
There's a Python project called gevent (http://www.gevent.org/) that works atop libevent. It monkey-patches the python socket module with a version that automatically cooperates with libevent's loop. It uses greenlets to make code written sychronously run asynchronously behind the scenes. If, for example, you want to use an Amazon S3 library that makes use of python's httplib (which would normally block and choke your event loop), it'll automatically work with gevent/libevent. If you need to handle lots of concurrent connections and don't need to be working in C, check it out.