I still really don't understand what's so hard about a couple thousand views. I've had a couple posts hit the front page (even #1 on HN and #5 on prog.reddit), and my server was fine. I use Apache to serve the resources and Clojure to handle web requests. Nothing fancy.
Wordpress without caching on a VPS that is probably not configured well. w3tc or wp-super-cache would probably fix his issues. Installing it while the site is hammered.... will be fun. Perhaps as challenging as solving the easter egg.
Any out-of-the-box Apache setup on, e.g., Ubuntu will, with absolute certainty, appear to crash if you subject it to several thousand pageviews in a short amount of time. KeepAlive is on by default, and that essentially amounts to turning any group of five or more interested people who share a 3 second window into denial of service attack.
This used to drive me freaking bonkers -- I was installing caching plugins, offloading static resources to other domains, tweaking the number of worker processes, etc etc. Nothing worked, and my blog regularly crashed almost every time it hit an audience larger than HN's. Turned off KeepAlive, bam, no problems since.
If keepalive fixed it, your configuration had other problems especially if you were using mpm-worker. email me at hnusername@hnusername.com - I'd be interested in talking further. Ubuntu and Debian distribute configs meant for home users/developers - those configs are certainly not made for production webserving.
How hard would it be for them to make it degrade by default? ie. Use KeepAlive until it starts having heavy load, and then disable it when the load becomes a problem.
ab with 500 concurrent, no keepalives: Requests per second: 13428.16 [#/sec] (mean)
ab with 500 concurrent, keepalives: Requests per second: 14403.98 [#/sec] (mean)
Keepalives allow the browser to maintain a socket between the browser and server and request multiple objects. So, rather than having to do a connect/teardown for each object, you remove that overhead. Some versions of Apache in conjunction with php and mysql_pconnect had issues in the past that might explain the OP's issues with keepalives, but, it has piqued my curiosity. In general, keepalives should reduce communication handling and allow more throughput.
exactly what I did... tried to switch apache to port 81 to be able to configure the cache plugin. But even this way, all the disk accesses were taking 30 seconds each...
The real error was posting it there in the first place and not directly on the Jamendo blog... Never again! :)
