One constraint I believe you may have missed is that an idle thread does actually consume a significant amount of memory. While most resources a thread consumes are fairly small, the stack is actually quite large and is allocated up front. The usual default ulimit stack size for Linux is 8192Kb. This means for every thread created, 8 megabytes is allocated whether or not it's used. On Linux the OOM killer will start to kick in, by default, once you've overcommited your virtual memory by 50%. You can adjust the ulimit stack size and overcommit ratio, of course, but either way you're playing with fire.
- 8MB of virtual memory is allocated. In practice, only a page or two will be allocated to hold the base of the stack, and of course any memory required for the request's data.
- You can decrease the default stack size, and would if you were trying to work with a lot of threads.
(Disclaimer: my assertions apply only to Linux, I'm not familiar with the vmm in other systems).
I think you may be misunderstanding how the vmm works. When you allocate memory by calling mmap(), sbrk() etc., that amount of memory is committed whether or not you actually use it. Because many processes will allocate more than what they actually use, for example excess stack space, the vmm allows you to overcommit by a certain percentage, defaulting to 50%. This will allow processes to allocate 50% more memory than the system actually has. If that is exceeded, however, Linux has an Out Of Memory Killer which will start killing processes.