If i'm reading this correctly for web applications setting RUBY_THREAD_DEFAULT_QUANTUM_MS=10 or maybe even RUBY_THREAD_DEFAULT_QUANTUM_MS=1 could be more optimal then the current default of 100, allowing for more throughput at the cost of potentially slower single shot response times?
I think you have that backwards; it should decrease latency and decrease throughput.
I don't imagine the overhead will be too bad though, so it may decrease latency while keeping throughput essentially the same.
Of course you'll want to benchmark on your specific load. For example, at my day job it'd make no difference because we don't use threads, each Passenger worker has just one thread and handles one request at a time.
I actually had a less than pleasant surprise when debugging sidekiq process with a higher than usual threads count, and noticing 100ms-multiples in my traces. 100ms before switch indeed seems to be too high for an app with a potential to misbehave.
IIRC it’s complicated because it can depend on the relative run times, and how fast the IO OPs are. By lowering the quantum you’re increasing the switching overhead (because there’s more handoffs), but it might allow the io thread to complete much sooner and stop contending with the cpu thread.
Python’s “new gil” (it’s some 15 years old...) uses a similar scheme, with a much lower timeout (5ms? 10?), but it still suffers from this sort of contentions, and things get worse as the number of CPU threads increase because they can hand the gil off to one another bypassing the io thread.
