This is a great idea. And unlike rate limits per user, you should get better utilization of the web server. A drawback is request latencies and queue sizes could get out of control without a feedback mechanism to the client. I would probably want some active queue management to return a 429 if something stays in the queue too long. You could also put a single rate limiter in front of the web server, to prevent server overload.
I return HTTP error 429 (Too Many Requests) when a single IP address has 100 requests queued. Anybody who gets into that situation is using the API incorrectly.
