There is a way to use fail2ban together with nginx's rate limiting to implement a temporary ban on addresses that are repeatedly abusive. As far as I know this is one of the least friction ways to achieve that.
The limit_req directive works by dropping the responses that exceed the limit but does nothing more than that. Fail2ban can be scanning the nginx log files for instances of someone exceeding the limit, and add them to e.g. a Redis set. You can then use a Lua plugin to track incoming request addresses and check whether they belong to the "banned" set (then you can drop the connection fully).
