The language's exception facility does what it says. We don't need a unit test to prove that in every particular case, any more than we need a unit test to establish that the language correctly carries out our assignments or function calls.
You picked two languages (Java and Python) with extremely weak guarantees about disposal of resources when exceptions are thrown. Any time I acquire a non-trivial resource I must make sure it's disposed of properly, via a `close` etc. method. (And these are most cases of interest; acquiring trivial resources e.g. memory shouldn't need error handling in Go either.)
This isn't theoretical. I review a lot of Python code and I would say in over 20% of cases I see a try block with a `finally` longer than two lines, it mistakenly uses a variable that might not be set when an exception is thrown earlier than the writer expected.
When a controller calls a couple of gateways and a database access layer, it is almost never holding resources. I would agree that testing error handling is more interesting when there are resources to clean up or fallback logic to implement. That's just very rarely the case. Mostly I just need to bail out of the request.