What if your browser receives Alt-Svc header and switches to http/3 on one network (say mobile data), but then you switch to a restrictive WiFi that has UDP disabled. All without restarting your browser/within one "session" of your http client. Wouldn't you start having connectivity issues that would be hard to troubleshoot? In that scenario, having http3 disabled is beneficial.
Changing network interfaces breaks connections and causes a new handshake. Browser session works at a different layer and doesn't prevent that.
QUIC actually lets you migrate between connections (because the packets are identified by a connection ID in each UDP packet rather than a 5-tuple). Clients will typically re-test a connection occasionally and downgrade as needed for this to work.