Do you by any chance have an Nvidia gpu? Nvidia doesn't like people passing through their consumer grade gpus to VMs and blocks those. You can however enable a "hiding" mode so it works regardless.
Maybe have a look at the Archwiki about OVMF Passthrough That's how I got it to work.
Yes, it's cursed Nvidia. Thanks for the tip, I'll try that out! Edit: Yeah, not exactly idiot-proof. Without a simple list of commands it'll probably take a couple hours just to establish beyond a doubt that my hardware even fits the prerequisites.
Maybe have a look at the Archwiki about OVMF Passthrough That's how I got it to work.