It’s not that the drivers are worse. It’s that the hardware acceleration of the router might not be available since it is all closed source/proprietary. If you care about having a better router experience, something like Microtik or an OPNsense box running on a general purpose computer is great. Then you can combine it with dedicated access points like Eero or Omada.
This is one of the reasons I've separated the router from the AP on my network. I'm running OpenWRT on a normal computer as the router because I'm familiar with setting it up and then using some TP-Link Omada EAP devices for the Wifi part of things. I don't like that they're completely closed as far as firmware/software goes but I'll live with that compromise because it gives me the ability to manage the wifi separately, and then the APs also support VLAN and some other features that lets me handle things nicer than I could with consumer grade routers (with or without stock firmware).
Can you suggest how to get the best router experience if I intend to use an alternative firmware? (I am thinking of OpenWrt right now)
I have some experience running m0n0wall (it wasn't deprecated back then) as a router and a Ubiquiti Long Range AP (bought old, used) as a dedicated Wi-Fi AP at my old home.
My personal current setup is 10 year old desktop PC with a dual Intel NIC running OPNsense as my router. This is more than enough power and aside from a relatively long reboot time (about 80 seconds or so) it works wonderfully. Connected to this router is my WAN (Frontier fiber) in one NIC port and a TP-Link unmanaged Gigabit switch with POE ports on the other. The Wi-Fi is provided by TP-Link Omada access points managed by a local hardware controller (TP-Link sells these and they support many more APs at a time than I ever would need). This setup has been amazing in terms of managing it and in terms of user experience. Fast Wi-Fi, built in roaming over my 6 access points, and a router that is powerful, fast, and very flexible to manage with a great community behind it. Would have loved an open source AP solution but Omada at least does work well just like most TP-Link stuff.
My sibling comment covers what I've done at a high level, what seems to work really well to me is to use a separate router and AP so that the wifi side of things is completely divorced from managing the router. That'll let you use something like OPNSense, OpenWRT, or PFSense to manage the traffic. If you don't already have a device to do that I'd highly recommend looking at ServeTheHome's reviews of some of the N100/N300 based devices that have been coming out lately, there's quite a few really powerful ones coming out that would work wonderfully for this (I'm using a Ryzen based normal PC that I've optimized for power since those didn't exist when I did this).
Some recent reviews that give an idea of what's out there (look further too, there's a few 4x2.5GBe + 2x10GBe ones too)
I'm actually running mine with the router in a VM in proxmox with a PCI-e passthrough NIC because I'm also running a few other network critical services that I wanted more isolation on (omada controller, mail server, ldap, etc.) but don't want the power budget for yet another server.
There are 2 very different approaches depending on the reasoning your interest in running alternative firmware.
Reasoning 1: FLOSS/Libre principles - find whatever wireless router has the best wireless radios but still complies to your particular set of openness principles. More than anything the radio performance will still be your performance limitation so the rest of the box ends up not mattering and you can use your software straight on it without much worry. If you're ideal FLOSS hardware doesn't support running your ideal FLOSS "smarts" directly you can mix this with the 2nd approach, otherwise just stick with the one box.
Reasoning 2: You just want better software - find the device with the best radios and see if it has a native "AP" mode (it probably will). The radios will likely outperform the rest of the device if you want to do any "smarts" with the traffic so completely ignore whether or not that specific device can run open software and get an x86 tiny/mini/micro PC to run some set of software like OPNSense or whatever you prefer. The AP is then a dumb passthrough and the PC is flexible so the two sides no longer limit each other (both in performance and lifecycle).