There are 2 very different approaches depending on the reasoning your interest in running alternative firmware.
Reasoning 1: FLOSS/Libre principles - find whatever wireless router has the best wireless radios but still complies to your particular set of openness principles. More than anything the radio performance will still be your performance limitation so the rest of the box ends up not mattering and you can use your software straight on it without much worry. If you're ideal FLOSS hardware doesn't support running your ideal FLOSS "smarts" directly you can mix this with the 2nd approach, otherwise just stick with the one box.
Reasoning 2: You just want better software - find the device with the best radios and see if it has a native "AP" mode (it probably will). The radios will likely outperform the rest of the device if you want to do any "smarts" with the traffic so completely ignore whether or not that specific device can run open software and get an x86 tiny/mini/micro PC to run some set of software like OPNSense or whatever you prefer. The AP is then a dumb passthrough and the PC is flexible so the two sides no longer limit each other (both in performance and lifecycle).
Reasoning 1: FLOSS/Libre principles - find whatever wireless router has the best wireless radios but still complies to your particular set of openness principles. More than anything the radio performance will still be your performance limitation so the rest of the box ends up not mattering and you can use your software straight on it without much worry. If you're ideal FLOSS hardware doesn't support running your ideal FLOSS "smarts" directly you can mix this with the 2nd approach, otherwise just stick with the one box.
Reasoning 2: You just want better software - find the device with the best radios and see if it has a native "AP" mode (it probably will). The radios will likely outperform the rest of the device if you want to do any "smarts" with the traffic so completely ignore whether or not that specific device can run open software and get an x86 tiny/mini/micro PC to run some set of software like OPNSense or whatever you prefer. The AP is then a dumb passthrough and the PC is flexible so the two sides no longer limit each other (both in performance and lifecycle).