There are indirect tax implications: Once you roll over your 401k into an IRA you can't use the Backdoor Roth IRA technique anymore (well, you still can, but then you would have to pay taxes).
Does it help to have a brokerage that has only 401k rollover IRAs and another brokerage that's used for backdoor roth? (I'm guessing not). I've been postponing a 401k rollover for the reason you state.
No. All of your pre-tax IRA accounts across all custodians are used when figuring out the taxes of the backdoor Roth. However, if you have a 401k at your current job that has good investments you may be able to roll your IRA into you 401k plan. That will clear out all the IRA assets to give you a $0 figure when performing the backdoor.