The idea that we should be less vigorous in pursuing the rights of women and minorities because managers will retaliate by discriminating against them is a form of concern trolling. Yes, policy changes come with unintended consequences. But this consequence is especially unlikely and silly, because it is culturally taboo, overtly unlawful, and hard to conceal from employees.
It's not diversity should not be pursued at the expense of something else. It's that you also have to plug those other insidious holes while you're at it, else it's more show than go.
Great, you hire lots of underrepresented groups, but then you can just push them out and keep the grinder going.