The complaint is that by reducing taxes for highest incomes there are fewer resources for the state to address the wellbeing of its citizens. It is not that the poorest pay less tax, it is whether social services and other government programs like education, environment, etc. are cut as a consequence of lower taxes on the rich (note the difference from raising taxes - where the wealthiest have greater capital mobility and access to professional services to shield incomes from taxation, so increasing taxes can fail to raise anticipated revenues when things go the other way.)
The only difference between an argument against a tax reduction and an argument in favour of a tax raise is what you consider to be the status quo.
The situation is more symmetric than you think.
Greater capital mobility cuts both ways: when you lower taxes you can also attract more rich people and their capital.
(I am not saying that this effect will necessarily occur, only that capital mobility comes into play both when raising and when lowering taxes.)
Oh, of course, the real solution to these problems is to tax land and not income nor capital: you can't hide land nor can it leave the country. See https://en.wikipedia.org/wiki/Land_value_tax
