Yet we don't allow random traveler to vote in local elections.
Modern travel is big specifically because the world is diverse place. If all societies would try to include anyone from anywhere, they'd become similar and traveling wouldn't be as great.
By that logic you'd have to argue that children, inmates, mentally ill and those that for whatever reason didn't register to vote in states where it's required are not the part of society.
Including outsiders in the society doesn't make societies less diverse. If anything it's making them more diverse because otherwise similar societies might include different outsiders and differ more because of that.
Not every argument is supposed to be double-sided.
Mix-mashing all societies just get them to lowest common denominator. Similar societies usually receive similar outsiders. Sometimes even outsiders from same place. Which just makes them even more similar. For example, north Germany and Bavaria are quite different places and have different culture. Yet same set of immigrants come to both. Local food is different, but kebabs are the same.
If each society can choose who they want to accept and who they don't want... Then we can agree that each society can judge for itself.
