Are you sure ethnicity was always mentioned as the factor that determine the in/out group? It could be projection, an assumption from your own cultural milieu.
For cultural appropriation, sure. A person born in the USA to a racially Vietnamese family isn't going to get shit for wearing Vietnamese traditional garb despite being raised in America. An American that doesn't pass as Vietnamese sure would.
That ethnicity can sometimes be a factor doesn't mean A) it's always a factor or always the primary factor or B) that we can assume it's how Vikings a 1000 years ago would have thought about it.
Consider also the cases of Americans who, despite being able to show "Irish ethnicity," are laughed at in Ireland for calling themselves Irish. Or people from around the world who take Islam as their religion and are accepted into Mecca despite their disparate ethnic backgrounds. I'd also point to multi-ethnic nations such as the British nation where Cornish, Scottish, Welsh and Saxon are all seen as equally part of the nation.
As I said in my other comment there are a variety of things like religion, cultural practice, common leadership/political belief, geography etc which can and do play roles in creating in/out groups.
I'm not saying ethnicity never plays a role in these things, only that there's little reason to assume that genetics would have been the main playing point in the mind of Vikings.