It's not about successful communication, it's about how you present an entire group of people. Using the term "the blacks" or "Blacks" makes you sound like Archie Bunker, dude. Negative social signal.
Maybe the goal is to send a specific type of social signal so people like you can self-select themselves out of his life? The term isn't pejorative. So, chill out.
It's more pejorative for the same reason that "White Pride" has negative connotations in today's society while "Black Pride", "Asian Pride", "Gay Pride" have positive.
White people weren't / aren't systemically discriminated against since before the founding of this country. come on son, log off lesswrong and learn some social context.