"The trouble with fighting for human freedom is that one spends most of one's time defending scoundrels. For it is against scoundrels that oppressive laws are first aimed, and oppression must be stopped at the beginning if it is to be stopped at all."
-H. L. Mencken
It is a well known problem, but one that you must deal with if you are interested in defending principles that you stand for. The ACLU has defended the KKK in the past. Most recently: https://www.aclu.org/free-speech/aclu-em-defends-kkks-right-...