You say that "some people who identify as the same religion [as me] have been homophobic" but I wonder if that's not minimizing the truth. If you're part of an opt-in demographic that's known for negative characteristic X, even if you yourself don't exhibit that characteristic, you're going to receive some transitive association -- and it's at least somewhat warranted because your participation is voluntary.
It's only known for that, because of a few high-profile examples. It's like saying that Muslims should just expect some transitive association with terror groups. I mean, they choose to still be Muslim, right?
