Just because you have privilege doesn't mean you're a bad person. It's not your fault. You shouldn't really feel guilty. It's just luck a lot of the time. Luck can take away privilege as well. One unluck car accident can make you paralyzed. There goes your able bodies privilege then.
And not everyone has the same obstacles (that's the point of privilege), so you can't say for sure that your white ancestors had the same struggles (say) to get university educated as a current black female person today.
No one has argued in this thread that you and I are bad peeple because we're priviledged. What they're saying is that if we are not aware of that priviledge you are living in ignorance and should not be surprised when less priviledged people don't take us seriously.
Take Rosseau's famous example of priviledged ignorance: "If the peasants have no bread, let them eat cake." If we are not aware of our priviledge we appear just as ridiculous as that French princess.
Also, I'm pretty sure my white ancestors never turned on the tv and saw a state governor standing at the door of a university trying to block white people from getting in.
And not everyone has the same obstacles (that's the point of privilege), so you can't say for sure that your white ancestors had the same struggles (say) to get university educated as a current black female person today.