Yes that is the physiology barrier that you describe that will be difficult to overcome.
The flaw is everyone thinks they will be a better driver than an AI, even though very few actually will be.
So if I had to choose my children driving with an "average joe" / friend / etc vs an AI, I would say: the data says the AI will crash 25% less, therefore it is safer.
I assume you are better then a drunk teen that comes from a party and has 3 months of experience, The most road deaths that happen here in Romania are caused by young drivers cumming from parties late at night, maybe drunk, the car is full with people so you have 1 crash causing a lot of deaths, So the average driver from the stats is a terrible driver because the stats are skewed hard by the inexperienced, drunk or tired drivers. You can have both this next two facts true at the same time
1 replacing all drivers with AI is x% safer
2 replacing you an experienced responsible driver with an AI is less safer
The flaw is everyone thinks they will be a better driver than an AI, even though very few actually will be.
So if I had to choose my children driving with an "average joe" / friend / etc vs an AI, I would say: the data says the AI will crash 25% less, therefore it is safer.