I am worried about that too. Right now the biggest challenge for autonomous cars is to recognize obstacles like pedestrians. I wouldn't be too surprised if we would end up with a situation where it's up to the pedestrian to get out of the way of autonomous vehicles because they have always right of way.
