We better design these systems not to fail then. Seems obvious but I think that’s the solution to what you’re describing. Having an unreliable human as a backup is not going to do it because as you say, and I agree, humans will become reliant on it.
The systems are not reliable yet. Fortunately humans are not totally reliant on them yet either so humans are still effective as a safety backup.
The trick is to make sure the crossover point (when humans become so reliant that they are ineffective safety backups) comes after the system becomes highly reliable. Unfortunately humans are already unreliable to begin with even without AEB, so we are just going to deal with a few incidents from autos, both those without AEB, and, until it becomes perfect, those with AEB.
