From a safety engineering standpoint, handoff is a very, very bad idea. When you automate the common operation of a system, how is the operator supposed to get a feeling for the system? And when the operator doesn't handle the system regularly, how is he supposed to handle a situation that's so dangerous and unusual that the automation can't handle it?
If you have years of experience driving a car, maybe it'll work. For some time. But what if your last manual intervention was five years ago? Or twenty years ago? The rarer handoff happens, the more stress it puts on the manual operator, and the more likely things go wrong.
It's much better to have a safe default reaction (stop the car, shut down the reactor, ...) that kicks in when normal, automatic operation can't continue.
You probably misunderstood what I meant by 'handoff'. It refers to the range of programmed interactions between an automated system and a human, for example, a warning light on the dash is a handoff. A warning sound or an automatic request for user action is a handoff. Pulling over and shutting down is a handoff. You say it's a bad idea. It would be a very very bad idea to build systems that do not have these responses programmed into them. It would be like throwing an exception without any exception handlers.
If you haven't already, I highly recommend listening to the Econ Talk podcast for much more depth from someone who has been thinking about these kind of problems for years (I can't really do it justice in a HN comment).
Thankyou! I've been saying this for a while, the current approach of 'driver assistance' or 'autopilot' is terrible. The more reliable the autopilot, the less attention the human 'driver' is going to be paying when the autopilot actually does say "okay turkey, you fly it" and shut off. And that is far more likely to happen in an unusual (read: hazardous) situation. You're self-selecting for a human panic reaction followed by a crash.
If you have years of experience driving a car, maybe it'll work. For some time. But what if your last manual intervention was five years ago? Or twenty years ago? The rarer handoff happens, the more stress it puts on the manual operator, and the more likely things go wrong.
It's much better to have a safe default reaction (stop the car, shut down the reactor, ...) that kicks in when normal, automatic operation can't continue.