Maybe the downside is that someone who sleeps less will also tend to wander during the night, which increased the risk of dying before the modern world.
I think that's down to risk management skills of an individual. It could also be the case that sleeping less means you're more aware of external threats and ambushes and you can be more productive.
