But putting aside the various incentives and psychological reasons why people tend to underestimate, I think there is a mathematical reason too.
Most complex processes with a large number of varied tasks tend to follow a log-normal distribution or something similar. And a property of this law is that when you go faster than expected, you don't go faster by much, but if you are late, you are late by a lot.
So, imagine the guy who does the estimate is completely unbiased and competent and tells you how long it usually takes (the mode), then there is more than a 50% chance that it will take longer (the median is greater than the mode), and on average, it will be even worse (the average is greater than the median).
But putting aside the various incentives and psychological reasons why people tend to underestimate, I think there is a mathematical reason too.
Most complex processes with a large number of varied tasks tend to follow a log-normal distribution or something similar. And a property of this law is that when you go faster than expected, you don't go faster by much, but if you are late, you are late by a lot.
So, imagine the guy who does the estimate is completely unbiased and competent and tells you how long it usually takes (the mode), then there is more than a 50% chance that it will take longer (the median is greater than the mode), and on average, it will be even worse (the average is greater than the median).