Like anything, look at the incentives. There are simply very strong financial incentives to generate ridiculously unrealistic low bids.
As for people talking about "why is the software industry like that?" I have great personal anecdote that to me explains why. Years ago I was tasked with a small team (~5 people) on a software project. We all got together and spent a good solid 3 days on planning and estimation, and our estimate was something like 5 months. Sales team freaked out, they said they needed it next quarter, so we had 3 months. So we looked at our original estimates and tried to find ways where we could cut scope, and came back with a "well, if we cut out literally anything remotely non-critical, and pray a little, I guess we can say 3 months." When all was said and done, the project took nearly exactly 5 months, just with a ton more stress and worse quality.
So these days, I'm more wise to the ways of the world. If someone says "we need you to find a way to cut the schedule by 25%", I basically say "OK, we can cut scope, but in my experience it always takes the time of the original estimate anyway." So then when things eventually go over budget, I don't stress at all - I leave work at a normal time, and just say to myself "everyone knew how long this was going to take, we just wanted to pretend it wouldn't take that long."
A big reason software is chronically over budget is that people simply want to pretend it will take a shorter time than it actually does, but they also know there is some real value to everyone engaging in that "mass pretending" in the first place.
"When all was said and done, the project took nearly exactly 5 months, just with a ton more stress and worse quality"
When in the heck did corporate America stop having men/women in the adult sense? I work in software/finance and it's disgusting the number of people who know better but don't have the backbone to stand their ground. People are too afraid of the mysterious ways of "blowback" by corporate whiners and politicians.
As for people talking about "why is the software industry like that?" I have great personal anecdote that to me explains why. Years ago I was tasked with a small team (~5 people) on a software project. We all got together and spent a good solid 3 days on planning and estimation, and our estimate was something like 5 months. Sales team freaked out, they said they needed it next quarter, so we had 3 months. So we looked at our original estimates and tried to find ways where we could cut scope, and came back with a "well, if we cut out literally anything remotely non-critical, and pray a little, I guess we can say 3 months." When all was said and done, the project took nearly exactly 5 months, just with a ton more stress and worse quality.
So these days, I'm more wise to the ways of the world. If someone says "we need you to find a way to cut the schedule by 25%", I basically say "OK, we can cut scope, but in my experience it always takes the time of the original estimate anyway." So then when things eventually go over budget, I don't stress at all - I leave work at a normal time, and just say to myself "everyone knew how long this was going to take, we just wanted to pretend it wouldn't take that long."
A big reason software is chronically over budget is that people simply want to pretend it will take a shorter time than it actually does, but they also know there is some real value to everyone engaging in that "mass pretending" in the first place.