Management doesn't invest in things they don't understand. They rarely understand tech (even if they are technical people, tech is misunderstood).
If you care about the outcome of a craft, you'll get craftsmen, and they'll produce things and you'll pick the best ones.
Imagine if a chef had to serve the first try at a new dish, if a photographer had to use his first shot, if a director had to use the first take, if a visual designer had to use the first concept. These processes are easier for people to understand because a layman can tell if two flavors didn't combine well, if a model blinked, or if something looks off. People (even technical ones) can't experience technical problems immediately or directly in the same way. Other fields solve for this with engineering, but ours rarely is allowed.
We often ship the first architecture that gets brainstormed, and the first implementation that works. It's never "We implemented the MVP using patterns/frameworks A, B, C, D, here's a table of the strengths and weaknesses as we see them, statistical summary of the time to complete each feature, and of defect rates. This next slide is a plot of the relationship between request latency, capacity, and estimated infrastructure cost of each implementation". It's rare to even have refactors scheduled. Just cleaning up your shit is "wasting time", never mind how long it takes to ship new features given how cluttered things are.
Sure, engineering is costly, so is not engineering - you just aren't projecting or measuring the latter costs. It's all trade-offs, if you aren't considering expected values of different options, with numbers, then you haven't optimized, you just rolled some dice. Sure, of course it's always worked, everything always have worked for the survivors. There's a big difference between what's unforeseeable and what has merely been left unforeseen.
If my house is on fire, I don't want the fire department to wait for the better firetruck for the job to show up.
Time to market and allocation of resources are serious issues for management. They may not understand the tech, but I've found engineers rarely understand the business context, either. "Good enough" is good enough. And we've seen process get more and more focused on this since the late '90s... Agile, Lean, etc. Get a product out the door, even if it's buggy and flawed. Get feedback from actual customers. Find out what's really a problem and what we only think will be a problem.
Right, there's a whole spectrum between "a prototype worked once on Bob's laptop, how do we get that live" and "We invented 12 new declarative DSLs that encompass our problem domain, and formally proved their implementations correct."
Engineering isn't the latter end of that, it's precisely just the process of figuring out the trade-offs, the diligence of figuring out where on that spectrum (highly-dimensional continuum really) of process your project should be to get the greatest chances of success. Of course it includes time to market and concepts of "good enough". But it's math and planning and risk accounting and not the cargo culting, handwaving, business platitudes, salesmanship and politics of authority that get utilized in the majority of cases I've seen for making those decisions. These result in monuments to compromise -- the output of arguments between "captain cowboy" and "doctor diligent" (neither of which correctly account for real-world incidence of risk or what that costs the organization which employs them), arguments of engineering without any engineering being done.
Engineering isn't disconnected navel-gazing perfectionism, it's how you prevent it. Engineering is both why we don't live in Kowloon Walled cities, and why most places have water and plumbing and transportation despite a wide variety of obstacles and economic realities that would preclude "ideal" solutions.
"Get feedback from actual customers" is, more often than not, dishonest rationalization. If you never use customer feedback in order to increase product quality (instead of just growing the feature list), "good enough" is just a code word for brushing all dirt under the carpet.
The fact is: software is a lemon market and teams/companies are rewarded accordingly.
Instead of the 80-20 split at some tech cos, perhaps there should be a 70-20-10 split, of main project-side project-refactoring/cleanup. I would wager Google for example would be almost immeasurably better off with that approach.
If you care about the outcome of a craft, you'll get craftsmen, and they'll produce things and you'll pick the best ones.
Imagine if a chef had to serve the first try at a new dish, if a photographer had to use his first shot, if a director had to use the first take, if a visual designer had to use the first concept. These processes are easier for people to understand because a layman can tell if two flavors didn't combine well, if a model blinked, or if something looks off. People (even technical ones) can't experience technical problems immediately or directly in the same way. Other fields solve for this with engineering, but ours rarely is allowed.
We often ship the first architecture that gets brainstormed, and the first implementation that works. It's never "We implemented the MVP using patterns/frameworks A, B, C, D, here's a table of the strengths and weaknesses as we see them, statistical summary of the time to complete each feature, and of defect rates. This next slide is a plot of the relationship between request latency, capacity, and estimated infrastructure cost of each implementation". It's rare to even have refactors scheduled. Just cleaning up your shit is "wasting time", never mind how long it takes to ship new features given how cluttered things are.
Sure, engineering is costly, so is not engineering - you just aren't projecting or measuring the latter costs. It's all trade-offs, if you aren't considering expected values of different options, with numbers, then you haven't optimized, you just rolled some dice. Sure, of course it's always worked, everything always have worked for the survivors. There's a big difference between what's unforeseeable and what has merely been left unforeseen.