"There is no theoretical reason that anything is hard to change about software.
If you pick any one aspect of software
then you can make it easy to change,
but we don’t know how to make everything easy to change. Making something
easy to change makes the overall system
a little more complex, and making
everything easy to change makes the entire system very complex. Complexity is
what makes software hard to change."
Worth pointing out that this study does -not- equate "architectural complexity" with abstraction. Many consider use of "hierarchies, modules, abstraction layers" to be 'unnecessary complexity' where as the thesis clearly states they "play an important role in controlling complexity." OP is not a call to get rid of software architects, arguably it says ~'hire competent architects as bad architecture negatively impacts faults, staff turnover, and product success.'
"Architecture controls complexity", and under-/poorly- designed architecture while superficially "simple" will give rise to un-intended complexity. Microservices are the du jour poster child here.
Not a direct response, but some thoughts that come to mind without any direct conclusion:
Good abstractions make simpler software. Leaky abstractions multiply the complexity of software by a lot.
Some respond to this by making "simple" software that dispenses entirely with abstraction. This ends up in a lack of architecture where complexity still multiplies, though perhaps less than the typical mix of mostly leaky abstractions and a few sound ones.
However, it's kind of the nihlism of software and throws away the opportunity for us to actually improve our craft... so I'm not all too interested in it.
The uncomfortable truth is that software development is a series of decisions made day after day, and many people simply cannot reliably make one good decision after another. You see this in poker and chess too, there are many people who will never be good at those games (like me), and there are many people who will never be good at software. Demand for good developers outstrips supply and then you have the whole measurement problem on top of it. At least with poker and chess we know who is good at it, and who is not.
People regularly forget Rule Zero of patterns: Don't implement a pattern if it doesn't solve a problem. That's the difference between unnecessary complexity and controlling complexity.
IMHO people aren't forgetting it — it's pretty commonly the other way around.
Anecdotally, the people I see advocating for spaghetti architecture are often utterly convinced they're solving some critical problem. Conversations with stakeholders then turn into fearmongering — often helped by the fact that most stakeholders don't care about architectural nuance — and it becomes easy to wave off any competing simpler architectures by deriding them as "taking on tech debt." In general, it's surprisingly easy to play corporate politics to bring a convoluted architecture into reality.
"There is no theoretical reason that anything is hard to change about software. If you pick any one aspect of software then you can make it easy to change, but we don’t know how to make everything easy to change. Making something easy to change makes the overall system a little more complex, and making everything easy to change makes the entire system very complex. Complexity is what makes software hard to change."
https://martinfowler.com/ieeeSoftware/whoNeedsArchitect.pdf