The disparity within AWS is the result of common inevitability that arises with managing technology built on over a decade of scaffolding. For sure there is a central planner within the company trying his best to put everything together into a cohesive whole. Technical debt is the most common name for this issue.
An "Architect" can't solve this problem anymore than an "Engineer" can fix the problem of technical debt because neither person can see the future. It's like pointing at everything wrong with a piece of technology and saying an "Architect" could've fixed that in hindsight.
"Architects" are not the solution to technical design and technical debt.
An engineering manager can place all engineers in a room and ask them to come up with an "architecture" that is the same if not better than anything a single "Architect" can come up with, and this "Architecture" will never solve the problem of technical debt for the same reason why we can't predict stock prices.
> In fact, this problem is probably more prevalent in greenfield over-engineered solutions than old legacy-ridden ones.
Yup. In that context, it is usually called Second System Syndrome (when discussing the person that is laboring under it's influence while in the act of designing) or Second System Effect (when discussing the resulting system):
I think if you ask, most anecdotal evidence from people who have worked with architects shows that companies with "software architects" don't necessarily have less "technical debt" then companies without.
Theoretically, those "software architects" would put a stop to "over-engineered" solutions, but why do these companies still have technical debt?
Most likely because technical debt is not solely just "over engineered" solutions.
An "Architect" can't solve this problem anymore than an "Engineer" can fix the problem of technical debt because neither person can see the future. It's like pointing at everything wrong with a piece of technology and saying an "Architect" could've fixed that in hindsight.
"Architects" are not the solution to technical design and technical debt.
An engineering manager can place all engineers in a room and ask them to come up with an "architecture" that is the same if not better than anything a single "Architect" can come up with, and this "Architecture" will never solve the problem of technical debt for the same reason why we can't predict stock prices.