"Whether a methodology works or not depends on the criteria: team productivity, happiness, retention, conformity, predictability, accountability, communication, lines per day, man-months, code quality, artifacts produced, etc. Every methodology works if you measure the right thing. But in terms of the only measurement that really matters—satisfying requirements on time and within budget—I haven’t seen any methodology deliver consistent results."
You star with a premise that requirements, time constraint and budget are all set magically right before the project began.
That's actually a good point. In my company most projects start with a fixed budget, a hard deadline and unclear always changing requirements. Everything is already set before the dev team gets involved.
That's a failure of the project manager. Before any deadline is set the dev team leads are supposed to meet with all of the leads on the project, who in turn meet and come to a consensus with their teams. The results of the negotiations are supposed to start at the top with the idea and goals, go all the way down to the people who will be executing them, and then go all of the way back up to the top over and over until everyone until a consensus is reached. Things may change, but in proper/formal project management a change management process is in place where the leads must come to a consensus before any change is approved. Good management enforces this process, and good leadership at any level makes everyone above and below them aware of the implications of what everyone wants. Everyone has to be on board and working towards a common goal, and come to agreement to how best to reach that goal. The biggest problem I see is at the organizational level, where the project manager is not the one with authority to control the process. When this doesn't happen, the people above and below them are not protected from the implications and things go bad for everyone. This is taught in PMP courses, but what isn't taught is what happens when the organization doesn't give the project manager authority but instead turns them into a project coordinator or project expediter. A formally trained, knowledgeable and experienced project manager with the authority to control the process can and will make sure everyone is on board, but they can't do that when their authority is undermined by the organization, client, or teams. Likewise for the team leads. This is the definition of cooperation.
Since when did a customer actual have a solid understanding upfront that was 100% correct. Thats just la la land, and one of reasons for the Agile approach. Iterating after discovery works.
You star with a premise that requirements, time constraint and budget are all set magically right before the project began.