That's because you are using an extremely small sample; of course the individual differences are going to matter. Now, get a random sample of 1000 teams using waterfall and 1000 teams using agile, and you'll get much more meaningful data. The larger the amount of samples, the more the individual differences are going to be smoothed.
Of course, not many people have the resources to do that kind of experiment.
> Of course, not many people have the resources to do that kind of experiment.
That's what I meant about it not being practical. Who would invest that amount of money? There's infinite variations of different methodologies as well.
I'm not sure what the solution is but it's tiring seeing bad studies used to promote certain approaches.
>That's what I meant about it not being practical. Who would invest that amount of money?
Tons of organizations and governments could.
20 10 person teams * 5 methodologies = 1000 people, for a 1 month project = 1000 * $100.000 = 100M dollars.
In the grand scheme of things this is insignificant amount -- in a world where businesses spend $15 billion for buying Instragram. A military could do that kind of spending for buying a single airplane -- and such a research could potentially have a huge impact / savings on future soft-eng projects.
And it could be even subsidized or be tax-deductible. Or could easily drop to like $70K or $50K per month compensation.
Of course, not many people have the resources to do that kind of experiment.