> In my experience, the most abject software development failures stem from building the wrong thing, not from building the thing wrong, and the former failure occurs outside anything that the authors even attempt to measure.
> ...
> ...the Accelerate view of high performance amounts to “the team can push a button and deploy changes to prod quickly”. If you care about any software quality outcome other than that, then Accelerate does not even claim to measure it.
This is correct, and I don't really think the authors try to claim otherwise.
The underlying reasoning is that if you cannot deploy quickly, then it will be very expensive for you to notice that you've built the wrong thing (because testing in the field with real users is an amazing way to get design feedback), much less course-correct quickly.
I've read the analogy of a big ship. If it takes you a day to turn, you will end up in the wrong place more severely than if you have a small ship and can turn on a dime -- even assuming that you're equally observant about heading errors in both cases.
So it's not so much that being able to deploy quickly guarantees quality, but it's sort of a prerequisite for being able to achieve quality. (As borne out by their further investigation into business metrics.)
Most of that reasoning is pure logic and doesn't need substantiation. The one blind assertion is that "deploying to prod is a cheaper way to validate design ideas than the alternatives."
Niwbthat might sound like a tautology too -- if you can deploy quickly to prod it's cheap to deploy quickly to prod! But what claiming is that going from expensive deploys to cheap deploys (a potentially significant investment!) is still cheaper than other ways of validating design ideas.
What do I base that on? Nothing solid. I feel like it's a recurring theme when reading about high-performance design/engineering teams throughout history: examples that immediately come to mind are China Lake, Skunk Works, Toyota.
Additionally, I trust that the DORA researchers have indeed verified that this is the case, as they say. I have yet to see evidence that slow course-corrections are an improvement. (Which is reasonable argument to make: if you're able to course correct too quickly, you might end up with "pilot-induced oscillations" where you overcompensate in a naturally damped system. Maybe there's an optimal deploy frequency that's impedance matched to the underlying system? But that's getting very speculative.)
Most of the article is casting doubt on the idea that the "DORA researchers have indeed verified that this is the case, as they say." Pretty convincingly too, IMHO.
At first I was going to disagree with you, but the closer I think about it, you're right.
My basis for disagreement was going to be that if "easy deployment" and "good economic performance" are correlated, the only sensical way that could happen is if easy deployment causes good economic performance.
But then I realised there's a trivial way for the correlation to go the other way: if we assume easy deployments are ineffective, then only organisations with good economic performance can afford to spend time on easy deployments.
(Of course there's always the confounding possibility. This can be analysed statistically (see Judea Perl) and it's not obvious that the authors have done this.)
So yeah, you're right. It's not clear that the authors have done that.
I think the assertion is "If you have everything else in the business right, then Deploying quickly makes it cheaper to fix issues and ship revenue generating features to customers, which will lead to more profit"
I think it's hard to say that there's a causal relationship though -- a lot of companies that can afford to invest in automation are those with already successful products + money to invest. This may change as it because easier to automate provisioning and deploying to environments (something I'm working on at my new company, jetpack.io)
But in this case (unlike in the classic "x, ???, profit" memes), there are connections that you can draw, that can plausibly lead to profit. If you've worked both with and without quick, easy deployments, you'll know that there's a difference.
> ...
> ...the Accelerate view of high performance amounts to “the team can push a button and deploy changes to prod quickly”. If you care about any software quality outcome other than that, then Accelerate does not even claim to measure it.
This is correct, and I don't really think the authors try to claim otherwise.
The underlying reasoning is that if you cannot deploy quickly, then it will be very expensive for you to notice that you've built the wrong thing (because testing in the field with real users is an amazing way to get design feedback), much less course-correct quickly.
I've read the analogy of a big ship. If it takes you a day to turn, you will end up in the wrong place more severely than if you have a small ship and can turn on a dime -- even assuming that you're equally observant about heading errors in both cases.
So it's not so much that being able to deploy quickly guarantees quality, but it's sort of a prerequisite for being able to achieve quality. (As borne out by their further investigation into business metrics.)
Most of that reasoning is pure logic and doesn't need substantiation. The one blind assertion is that "deploying to prod is a cheaper way to validate design ideas than the alternatives."
Niwbthat might sound like a tautology too -- if you can deploy quickly to prod it's cheap to deploy quickly to prod! But what claiming is that going from expensive deploys to cheap deploys (a potentially significant investment!) is still cheaper than other ways of validating design ideas.
What do I base that on? Nothing solid. I feel like it's a recurring theme when reading about high-performance design/engineering teams throughout history: examples that immediately come to mind are China Lake, Skunk Works, Toyota.
Additionally, I trust that the DORA researchers have indeed verified that this is the case, as they say. I have yet to see evidence that slow course-corrections are an improvement. (Which is reasonable argument to make: if you're able to course correct too quickly, you might end up with "pilot-induced oscillations" where you overcompensate in a naturally damped system. Maybe there's an optimal deploy frequency that's impedance matched to the underlying system? But that's getting very speculative.)