Obviously, it's a biased study that shouldn't be taken seriously, but while we're on the topic of Agile:
Consulting Agile, grew out of web development consulting firms as a defensive methodology for managing shitty stakeholders. They incorporated some genuine industry best practices, but they also have a lot there that's simply meant to blunt the worst impacts of having to deliver software in an environment where the person paying you doesn't understand the basics of software development (or project management really).
It actually works pretty well in those environments - you don't get hyper-productive engineering teams, but you do get somewhat consistent delivery and a stable enough working environment that the worst outcomes (not delivering anything at all) are avoided.
No large tech company that I'm aware of implements this type of Agile but plenty of big non-tech companies do. If you have a CIO in your company and the business refers to software engineers as 'IT' there's a good chance you do this. This is inefficient but it's probably good enough for those companies, especially if it's established, the switching costs alone would be a nightmare.
If you're using it as a startup then something has gone terribly wrong and you should seriously reevaluate what you think the future of the company will be.
It's not just a biased study, it's an article about a study of which the data is not published, so it's impossible to tell if there's any merit to it.
The excerpts of the study published in the article do not inspire confidence either:
1. Agile ("Agile Requirements Engineering") is defined as: Development starts before clear requirements, no complete specification, significant changes late in development.
I think even those who are opposed to Agile would agree that's a severely lacking if not completely incorrect characterisation of Agile development practices. It sounds more like a list of things that would cause an Agile project to fail.
2. Impact Engineering is defined as: Use of all engineering practices studied which increase success rates.
Why doesn't the author dare to give any properties of Impact Engineering up front? This sounds like he's just baiting for people to buy the book so they could learn what magical engineering practices are studied therein. It smells scammy.
Yep, and the big companies basically use Agile as an excuse for not thoroughly planning and vetting ideas. Just build it and figure it out as you go. Then rebuild everything in 2 years since you're left with outdated and unmanageable spaghetti code and you burnt out all your SMEs so they took the knowledge with them.
Several years after a project completes when the system is finally sort of working and the end users know what to expect new management will come in at a high enough level and demand everything be rebuilt using their tech stack of choice because they need the resume entries.
Then of course you'll get a buggy second system that is back at square 0 which the business doesn't understand/hates and the developers hate because they're constantly fighting fires.
Yep, this is what my experience is. It could be a cost explosion too, like using expensive mainframes and migrating to something else before the MIPS cost explodes.
Yes, I've heard of some systems successfully migrating from the mainframe at my company. It tends to be a painful process. There are still some apps that need to be migrated.
Waterfall and spiral model failed in the general case for the reason of plan everything in advance, so unless it's a safety critical system I'd say Agile is still better.
I say this because no matter what methodology is used there will be spaghetti code regardless, mid tier devs like to over engineer solutions, junior devs don't know better and do whatever is first on stackoverflow.
Business execs also don't ever know what they want so change a system half way through when they see prototypes so then you still end up with Franken-systems.
To that end, you might as well start with a half spec'd system as it saves time.
You make it sound like Waterfall had full plans and completely vetted ideas.
It didn't.
> Just build it and figure it out as you go.
That sounds like hundreds of waterfall projects that I was pulled into after they hit the 1/2 way mark and we're running behind because of unexpected scenarios.
It had better plans than the way Agile has been done on the teams I've been on. You can certainly misuse either. In my experience, planning is neglected more Agile.
Having been on the “customer” side in such companies, I’d say the improvement is the result of more frequent interaction which improves mutual understanding.
The internal customers are forced to think through requirements in detail (albeit in pieces). The developers are forced to explain what they are doing.
Too many projects I’ve seen suffered from a lack of communication. Customers would hand things off (“just build x”, lots of customization requests that aren’t thought through). Developers would start guessing and working in the wrong direction, often going on complicated tangents as a result of the customizations).
Consulting Agile as you call it really helps both sides to understand what’s actually needed and how to get there.
I have observed that there are two kinds of Agile.
"Agile in doctrine" is the kind all of the anti-Agile crowd rail against because they've been burnt by working for some company or manager who drank the Koolaid without actually understanding what Agile is. Blindly following some consultant's Agile playbook is not Agile.
"Agile in spirit" is a bit like The Zen of Python: X is usually better than Y but there are times where it makes complete sense to do Y. In order for Agile to work, you need to have leaders and CIs who understand that. Agile emphasizes communication and flexibility. Agile is advisory, not prescriptive.
I largely agree with you, but when it comes to evaluating competing methodologies, what you refer to as “agile in spirit” turns quickly into No True Scotsman. “If you project failed you mustn’t have been doing agile correctly.”
There is some virtue to your criticism, but it is also not entirely true. There are things you can check to see if a team is doing Agile for real, like, what's the last process change you experimented with to see if it improved your outcomes? What experiment failed and was dispassionately removed from the process because it didn't work out? When your team's workload nature changed (e.g., from new development to maintenance), how did you adapt your processes to match? Do your processes come up from the team and their experience or do they source from above with effectively no feedback?
At that point, if you are doing those things, there is a certain amount of validity to the criticism that if it didn't work, you really weren't doing it right. Either something external jammed you up so you weren't able to adapt and truly follow agile, your team was personally unable to execute on the adaptation for some reason (structure, personality issues, experience levels, being simply too large to be able to be this flexible because large teams simply need more structure), or the task was simply too hard in the first place for some reason (such as "it doesn't matter how agile and smart the team is, you're not getting 4 people to produce a standards-compliant browser that is also an office suite in six months").
To me the single most common evidence of failure to perform "Agile in spirit" is management holding specific teams responsible for missing a deadline after a feature or due date was changed. It shows that the management doesn't understand software development. If you think you can add features or move delivery dates to the left without consequences, you've misjudged your entire industry.
Yeah, my former wannabe-boss was very surprised to realize that splitting the company in two and selling one part during my onboarding period might have a negative effect on my productivity.
We're not doing a very good job of picking the right people to manage things; quite the opposite, in many cases whatever success is despite the leadercrap we have to deal with.
As another commenter said, it’s difficult to compare “Agile in spirit”, but I’ve found that remarkably successful
Evolving the methodology to suit the team, product, and company in an additive way seems the best outcome I’ve seen.
Rarely will I have daily meetings, we can slack. Maybe twice a week hop on a call.
For story points I’ve seen models of using Fibonacci sequence, having 4 categories rated which then convert to points, or even (less desirable) 1:1 mapping to days of work. Personally I like the 4 categories approach, it feels less subjective as a whole despite each category being subjective to a degree
And in the end, if process gets in the way of productivity, I say go for it, we’ll either go back and document what happened or refine the process later (though rarely if ever have we skipped QA for good reason)
> Evolving the methodology to suit the team, product, and company in an additive way seems the best outcome I’ve seen.
This to me is the core of agile: the people doing the work have the flexibility to adjust to match their goals and needs. So much of the consultant-industrial complex comes down to finding ways to helping business people who aren’t willing to give flexibility or clear requirements ways to not do that which don’t sound like saying no.
The problem with "Agile in spirit" is that if adopt it and don't get the desired results, Agile aficionados will quickly remind you that you weren't actually doing Agile.
This is a common refrain, but I don’t think it’s true. “Agile in spirit” people will recognize the failure and look for ways to adapt so similar failures are less likely to occur. Rejecting that opportunity to reflect and adjust is where “not actually doing agile” comes into the discussion, and it doesn’t really have anything to do with “agile” at that point.
If you applied the same reasoning to any other iterative self-adjusting process, you’ll find they all tend to fail before the iterative aspect has been applied. Because that’s their nature, they’re iterative processes as a mechanism to respond to failure.
The by far strongest correlate of a complete clusterfuck of a project is how often, per meeting, agile is invoked.
In fact, I've discovered you can use agile as a sort of meeting hand grenade, if you don't like the direction a meeting is headed, like they're about to decide on something stupid, you can just throw in "wait, is that agile though?" and the rest of the meeting will discuss methodology, never arriving at any sort of conclusion.
Yea. I do Fractional CTO consulting and usually have good results with developers because I focus heavily on removing bottlenecks, unhelpful processes and meetings, etc.
The challenge is always when you run into sacred cows from other levels of management.
- Hard deadlines while acknowledging we don't have enough information to commit to those deadlines
- Story points as a measure of time
- Absolutely overloaded teams being asked to deliver new features without being given time to improve underlying issues in the system...and then having to constantly stop what they are working on to deal with production issues caused by those underlying problems.
It's like clockwork sometimes and the conversations are always hard.
So kind of like the stereotypical communist meetings where they spend all their time in meetings debating each other and obsessively discuss the finer points of socialism instead of ever accomplishing anything.
My team were employees. But the PM and several developers all came from a Canadian consulting firm. We were being project-managed from the outside. And any time some piece of functionality needed to be added, the new boss kept asking, "Why don't you just use a stored procedure for that?"
In the end, the boss decided to just fire everybody and replace us with Oracle stored procedures. Not even kidding, this is not a "things that didn't happen for $400, Alex".
I'm not really sure what to make of this article, it says itself it's a study commissioned by promoters of another methodology. If they're consultants trying to pitch their consulting services, they need these types of things to sell.
But my fundamental reaction is, what does failure mean. Because in my world view failure should be expected and accepted and learned from. It's entirely possible to spend a bunch of time avoiding every possible failure mode, and really not delivery much value at all... but we've successfully avoided the work being considered a failure.
I've been reading that 50+% of software projects "fail" ever since I started programming in 1992 (long before the term "agile" crept into our lexicon). They consistently fail because they consistently have deadlines but no actual definitions. That was true before Agile, was true during Agile, and will be true after Agile.
I sometimes wonder if our definition of failure is flexible and moves so that half-ish of projects are below average thus failures. I once had to do a post-mortem on why a project went 5% over schedule -- that's a pretty soft definition of failure.
> But my fundamental reaction is, what does failure mean. Because in my world view failure should be expected and accepted and learned from.
That's fine in some domains. It's not acceptable in others (i.e. you cannot just try things and see if they stick).
But regardless of that, you still want to minimize failure. So if one methodology (Agile) pushes to spend less time on fixed and clear requirements upfront, and this leads to higher failure rates, this is still an issue.
Agile is about reacting to changing requirements. If the requirements never change, then agile doesn't promise to be better than other methodologies. It came from a rapidly changing crashing markets in the .com bubble. How would the other methodologies fare if requirements changed two or three times in the middle of the project? That's when you need to be agile.
How did they even measure "success" if the requirements weren't given up front? What did they expect at the end?
Agile means delivering very small increments to users at a frequent pace, allowing them to provide feedback early and often. When done correctly, this eliminates a significant amount of guesswork. However, if the client (or, indeed, your own team or management) does not fully understand the process, things can go spectacularly wrong in many different ways.
We still need to explain what "agile" is. Ask yourself why do we "need" to deliver small increments at frequent pace? Isn't it to be able to react to changes?
It's main purpose is to react to changes, and make the whole process sustainable. It's literally one of the 4 main lines of the manifesto and the second of the 12 principles, and included in the meaning of the term "agile" itself, while guesswork isn't mentioned. But I can see how trying to predict what the markets will look like in 6 months involves a lot of guesswork, which is eliminated. I hope you can see that the only reason why guesswork is tricky is that there are a lot of changes in the meantime, so the ability to react to them is the deeper, actual main purpose.
I mean you are entirely discounting that it also allows engineers to pick their tasks and pace them as they see fit, as well as have input into the creation and prioritization of those tasks.
Everytime I hear an engineer complain about this, I ask them if you wish the PM would just write a PRD full of guesswork tasks and you execute on them unquestioningly.
Some engineers complain about having the ability to pick their tasks and have input in prioritization and planning? This has always been the most attractive part of agile for me. Self-organized team. Built around motivated individuals. Striving for technical excellence.
That's cargo scrum, very common disease. Teams without autonomy or self-organization. The queue should consist of higher level tasks, milestones, and the team should rework them into the small tasks that will actually go into sprints. Of course there is a motivation to pick the higher priority tasks first, but team members should be able to pick a task with which they can most efficiently contribute to the sprint goal, it's not always the top one.
Projects with fixed, clear and unambiguous requirements (and the necessary counterpart - fixed budget) I worked on typically turned into a nightmare, because they were not actually fixed, clear and unambiguous. You could pretend everything was fine for like 80% of the allocated time, the next 80% were spent in bickering over possible interpretation of some not-so-unambiguous line in the spec, death marches, blaming each other (client and supplier, but unfortunately also within the company).
The main innovation of agile is showing failures early and thus being able to quickly react and correct course, meanwhile in the waterfall the project spent most the time in a quantum superposition without possibility for correction.
> It's not acceptable in others (i.e. you cannot just try things and see if they stick).
Why not?
> So if one methodology (Agile) pushes to spend less time on fixed and clear requirements upfront, and this leads to higher failure rates, this is still an issue
That is an incorrect characterization of Agile. Maybe some "agile" companies use it as an excuse to avoid requirements, but that is definitely not part of the process.
If you fulfill the role of a supplier of a component in a bigger product (especially in an industrial setting), you cannot "try to build something".
People are not happy if you tell them that you want to iterate on their control system and deliver incremental value, or tell them that it's not possible in the contractually agreed way.
> That is an incorrect characterization of Agile.
These are also the cases where you just have to work out overarching requirements upfront.
Ah, fair. This is a startup site so my mind was on organization that have the freedom to pivot.
It is supposed to be that new requirements come from the customer though, working closely in conjunction with engineering team reps. So it’s more like “deliver MVP control system software, test it out with actual users, then incorporate their requested changes into the next sprint.”
I’m not surprised if most so-called agile teams didn’t do that though.
Yes, that's possible for product development in a startup (even necessary, because you don't know what you need to build), and in web dev, end user facing apps,...
For many other domains, the full incremental / Agile loop is not possible, and becomes disaster if it results in skipping upfront planning:
The counterparts you need to integrate with are often not done yet. Maybe the hardware is not finalized. The end user is years away. Maybe each test run is hugely expensive.
There is no MVP here, as in the system works as expected or it doesn't.
You cannot meaningfully iterate the core software running an MRI machine, a surgical system, financial clearing, pick-and-place machines, automatic rail protection, etc. towards correctness with the end user.
My startup is a hardware company where we are our own customer. You absolutely can iterate these kinds of projects instead of top-down waterfall planning based development. It just requires dog-fooding the product and using mockups or simulations for parts that haven’t been implemented yet.
> So if one methodology (Agile) pushes to spend less time on fixed and clear requirements upfront
Fundamental to agile is to have clear requirements, the time is not fixed.
With reference to the quality (scope), time, and cost triangle, each of the major methods picks one point to emphasise, another to fix (not allow to change), and another it manages the change of.
Agile wants speed so time is its emphasis, it does that by fixing cost (over a release period) and allowing changing scope, and it does that by limiting the features that go into a release. That doesn't mean that the features required should be unclear. I will deliver these 5 features out of 30 in the next 2 weeks at X dollars. If one of the features isn't ready, we'll drop it. No, I'm not doing features 6,7,8 till later. Yes, you can change you mind about what you want for later releases.
Lean emphasises quality and fixes cost while allowing time to change. You can get your burger for X dollars but you're 5th in the queue, you'll have to wait.
Traditional is about predictable cost, it (tries to) fixes time and allows change in quality to remain on budget. If everything goes to plan you should come in on time and on budget but maybe with less features or worse features than you wanted.
Of course, we know that traditional done badly produces over-budget, over-time projects that don't do what they were supposed to. It seems that agile methods are being used as cover for bad management because, as this entire thread is evidence of, there is widespread misunderstanding of the basics of project management. If you're going into an "agile" project and the requirements that you have are unclear then you need to kick up a fuss and hold the PM and the product owner to account. They should certainly be clear by the beginning of each sprint.
The article also doesn't go into what kind of software was being produced, etc.
If you're building control systems for the ISS, or an interplanetary probe, of course you need requirements hammered out pretty well before you start development; you can't just continually push updates out incrementally.
But if you're building Yet Another CRUD App at your startup's third pivot, then getting every requirement written down beforehand is virtually impossible.
This is the issue of agile. It is a blanket prescription for every scenario which of course … fails.
If you are talking about customer expectations, yes, you can manage customer expectations. As long as you are honest and upfront, customers tend to be forgiving.
On the other hand, if you are talking about a core API or building block that downstream APIs will depend on, then no. You can't deliver a API that writes to the database/storage layer that only works 90% of the time.
> "Our research has shown that what matters when it comes to delivering high-quality software on time and within budget is a robust requirements engineering process and having the psychological safety to discuss and solve problems when they emerge, whilst taking steps to prevent developer burnout."
Isn't "Christian" one of those predicates where identifying as such (public profession of faith) is what makes you one? I mean unless you're going for a denomination that requires baptism.
Yea, that's what it has devolved into. At least Catholicism can protect its brand against crazy stuff like that. The agile guys should have done something like that from the start... if they had known what was coming.
I mean.. "science" is what works too. If something turns out to be false, we throw it out and say it's not science. This is good. I don't see a problem with this at all.
But pushing useless stuff that doesn't work and then leaving, yea that's not agile :P
Science has proven methodologies to it and those are explicitly taught, followed, and reviewed. Agile coaches offer no real guidance if the bulk of the advice is "do what works". No shit...
The methodologies were discovered after the fact though. The only real scientific method is "if you are a winner you are with us". In fact, plenty of scientific methodologies are NOT tested and probably super bad, like the modern peer review system, or grant writing/proposal/reviews.
Neither of those are methodologies for actual science, but rather publication and funding issues. The methodology most closely related to peer reviews would be idea of reproducibility. There are plenty of peer reviewed things that fail reproducibility.
I worked on one team that had this and it was amazing. It implicitly helped with burnout too. It was an "Agile" team, but we followed more of a scrumifall model that worked well for that project.
I've been on a bunch of Agile teams (some more than others) since then, but the requirements processes are all garbage. Then the leaders wonder why things take so long, there's so many bugs, burnout is high, etc. It's really not surprising.
It was an app that was heavily focused on business processes, like routing items between different groups. We had spreadsheets and process maps for each business process. This means the business actually had to think about the possible routes it could take and the data required on each step. So the spreadsheet would tell us all the possible data exposed on the step including how it was treated (R, RW, required, formats/types, etc). It was magical.
Once a process was created the business would have to sell any process changes to the product owner by either showing it was a regulatory/legal issue, or using a cost study to show it would save money. So not back and forth or superfluous changes.
This is the type of engineering method that I try to instill in my teams. It's very dependant on good practices at every level of requirements management.
...Frankly I think all the standups & the pressure from constantly asking me if I am contributing (every single day?!) instead of trusting that I am... burns me out.
I don't mind 2-3 standups per week, 15 mins max.
But 5x 30 minute standups... Is exhausting. It feels like a lack of trust from product managers.
Oh yeah... and unqualified, zero engineering experience, diversity-hire product managers. That's super annoying-- being managed by people who did not obtain their role via the route expected of an engineer (having demonstrable engineering skills)
If you read about agile, standups are supposed to be conducted among the dev team members who are working together on something, and primarily for their benefit, not as a status update to or for someone else. The PM or whoever is considered the "product owner" in scrum could listen in and help unblock issues, or report delays to other people, but not demand more information or change things up that have already been planned. Of course, it actually working this way is rare, although I have seen it.
I haven't really seen this, but a poorly performing product manager or product owner completely breaks the scrum or agile model (to the extent it works at all). They are assumed to basically know the domain and know what needs to be built at a high level, and the software requirements originate from them in the scrum model. If they don't know what the requirements should be or how to communicate them or how to collaborate with the engineers on the requirements, it is completely garbage-in-garbage-out. On the other hand, working with a product owner who is a domain expert and happy to help define software to solve their needs can be a joy.
The equation of "agile = standups" has done more harm to agile (and projects using it) than anything else. I run a company and we use a lot of agile project planning methods, but threw out the standup. Especially in the era of remote work and varying timezones, it makes no sense at all.
That's all kinds of wrong. A stand-up is supposed to take 15 minutes max, and if no one is stuck with anything, it shouldn't take more than just a couple of minutes. It's not supposed to be a "status meeting" at all.
> However, while the Agile Manifesto might have its problems, those stem more from its implementation rather than the principles themselves.
Is the author an Agile coach? This is the classic response when people observe that Agile has failed to transform their team: you're doing it wrong, the problem is you. When systems seem good in theory but tend to fail in practice, you should just try to learn what you can from them and move on to the next thing.
No, it really is true. Agile is widely misunderstood. The way scrum is implemented in 90+% of the cases is a complete mockery of what it was supposed to be. A lot of coaches completely miss the mark as well.
And the fundamental problem is that agile is completely incompatible with the way most enterprises are built. A truly agile team talks to the users of the product, and they have complete control and responsibility over the development process. That just doesn't happen because managers want to manage, and as much as developers would like freedom, they'd often rather not have the responsibility.
A lot of us were doing some form of agile before someone came up with the word agile.
I'll give you an example:
Imagine the process of creating a retro video game.
First, you might write a function that clears the screen.
Next, you'll get something simple drawn. Maybe just a simple block.
Then you write a function that allows you to move the block with a controller.
Then you swap the block with some sort of character animation.
Next, you'll write a routine to draw girders.
Then you'll implement gravity.
Etc., etc. You'll just keep iterating until you have a well-balanced and engaging game.
That's agile. You may have had a lot of the game designed beforehand, or none. In fact, you could even create the game design documents with the same method. Make a quick first draft, get feedback, make adjustments, and repeat. And then keep going back to it whenever you notice something doesn't work or some better idea comes along during the course of development.
Originally, agile had nothing to do with rigid meeting schedules. Or "adapting to changes," or whichever buzzwords people like to throw around.
Electric cars have electric motors. So if you attach an electric motor to an F150 without removing the gas engine, you have an electric car. And it has terrible gas mileage.
> a complete mockery of what it was supposed to be
I started programming long before "scrum" or "agile" or even "XP". One consistent observation I've made around every methodology that comes along is that management manages exactly the same way they used to manage after they roll out the methodology as they did before the methodology.
Honestly, 99% of the time people are doing it wrong. People are either too strict with it or too "loosey goosey" with it.
*EDIT* It doesn't have to be correct, it just has to work for the people on the team. If it works, then use it, if it doesn't then change it. That's agile.
If people can only ever get into the goldilocks 'not too strict, not too loose zone' 1% of the time, then it the process is by default not fit for purpose in almost any circumstance.
Likely the 1% of the time it works is the rare case that you have a small and exceptional team working together, in which case the project management methodology is unlikely the reason they are working well.
You don't have to, just find what works for your team and use it. It may not be perfect or exactly how it is supposed to be but if it works then use it.
No, they need to come up with that on their own. What exactly "wrong" means? And why do you think you know it better than them? What if they find some better way that you didn't know about? Agile says that it needs to be sustainable also for the sponsors. Show the team that if the sponsor isn't able to sustain the next sprint, there won't be a next sprint. The process is up to them.
That certainly applies to Scrum but agile is more of a religion than a system. It's a bunch of extremely vague "principles" whose meaning is often anyone's guess.
What constitutes agile is basically a matter of opinion.
The study isn't shared, so we can't see the methodology to determine its accuracy. We see the results of the survey; but that's not a study, and the survey itself isn't shared with us.
I don't dispute that agile has its issues -- and I'd be surprised if 'agile' failure rates weren't an issue. We can empirically see the result of 25 years of 'agile', and it's not somewhere we should aspire to be.
The issues comes down to this:
1. Is adopting an 'agile' methodology sufficient to ensure (insert goal here)?
2. Has the software industry adopted a method of operating that creates a statistical quality control; and adopted practices that investigate and remediate special causes and common causes of variation in quality?
To spoil it: for #1 the answer is 'no', and for #2 the answer is a resounding 'no'.
There are practices that are ingredients to the world described in #2, but we as an industry have not yet adopted a systematic way of thinking and resolving problems in the software development process.
I think we'd be better off adopting Deming's method of management and his System of Profound Knowledge and be far better off than adopting the agile fad of the week; even as I admit I still have open questions on how to apply his work to Software. (See "The New Economics" 3rd Edition, and "Out of the Crisis", both written by Deming).
In any discussion on this site or others about Agile, you will find a flurry of PM's rigorously defending it with the typical line of reasoning -
If you tried agile and it failed, it wasn't actually agile.
When you show how it was actually agile, you didn't do it right.
When you show you did it right - go back to argument 1. It's a really circular no true scotsman style of argument.
Obviously this "study" is not good or even really a study, but I think the general arguments it makes aren't tremendously out of line. I have been an IC on agile teams, an IC on more traditional teams that I guess would be called "waterfall," and I've been tasked with implementing a scrum process both as a team leader and as a PM in my career - I've personally never seen it work on Ops/Infrastructure teams. In fact, I'm convinced it can't. It's far too difficult to determine the scope of work, far too interrupt driven, far too difficult to measure actual velocity, way too easy for engineers and managers to bullshit the system.
All the arguments I see in favor of it over the years strike me as pretty dogmatic. The way I like to manage projects these days is a kind of kanban system with a task queue, and 1-2x a week brief cadence meetings to discuss priority/alignment and any blockers.
edit: should probably clarify I'm using agile == scrum here, which I know isn't technically correct, but that's how I've seen it implemented in 99% of cases so far in my career.
> The way I like to manage projects these days is a kind of kanban system with a task queue, and 1-2x a week brief cadence meetings to discuss priority/alignment and any blockers.
To me, distilled down to its core, agile is all about faster feedback loops and course corrections.
If you’re regularly getting good, actionable feedback on your work, and maintaining alignment, I think you’re living up the spirit of agile.
There are plenty of “agile” teams going through the motions, doing the ceremonies, but not actually reaping what the real benefit is supposed to be: frequent, actionable feedback that helps guide and align the next phase of work.
I agree with you completely and what you said brings to mind something I wrote on my blog a few days ago:
> I posit that if I showed the Agile Manifesto bullet points (without the title) to a group of people with cringe-worthy titles like Scrum Master, Agile Delivery Manager, Agile Coach, or Head of Agile Delivery, and asked them “Is this agile?” almost all of them would say it isn’t. I can only imagine the effect that revealing the title and explaining that, actually, this really is the agile, would have on their Jira-addled brains.
>If you tried agile and it failed, it wasn't actually agile. When you show how it was actually agile, you didn't do it right. When you show you did it right - go back to argument 1. It's a really circular no true scotsman style of argument.
It'd be interested in seeing that argument in the wild. Mostly I see complaints like yours when people point out that an hour-long daily that the manager uses to harangue the developers isn't actually agile, and that is perfectly true. Or that a blame-assigning retro or measuring dev productivity using story points is not actually agile, which is at least true insofar that it clearly is forbidden in Scrum, and stupid in any case.
Though I find it interesting that you end with kanban -- an agile methodology. It seems you do like to use agile after all, just not scrum.
I agree, and I guess it should be clarified I'm mostly criticizing the "traditional" implementation of agile as I have seen it which is scrum with sprints, retrospectives, etc. I find it extraordinarily counterproductive, but obviously some have had success with it or it wouldn't be so widespread.
> The way I like to manage projects these days is a kind of kanban system with a task queue, and 1-2x a week brief cadence meetings to discuss priority/alignment and any blockers.
Sounds pretty agile to me.
While the "defenders" of agile might be employing No True Scotsman fallacies, it's detractors (which I personally seem quite a bit more numerous on HN) are often doing the same in reverse: refusing to define what "agile" actually means, and just throwing together a bunch of anecdotes and feelings about excessive meetings and estimates being misused to measure productivity.
> refusing to define what "agile" actually means, and just throwing together a bunch of anecdotes and feelings about excessive meetings and estiybeing misused to measure productivity.
I mean - this seems necessary when the agile manifesto is extremely vague and practitioners can't seem to agree on the One True Way to do agile - and in the absence of formal studies and analysis, what else is there but anecdotes?
I like to think about it like this. If you are building a skyscraper in Manhattan you need a plan. It has to be engineered. The materials must be specified and ordered long before construction starts. There are engineering practices and building codes that create a measure of safety. There is an order to the construction process. The project plan would likely be called “waterfall”. If you attempted an agile approach of dropping off a team and saying for our first sprint let’s just dig for two weeks, then we’ll plan what’s next is not going to cut it. Agile on the other hand is great for smaller, less complex efforts where the desired outcome is “fluid”. Different methodologies work best for different things.
All I can say is the big waterfall projects I was on pre-agile had excellent planning for the wrong things, and took up most of the time allocated for the project. So we were out of time, the quality and people suffered. Another project was cancelled because we took so long planning, the company lost interest.
I don't think that's what a selection bias is. If I have two cars, and electric for getting around locally, and a guzzler for road trips, you wouldn't call it selection bias if I always chose the guzzler for the road trips.
You'd just recognize that they are better for that task.
There is choosing the right tool for the job, and then there is evaluating the tools against each other on the properties specific to the job.
If after making your tool selection you went on to author a study about how gas cars result in more frequent bathroom breaks while driving, without controlling for the length of the trip, you’d be doing what these folks did, and that’s selection bias.
Also, how is success vs failure defined? Whether you implemented what you intended to on time and on budget?
That's a reasonable definition, but it doesn't capture whether the project actually achieved the business goals it set out to achieve. For many projects, it's better to fail quickly than to succeed and then discover afterwards that it wasn't worth doing the project.
There are more direct selection biases in the study: "projects which had clear requirements before starting development were 97% more likely to succeed" because hitting a moving target is more difficult, with all methodologies.
I don't agree that on time and on budget are the correct measures for determining overall project success or failure. By these standards a multi-year project delivered a single day after the original estimate, or a massively profitable project that went a dollar over the original budget are counted as a failed project. Ideally some actual business outcomes need to be included before determining if a project succeeded or failed.
I think this problem persists for a lot of research and discourse about software project success and failure where we conflate whether we estimated accurately with whether or not the project succeeded or met the needs of the business.
Also, a project that was cancelled after two weeks because it was determined that it wouldn't be valuable enough is a failure while a project that spent two years building something and then didn't make a single dollar is a success as long as it was budgeted for two years.
Clear requirements are needed regardless of methodology. The simpler (briefer) the requirements, the more likely to be understood and followed. Iterative development processes can leverage more concise (and thus clearer) requirements
> One standout statistic was that projects with clear requirements documented before development started were 97 percent more likely to succeed.
Also, if a project doesnt have requirements, how can it either fail or succeed?
EDIT: agile is one type of iterative, and iterative exists in contrast to waterfall. go ahead flame away :)
I can count on one hand the number of "agile" projects I've seen that were actually iterative, regardless of what the manifesto says. It would be a mistake to confuse what this article is talking about (projects that profess to be agile) with anything based on small, self-organizing teams.
"One standout statistic was that projects with clear requirements documented before development started were 97 percent more likely to succeed"
The issue with a majority of projects are the requirements aren't always clear from the start or even if they are it tends to deviate while the project is in progress. Agile tries to mitigate that by allowing teams to be able to react more easily to changing requirements.
That being said not many companies understand or implement agile correctly, unfortunately.
> Agile tries to mitigate that by allowing teams to be able to react more easily to changing requirements.
This often breaks down into a practice where maintaining requirements is not given enough priority and in very short time the actual product has deviated from the documented requirements to the point where they're basically useless and there's no clear goal for any of the features.
Most times I've been in a doomed to fail agile project, it was almost always a manager or higher who overpromised with deadlines, not accounting for real world blockers. One of said projects has insanely awful app store ratings for their mobile apps.
Article source seems to be mainly an ad for some agile alternative which makes the whole thing suspect.
Most of the "results" can boil down to: scope creep kills projects. That's entirely independent of whatever methodology your team follows and they don't make it clear how their "Impact Engineering" concept would address it
> At least within my organisation, we do not design anything up front. We’re agile
Agile doesn't mean forgo design completely.
> We don’t think about proper api modelling. We’re agile.
See previous point.
> We also do barely any testing. We’re agile.
It's difficult, if not impossible, to be agile without testing. If you want to move fast, you want to be confident that your latest change didn't break anything.
> We do rewrite the UI a dozen times based on user feedback. After all, we’re agile.
Sounds like you built a complete UI before any users could give you feedback. That's the opposite of agile.
I like how you quoted all the text except the part where I said “I’m sure agile done right is great”. (And that most organisations don’t do agile right).
But in the name of agile, business tends to get away with a lot of bad decisions, at least in the places I’ve worked.
Is it all as bad as it sounds? No, but the times I’ve heard we’re not thinking about design upfront under the name of agile is staggering.
Also what I dislike about the agile way of working is that whenever somebody presents any argument against it, people always say “that’s not agile”. Great. But if it is how people DO agile, that argument is moot.
It’s like REST. Rest is great. But nobody (almost nobody) actually does Rest like described in the original whitepaper. We can’t just ignore the issues resulting from this discrepancy.
Which is funny because agile allows for almost anything. Even the Scrum Guide says that the process must be adjusted to the needs of the project. It's not even an option. Yet scrum meetings are treated as if they were religious gatherings.
Can you point to any of the 12 principles which says that we should do "barely any testing"? I can point to some that say "deliver working software" and "it's the only valid measure of progress". If testing helps you deliver working software, then do testing, that's agile.
Sarcasm does always, without exception, sabotage a factual discussion into confusion about what people are actually saying (because that remains unclear even when you fully recognize the sarcasm).
who really does "Agile" or "waterfall" or anything ? I'm in this job for 30 years. I've never seen any formal methods my whole career. it's like here's the thing we need or what the customer needs, gather some vague requirements, write some prototypes, do internal demos, now more requirements are there, more clarity, iterate on the prototypes, etc., repeat. use issue trackers, use "stories", use kanban boards sure, but are we adhering to some rigid notion of "agile"? no way. I can't imagine the "waterfall" method actually working either.
Agile in itself is not a development methodology - it's a communication one. A small team working on a stealth R&D project, where everyone is same rank and in the same boat, would be suicidal to use Agile. This is what Kanban is for.
Agile is an overhead. It doesn't make things faster, or more efficient, quite the opposite. It's there to avoid arbitrary deadlines coming from the management and developers not working on an island, with no visibility. That's a recipe for some nasty surprises. "But we thought you were working on THAT and it would be ready NOW!"
It seems like this is in service of promoting some new competing methodology. I also can’t find any details on the research methods, but it seems like it may be relying on software engineers’ judgment of the success/failure of projects they’ve been involved with, which…I think is pretty questionable. The reality is that there are a lot of software engineers who might describe a project that was hugely profitable as a failure because it has a shitty database schema and lots of dead code because of a last minute change.
It looks like, over a span of four days, they had 600 engineers (heavily biased towards UK engineers) answer survey questions. The vast majority of the higher failure rates were for the whether "Development starts before clear requirements, no complete specification, significant changes late in development."... which is placed under "Agile Requirements". Of course, these are things that agile recognizes as factors that create risk, and the methodology is about trying to mitigate said risk as best as possible. So, it's not an indicator of methodology but of the context. Yes, if such risk never presents itself, you are much less likely to fail, regardless of methodology.
I've always liked the Agile Manifesto, but it's fairly vague, and leaves a lot of details to be provided at implementation time.
I suspect that a lot of Agile is actually "agile™." A branding facade, over an unstructured, ad hoc development system. Not even Waterfall, which is actually a very robust system (just robustly inflexible, which often gives bad results).
I like the idea of evolutionary design, and adapting to change, but I have found that it needs to be done carefully, and that having experienced developers; concentrating on results, is a must.
As always, I think the proper answer is "It depends." The search for The One True Methodology is one that will never be satisfied. Even different projects, under a single organization, probably need to take different approaches.
I just believe that we always need to keep our eye on the end result, and all development needs to be done, with that in mind. I think that (for me), Agile is accepting that we don't actually know what "done" looks like, when we start, but we have to have a heuristic to help us to understand when we're done.
The article does state that the referenced study may be seen as endorsement of this new shiny approach without clear analysis of the problem. What is true, in my experience, is that Agile is poorly implemented across the board. Sprints are just biweekly massive status updates and no developer talks to the customer/stakeholder directly. So, criticism of implementation should not have been underrated in the article.
That said, in the referenced study, they are calling some ad hoc development practices without requirements "agile", because they feel like calling it that, and then blaming the authors of Agile Manifesto for not delivering on their promise. Clearly, today, a lot of software management malpractices are called "Agile" and then the fault for the failure is pointed outside of the organization. It's the management's fault, not the Agile methodology.
> One standout statistic was that projects with clear requirements documented before development started were 97 percent more likely to succeed. In comparison, one of the four pillars of the Agile Manifesto is "Working Software over Comprehensive Documentation."
Someone does not understand (or is intentionally misreading) the meaning of "Comprehensive Documentation".
The Agile Manifesto is not discouraging having requirements before you start implementing. It's against having a multi-month planning phase that produces hundreds of pages of detailed specs before you start implementing.
Agile development was a reaction to the status quo where that was done - and a staggeringly large percentage of projects failed.
> One standout statistic was that projects with clear requirements documented before development started were 97 percent more likely to succeed.
Having worked with both this approach and agile, I'm not surprised by these results. However, I will also say that writing clear requirements is easier said than done - when those requirements are vague or of even mediocre quality it can very easily send a project off-course immediately.
Part of this stems from, in my view, a lack of accountability when it comes to waterfall approaches. If the BAs responsible for drawing up the requirements burn through time and budget that ultimately gets felt by the development team. It requires Engineering Managers to carefully vet those requirements to ensure they're appropriate start work, which results in a tremendous amount of back-and-forth.
Yep. And my favorite is when developers spend far too much time discussing the story point estimate, having long discussions about why something is higher or lower...and then not writing any of it down other than "3" or "5".
Because agile has itself been made so heavy weight that it is no more agile. You have product managers and then you have scrum masters and then there are meetings about whether how pure and religious we were about our agile practices, how pure we were, would gods of agile be pleased with every single second that we spent in past few weeks or our hearts had some impurities deep down somewhere? And no kidding, there are then meetings about those meetings and their outcomes and action items and their evaluation.
The obsession with the process itself is a sight to behold. It is now a whole cottage industry, major technology journal and websites have whole sections devoted to just Agile. Consultants, Coachs, Gurus and what not.
I’ll never understand the hate for agile on this site.
The process is at its core, enumerating tasks to get to a. Goal, prioritizing them, and letting those who do them do them at their own pace and order across a larger team.
I’ve run teams that way for decades and never had a complaint.
I think the hate stems from so many people working in environments where either the company has bureaucratized their nominally agile process to the point of absurdity or just having superiors that are in over their head, which makes any kind of methodology a farce; bad leadership can only be mitigated so far by processes, especially of someone has to take responsibility for said processes and that often by default is the mentioned in-over-their-head superior.
Confounding variables. It's possible that riskier projects use agile because they're less well defined to begin with.
The stats I'd really like to see is how many people, how many months, how much money. Hypothetically, one successful waterfall project could perhaps fund 3 failed and one successful agile one with comparable output.
My takeaway from the article (which has a "people before process" feel to it):
> Projects where engineers felt they had the freedom to discuss and address problems were 87 percent more likely to succeed.
Also beware capital-A "Agile", it defines a process devoid of project characteristics, making it not.
Yea, failing fast and cheaply can be enormously valuable. In fact, of the projects that "didn't fail" one can wonder how many of them are still in a death march.
"projects with clear requirements documented before development started were 97 percent more likely to succeed."
I believe Barry boehm is on record stating that the biggest mistake he made in designing a software creation process was believing people who told him that you could get firm requirements up front.
So yes, if you're in that rare case where you can start out with clear and non-contradictory requirements, probably anything you do is going to succeed better than when you don't.
There's a good book on risk management that goes into this more, Waltzing With Bears, by Demarco and Lister
Agility is implemented as a way to increase productivity. It does it with more detailed scrutiny of what workers do.
Management understood very well that it's taylorism applied to our profession. Taylorism started with measuring every movement of the workers and have their expertise removed from themselves. Daily stand-ups and points kind-of do that, even if there is no magical 4x productivity gain to ever be found, unlike was what won in manufacturing (cf. Braverman). It's a game played against workers, if they self-organized there would be no Agile.
You get into problems when you turn these things into fixed recipes. Prudential judgement is something you must learn and keep learning all your life, and the heart of prudence is humility, which is to say, a submissive attitude toward the truth in place of willfulness (something that comes easy, and is celebrated by our morally bankrupt culture). There are general principles, yes, but the application of them in the concrete and particular, as it were, is subject to prudential judgement.
Even with agile, modeling the business and documenting requirements is still important.
The difference is that 30 years ago we’d spend 6 months defining requirements in functional and detailed requirements and the larger the problem, the more fragile those documents became.
And proper story development is hard. Most “agile” implementations don’t bother to get stories “completed”. It’s an afterthought.
It's funny, the root of the idea is that Agile allows teams to change how they are doing things midstream so we can constantly improve. But I have never met a scrum master or PM who has both the authority and the desire to advocate for changing how work is done. So all it ends up being in practice is a series of rituals that people sleepwalk through.
Dilbert captured it best: "Our boss can't judge the quality of our work, but he knows when it's late". For everybody doing programming work, there are ten people tasked with making sure it's done by the "due date", whether there's any value in that due date or not.
"Ensuring the thing never fails" is a question of having plenty of tests, which can be practiced as part of an Agile methodology. In fact, it sounds more agile to write tests while (or before) you develop a feature.
I say this neither in attack of, nor in defense of, either one. Make your own decision. But those two things are now diametrically opposed to each other, so this is the sort of word carelessness that will destroy all ability to converse or think about the characteristics of each.
I have only seen three principles work consistently in software engineering: tight iterations, divide and conquer and fixed max sizes. All of these principles work because development is a NP problem and the best we can hope for is a local optimum. In general you see best practice anti-patterns come up that break one or more of these things and math takes over. Take a product backlog for example. In general these backlogs become the one place that everything is put but then you break divide and conquer and fixed max sizes. Since scheduling is an NP problem, if I have an unbounded backlog I have just created an NP problem to solve. the correct way to do things is to admit that you will not find the optimum solution, so the best you can do is break the problem into pieces, assign teams and minimize communication between the teams so that the pieces don't get combined into one again. The teams then need to do the same if the individual pieces are still too big. Massive monolith teams break all the principles and are basically guaranteed to fail. Or they are guaranteed to fail until someone proves that P = NP.
This very obvious PR stunt appears to be based entirely on redefining "working software over comprehensive documentation" to mean "don't gather requirements and don't write a spec".
There is no scientific evidence in support of Agile. Its supporters, usually Scrum Masters and the like, will go to any length to save their pointless job.
sigh. It's nearly impossible to have a meaningful conversation about anything to do with "agile" these days. And that's because there's approximately zero agreement about what "agile" even is, and all of the various terms are overloaded with 100 different meanings, and every "agile" implementation is different from all of the others.
I'll just note that, in my experience, a few things are true:
1. The Agile Manifesto, in and of itself, is great.
2. There is no such thing as "agile development", as a methodology in its own right.
3. There are many "agile" methodologies, including Scrum, SAFe, Disciplined Agile Delivery, Agile Unified Process, Crystal, XP, etc. And then on top of that, every company in the world has implemented their own poorly specified, half-assed, bug-riddled, barely comprehensible implementation of "agile".
4. Approximately everybody misunderstood / misunderstands (perhaps intentionally at times) the Agile Manifesto and bends it to suit their own biases. A classic example is illustrated in this article:
One standout statistic was that projects with clear requirements documented before development started were 97 percent more likely to succeed. In comparison, one of the four pillars of the Agile Manifesto is "Working Software over Comprehensive Documentation."
NOWHERE in the Agile Manifesto will you find any instructions saying "Don't do requirements engineering" OR "Don't identify requirements up front". I promise. If you don't believe me, go read it right now and see if you can find those instructions.
What you will find instead, is the quoted admonition to NOT put documentation (possibly including requirements) OVER actually delivering software. But the Manifesto itself clearly says "there is value in the items on the right". Where "The items on the right" include:
- processes and tools
- comprehensive documentation
- contract negotiation
- following a plan
Unfortunately we, as an industry, collectively managed to myopically focus on items on the left and the "we value the items on the left more" phrasing and threw the baby out with the bathwater. "We are doing Agile" became the excuse to abdicate with regards to the necessity of doing requirements engineering, architecture/design work, documentation, etc.
Basically, we swung too far in one direction, and need to move back towards the center. No, I'm not calling for a return to waterfall or anything. I'm saying that we can embrace the principles of the Agile Manifesto and STILL DO requirements engineering, architecture, and all of those other things.
A good starting place would be to stop talking about "agile" like it's a discrete methodology (as opposed to a family of loosely related methodologies) and - as organizations - pick an actual methodology to implement and then actually embrace the "empiricism" attribute that Scrum emphasizes (whether or not you're using Scrum per-se)... measure things empirically and actually make adjustments based on THE REAL WORLD not somebody's whack ass theory. Reality trumps everything and I desperately wish we could get everybody to acknowledge this.
What I appreciate about Scrum and other methodologies is that it can be used as a starting point from which customization and adaptation can grow, rather than starting from practically no documentation or process and building a monster of a system that ends up mostly fitting the existing broken system.
Yes, exactly. That's why I advocate starting with some known process, even if you modify it. Starting from scratch, IME, tends to lead to just creating a non-coherent mess.
This is such a strange phrasing. The problem with "agile" is when it becomes manager-heavy and meeting-heavy.
The last firm I was at was ridiculous. One project had about 30 hours of meetings a week. There's no time to get any work done. It's just lopsided manager bullshit.
That's the actual criticism, not some strange dichotomy of things being successful if only it was meeting, manager and also authoritarian focused with some specs decreed by some deputized opinion makers with holier than thou thoughts they wrote down two years ago.
I can't find any link to the actual study or even a description of its methodology. The research appears to have been done for the book Impact Engineering, which introduces the new methodology, and yet the research also claims to have measured projects following that methodology. From the marketing summary here [1], it seems like what they actually did was to isolate which individual virtues of various methodologies had an impact on project success (which also goes without a definition of course). Then they choose which of those individual virtues would be promoted by their new methodology, and use projects adhering to those virtues as a proxy for scoring their new methodology. And then claim the result as statistically significant. It would be interesting to see if this was the actual methodology, because if so, that's clearly nonsensical.
Also, it would be highly relevant to know the definition of success here. Evaluating an Agile project's success as "completition of the project's initial requirements on time" would be completely asinine, given the entire point of Agile is to adapt to changing requirements.
This looks like more of an "ad" (or a very directed study by a competing methodology), but excess pragmatism can ruin even the most sensible ideas.
Agile, testing, design patterns, best practices can all tank and bury a project if applied excessively "by the books" without consideration of the actual problem to solve.
I've worked in teams that had about 10 people actual doing dev work that implemented the full suite of Agile "principles" as rules. Daily standups, grooming, retros, pointing as "poker", 1:1s every week. The result was that we had barely time to actual work since the week had 10-20 hours of meetings. Most retros and standups were literally just us saying "same as yesterday, only had a few minutes to work on this" the whole week.
Testing is the same. If applied without consideration for the actual problems, reaching that 90%+ code coverage is easy if nobody cares about how hard and time consuming it will be to change code later. Specially when a feature is in very early development.
I think all those things are good, but what I see sometimes is that they are applied as absolute rules that cannot be deviated from, which inevitably leads to poor results.
I'm now working in a "light Agile" environment with just 2-3 meetings a week, barely 1 hour total, and much less strict PR/testing requirements (we focus on testing the important functionality, not line coverage count) and it is so much better. Some of the same co-workers that were under the more strict rules are now twice or more more productive then before.
Complex things (life/political systems/software) cannot be built based on a manifesto or commandments. See where Christianity and communism got us.
When you reduce complexity into simple but interpretable phrases, we always open doors to priesthoods (Agile coaches, the church, The Communist Party) who claim to know the right way and try to impose a rigid structure on everybody else in their respective communities.
I wish a more scientific method to software development could have been possible. And Agile is not it.
Well, yeah. It's impossible to do Agile right. Why would a practice that's impossible to execute correctly have anything other than a higher failure rate? It's Agile's fault and it shows.
Isn't it the whole point of failing fast? It seems the industry rebooted back into "any project must be a success" mindset, which we all have been walking away from for the past couple of decades.
Truth is, there are shitty goals, wrecked scoping, high-risk biz hypotheses, managers cheap skating on resource and talent side, and many, many other things that deserve early failure. All these do fail much faster today, provide valuable early feedback and lead to changes.
I'm sorry I didn't RTFA, but I wonder if there's a correlation / causation issue here. Did the "agile" projects fail because they were agile, or because their project management methodology was prescribed by management than is higher up than appropriate. Maybe they happened to choose agile because it's the flavor de jure at the moment, but it wasn't the "agile" that was the problem exactly, as much as the undue influence of the wrong layers of leadership.
"Agile" and "failure rates" don't have strict, universally accepted definitions, so any study (even if definitions are described) is essentially meaningless.
Consulting Agile, grew out of web development consulting firms as a defensive methodology for managing shitty stakeholders. They incorporated some genuine industry best practices, but they also have a lot there that's simply meant to blunt the worst impacts of having to deliver software in an environment where the person paying you doesn't understand the basics of software development (or project management really).
It actually works pretty well in those environments - you don't get hyper-productive engineering teams, but you do get somewhat consistent delivery and a stable enough working environment that the worst outcomes (not delivering anything at all) are avoided.
No large tech company that I'm aware of implements this type of Agile but plenty of big non-tech companies do. If you have a CIO in your company and the business refers to software engineers as 'IT' there's a good chance you do this. This is inefficient but it's probably good enough for those companies, especially if it's established, the switching costs alone would be a nightmare.
If you're using it as a startup then something has gone terribly wrong and you should seriously reevaluate what you think the future of the company will be.