I've been through about 5 Agile shops now. I would say one of the 5 ran Agile well, the rest were a mess that just makes it frustrating for everyone.
What we mostly have now is watered down agile.. the worst bits of agile like never actually "designing" anything, but we don't really have any of the good bits of the older management processes left over.
I don't really want full waterfall again but I would really love to see some sort of lite versions of Functional specifications & design specifications come back.
Agile works best IMO in the Pre-1.0 phase of a product and gets progressively worse the further you get towards maintenance of a product. I also think Agile works better and better the closer you are to the UI, and worse and worse the further you get from the UI. Solid back end design and infrastructure rarely fits well into "Sprints" and solid back end infrastructure is not sexy at all in a demo, so Agile can demote back end design to the point everything gets done in a way that is too MVP. Then later on when scalability, performance, bugginess, and maintainability come into play you're into that "established product" phase where Agile doesn't do as well. So you've got problems in an area of the product Agile has trouble with happening in a phase of development Agile has trouble with.
Agile also seems to really require absolutely top notch product management folks, and they are seriously rare.
The thing is of the 5 shops:
* 3 Were acquired
* 1 Went IPO
* I work at the 5th currently
So the companies always had a successful exit even if the codebase passed onto the acquiring company or to the public investors had major issues.. so at least in my experience there's no feedback loop to correct the issues.
I cannot agree more with the assessment that agile works well with front end and poorly with backend.
Agile with the backend/infrastructure actively encourages taking on technical debt just to get something, anything up and running as quickly as possible. Which is great for PMs' careers but terrible for the developers who have to maintain those systems, and ultimately the organization as a whole. Maybe it's even great for job security or to get an IPO/acquisition as quickly as possible, but it really does result in much more work in the long term.
The issue is that the people who push for these terribly designed backends that suck up unnecessary human-years worth of work to maintain, patch, and fix are never held accountable. And if you're stucking as a developer working on these systems, you're screwed (and it can even be bad for your career since you're not shipping anything new). I can't tell you how much of a relief it is to just walk away from that kind of mess... which really, makes the situation even worse, since everytime that happens organizational knowledge gets lost and some poor soul gets to start working on something terrible with much less context
I think it depends. It does protect from overcomplicated designs that never come to fruition in accordance with Gall’s Law (See Googles Omega). And nobody will care how much extra debt you saved down the line if company seizes to exist because it was outpaced by competition so there’s definitely some trade offs to be made.
The last part is more of a problem with hard maintenance work being unappreciated compared to launching new shiny shit which is entirely different can of worms.
but I would really love to see some sort of lite versions of Functional specifications & design specifications come back.
None of those things are disclaimed by Agile. Agile never meant "don't do any design, requirements gathering, or specification." If the people you're working with think that "being Agile" means you can't do those things, then quit and go work somewhere else.
Remember, "there is no such thing as Agile." That is, there is no specific process or methodology named "Agile" which is prescriptive and says "don't do design". Agile is an umbrella term that encompasses a wide spectrum of different methodologies, including Scrum, XP, AUP, Crystal, DAD, SAFe, etc. Those different methodologies may argue for more or less emphasis on up-front design, but I'm not aware of any of them that disclaim it completely. If your people are saying "Agile doesn't let us do design" then they're full of shit.
The Agile experiences I've had that were more satisfying were actually the ones that were closer to "by the book" Scrum or Kanban.
The specific things that I've noticed get left out that make Agile miserable are:
- Story pointing goes away
- Velocity never gets calculated
- Retrospectives get removed
I think Agile shifts responsibility off of management and onto the engineers in even the best run team. PMs and management don't have to commit to hard decisions early in the development cycle because the methodology says that overcommitting early will lead to failure.
Velocity and Retrospectives are two of the best features for me.. Velocity is a check on managers over scheduling developers, and Retrospectives force everyone to look failures right in the eye and adapt. When you remove those two things Management can over schedule the team without even realizing it and no one has any data to show it. When you remove retrospectives no one gets to discuss bad parts of the process or bad parts of the planning.
That's just one part of being satisfied though.. if you're doing perfect agile and killing it but you're building a poor product that's not going to be terribly great either.
Most of the reality for me has actually been good products that were right for the market but ended up implemented pretty poorly due to failings that could somewhat be blamed on Agile.. it seems to be getting worse and worse over time in my experience. More technical debt, poorer scalability, more bugs, less QA (you could write a book on QA and Agile).
I did quite a bit of UI early in my career and moved towards back end over time. I was mostly all back end by the time Agile really took off. I am generally pretty depressed with the quality of the back end code under Agile. You get horrible MVP database schemas and awful quickie framework dependent code that is horribly inefficient and really hard to remove later.. but that path is super quick for that MVP "gotta demo" mentality. I think this stuff really kills the morale of back end developers and leads tons of people to move on.
>So the companies always had a successful exit even if the codebase passed onto the acquiring company or to the public investors had major issues.. so at least in my experience there's no feedback loop to correct the issues.
In a sense, it sounds like there is nothing to correct! Would moving more slowly and deliberately have improved the exit, or worsened it? Perhaps code quality has only a tenuous link to business outcomes.
I periodically think about this line I found on the C2 wiki:
"I believe it is time to explicitly state the long held secret of software, we do not need to do design; design is ineffective and costly." --Wayne Mack
It's really hard to argue with a money printing machine. That theoretically it might print faster or at least longer for a bigger integral, after doing things that seem to slow down printing now, is also hard to believe if the person you're arguing with has never experienced such a non-linear outcome.
That sounds like a line by someone who has never seen the disaster that some incompetent people can do. Design is not necessarily something done by a kind of mythical "architect" (whatever that means...); instead, a good programmer simply very often write software that has a good design.
And I would say than in most areas, design is effective and required.
NT works. 9x is long-dead (and even 9x had tons of design, but there was just a gap too immense on some fundamentals to get anywhere)
I would even make the hypothesis that good design is always eventually effective and required, but the period of time after which lack of proper design hits you badly varies, and in some fields it is actually longer than the lifetime of the product...
I've come to expect that code quality has a strong link to success but in sometimes its inversely proportional. What I mean, is that at the Seed/Series A timeframe, I doubt you will see a strong performing company, with a codebase with high quality. I also the odds are against seeing a tech company with billions in revenue (not worth) have a really poor codebase. I can think of plenty of examples where the desire to 'do it right', has cost a company their marketshare.
As much as Netscape vs Microsoft hurt Netscape, the time it took to do NS7 and the quality it shipped with, pushed them back 5-6 years.
One thing I learned a long time was this: A "bad" process that has complete buy-in and everyone participates in will always outperform a "good" process that has lip service buy-in and disagreements about who does what.
A team that has skilled individuals exercising judgement and thinking critically about tools (for project management as for everything else) will alway outperform one that’s subordinate to a process.
Absolutely. I let the pms do their thing and follow the rules. Every organization I've worked with is thrilled when developers get buy in, but guess what your process is still pretty crappy and we just want to make you feel good about it.
Positive criticism always falls on deaf ears - "why not just give it a chance" blah blah.
Our agile process has complete buy in from developers.
Yet the product is markedly inferior to the same product we created with fewer developers in a shorter period of time, while also creating the backend services, right before we created this product, when we were not following ‘agile’.
Because the current management approved Agile is just a bunch of rules that are enforced upon us. The previous process was basically an ad god process we had built up using best practices, adapted based on our own experiences with the unique needs of our team. In a sense, we were far more agile when we were not doing Agile.
I'm ready for what comes next after Agile.
I've been through about 5 Agile shops now. I would say
one of the 5 ran Agile well, the rest were a mess that
just makes it frustrating for everyone.
I can't tell you what will come after agile, but I can tell you that it'll probably have around a 20% effective rate all the same.
Also there's nothing in agile that would prevent you from designing things upfront, I believe the exact phrasing is "Responding to Change Over Following a Plan". I.e., build extendable / adaptable designs (that can change), do the amount of specification upfront that provides value, but don't ticket-plan 2 years of development upfront.
I'd also add: much of the culture and thinking surrounding Agile grew out of software consultancies: organizations that have the privilege of green-fielding new projects and then "abandoning" them; either because their contract was over, or there's no desire (or capital) from stakeholders to improve the software. So it doesn’t surprise me that their model breaks down for software that undergoes continuous modification for years, as that’s an atypical case for consultancies.
The outcome can only be as good as the sum of its part. The best methodology with the worst people will still result in a poor outcome. The best people with the worst methodology might still manage do deliver something useful. The best outcomes come from trusting good people do what they're good at. Why trust a manager over an engineer?
Scrum != agile. You don't need sprints to be agile. If sprints aren't working for you and your team, then stop doing sprints. If your team can't decide to stop doing sprints, then you're not leading your own agile process and you're not really doing agile.
Let me clarify that -- there are people who are using Scrum to do agile. Scrum is not _against_ agile. But if you are doing two-week sprints because management is chasing the latest fad, without empowering the team, you're not agile, you're just in a Scrum cargo cult.
Exactly this, agile is all about empowering devs, trusting them and allowing them to work.
Sprints are about enforcing feedback from the customer into the loop. If you have a half decent customer they should be happy with any length sprint and releases that make sense.
No, sprints are about creating artificial deadlines to stress developers to work more hours to meet so-called commitments/forecasts.
It does not work of course, but that is what it is about.
Well, there's supposed to be -- Technical Quality - Refactoring.
Common to all flavours of Agile (I lecture agile techniques) sounds like you guys are missing out important parts of agile... but I totally agree with what you are saying. I'm old Skool and grew up with waterfall. I find it really weird teaching students who don't have a spec or UML diagrams or any idea what a software pattern is. Hell, some of them don't bother with a Model layer - not enough time to implement it in a sprint, so why bother? Lol. They get code out the door, but quality code? A-hem, it is possible, but do people do it?
Of the agile shops I've done similarly only one did it well, and I won't lie - there was way more process and documentation than any waterfall shop I was ever a part of.
I've been through about 5 Agile shops now. I would say one of the 5 ran Agile well, the rest were a mess that just makes it frustrating for everyone.
What we mostly have now is watered down agile.. the worst bits of agile like never actually "designing" anything, but we don't really have any of the good bits of the older management processes left over.
I don't really want full waterfall again but I would really love to see some sort of lite versions of Functional specifications & design specifications come back.
Agile works best IMO in the Pre-1.0 phase of a product and gets progressively worse the further you get towards maintenance of a product. I also think Agile works better and better the closer you are to the UI, and worse and worse the further you get from the UI. Solid back end design and infrastructure rarely fits well into "Sprints" and solid back end infrastructure is not sexy at all in a demo, so Agile can demote back end design to the point everything gets done in a way that is too MVP. Then later on when scalability, performance, bugginess, and maintainability come into play you're into that "established product" phase where Agile doesn't do as well. So you've got problems in an area of the product Agile has trouble with happening in a phase of development Agile has trouble with.
Agile also seems to really require absolutely top notch product management folks, and they are seriously rare.
The thing is of the 5 shops:
* 3 Were acquired
* 1 Went IPO
* I work at the 5th currently
So the companies always had a successful exit even if the codebase passed onto the acquiring company or to the public investors had major issues.. so at least in my experience there's no feedback loop to correct the issues.