The root problem in my estimation is that people want a boilerplate bureaucratic process which they can follow and achieve good enough™ results.
A lot of the activities done in agile or scrum or whatever processes are potentially useful tools in specific situations. But people don't want to understand the history and reasoning behind the tools nor do they want to apply them judiciously as appropriate.
They want a one size fits all process that produces results, if not optimally, at least consistently. And whatever Agile flavor of the month is popular is just that, it's good enough. And the benefit is that you get this fun mind trick to deflect blame ("I don't know what happened we followed the process. People don't fail processes do.") or ("I'll tell you what went wrong we didn't follow the xzy process. If we had just agiled a little harder this project would have shipped on time").
IMO the main problem is in the very way organisations are structured - it is about power, and people (really meaning management) want to achieve good enough results, given that their control is maintained.
Agile as aesthetics is cool, but actual implementations move power away from management and we can't have that. So we will rebrand project managers to POs and whatever managers to scrum masters, and will follow the rituals as long as they don't get in the way of normal run of things.
Agile was grounded on solid principles, but is very ideologically naive, which allowed its easy cooption by consultants and management.
This is a very good summary. The universal formulation of this is Pournelle's Iron Law of Bureaucracy [0].
Pournelle's Iron Law of Bureaucracy states that in any bureaucratic organization there will be two kinds of people:
First, there will be those who are devoted to the goals of the organization. Examples are dedicated classroom teachers in an educational bureaucracy, many of the engineers and launch technicians and scientists at NASA, even some agricultural scientists and advisors in the former Soviet Union collective farming administration.
Secondly, there will be those dedicated to the organization itself. Examples are many of the administrators in the education system, many professors of education, many teachers union officials, much of the NASA headquarters staff, etc.
The Iron Law states that in every case the second group will gain and keep control of the organization. It will write the rules, and control promotions within the organization.
There is also what Stafford Beer writes about when discussing viable systems - the goal for most organisations, is not their stated goals, but survival and perpetuating the system. The stated goals are a byproduct of that, if and when achieved.i.e. in a lot of universities education is a byproduct, in a lot of hospitals, healthcare is a byproduct, hell in a lot of governments, governance and actually improving things is a byproduct, that will not happen unless there is an absolute need for it...
In a way it's like running a tricky multi-objective optimisation, and the AI will find a way to cheat the stated goal.
This whole trail through to the top comment is spot on.
It's also helped me work out a corollary in my mind that has puzzled me for a bit.
My sense, in the Marxist tradition, is that modern organisations are dependent on the extraction of value from highly capable technical resources and, especially outside the tech bubble, they largely resent this dependency.
Let's say developers are an example of a highly capable technical resource, though I am by no means limiting the scope.
This results in a series of mechanics that lead to developers being alienated:
- From the product of their development (ownership of IP and the resultant value of their code, distance from seeing the positive impacts of their work or talking to those it helps)
- From the act of developing itself (by its reduction to commercial use and control over how it is done, approval gates, arbitrary coding standards, ticket systems, scrum processes, project managers and product owners, Jira, timesheets etc.)
- From their fellow workers (stack ranking, power dynamics, labour competition, structural organisational tension)
- From their human nature and natural talent (by the reduction of their humanity and passion and capability to a mere "developer" or "engineer", use of stereotypes, reduction of humanity to output/LOC/story points delivered, corporate gaslighting at questioning this state of affairs etc.)
And most non-developer people that have worked in an average organisation and spent much time with developers have seen all of this at play, and heard how much developers hate it. Yet many still refuse, even in the face of self-interest (e.g. faster delivery of outcomes for a non-technical manager), to empathise and accept the reality of the experience enough to support better workplaces for developers. A tangible recent example is the insistence with all sorts of reasons on getting developers back into the office where they can be watched despite demonstrably lower productivity and engagement.
My sense of this is that what developers can bring to the modern world is the closest humanity has got to magic. And this dependency is resented. And this resentment leads to workplaces in which this resentment is externalised in the form of debasement (e.g. caricatures and other forms of ego compensation - "they're just the boffins, they don't have people skills!"), control ("the boffins can't really be trusted - better add some process and oversight, and given they don't have the people skills better make sure they aren't anywhere near management/clients!"), and dependency inversion ("sure the boffins can do their coding stuff, but they'd be nothing without us to help babysit and organise things, they don't get the way the world works, clearly it's they who actually need us!"). And this environment in turn leads developers to internalise this systemic resentment as a resentment of themselves, their capability, and their work, aka burnout.
But one question has been bubbling away for me for a while.
How do so many organisations arrive at a system in which it's almost a badge of honour to not be one of the doers? That those who can't do, should, as a moral claim, oversee, and manage, and lead? And we should keep adding more of those people until the doers can't possible do. Even when that produces lower tangible results.
Maybe at one point I internalised the Office Space / IT Crowd idea - the non-doers are "people people" who didn't spend decades at their PCs honing their craft but instead went to wild parties and focused on normal people stuff (the implication of course being that developers are lesser than normal people). Maybe the developers really can't be trusted. Maybe they do need to be managed and watched and distanced. Maybe the code monkey caricature (before being reclaimed by those it was used to demean) is right.
But now I wonder.
What if those people are empowered by the systems into positions of power over developers because in the first instance they affirm the original resentment: what if putting non-doers in charge safely perpetuates the idea that the developers belong at the bottom of the pyramid and affirms the extraction of their labour in support of the salaries and profits of those above relying on it? Or to put it another way, what if the code monkey caricature is effectively a justification of Marxist exploitation?
And what if the second order effect here is that some, let's call them the senior management class - leaders of the Second order of Pournelle's Iron Law of Bureaucracy in the example above - are more conscious of these dynamics and consciously perpetuating them. What if they are knowingly hiring more non-doers to help keep this balance and control. What if it's not accidental, or people skills, that means non-doers are in charge? What if the very reason they're there in the first place is to be in charge even in positions outside of formal leadership (or at least indirectly support the power of someone else who brought them in for that reason) - after all they're not there to do.
So to close out a long post, a corollary to Pournelle's Iron Law of Bureaucracy: in any bureaucratic organisation the ability to be able to contribute to the goals of the organisation will be an insurmountable barrier to having influence or control within it.
> How do so many organisations arrive at a system in which it's almost a badge of honour to not be one of the doers?
I believe this is a holdover from Western society's feudal roots*: the people who do things, especially who do things with their hands, are considered to be lesser beings than those who, by divine right, rule over them.
The ideas and values of feudalism are deeply embedded in our culture, and it's very hard to escape, for instance, the idea of the Good King, ruling benevolently because he is divinely granted better discernment to know what is good and right. (And also the Iron-Fisted Tyrant, which is a seductive fantasy to a different type of person.)
Some of these ideas come to us through the medium of things like just world theory and other Protestant notions, but at their core, they're still basically the same thing: the idea that the people who rule do so because they are Better People, and the people they rule (the ones who have to work for them) are in the position they're in because they are Lesser People.
Thus, rising to management becomes a proof that you, too, were a Better Person all along.
* Not saying other societies don't have feudal roots, just that I'm familiar with Western society
> What if they are knowingly hiring more non-doers to help keep this balance and control. What if it's not accidental, or people skills, that means non-doers are in charge?
You bet this is the case. Every manager is, consciously or not, empire building. Hiring more underlings so they become more important. This not necessarily mean that they are evil schemers - on average, people are pretty good at convincing themselves that what they do is good and necessary.
Note that Big Corps have much more in common with communism than with capitalism. The way people think about capitalism, where everything is hyper efficient, is incompatible with the huge amount of waste that goes on in big organisations (this is a gross simplification, but you get the point). Communism by its nature means a large centralized bureaucracy, with ironically even worse denial of personal agency and thus more alienation.
Valve tried the whole "let's get rid of managers" and "taking back power in dev's hands' thing. It always devolve into doers-turned-barons who screw everyone and everything with power struggles and political intrigues. And I say this as someone who finds most managers useless and incompetent.
This is the main problem with Marxist theory. It is a well contrived system of beliefs mixing strong emotions, down to earth realism and an overcomplicated and oververbose theorical development that justifies the existence of its own expertise. All of this form a strong opium for most intellectuals.
In the end it loses touch with reality and its scientific pretense very quickly as the predictions don't meaningfully materialize and it fails to deliver on its promises.
History does not follow a direction. The workers never formed a group united just by virtue of being workers. Humans are not malleable not do they have a good natural state. Reforms have always outperformed radical and violent revolutions. Marxist revolutions were never lead, instigated and started by the workers. It miserably fails at deciding on who actuallt gets the power ans make decisions.
A lot of people blame "capitalism" for societal problems but there are no other reasons given except that those are unique to and caused directly by capitalism; capitalism being whatever exists just by virtue of being outside a marxist/socialist society. This is made evident by the fact that marxist revolutionaries are always confronted by the same problems that those who they overthrown had; and so they become labeled as state-capitalists to excuse their faillings.
I think that some of this is true, but, in the Marxist tradition, it strives to define classes of people to hate, and ignores the key aspects of capital, risk and relationship in things succeeding.
True but not only management is to blame. Proper agile requires a certain amount of common sense and a willingness to spend some thought on improving the process.
There is a group of developers that are either not able or not willing to think about these things. This group just wants a set of rules that tells them exactly what to do in every situation so they can switch off their brains, don't have to carry any responsibility and just do what they are told.
> There is a group of developers that are either not able or not willing to think about these things.
In most cases because process improvement suggestions by non-managers (or in the worst cases, by anyone non-external) tend to get ignored by the upper chain or, worse, being stolen by someone along the chain.
Or, to put it shortly, why invest mental energy into improving processes when it's either being discarded anyway or used for the promotion of some manager without delivering any benefit to oneself?
And to make it worse, some organizations are actively set up that way to discourage internal improvement ideas so that the uppermost management is free to hire the infamous Big Four "consulting" agencies to mask what upper management wants as "external and independent" expertise that must now be followed. The Big Four (and their smaller fellows) are a fucking scam set up to enrich the top level at the expense of everyone below.
I'm not sure I see the point here. You improve the process because you're subject to it, and because the current process has some downsides to you.
If you're trying to improve a process just for the recognition, then you're by definition not subject to it, so you're playing the role of a manager. And you excluded that yourself in your first sentence.
> If you're trying to improve a process just for the recognition, then you're by definition not subject to it
Simple: when hired as a developer, I expect being compensated in one way or the other for work that goes beyond scope - especially when it is like in most private companies where you need some form of achievements to show to get a raise or a promotion. And do not forget: when you improve processes, usually your workload will be increased as a result, so in the worst case you'll end up with your manager getting the raise for your suggestion and double the workload as "thanks".
This falls squarely into stepping out of the developer role and taking on the role of a manager though. It's really not about the people who don't want to think about those things.
>True but not only management is to blame. Proper agile requires a certain amount of common sense and a willingness to spend some thought on improving the process. There is a group of developers that are either not able or not willing to think about these things.
You can fire or train those developers. You can't do that to the CEO.
Agile tranformation requires buy in right to the top. I've seen it work from the top down, never from bottom up and never from middle up/down either.
Oh, there is definitely something to that. There are people, not just managers, in general who for one reason or another _want_ someone to take charge, be responsible and tell everyone what to do. Which is not bad, per se, it gets bad when the insistence is that this be a boss assigned to you, rather than someone you choose, but that is broader than just agile and the topic in question.
The generic law for that is: any methodology that intends to take power away from the managers will be thoroughly corrupted and inverted after adoption
To me that sounds like socialism - a perfect system if it just weren't for those pesky people and their deeply ingrained selfishness and desire to gain more status, power and wealth.
Agile is a system that ignores how power and status plays work in companies which almost always makes it into a miserable system for everyone involved.
Well agile is rooted in anarchism's self-organization. It is meant to evolve in a collaborative shell. I think the main factor of trouble is that we want to implement Agile principles in competitive driven companies. They are not looking for customer's satisfaction, they are looking for customer's money.
I would add, if something doesn't work then change it.
This is the one thing that "failed agile" hits against. Large organisations want everyone to use their same crappy Jenkins pipeline. Which means when it keeps breaking then no one can do any work. No one can fix it, and no one is allowed to switch it for a service that isn't down for half a day most days. That isn't "agile".
It derails everything else in the process, what's the point of a retrospective if the most painful parts was because someone has decided outside of the team that you have to use OpenShift for your Kubernetes, but then hires folks who don't know what they are doing and you can't just shift to a managed service until they sort their shit out?
What's the point of standups to get updates if most days people are still stuck on their ticket because the mandated CI pipeline is stuck due to a lack of workers?
I see it in all these complaints about agile, that they don't grasp that they were never given autonomy in the first place. It was always only lip service.
This is true, measurement is important and if it is worse can you tell if the original bad part improved and look at a hybrid of actions, however the ability to change in the first place is always lacking with people who say that Agile doesn't work.
Note that ALL of these can be summed up in one phrase that I tell my teams often…
TIGHTEN
YOUR
FEEDBACK
LOOP
This is literally the key to life (besides 42) and everything. Want a better relationship with your partner, pet, friends, boss, code, family then tighten the loop.
That’s very good advice for day to day tactical approach:
- small changes, fixes, additions
- tests, optimizations, refactoring
It’s what you do to move forward iteratively with approaching deadlines.
But to not get stuck in local maxima you need some time off from feedback loops and tactics. You need time to think, design, experiment, sleep. You need time for creative, strategic thinking. Processes, rules, feedback loops, deliverables etc. are all a hindrance to that.
Also, getting an app up and running. Often times it's better to hack away at a (truly minimum) mvp and then iterate on that. Than try and iterate your way to an mvp.
That is exactly the failure of all the various management methodologies - they are built on the false assumption that all feedback is valuable and if something isn't going right you must not be getting enough feedback. What is the value in getting bad data faster? Can we not trust a developer to build something without deferring on every decision?
We need to get back to engineering as an art form, not factory work.
My company have mandated that my team use sprints, despite us having previously proved that sprints are a bad way to handle our workload (we used a kanban-like system for a glorious year or so and we were actually way more productive in a measurable manner, and also a lot less annoyed). The reason? Predictability.
Except you can also measure that we're not hitting any predicted goal times, ever, because our sprint system is incompatible with the nature of our work supporting live systems and simultaneously developing new features. We're regularly taking on work from outside the sprint half an hour after sprint planning, because something exploded and we're the ones who have to fix it.
It's completely impossible, but the vision of predictability has blinded management.
The number of times there's a card on the retro board under "Stop doing" that just says "sprints" on it...
I believe this has to do with managers being required to constantly estimate cost, time, and ROI with precision, minimizing risk.
I like Basecamp’s shape up methodology: make bets of efforts that are worth 6 weeks of the team’s time, commit to the best one, timebox it. If it takes longer than 6 weeks, throw it away. This way the risk is limited to six weeks.
Few managers would be willing to stake 6 weeks of a team’s time though.
That needs to be balanced as well, same as all things. A retro every two weeks is a lot and brings too much overhead into the process. Same with the constant refinements and groomings and plannings and whatever. A team or a subteam needs to have uninterrupted autonomous time to dive deep and get things done. That's fulfilling work, from my perspective. Rhythmitize work into periods of quiet, focused implementation work and team work where people investigate, research and plan new features and refactorings.
- (1 hour) Retro every two weeks (sometimes we skip it or it becomes a show-n-tell)
- (30 minute) 1:1’s monthly for seniors, every other week for juniors unless requested more frequently
- Optional (1 hr) themed mob sessions / office hours throughout the week. You bring a problem and the group that shows up will work on it with you. Seniors/IC’s are required to hold office hours at least once a week.
- NO OTHER MEETINGS; engineers are asked to block out their lunch/personal time and refuse any non-essential meetings. They have the right to request meetings be rescheduled to appropriate times or moved async.
If all you concentrate on is short term improvements then... sort of?
Making iteration faster let’s you see the results of changes you make faster, that’s the benefit. If you’re iterating faster than you can make meaningful changes then you won’t see any change by definition.
Iteration length is also context dependent. For a multiplayer game I really like being able to see my changes immediately when working on it, I love being able to iterate with other devs on some idea where each iteration is in minutes and we’re chatting about it together but we only test the game itself “as it’s meant to be played” with anyone outside the team once a week. That’s part logistical and partly anything shorter than a week doesn’t usually show enough progress. If there’s some bigger change we’ll also keep our work iterations but pause the larger play tests until it is done.
Short as possible is good advice but you’ve got to look at your own context and make sensible decisions. You can also look at it like following a gradient, if you’re unable to make bigger changes then you’ll just be optimising towards some local maximum.
Micromanagement itself is more about the amount of agency to decide what to do at each iteration.
Even worse it means you are always reacting to the latest or "loudest" issue (like the CEO wanting a feature implemented that no one will use but will impose a maintenance burden) and never standing back to evaluate what is valuable for the majority of customers or product.
It doesn't necessarily mean that management is giving the feedback.
It could be coming from tests, from CI, from the QA team, from customers, from other engineers in your team, or, failing all that, then yes from management.
But if management is giving feedback that is hard to reconcile with everything else, at this point you can point to CI, the previous tests that were added based on other requirements, feedback you got from customers etc. as justification for your existing actions. It means you're operating with a lot less assumptions because you get data validating what you're doing (and where you should fix things) every step of the way.
If I had a perfect understanding of both technology and the problem domain, I wouldn't need feedback. Yet, usually we find ourselves in a different situation of less than perfect understanding of both, so we seek feedback to avoid going in a wrong direction. The less understanding, the more of it we seek. But there has to be a logical limit to that, otherwise we could come to a conclusion that things could be accomplished by someone with nearly no expertise, given enough feedback.
The secret lies in the fact that neither giving, nor receiving feedback is free.
I would like to interpret “feedback” in the parent comment as communication. And I think we should distinguish between frequent communication and micromanagement. One may report to their manager frequently. Doing so has the benefit of letting manager communicate back any context they might have regarding the current status. Perhaps, that problem has already been solved in some other situation. Or, the manager has experience in a similar domain and can give direction. Unless the manager dictates a direction, doesn't leave enough room for decision autonomy, or begins working on a solution rather than leaving it to the engineer, I wouldn't call it micromanagement.
In my experience, if the management, processes and teams aren’t trusted then it can turn into unwelcome micromanagement.
Engaging with people, who have different motivations and reasons for being in a job, and stripping away often unhelpful implied titles of rockstar programmer to QA robot, can help everyone to see the big picture and in turn to see how their contributions, their cogs, fit into that picture.
Production-line programming can be left to new automated systems, whilst we can embrace and capitalize on people exercising their intellect for productive, feedback-oriented work.
Be careful, because tighter feedback loops might result in a lower signal to noise ratio. When noise is reacted as signal, things might become worse. To be clear, I'm not arguing against tight feedback loops. Our current feedback frequency might be looser than the optimal, after all. We need to be more cautious with what we are doing, though.
Common sense is a dumb term because it tends to mean whatever the person invoking it considers good.
Having worked as a programmer since 1999, I can say that these things were not part of the processes and ideas when I started, at the places I worked. Interpret that any way you want.
I like agile/ scrum/ whatever and what I mean when I invoke those terms, is the list enumerated above.
I dislike the bullshit derivative terms, however. The burn-down charts, estimation poker, story points and all that. (Sure - the poker might trigger a thorough discussion about controversial estimations, so I guess I'll allow it.)
I also really dislike the concept of the daily standup for non-juniors.
> Sure - the poker might trigger a thorough discussion about controversial estimations, so I guess I'll allow it.
I believe all these tools have their place. But they are more frequently used by management to exert power over engineers. “What a dumb take. That feature obviously will not take that long. You are over-cautious as always.” “We had decided that this feature was 2 story points. Why did it take that long?” No, dumb fuck. It was your decision, not mine. I had argued against it, but you didn't listen.
That's not my experience. But if it does happen, does it matter? All that happens is that less work goes into the next sprint, right?
(I'm suggesting that the problem in your process might lie elsewhere - with mapping points to time, maybe, because the points system should be self-correcting to that kind of gaming).
I like this focused list. I’m always in favour of understanding the need for methodologies yet at times too many people use a methodology name as an excuse and alternative to saying “I want it now, I don’t care how, and I’m not interested in excuses nor if there are problems. But make sure there are no problems.”
Common sense, teamwork and cooperation can help us embrace the salient features of different methodologies without the barriers and no-entry poses sometimes presented by leaders or masters in the name of leading-edge box-ticking, often forgetting that we’re working with people. I’ve not seen many unhappy yet productive, professional and inclusive workers, in any field. Explanations, regular feedback and carrying a team with you can help to reduce stress of everyone in the team.
Processes are key to this, irrespective of the ideology.
In theory you can go even smaller: a way of measuring output, and a regular retrospective to improve the process. In other words, a feedback loop that improves your process over time.
Of course it would be dumb not to include all the stuff you mentioned at the start, because we've learned from previous projects that it works. Might as well stand on the shoulders of giants. But it's not the absolute, minimal core.
Personally I'm not convinced that "tighten your feedback loop" is good advice, because I interpret it as "shorter measurement intervals", and the shorter they are the more noise there is in the measurement. But the beauty of iterating the process is that we can try it and see if it works.
95% of the time the reason teams "need" that is bullshit internal politics and everyone is better off if you can structure the organisation to not do that.
When I worked for a larger company with a pretty mature product, this was true - deadlines were internal and arbitrary. It also helped that the product was more of a “nice to have” for our customers than “absolutely crucial” - they didn’t truly NEED anything it did (it was software to make social media marketing more efficient).
But I’m now at a smaller Enterprise SaaS startup, where the product is absolutely crucial to the customer’s work (core operational software for public transportation agencies). The reality is that the product isn’t yet mature enough to close big sales without making sales commits. I dislike sales commits, but we’d legitimately lose every big RFP unless we commit to adding the ~2-5 features they really, truly need that we don’t yet have. And sales commits come with real, external due dates, that you need to be able to meet if you don’t want to piss off your customers. You need to be able to say “yeah, we can build X by date Y”, and then actually deliver on that.
This is one of the reasons I’ve stayed where I am for so long. I get paid a nice six figure income to work on a training platform that is useful, but not critical. It is a low stress job with little to no critical dates.
> The reality is that the product isn’t yet mature enough to close big sales without making sales commits. I dislike sales commits, but we’d legitimately lose every big RFP unless we commit to adding the ~2-5 features they really, truly need that we don’t yet have. And sales commits come with real, external due dates, that you need to be able to meet if you don’t want to piss off your customers.
That's just pushing the issue one level higher. Most likely both the customer and you would be happier if you were able to iterate with them rather than building the thing they thought they would want however many months back. (And while it may be harder to set up that kind of contract, it's absolutely possible)
> Most likely both the customer and you would be happier if you were able to iterate with them rather than building the thing they thought they would want however many months back. (And while it may be harder to set up that kind of contract, it's absolutely possible)
Possible? Sure. Just like it's possible that I could win the national lottery.
In practice, multiple companies bid for contracts, and all of the bids have to be of the form "Deliver $X in $Y months for $Z dollars".
No company is going to accept "Choose us, and we'll work until you're happy with the product or you run out of money, whichever comes first".
Agile doesn't work where there's competition for the project. It can't, because the buyer of that project cannot evaluate "we'll work until your money runs out" against the other bids that state "$X in $Y months for $Z dollars".
Agile works well for internal teams that are sheltered from having to provide competitive bids.
No, not necessarily. It depends on how often and badly the org has been burned on past project failures.
Typically, the intermediate step is to tie some compensation (such as a bonus) to a particular metric, but why not just go whole hog?
Even if X% is projected to be larger than $Z (and in fact, the better the project succeeds, the larger the %X total ends up being, but you can cap it, or make the % progressive up to a point and regressive above it), the advantage for both client and vendor being aligned on the upside is considerable.
The problem is winning the RFPs. There are some core features and integrations that they really, really want. If we don’t have them, and won’t give a firm commitment to building them, we lose the RFP.
“We’ll work collaboratively and evolve the product over time in yet-to-be-determined ways” can work when you have an established relationship and 2-way trust, but there’s not much of that during an RFP. If a competitor commits to their biggest asks, and we don’t, we lose the RFP.
Its an example of bad leadership. You see it often. Higher management thinks product x can be ready in a y period. Reality is much different. One of the key reasons why companies and products fail.
This is not true for most. Imagine going to a contractor to get a house built and they tell you that they can’t tell you what it will cost or when it will be done.
You have obviously never built a house in the U.S., otherwise you would never make such a statement.
As almost everyone knows, construction delays and cost overruns are the norm, not the outlyer. Construction projects here very often suffer dearly from the "2 more weeks and/or a couple more thousand dollars" overrun issues.
In fact, I'm sure at least several people who have been trapped in a home renovation nightmare LOLed deeply after reading your post.
I’ve worked on the building of major resorts that many people in this thread have probably been to. I’ve seen this done well before, and I’m happy to compare credentials if that’s the route you want to go. Then we can see who LOLs.
A major resort is a project that has very large contractors involved and where a part of the upside for the contractor is how well they can manage the buffer they built into the quote for the project. The smaller that buffer the bigger the chance they will win the bid but also the bigger the chance that they will end up losing money on the deal.
Houses (not housing tracts) tend to be built by much smaller contractors who are in absolutely no position to absorb any setback without going bankrupt, you can't apply the rules for 'big business' to small time operators and expect the exact same outcome.
Constructing resorts also has all kinds of economies of scale that building individual houses does not.
That’s fair, I may have picked a poor example, but I maintain a business needs to be able to have a reasonable estimate of the cost of doing something, and often needs the ability to hit a target date.
This is very doable in software, and I’ve seen it done for decades from all levels of the org chart.
I don't know about the US but in Germany this is already the case. You get a rough estimate but nothing is set in stone.
Overshooting deadlines and increasing costs due to materials not being available or them suddenly costing twice as much is normal.
Also coordination between all the various contractors is always getting messed up because the person putting up the scaffolding didn't show up, so the plumber cannot work and cannot come back for 3 weeks, which messes up the plan for the drywall, which gets pushed back, and so on.
But they give you an estimate for cost and time, and are reasonably close to it. I.e. it doesn’t cost 3x the estimate!
They could probably do it more accurately but it would lose them bids, so they knowingly low-ball it. You shouldn’t have that problem inside a company doing software.
Of course you have that problem inside s company building software.
Engineers low ball partly because they don’t know what’s involved in a new piece of work. Like walking between two points on a map; they don’t know what they’ll run into on the ground.
Managers and stake holders then bully the engineers into lowering the estimates because even that’s not quick enough. Worse, they scale the team without discussion without realising that will make the project harder to deliver.
That's pretty much every contractor that I've ever worked with. It's all based on estimates with set minimums, buffers in time and money to deal with the weather and other outside influences and even provisions to deal with the changes in price for raw materials.
The only way you are going to get a hard quote is if you buy a prefab or a house that the contractor has already put up multiple times (and even then there will be some room for adjustment). Oh, and foundations are a completely different story.
Yes, I think we are saying the same thing. You use buffers and other means of flexibility to hit a target launch, and it works most of the time but not all the time.
Makes sense only if you assume building software is like building a house. Imagine getting an author to tell you when the novel they are writing will be finished?
Or this: imagine seeking an estimate on when a house will be built, trimmed, painted, furnished, decorated, and filled with every item needed practically and for comfort by the homeowner.
It’s a living process that requires continuous reassessment of priorities and scope, and in fact never finishes when you consider the life of the home.
So you think it’s reasonable to have no idea what it will cost, or when it will be done, when building software? You think it’s possible to run a successful business that way?
> So you think it’s reasonable to have no idea what it will cost, or when it will be done, when building software? You think it’s possible to run a successful business that way?
It absolutely is; look at some of the biggest and most successful software companies in the world. No-one has any idea what facebook will cost or when it will be done (Zuckerberg has said that he was off by at least 5 orders of magnitude IIRC). No-one has any idea what google will cost or when it will be done. No-one has any idea what amazon will cost or when it will be done. No-one has any idea what netflix will cost or when it will be done.
Google and Facebook are companies. They aren’t features. I know a lot of people at both companies, and they absolutely gave dates and estimation for many things.
I make a point of stopping project managers when they use the analogy of building a house to push back on estimates. What was the contractor doing before the current job? Building a house. What will they do after this job? Build a house.
By contrast, software development is doing stuff that hasn't been done before. You may be lucky, and have done something similar. If it's really been done before, you would ideally be using something off the shelf.
For me, a project manager who uses this analogy has no place running a software project.
Construction companies can do all sorts of projects, and still usually have to meet a deadline. Do you think every bridge or skyscraper is the same, just because they are all bridges and skyscrapers? And do you think mechanical engineers are also not estimating delivery dates just because they usually create new stuff? Because that's not the case.
Also, tons of software dev isn't actually about pushing boundaries or doing something that hasn't been done before. In fact, I'd say most development is about implementing known solutions but in a customized way. Just like every other engineering field.
The analogy was house building. Generally a house builder will not be building skyscrapers next month.
And the project managers who use the analogy don’t make the comparison with significant and novel civil engineering projects. They make the comparison with having their kitchen extended.
A few months ago i worked at a company which estimated 2 months, after 1 year and a half it’s still not done, i quit a couple of weeks ago
But the client, a really bigcopany, it’s angry but stillwaiting, lying to tjeir upper managent to get a few bonuses
Imagine wanting a two floor house with a closed kitchen and once all the wall built you request your contractor to have a single floor one with the garage underground now and an open kitchen with island.
With a loose enough contract that they can ask whatever for free.
My current team only estimates projects not tickets and does not do scrum, and I don’t notice a difference in performance from scrum teams I’ve been on.
Furthermore, I worked on a 18 month project that was estimated by someone who was not on the team that implemented it and it was implemented on time without overtime, and without any special process. Maybe that was unicorn, but it happens.
I think just having a manager/tech lead that cares and maintains priorities is all it takes to stay on track.
Processes are great tools in the hands of good managers / leaders. Good processes in the hands of bad ones, or bad company culture, does not make them good managers.
Outside of software, you see the same thing over and over again with Lean Management and TPS.
> but it misses the thing many teams need and that is a process to reliably hit dates they commit to
40 years of IT experience lead me to believe that this is only possible for the most trivial of projects with nailed down specifications and where the party building it has relevant domain knowledge. In practice such projects don't really exist. Trivial IT projects are rare, specs change, scope creep is a thing and domain knowledge is built up slowly over time.
What is necessary is that the work does not occupy more time and effort than it strictly requires. That is still a hard problem, but much less hard than to find a way to hit dates that you commit to, unless you give yourself ample room to deal with the realities of complex IT development work.
I think we’re on the same page, you have to give yourself room to deal with the complexities. It’s not perfect, but it’s a far cry from those saying there shouldn’t be any estimates or date targets with predictable delivery.
+1 the problem is the management’s desire for modern day taylorism. Once that line of thinking sets in it really doesn’t matter which process you pick it’s still going to suck
Having lived through a lot of different orgs, following all the rigor of textbook scrum is like training wheels for effective agile. Running with true agility and without the ceremonies requires a disciplined team. I've seen some undisciplined, immature and just flat out mediocre teams that will just flounder without the training wheels. I've seen plenty of highly disciplined teams that have really internalized the meaning of agile delivery and don't need to be told all the steps and will actually self-organize. You just have to calibrate your level of rigidity for who your team is.
i think this is a very insightful take which i totally agree with. Although I have also seen many high functioning teams who could have reached self-organizing/self-whatever, but they arent allowed to raise to that next level because the scrum orthodoxy is mandated/unquestionable. Its frustrating.
Yeah I think the common storyline is that a manager who was trained in an immature organization and has seen the training wheels approach be successful decides that they can repeat their success by following the same steps every single time. To be a good manager to have to manage to your team, not to a set of rules. That's not easy for a lot of people.
> They want a one size fits all process that produces results, if not optimally, at least consistently.
There's an additional perhaps even more important aspect contributing here: the sociality of language and shared values in technology as social communities.
IMO If it hadn't been agile it would have been something else; we build common understandings with the terminology we have available, in order for our activities to make sense.
It doesn't matter if the terminology is applied correctly, viz. adhering to the theory (although many escape this accusation by a tacking "hybrid" onto everything). What matters is practical applicability: when we use these terms we're more often understanding rather than misunderstanding each other.
So the terms are used in more abstract ways than intended. (E.g. agile as methodology in health bureaucracy is kinda weird.)
I said to our CTO that I found the Agile practices we had recently adopted a bit annoying, and his reply was “look, Agile is bullshit, but what it gives us is standardisation across teams.”
People want to minimize complexity in their lives, the less you understand the topic the more you want some magic 1-click/effort solution. It can be software dev, politics or anything else.
That's why populism is so damn effective on masses who often wouldn't be even able to articulate properly current internal and global political/economical/whatever situation, yet have very strong opinions how these things should be.
> They want a one size fits all process that produces results, if not optimally, at least consistently
I disagree with this statement. Waterfall produced good enough results. What they really want is the most meeting and process heavy approach irrespective of results, because those who advocate for Agile are the people who have no real job and this gives them enough bullshit to bs themselves through every day of their work week.
Waterfall was good in a time where requirements were more static. It's not great when you don't know the requirements fully or they may change during the project.
I'm not a fan of agile either though. I'm not a team player and agile is way too heavy on the framework side for me. Too many procedures and too much fuss around the work. Just give me a task and let me get on with it. But some parts of it are ok like close collaboration with the (internal) client.
Luckily I work for a company that thinks logging our hours in Jira means we're agile so I don't really have a lot to do with it.
This shows the uncanny parallel between Agile and religious sects. If you agiled and failed, you should have agiled harder. Or, even better, part with your hard-earned cash on Agile coaches to show you the one True Path™.
There is only one canonical work, be it a relatively short one, the Agile Manifesto. However, there are multiple interpretations, with each faction claiming they are the one True Path™ to reach enlightenment. Each of these factions has culturally appropriated existing concepts, such as daily stand-ups and backlogs, and claims these are unique to their implementation of Agile.
That leaves you to wonder how developers managed in the dark ages, before the Agile revolution. Did developers chuck out unassigned tickets with their poo buckets on the streets? Could they only deliver Babylonian towers as they lacked a way to communicate with each other daily?
Organizations that adopt one of the better bureaucratic processes like Scaled Agile Framework (SAFe) do actually ship on time. What they ship might have quality problems or fail to meet customer requirements, but at least they ship something. For some organizations that alone is a huge step forward and can put them on a path to additional improvements.
Not least - the Agile Manifesto states "favour people over processes" - by definition, there isn't a fixed process that you should be following as it will change whenever people are added to or leave the team. This is the exact opposite of what companies want - consistency with (most) people being interchangeable.
The problem is making people who can't do the actual work into managers.
It's really a class problem. Rich people don't want to associate with those filthy poors unless they get to be above them, so they created MBA programs and a managerial class.
Excellent rebuttal of those calling for the end of Agile whilst simultaneously not know what Agile even is.
My addition-- If we're going to end cargo culting, we should also end all the cargo cults of Silicon valley style startups.
1. Stop the cult of "Grind". Working more hours is not more productive for most people
2. Stop the cult of "build faster with tech debt" -- internal quality matters and costs you velocity. Those 10x engineers are 0.1x when you consider their externalities
3. Stop the lie that you'll get rich off startup options. Statistically improbable and options which expire 90 days after your employment ends are not equity or property as most people understand it. They expire/evaporate unless you shell out the exercise price, and the tax money on the gains... -- RSUs or extended exercise windows are though.
4. "We can fix it later with an order of magnitude more VC money" -- Similar to the tech debt item, a company w/ a game plan of "more VC money" is outdated and missing the change in environment, instead be profitable from very early.. ideally from your first customer.
5. "We need bespoke software because our Product is so unique." -- Nope -- use all the tools you can until you have scale, then start to build whatever is least economical to buy.
6. Doing it like Google (or other respected company). You're not google. Oh and if you survey for "Best practices" -- it's not best, it's average. Average isn't going to cut it anymore.
All of these are great except (depending on what exactly you're thinking of):
> 5. "We need bespoke software because our Product is so unique." -- Nope -- use all the tools you can until you have scale, then start to build whatever is least economical to buy.
You're talking about software startups, right? If their product is not "so unique" then why are they building it? If you don't need custom software, why the hell are you hiring software developers? This one is just weird. While I understand the dangers of Not-Invented-Here syndrome, the pendulum is currently WAY too far the other direction. Almost every "modern" software project is way too reliant on dozens of libraries that are philosophically incompatible, spending more time on adapters and API-specific integration than they would spend simply writing what they actually want to begin with.
When I arrived at the startup I'm at now, we used a "bespoke" backend for interacting with AWS SQS. It was a pile of code that was written that had no value to the function of the startup.
Switched it to an OSS bus and dumped a bunch of bespoke code. I regret not pushing harder for the CTO to avoid even more bespoke code for some of the admin interfaces that we ended up building for internal tooling which probably would have been better in something like NocoDB.
The value of the startup is rarely in the bits and pieces themselves (very few are), but the process or data flow that the startup models with those bits and pieces and some pain point or business value that gets added in doing so.
One thing I have come to appreciate in startups is that it's better to even spend $10,000 to learn fast rather than to wait 3 months to come to the same conclusion. To put it another way: if you could see 3 months into the future for $10,000, would you pay it? Instantly. So if there's a shortcut that can -- with a high degree of certainty -- remove obfuscations of the path to take, you take it.
I used to work for an automated property valuation platform. It was basically a property valuation workflow tool that presented valuers with the right data at the right time so they could drastically reduce the amount of time spent valuing each property and theoretically produce much more accurate results.
We spent so much time working on features and workflows to interact with clients rather than building the platform for our actual users, the valuers. We could have picked any CRM off the shelf, adopted its way of working, and saved ourselves a lot of time. There is no scenario in which the CRM subscription would have exceeded the cost of the developer's time to produce the cobbled-together half-baked solution we ended up with.
What I meant by bespoke can be illustrated by thinking about an example...
Why are we still hand writing HTTP JSON APIs out of base language primitives in 2023? You can generate correct docs and a skeleton implementation in with JSON Schema and Open API for example. Why spend weeks-months writing endpoints + validation code + docs, when you can define a JSON Schema in slightly more time than it takes to discover what the PM wants to build. You can take it a step further and add the validations to the JSON column in your datastore and now you're correct all the way to the DB.
I feel similarly about most folks who think they can write a better queue than their favorite cloud platform (eg AWS SQS) . Just use the tools that you can buy for pennies on the R&D dollar. The reason a team cannot compete with that is due to amortization. AWS can amortize the cost of development and maintenance across 1000s (millions?) of customers.
This is what I mean by writing bespoke software. not sure if that lines up with everyone else's interpretation.
You can build bespoke software without using bespoke software (other than your own to test or dogfood if applicable of course).
E.g. if it's basically a webapp, the chances that it's such a snowflake that existing frameworks are inappropriate and you need to roll your own are extremely slim.
(On the other hand, you won't think you're making a game and accidentally create Slack that way...)
i don't think gp is talking about building business logic or core product. but like maybe don't roll your own monitoring or log uptake or kanban board / task management solution.
i worked at a company that decided they needed to build their own rdbms because none of the off-the-shelf solutions were going to be able to manage their unique scale. come on.
A mess is debt, because you have to pay interest on it. If you don't pay your interest, it will compound and completely kill your velocity. Nothing in Ward's transcript disputes that, nor would it matter if he did (Death of the Author).
Ward seems to be clarifying the difference between useful technical debt, which gets your product out the door faster, and wasteful technical debt, which is writing your code poorly when you could have done better.
If it's slowing your velocity down, making things harder than they should be, it's technical debt. I suppose if you had a "mess" that was self-contained, bug-free, and not slowing you down at all, then that's not debt -- but is it even a mess?
you've described a mess correctly, but misused the term/jargon.
What you've described is poor quality implementation. There are many (infinite?) implementations that could satisfy the product manager, but each has it's nuances of correctness in non-tested cases (if you have tests), in runtime effects, race conditions (across processes), thread safety, ease of refactor, readability etc.
All these are markers of software quality, but failing to do them is not "techdebt". Tech debt is a very specific thing as described in the video above
I've hear this before and I found it nonsense. In short, I don't see any advantage to distinguish responsible) deliberate debt from irresponsible/involuntary debt. A debt is a debt.
You take a loan to buy a house, knowing your salary will allow for the extra expense and you have backups plan. That's a responsible debt. Like, hardcoding some variables in a prototype - eventually that needs fixing. Everyone agrees this is technical debt.
But you can also take an irresponsible loan, eg for some far away vacations you can't afford to. Is that debt? Sure it is, and you better figure a way to solve it once you realize your mistake!
The reason for the distinction is that you basically cannot do anything about what you do not currently understand about the product (besides ship and iterate).
But you can do a lot about learning how to write quality software. And the more you do it the faster you'll get at it. It's only rational if you believe you'll have a long career programming to train how to make quality software, because many quality implementations are equivalently difficult to implement than low quality ones. For the same resources you get greater returns, and those returns compound as you are able to out ship your competitors and get more and more product knowledge/feedback.
From the transcript:
> I'm never in favor of writing code poorly, but I am in favor of writing code to reflect your current understanding of a problem even if that understanding is partial.
Oh I see: the distinction is to stop people frome shipping lousy code and saying "it's technical debt, we'll handle it later".
Ive been at the other end, when we're looking back at code and talking about technical debt, and there's been some semantic arguing about whether it was technical debt or not - I thought giving everything the name of debt would make the mission clearer (improve the code and reduce debt, which is a nice metaphor to communicate with others).
I'm still not entirely convinced by the distinction (we can just say: avoid taking debt if you can avoid it!) , but it makes more sense now.
> Stop the cult of "build faster with tech debt" -- internal quality matters and costs you velocity.
Well, internal quality costs, and the price is expressed in time.
As someone else said: "shipping is a feature", and especially for first versions! The quality of the code is only important for that code that actually brings value to the user, and you need to ship in order to know which is which.
The problem with technical debt stems from the diffused myth that the development of a functionality ends as soon as it is released. No, you must continue to work after the first release, in order to polish the code while integrating the feedback from the user. This is the core of Agile.
>The way I assess where lines cross is by canvassing the opinion of skilled developers that I know. And the answer surprises a lot of folks. Developers find poor quality code significantly slows them down within a few weeks. So there's not much runway where the trade-off between internal quality and cost applies. Even small software efforts benefit from attention to good software practices, certainly something I can attest from my experience.
If you expect your business to exist more than a few weeks from now, opt for quality implementation over "ship it" hacks that cause compounding costs into the future.
> Developers find poor quality code significantly slows them down within a few weeks.
By the way, I'd like to know which kind of software those "skilled" developers are used to work on. If it is the umpteenth CRUD web application, several weeks can correspond to a few releases. In other fields, a few releases is a matter of years, and even only a few months of customer feedback is a treasure that smart developers don't give up easily.
There are hacks like "everything in a huge global object", and technical debts like "no unit tests for internal detail and no detailed software structure documentation until we know which code is actually worth to maintain".
I think both the rebuttal and the original articles are good, but talk about different things. The original one seems to talk about the way Agile tends to be done, especially via the world of corporate, consultant agile, certified scrum masters and so on, whereas the rebuttal is talking from the point of view of what Agile should be. Two very different things, both very valid, and in a sense the discrepancy between the two tells you a lot about the state of agile itself.
That said I love the thing about ending Sillicon Valley bullshit - IMO it has led to a lot more hurt than good.
Fucking "Hustle" -- a word used by scammers and conmen that now somehow means "work smart" but actually means spamming people and doing other sleazy things to get money
not sure if you're ironically mocking the exact kind of answer you'd get in an "I'm so excited!" style culture.
Or if you're serious... I'm not angry I'm just tired of the doublespeak by people who, for example, say you gotta take a vaccine or you're "anti-science" but then turn around and denying much of the academic information we know about software engineering practice. It's self serving cherry picking.
"Then build it, however long it takes." (from the referenced article)
This is always my favorite fantasy. No one would have issues with "Agile" if you had unlimited time and money...but that's not how the world works. Hence why you break everything up into smaller chunks. It gives you clearly defined guardrails so that projects don't spiral out of control. EDIT: AND the people funding said projects know where there money is going. That's important.
The best effects I've seen from agile practices have been getting everyone to frequently think about deadline/feature trade-offs, reducing the occurrence of surprise bad news. I'm sure this is possible without "agile", but it's a thing I've seen it aid with.
The worst is probably standups-gone-wrong. There's little more demoralizing than a 20-person 3-team standup where you couldn't possibly care about what anyone says (because it's irrelevant to you) except three or four people, and you already know what they're going to say because you all, you know, talk, and then also having to turn yesterday's work into some epic (LOL) tale of adventure and adversity because your manager's in the meeting and that's the kind of dumb-shit tone they've set for it ("now—justify why I should continue paying you" with pointed, bordering-on-hostile follow-up questions to anyone without a sufficiently-convincing theatrical performance) and you'd like to keep your job. Starting every day with a half-hour of that shit. JFC.
But, I'm pretty sure, overall, agile is one of those things where most places that it works, any halfway-reasonable system would have worked just as well, and places where it doesn't, no system was going to work.
> and then also having to turn yesterday's work into some epic (LOL) tale of adventure and adversity because your manager's in the meeting
I find myself going over what I’m going to say in my head so it sounds good when it gets to my turn. I then end up not paying any attention to what anyone else is saying.
Yeah I wish more people would just say "I'm working on the same thing I was yesterday, no blockers, just need a few more days" and legitimize that if that's what's actually happening
I’m not sure why weekly status meetings lost favor.
Meet for 1hr every week as a team, and go over what everyone is working on and see how we’re progressing, and forecast where you should be by the next week.
The moment a developer feels they may not be able to deliver what was expected by next week’s status meeting, they share that with the team in whatever asynchronous communication tool your team uses, whether it’s chat, email, etc.
Regular problems and communications are raised in the same group chat/group email.
It doesn’t matter how small the daily standup is. It could be 5 minutes. But it’s a daily context switching exercise. I think it’s far better to have a single weekly, but longer meeting, so there’s only 1 context switch in the entire week.
At most places I’ve worked, long standups turn into “watch somebody else use Jira” meetings that all participants ignore until they hear their names. Not a great use of anybody’s time.
2. Those participants are not good employees then and should be told to pay attention or face further action. Meetings are for everyone to get on the same page, I’ve had too many engineers who think they have better things to do then implement things wrong or ask questions we covered in the meeting. Not something I’ll deal with anymore.
If one of ten employees isn't paying attention in a meeting, good chance it's the employee's fault. If most of the employees in a meeting aren't paying attention, it's probably the manager's/meeting-runner's fault. In between those, it might take some digging to figure out who needs to adjust to fix the problem.
When I’ve experienced this, I’ve availed myself of my personal favorite cargo cult rule: if you’re not getting value out of a meeting, you can leave.
Cargo culted from Urban Airship, and almost all of their meeting rules have been well received several places I’ve introduced them. Although admittedly in my experience this particular rule is best invoked with some extra social subtlety.
The best daily standups I ever had went like this:
Everybody is physically standing up, in a place where there are no computers.
Going around the circle, everybody answers three questions:
1. What did you do yesterday?
2. What will do you today?
3. Any blockers?
Any long discussions were tabled until the standup was finished. It took about 5 minutes to get through a team of 6, and was a great way to kick off the workday. On Zoom, you can't get quite the same kind of effect. But you could still keep it short, and you could leave your ticket-board out of the meeting.
I find that the no-Jira standup is better, because it makes people more honest about what they've actually been doing. They don't feel the pressure to fit some kind of Jira narrative, and will instead just talk freely about whatever's on their mind. Much better from a team collaboration perspective.
“Good standup” also surfaces what people are working on because it might be important for the rest of the team to know for innumerable good reasons. It can serve as an opportunity for many-birds-one-stone, or even as an opportunity to interrupt work that doesn’t need to be done because it’s redundant to other efforts or because other efforts made it moot.
But yeah, you won’t get that from standups where people aren’t invested in what others say they’re working on. That’s just people blabbing for a (hopefully short) ritual time.
Because everyone may not be available all the time except standup when all hands are on the deck. Or maybe the discussion is better done face to face. Or maybe the issue happened just before standup and the timings aligned with standup. Or any of the myriad reasons. The example also doesn't preclude raising issues outside of standup.
In the few cases those need some synchronisation, further discussion and knowledge-sharing across everyone in the team, it's a time set for that to happen.
Anything else can be discussed asynchronous, when the need for some synchronisation happens the stand-up is a good way to tackle it in a iterative way. Everyone knows what to expect.
By the way I'm not disagreeing with you, my current team (and the best teams I've worked with) don't need a daily stand-up, just a recurring time slot available for a synch time when people feel the need. People can show up when they want.
"Anything to bring up on those blockers, any help needed? No, ok, next: anything you want to bring up in the work in progress? Ok. Any further topics? By the way, if these topics are not relevant to you, you're free to go".
> I’m not sure why weekly status meetings lost favor.
Likely because, at least in my experience, they suffer the same problem as daily meetings: They provide no value. If you aren't already in the know about status before the meeting comes around, you didn't care about it in the first place and won't magically start to care when the meeting is in progress.
Those who have meetings for the sake of process adhere to daily status meetings because that is what the process literature suggested to them. Anyone who questions that further questions what is to be gained by having them at all, leaving little in the way of middle ground.
I've seen 1hr 11am weekly status meetings turn into "does anyone have a 1pm?" as Sr managers drill into every developer over the minutia of their status reports. Single handedly the most unbearable meeting series I've ever attended.
At least standups have structure and they are "supposed to be short", which helps a lot in keeping managers nitpicking at a minimum. Open ended 'status meetings' are horrible. A weekly 1 hr "standup" with a "no managers are allowed to speak", "stick to what you did, what you'll do, what are you stuck on and need help", isn't that bad, but you have to have structure.
Plus, if you are doing "Scrum", in a retro you should be able to suggest just having a "mid point" sprint review and change the process. I realize that's easier said then done, but its a good smell test on your org's attitude towards their process. Just make the suggestion and from the reaction you can tell if you can change the process like you are "supposed to".
As far as I know, there's no such role as "manager" in a Scrum team.
The Scrum way of working is a joy, which I have gotten to experience in my career just once. Practically it also seems very difficult for an organization to support, at least many have failed.
Scrum teams need to be autonomous, or the process becomes torture. Scrum as a process is rather simple, but the context is exactly a team working on their own. SAFe and such seem more like consultant slideware to me .
> Meet for 1hr every week as a team, and go over what everyone is working on and see how we’re progressing, and forecast where you should be by the next week.
So you're spending longer and getting less accurate results?
> The moment a developer feels they may not be able to deliver what was expected by next week’s status meeting, they share that with the team in whatever asynchronous communication tool your team uses, whether it’s chat, email, etc.
What prods them to think about that though? Often there's no clear threshold, and if you're supposed to keep it on your mind continuously then that's taking away from your capacity to think about what you're building. Having that daily sync point to take stock is very useful.
> It doesn’t matter how small the daily standup is. It could be 5 minutes. But it’s a daily context switching exercise. I think it’s far better to have a single weekly, but longer meeting, so there’s only 1 context switch in the entire week.
I mean hopefully you're switching out of work context when you go home or go to lunch. It's a tradeoff, but a few minutes a day isn't so intrusive IME.
Weekly isn't terrible, but I'd look at it from the other side: how long is it ok to be blocked for before you want to make sure you remember to raise things? A day works best IME. Maybe even twice daily would be better.
> I mean hopefully you're switching out of work context when you go home or go to lunch. It's a tradeoff, but a few minutes a day isn't so intrusive IME.
I don't think that is what was meant by context switch persay. It's like reading a fiction book, the context is the story. If you put that down, you might have to come back and re-read the last few pages to remember where you were exactly.
Worse yet for meetings, personally, about 15 minutes before I have to stop doing mostly anything so that I'm not too into it to then be late. About 5 minutes before I literally do stop everything otherwise I'll get into something and look back at the clock and it'll be 3 minutes past the hour.
A productivity hack I'm trying is to set an alarm for a few minutes before a meeting so I can do this more reliably without so much of a context switch.
All in all, given about 10 minutes before the meeting it is intentionally doing little so you can click that button on time, and then there is the time needed to get back into your work (most literature says it takes about 20 minutes to regain programming context), any meeting costs about 30 minutes of context switch overhead. It's a lot of time given a 8 hour work day.
You don’t wait for the meeting to raise things. You raise them when they happen. We have enough async communication tools that you can raise issues without having to wait for a meeting, and yet without disturbing someone who may be focused on something specific.
Really easy to thrash at something for days though thinking you are ‘almost there’. And someone else on the team might be able to help either directly or help you think of the solution if you just spoke out loud with the team sharing information.
You hit the nail on the head here. The whole standup process is extremely degrading. I don't think I'd ever work at another company that does a daily standup, the most I can take is 3x/week - and ideally as short as possible.
I suspect it depends a lot on the standup. My favorite place I worked had a great one. 10-15 minutes every morning, a brisk and helpful way to get focused, line up the help you needed, and kick off the day. I miss it deeply.
Conversely, I did a project once where we had not a single stand-up in 9 months. CEO would come up with some designs, I'd asynchronously comment on them with questions or concerns, then slice the designs into tickets that'd I'd pump good amounts of context and detail into, estimate and prioritize, then chuck my engineering team at them. We'd all asynchronously work on them. If they had questions, they'd slack me. This across five timezones.
Once or twice we'd have a miss on a ticket probably because I didn't define it well enough, but I'd just point out the issues in code review and they'd rewrite as needed. Oh well.
Meanwhile we all have a phenomenal quality of life because our entire day schedule is set on our terms. And we delivered a whole ass SaaS product from scratch in like 9 months for around 2,500 engineering hours if my brief math is right so the founders were happy.
2500 hours in 9 months could be two developers logging hours on the project. The project may have had stretches when dependencies limited the amount of work that could be done, and often one assumes at least an hour of non-billable time lost to miscellaneous business shit, per IC, per day.
The rest of the team (not logging engineering hours) may have included a (probably part-time) designer and likely some kind of project manager (likely also part-time—that is, managing 2+ projects at once for different clients).
7 (billable, on average) hours * 47 weeks (assuming 25 weekdays lost to holidays, sick days, and vacation, over a year) * 5 weekdays only yields 1,645 hours per worker per year, anyway. Divide by four, times three, to get ~9 months, and 2500 hours just looks like two full-timers on the project, really, if we're talking billable hours, but that's also (it reads like) not counting support roles and non-dev contributors like design or PM or maybe testers. Or it might be one full-time and two half-time devs, or whatever. I'd guess not more than four developers contributing significant time, anyway, based on the numbers and with fairly pessimistic assumptions for how many hours could be logged a week, and maybe as few as two.
I'm rough on the math cause I don't have the billing sheets in front of me, but the other person was pretty close.
The team I managed was me, functioning as frontend lead / architect / engineering manager as well as individual contributor. I then had three junior / early mid-level frontend engineers working with me.
I was pulling 40-60 hour weeks (contract so I didn't mind, just meant more money), the other devs were between 10-40 depending on how much work I could lay out in a sprint. Timezones were Taiwan, somewhere in Europe can't exactly recall, NYC, and California iirc.
That accounts for the frontend. There was a data system that was mostly built when I showed up, by one full time engineer at the startup. So that would be another 40 x 4 x 9 engineering hours on whatever he was working on in the meantime.
Then there was an API that would middleman requests to the data system and build out models and handle customer accounts and shopping carts and other data adjacent requests. That was one full time backend engineer over whom I didn't have responsibility, so add another 40 x 4 x 9 for that.
Finally, the designer was just the CEO. He would draw things up on figma and I'd slice them into frontend tickets. Now that I think about it I think the API engineer was consulting the same designs as he built out the API, I'm not remembering exactly.
So technically my original estimate is just for the frontend, it cost more when considering the whole system.
We "released" every merge into the main branch, internally. The CEO would constantly be giving feedback. It caused some friction occasionally, the classic "right the alignment is slightly off but what do you think about the actual interaction" type shoulder bumps, nothing major. Good feedback loop that basically daily aligned where I should be directing frontend priority.
There was a major customer facing release where a couple core customers were brought off the no code platform onto the new system, that cooked for about two weeks while we collected feedback and engineered against it, then another major release where I took advantage of the respite to do a major rewrite of the naively implemented request management system and fixed up some auth stuff, then another big release and then everyone else is brought over.
I still work a couple hours a week on the projecy but no longer as a leader. Looks like releases are still just kinda whenever, which I think is fine. There are "sprints" and have been the entire time but always quite informal.
I think a lot of this is just personality driven. I don't find meetings to be a way to get focused or a way to "kick off the day"; indeed, I always need a break afterward. Yes, this applies to a good 10-15 minute standup. And I don't really know what people mean when they say things like "line up the help you need". Why would I wait until a meeting the next morning to reach out to relevant people for help?
However, I recognize that lots of people don't feel the way I feel about any of this.
> Why would I wait until a meeting the next morning to reach out to relevant people for help?
It's hard to notice you're stuck when you're stuck. For a lot of programming problems putting your head down and ploughing through is the right approach - but you do need to come up for air occasionally. And if you need help then it can be hard to ask without feeling like you're interrupting someone else's work, so having a regular point where everyone's already not going to be in the middle of deep work is useful.
Again, I recognize that this is a "to each their own" situation, but I don't relate to this at all. I know when I'm stuck and I know who to ask (or who to ask who I should ask) and I have good asynchronous communication tools where I can ask people things without immediately interrupting them.
A lot of people believe they know the optimal time to reach out. But with my manager hat on, it's also true that a lot of people wait well beyond what's optimal for the organization. Things that can get in the way include pride, anxiety, laziness, a tendency to procrastinate, underestimating the resources available, and overestimating the cost of interruption, and underestimating the cost of delay. Maybe you are the rare sort of person who is perfectly calibrated. But if so, know that it's rare.
No, I'm not that rare sort of person. But it also isn't in the organization's best interest to force everyone into a single style of work, rather than supporting people in their own preferred style of work. That's why management is hard! If it were as easy as "having a status update meeting every morning is the optimal process for everyone", we could pay managers a lot less :)
I think there's conflict between "own preferred style of work" and what's effective for a given team. If there's a team of people who are, say, doing fine communicating only via pull request, then I wouldn't mess with them. But if somebody of those habits wanted to join a team that was into pairing and a faster release cycle, then I would suggest that new person look elsewhere. So although I don't think it's in the best interests for a large org to force everybody to work similarly, I do think it's in an org's interests to support teams finding an effective joint work style, even if that means a given individual might not find it optimum or might even have to find something else to do.
For what it's worth, treating my social anxiety made a big difference for me in this. A bad meeting is still draining, but good ones can be positive and energizing.
> Why would I wait until a meeting the next morning to reach out to relevant people for help?
Having a specific time for it makes it easier for people to do. It normalizes it, so it lowers the bar to ask for help. Also, note that this comes after a discussion of what people are working on for the day. So it's natural to say "Looks like I'll be getting into the foobar system today; who worked on that most recently?" And it's also natural for people offer help without being asked, so that lowers the help bar futher.
See, I used to think like this, that I have social anxiety. But as I've grown older and participated in more kinds of social situations over the years, I've learned that I don't, actually, have social anxiety problems; I love socializing, I think even more than average (the pandemic social starvation taught me this lesson).
What I have is a personality, which means that I have a set of preferences for my interactions, just like everyone else. It's fine that some people prefer group meetings, and it's fine that I prefer documents, chats, and 1:1s. We're just different people with different personalities. What I don't like is the group-meeting-preferrers seeming to believe that their preference is normal and my preference is aberrant. I find that condescending. I'm happy to accommodate other peoples' preferences, but I don't need to change my own preferences to align with theirs in order to do so.
> Having a specific time for it makes it easier for people to do.
Again, I don't relate to this. I agree with you that it makes it easier for some people to do, but I'm not in that group of people. My point in this thread is that people are different.
> So it's natural to say "Looks like I'll be getting into the foobar system today; who worked on that most recently?"
It is much more natural for me to say "`git log`, what have people been up to in this system recently?". I prefer to read text than listen to verbal explanation. I also don't usually want to be offered help without being asked, I just want to go read and work.
Again, I'm not at all saying this is the one true way, but please believe me that I just have different preferences that make status meetings less interesting to me (and I think a lot of other people) than they seem to be to you (and many others).
I believe that you have different preferences. And I'm happy to take your word that you don't have social anxiety. However, loving socializing doesn't prove it. I loved it and also often got anxious. Now I love it as much but am not as anxious, which has made the social parts of collaborative work easier for me.
As to the second bit, I'm sorry to be contradictory, but having a specific, in-person time where asking for help is normal and expected lowers the barriers for literally everybody, because the particular barriers being lowered include costs like "figuring out how to contact the person", "being concerned about catching them at the right time", "contacting the person", and "delay in receiving a reply".
If you have a different approach you like better, that's great! The way a team works should match the needs of the people on the team, and of the team culture. There are plenty of teams out there, and good work gets done lots of different ways. I hope everybody ends up on one that works for them.
That said, "personality" is not an immutable set of characteristics, and not all of the way people express their personalities are equally good for collaborative work. As an introvert who also wants to get things done at the scale only a team can provide, I'm just always going to have to stretch myself.
I may also decide that I can only stretch myself so far, and so my limits may mean that there are kinds of work I'm just not well suited for. E.g., large companies demand too much of what I think of as "political" behaviors for me to find the work sustainable. So although I think we should generally try to honor each other's preferences, we must also be honest about the fact that not all preferences are equally suitable for every kind of work.
I disagree with your (in my view) far too prescriptive certainty that a lot of synchronous communication is universally best for collaboration. It is a common view (especially among managers) that I believe is simply wrong, and based on vibes rather than empiricism.
It's not universally wrong though. All I'm saying is that it isn't universally right. It just isn't universally anything. A good manager or lead should be able to look around on some teams and realize, hmmm, we aren't getting much value out of all these synchronous touch points, and on other teams they should realize, hmmm, we could use some more frequent synchronization here. But a manager or lead coming in with their own prescriptive preference for daily synchronous meetings, and failing to notice if they aren't valuable, is not doing their best work, IMO.
I have been on both kinds of teams. Indeed, right now I'm on a team for which I think daily morning meetings are working really great, regardless of my own personal preferences. But I've also been on teams where they are working terribly and only the manager is happy with the situation, and that's bad.
For me it's based on a pretty simple feedback-loops model. The length of the feedback loops limits all sorts of important things, especially external responsiveness, adaptation, learning, and error correction. Collaboration requires co-laboring.
There are circumstances where collaboration isn't particularly necessary, of course, so maybe that's what you're looking for.
I agree of course that there are plenty of ways to have bad collaboration even with short feedback loops, as in your last example. It's not a panacea. Nothing is.
We talk for a full hour every day, but there are only usually 5 or 6 of us, and we are all invested in every feature, so the meetings end up being a fairly detailed review and feedback on everybody work.
I got pushed to a team that did this. I decided to try to extend the meetings for as long as possible with the goal of eating so much time nothing got done anyday. The daily stopped shortly after
My current employer doesn't do any stand ups at all. I've been there four months and not a single all-devs meeting. Sure we avoid degrading. But I've never been part of a team that's more dysfunction than this one.
I wiah. But it's not even on the radar. In fact, I'd be marginalized more than I already am. I said to someone on the leadership team, "How can we have a team without a single team meeting?" I got no reply. I'm not even sure they heard me.
The major problem facing the industry is doing agile just because everyone is doing it, not because it is the right tool for the job. No matter how much money or people you throw at it, it's not going to work.
> The major problem facing the industry is doing agile just because everyone is doing it, not because it is the right tool for the job. No matter how much money or people you throw at it, it's not going to work.
I recently started a role running an Engineering org at a startup. The first two questions I asked everyone are why are they doing scrum and what do they want out of the engineering process. Answers to the first question were all over the map, and answers to the second were almost identical.
> you already know what they're going to say because you all, you know, talk
Standups are a pet peeve of mine especially. If the team isn't communicating regularly, then there's a larger issue that standups won't solve. If it is, then standups are a complete waste of time.
This is where a decent Scrum Master or Agile Coach makes such a massive difference... I've seen what you're describing so many times and it's a simple fix to get people to stop focusing on what they've done and to instead focus on what the team needs to do (ie focus on work not yet complete for the team and not what on the individual has done/will do)
In practice, any standup format where developers show up and basically perform to some standards set by a “master” or “coach” is kind of doomed to feel a little bit like a circus show.
The problem is the existence of someone else running the show in these meetings. The best standup I’ve had were those run by the engineers themselves. Adding a “coach” or “master” just makes everything worse unless the team is really bad at communicating with each other, in which case you have bigger problems.
Lord knows what people are calling coaching these days. But if they're actually coaching, then yes, getting the team to run the meeting was exactly my goal when I did that kind of work.
But that can be tricky. The biggest issue is to train managers to not make it about the manager. For that, an external coach is especially helpful, because it's the rare engineer who can sit their boss down and say, "Look, you're destroying this meeting. How about you not show up for the next two weeks, and then optionally show up but never speak for a month after that?"
But even once you've created a space where power doesn't distort everything, there are a lot of things that can go wrong. People who talk too much or too little. People are unkind, rude, cutting, or outright mean. People who talk about the wrong things. People who don't even know what to do if there isn't a manager around to try to please.
The truth is that for most people in the industry, their median meeting is bad. People with shorter careers may never have experienced a good meeting. So expecting them to just figure out how to have an excellent meeting every day is asking a lot. Especially a stand-up, which has pretty sharp constraints. If a team already knows or quickly figures it out, great. But a coach can really help.
> whenever agile project fail, agile shaman say "you didn't do agile right!" grug note this awfully convenient for agile shaman, ask more shiney rock better agile train young grugs on agile, danger!
I know you're being somewhat flippant but if agile shaman blame team for not doing agile right team should throw agile shaman off nearest cliff and find somebody who's not doing that job as some kind of dodge... too many self declared agile coaches and scrum masters out there who have no idea what they're doing - it usually starts because the organisation gets their first feed of them and agile through a consultancy who just tells them what they want to hear rather than they reality, and the senior management usually can't get out of that headspace.
pure marxism has never tried! if only we try the pure version it will work better! Marx had some good idea, communism stinks. Agile manifesto has good ideas Agile stinks. Maybe not as bad, but let's move on
I've found a direct correlation that the more closely a workplace conformed to the Agile manifesto, the better it was to work in. It absolutely does work in practice, and where it doesn't work I have always found specific, actionable places where they were doing it wrong. At some point you do have to actually spend 5 minutes to try to understand and follow the methodology - I don't think there's any methodology that claims you can ignore everything it tells you, do the oppsite of what it says half the time, and it will still improve your organisation.
(Unfortunately I haven't found any correlation between how much an employer talks about Agile and how closely they actually follow it)
All those people do is make me disgruntled knowing it's someone's _job_ to run this bullshit circus. Automate the lot of it away, don't make it someone's career.
People care and should care about what they get rewarded for. If promotions/raises/bonuses are based on the manager's view of your personal work, as is the case at most companies, then that is exactly what a sane person should focus on. Removing the manager from the standup or adding another layer of a Scrum Master won't change that but merely change how they get that same information. Standup at least provides an equal opportunity for everyone to sell themselves in those cases.
As the other comment said I'm simply noting the unfortunately reality that what matters is rarely actual delivery. Trying to avoid that reality won't make things more tied to delivery but simply more opaque as to what they're tied to.
There's exceptions but in my experience those teams don't bother with these types of deep discussions on process. The process is somewhat irrelevant since the expectations are fairly clear and what they will be measured on is also fairly clear.
lol, this is exactly the team I'm on that I got moved to because I hate scrum and they thought they were doing me a favor after a product focused reorg.
just...hours of useless status meetings.
It didn't start like that, but they overhired, and the time not in status meetings is split between a dozen slack channels. bleh.
> getting everyone to frequently think about deadline/feature trade-offs
I never let anyone else impose a deadline and a list of features on my schedule.
If the features are mandatory, I will provide the deadline. If the deadline cannot be missed, I will accept a sorted list of features (by priority). In both cases, I'll do my best to cram the most value possible in the next release, but the sooner I need to commit to its content, the poorer it will be.
If the deadline and the list of features are both set in stone, then it's not my problem.
> break everything up into smaller chunks. It gives you clearly defined guardrails
This is only true if management respects the estimates. In my experience, they do not understand why Fibonacci or t-shirt sizing, etc. are used. They mentally convert the points to some "X days" and then beat you in the head with it. They want to micromanage and they do not understand they are building software. You can't break certain tasks down further and you can't parallelize many tasks. You can bring up Fred Brooks but it's a hopeless cause. They will never get it. Then as you inch towards their arbitrary deadline (that they tell no one, by the way, because it's better the developers don't know about deadlines until the last fucking minute because "agile" and not "waterfall") they suddenly act shocked that 9 women really can't make a baby in one month.
They mentally convert the points to some "X days" and then beat you in the
head with it.
That's actually why I really dislike Fibonacci estimation. If we just estimated days or hours, then I could say, "Okay, this is going to take me until Wednesday," my manager would reply with, "Wednesday? It's going to take that long?" and then we could have a discussion about why it's going to take so long. Or, worst case, manager says, "quanticle, I really need this to be done by Tuesday," and I know I'm going to be staying late until it's done.
But with Fibonacci numbers, it's like, I estimate a 3, manager nods and agrees that this task is a 3, and then it turns out that when I said 3, I meant that it'd take until Wednesday, but manager thought that a 3 point task would be done by Tuesday, and now manager is mad at me, and I don't know why.
The old way sucked, but at least it sucked in an up-front and transparent way.
> and I know I'm going to be staying late until it's done.
I cant tell you how many times I spent long days, 16 hours at a time, sometimes over the weekend etc. Only to have the manager stroll in monday talking about how great their weekend was, and barely acknowledge the item was completed. And then I find out the customer that "needed this so bad" didn't touch it for the next month. Or were never told it was deployed etc.
So many deadlines are fake that it's hard to trust any deadline.
Fake or not, at least you were told what the deadline was. The agile approach to this dysfunction is for you to be pulled aside at some point and told, "You're not delivering enough points," with the manager remaining extremely vague about how many points is "enough".
I think you illustrated the point of the deadline well. How else were they going to get you to work 16 hours days? No way to make someone eat the cost of estimation error by pretending there is no error and making job performance and team esteem dependent on "not failing the sprint".
For me the most absurd conversation around Fibonacci estimation is the burndown (etc) charts and group estimates.. you can't add non-linear values and expect the sum to mean anything!
I've had managers fight over how the upcoming sprint is doable because it has around as many total points as the last, ignoring that they've defined the individual five-pointers as being significantly more effort than five individual one-pointers. And they either fail to grasp the issue or deflect when it's pointed out.
That brings to mind the other issue I have with fibonacci estimation: it makes improvement impossible.
In any high-performance organization, there is periodic self-reflection and self-improvement. Professional sports teams will, for example, sit down and "watch the tape", reviewing video of their most recent game(s) to see what they've been doing well and what they've been doing poorly, drilling down into specific areas where they need further practice.
Agile pays lip-service to this practice with its end-of-sprint retrospectives, but I've never seen an agile team do anything approaching a ticket-by-ticket breakdown, going over why the estimate was wrong, and what could be done to improve estimation in the future. One of the reasons that teams don't do this is because Fibonacci estimation makes it impossible to do this sort of thing. If I estimate that a task was a "3", and it took me two days to finish, did I estimate correctly? How could I have estimated better? These questions don't even make sense without a common baseline for what these numbers mean.
That's what I find most frustrating about Agile, as it is practiced. It's not the fact that the estimates are wrong. It's the fact that the estimates are so meaningless that they're not-even-wrong, in a way that makes it impossible to find and adjust for biases.
> manager says, "quanticle, I really need this to be done by Tuesday," and I know I'm going to be staying late until it's done.
At some point you have to grow a pair and create the workplace you want. (Or, y'know, unionize). Tell them to fuck off.
> But with Fibonacci numbers, it's like, I estimate a 3, manager nods and agrees that this task is a 3, and then it turns out that when I said 3, I meant that it'd take until Wednesday, but manager thought that a 3 point task would be done by Tuesday, and now manager is mad at me
They're explicitly not allowed to do that. If they're claiming to follow Scrum then that's a promise not to do that. At some point all I can suggest is "don't work for a liar". Of course that may be easier said than done.
> The old way sucked, but at least it sucked in an up-front and transparent way.
I agree that pretending to follow good practices while following bad practices can be worse than honestly folowing bad practices. But that doesn't mean that good practices are worse than bad practices!
At some point all I can suggest is "don't work for a liar". Of course that
may be easier said than done.
The point of my anecdote was to illustrate how Agile makes it easier to lie, and makes it difficult for honest people to call out liars. If a manager tells me that the deadline is Wednesday, and the deadline passes without anything bad happening, then he or she looks bad. If a manger estimates a "3", and I estimate a "3", and then it turns out that our definitions of what a "3" is differ, then I look bad, regardless of whether the work was, in reality a "3", a "5", or a "3.1415926".
But that doesn't mean that good practices are worse than bad practices!
The worst practice of all is a bad practice that masquerades as a good practice. I'd much rather that people be honest and up-front about the deadline (even if it is BS) than try to hide behind some overly complicated poker-game kabuki.
> The point of my anecdote was to illustrate how Agile makes it easier to lie, and makes it difficult for honest people to call out liars. If a manager tells me that the deadline is Wednesday, and the deadline passes without anything bad happening, then he or she looks bad. If a manger estimates a "3", and I estimate a "3", and then it turns out that our definitions of what a "3" is differ, then I look bad.
If your manager says they're following agile/scrum and says you took too long on that "3", they're lying and you should call them out; you don't look bad, they do.
> I'd much rather that people be honest and up-front about the deadline (even if it is BS) than try to hide behind some overly complicated poker-game kabuki.
If you've got a real deadline then be honest about it. But most everyday tasks don't need a deadline; agile isn't about pretending the deadline isn't a deadline, it's about actually not having a deadline.
I think that's the crux of our disagreement. I think most everyday tasks do indeed have a deadline. Even if that task specifically doesn't have a deadline, it's often in service of a broader task which does. The premise that all agile methodologies are built on is that customers are going to be more willing to drop features rather than add days to the schedule. I think experimental evidence has proven that premise false.
And frankly, it was kind of a ridiculous premise to begin with. Can you imagine if other professions adopted "agile" methodologies? Can you imagine your auto mechanic telling you, "Okay, well, putting the dashboard back on is going to take me until tomorrow, actually, but, if I only attach half the bolts and leave the speedometer disconnected, I can get it to you by the end of the day. Would that be all right?" Or a home-builder saying, "Okay, well, the drain pipes are proving trickier than anticipated to install, but I can give you the bathroom without the drain pipes, and install those in the "version 2.0" release. Would that be okay?"
> Can you imagine if other professions adopted "agile" methodologies? Can you imagine your auto mechanic telling you, "Okay, well, putting the dashboard back on is going to take me until tomorrow, actually, but, if I only attach half the bolts and leave the speedometer disconnected, I can get it to you by the end of the day. Would that be all right?" Or a home-builder saying, "Okay, well, the drain pipes are proving trickier than anticipated to install, but I can give you the bathroom without the drain pipes, and install those in the "version 2.0" release. Would that be okay?"
You're talking about stuff that's a regression from the previous state of things, which is rather different. Where I live it's totally normal for a dentist to say "ok, I only had time to fill two teeth today, come back next week for the others". Or a tutor to say "ok, we'll stop there for today and finish the chapter next week". Or a cleaner to say "well I've got time to do the balcony or the fridge but not both, let me know which you want today and I'll do the other next week".
I prefer 1 or 0, do or do not. The rest of the sequence is meaningless. If you are truly in a position where two different features will bring the company £5mil of revenue and one is a 3 and one is a 13, just do both, no need to prioritise the 3.
At the end, those complexity number end up representing some time estimates anyway.
Cause you will get the velocity graph, showing how much points have been completed this sprint and how it compares to previous sprints. The graph is expected to stay somewhat level or increase as the team efficiency increases. It will decrease when members are away on holiday.
If your team capacity stays equal and last sprint your team completed 20 story points, it’s kinda expected the next sprint another ~20 story points are completed. So in that case 20 story points will represent about 2 weeks of work for the full team.
I also think for many people complexity translates into time needed to finish work. A more complex task will naturally take more time to complete than a simple task.
So from my point of view, it’s fair (if after a couple of sprints) a product owner might get an idea that 1 point is equal to -say- half a day of work.
PS: I am not a product owner and not a big fan of ”Agile”. I think daily stand-ups might be overkill in many situations (in my current remote team it’s more like a weekly stand-up) and I generally hate retrospectives. I just want to get shit done (which is luckily the practice we have in my current team). We do have a task board, but we don’t do the whole assigning points stuff. And I am mostly free to pick-up any task I feel like working on. This is much more agile to me than “Agile”.
That’s generally true for that specific team in that configuration, but as soon as you start plucking people off, and adding other people to the team everything is up in the air again.
Yeah, my last gig had high turnover (not a coincidence with the dysfunction) and it made it particularly apparent that the estimates were a waste of time, but we kept on trucking.
Okay, but I could equally say, "No one would have issues with 'Agile' if developers could get infinite work done in a week... but that's not how the world works."
The fact is, there's a limited amount of work that can be done in a limited amount of time with a limited amount of money. Developers want the time and money to be unlimited, while customers want the work to be unlimited. But none of that is how this works: all three of these things are finite.
The point of "Customer collaboration over contract negotiation" is that the customer is involved enough in the process to see that you're producing close to optimally with the time and money they're giving you.
Committing to an amount of work in a sprint doesn't work, because it always ends up with customers trying to get more work into the sprint--more production than ultimately is possible. And, it's worth noting, setting boundaries around that and guarding to prevent that, starts to just look like "contract negotiation" with a shorter contract. Sprints aren't agile, they're Agile: a marketable idea that doesn't work.
Part of the problem is that customers see it as their job to get more work for less time and money, so they try to set aside what developers are telling them, the "work is limited" part of the trifecta. They see developers as their subordinates, so they think their desired outcome, where time and money are limited but ability to produce is unlimited, is reality. If developers and customers are seen as equals, both bringing needed value to the table, then this doesn't happen, because everyone agrees on the limitations of the situation. But as soon as you start treating customers as superiors, their view of the limitations becomes dominant, with predictable negative outcomes.
I am really struggling with this currently, I have agiled myself into a spiral of never being done. I am wishing I took a more waterfall approach so there was a fixed endpoint.
For personal projects in the future I will adopt a more rigid design phase as you don't have designers and managers enforcing deadlines and sanity checking whatever tangents you go off on.
You also have an agile option: release early and often! Use the feedback you get from users/customers to decide what to do next and when to stop.
The problem with waterfall approaches to "fixed" endpoints is that at the beginning of a project you know the least. It can be comforting to imagine that you can fix an endpoint up front. But most projects worth doing result in a lot of learning along the way. I have heard of very few projects that did their 1.0 release and were so successful that they just stuck with that release forever.
It’s hard to have this conversation given the countless definitions of “Agile” but I think the majority of what I’ve experienced does not help with the concerns you raise.
Proper project management to enumerate concrete goals with well-defined timelines is what you need, which is obviously not a novel part of any Agile interpretation.
Often the desire for Agile comes from not knowing how to manage a project and reaching for the popular thing that people think will magically solve their lack of expertise.
> Often the desire for Agile comes from not knowing how to manage a project
Corporate Agile takes thr worst of Waterfall and Agile and combines them.
It frees the management from budgeting for discovery and research, which are nessesary in waterfall. They can come unprepared.
Then they demand all the things they expect from waterfall - hard deadlines, no surprises. Then thry change scope on the fly because 'its agile, right'.
Agile does more to excuse poor managment than it does to enable developers.
I think this mostly nails how Agile is largely practiced. Being prepared with a clear, concise, well-communicated vision of what you want to build is essential to developing quality software, regardless of the adopted process.
Agilists frequently argued investing time and energy into crafting requirements is an “anti-pattern”, so being unprepared became the Right Way To Work TM. This is 100% dead-ass wrong.
Yah. You run into a different issue even if you've got that. There's a lot of example of scifi/fant sagas where, as the author gains fame and (so it is assumed) gains more ability to say "no" to their editor(s), the series goes down in overall quality. AFAIK you can also see this is military R&D.
(IMHO, you can see the flipside of that in a lot of web fiction with 1/wk+ release schedules)
I'm pretty sure I'd never want to try touching a codebase built with unlimited time and money - I'd expect waaay too much code, mostly.
IMHO, even if you have (functionally) unlimited time and money, spending unlimited time and money is still a bad idea.
(Balance comes from opposing forces. If you don't have an opposing force, you can't be in balance. Budget and shipping can be treated as opposing forces.)
You can do this if you hire the right people. The only “process” that always works is “hire right people and the F out of the way.”
The problem of course is that there are waaaaay too many software developers and roughly 93% are terrible at their job (this is just simple math, take anything and put millions of people to do it there will only be 5-7% that are actually good at it).
FWIW, the early '70s paper the DOD misunderstood as suggesting 'waterfall' had already said the right way was to build one to throw away -- but use the prototype to derive the usability, requirements, and design docs first.
If you had unlimited time and money, then you wouldn't need Agile. There's no intrinsic value in having 50% of the system working at 40% of the deadline.
> There's no intrinsic value in having 50% of the system working at 40% of the deadline.
In practice that's usually more valuable than having 100% of the system working at 100% of the deadline. Almost all deadlines are fake, and most requirements are too.
Agile, and especially SCUM, are a scam. Well intentioned, yes, but mostly serve to create an industry around a problem that often doesn't exist at all.
That's not even my main objection to Agile.
The worst part of these methodologies is that there's no evidence that they work. It doesn't matter whether an individual team thinks it works. Does it work for many teams? What about most teams? What for what kinds of teams does it work best? What kinds of teams does it have no impact on or a negative impact for? Nobody actually knows, and seemingly nobody is interested in studying it. Even companies themselves aren't honest about whether Agile is working for them. Burndown charts and the overall number of tickets being closed is about as far as it gets, except management inevitably gets their hands on these things and uses them as a means to extract more work from their employees. What suffers? Quality. Agile doesn't care about quality.
tbh, Agile as preached and Agile as practiced is not exactly similar.
I think there is even a good video describing that, from 7 years ago[0].
agile does not at all mean backlogs, or sprints, or specific tooling or even CI. Agile boils down in essence to “deliver often, do the minimum needed, prioritise prototyping”.
"Manifesto for Half-Arsed Agile Software Development
We have heard about new ways of developing software by
paying consultants and reading Gartner reports. Through
this we have been told to value:
Individuals and interactions over processes and tools and we have mandatory processes and tools to control how those
individuals (we prefer the term ‘resources’) interact
Working software over comprehensive documentation as long as that software is comprehensively documented
Customer collaboration over contract negotiation within the boundaries of strict contracts, of course, and subject to rigorous change control
Responding to change over following a plan provided a detailed plan is in place to respond to the change, and it is followed precisely
That is, while the items on the left sound nice in theory, we’re an enterprise company, and there’s no way we’re letting go of the items on the right."
Nah, most of Scrum is fine [0], even insightful I'd say, but yeah, the certification industry that drives its adoption is a scam.
The problem is that nowadays people have to be in the software field whether they want to be in it or not. So you have the perfect storm of people who believe in good old fashioned taylorism ("scientific" management) clashing with people who ultimately reject much of that thinking. Basically, traditional management realized that it will be obsolete and power is shifting to a bunch of "nerds". So what basically happened is that they appropriated a lot of the nerds' concepts and methodologies in order to regain power.
[0] Check this out, http://scrumbook.org/ - Take your time to look at the interlinked patterns and the rationale behind them, there is some worthwhile stuff in there.
> The worst part of these methodologies is that there's no evidence that they work.
I think it's worse than that. The management that promotes these bastardized "Agile" methodologies are often hostile to any empirical data that raises questions. It's not for a lack of evidence, it's that the methodology is elevated to untouchable status and any evidence to the contrary is explicitly suppressed and ignored.
Take the example of "story points". If all planning is based on story points, and all sprints are based on executing those stories, surely each sprint retrospective would look at how well the original points aligned with objective reality - use the empirical data and compare it to the estimated story points in order to refine the theory. This is the basic scientific method! Yet I've found that teams systematically avoid any such analysis. In the worst cases, I've seen teams explicitly forbidden from discussion story points in the retrospective ("How dare you attack the methodology with empirical data!").
As someone from the hardware world, I feel like very few of the agile evangelists have actually worked on a waterfall project. Everything they say about waterfall seems to come from hearing polemical takes on it. I have never been part of a waterfall project where the team didn't adapt partway through the project as requirements or understanding about the project shifted.
I cut my teeth at a waterfall shop on the software engineering side. In hindsight, what was really killing our productivity wasn't the process but a bunch of dumb politics in the org. I'm not sure Agile/Scrum could've saved them from that.
With that said I totally get why designers hate Agile - it forces them to make design decisions they're not ready to make and once there's functional software in people's hands everyone groans when design suggests non-feature changes.
I don't know about anybody else but for me the private conversations about agile have shifted substantially over the last 20 years. Early on, and once in a great while since, the conversation is that Agile is just a flashlight, or a mirror.
All of those dumb politic things get no pushback until the bitter end of the project when everyone is trying desperately to ship it before the board gets pissed off enough to start rolling heads. Then you get some death bed conversions that don't really change the fact that the real change was needed six months ago and now it's all going to be bailing wire and masking tape.
The fast feedback loop makes the bullshit quite apparent early on. And if you're very careful, you can arrange for some of the features that never should have been agreed to in the first place to be at the end of the line, where you can say, "look, we can ship what we have now, which does most of what most people wanted it to do, or we can keep farting around with this for another three months".
A big part of where this breaks down is that you have two groups, one of which hates to change and one of which has had to become more comfortable/resigned to it, and if you put them both in the same pressure cooker, group A is still going to evolve slower than group B, except now group B has concrete evidence about what a group of children group A is. Now both sides are looking in the mirror and neither of them like what they see.
The problem with knowledge is that once you have it, it's hard to go back, even if you don't like what you know. So I'm not sure what the road out of this is, other than the ones that sort of already exist, like OSS, which creates little pockets of what should be sanity (but often are just a different flavor of crazy).
I feel like when Agile first came out the local agile guru was basically an older programmer w/ a ton of years and wisdom under their belt who knew first hand why doing things a particular way was best. It's since becamse a 2 week course for anyone to enter the tech field. Feel the same way about the influx of cybersecurity experts. Those used to be really good devs who also knew a lot about security, now, don't even get me started.
It was started by people doing a giant business consulting project, for Chrysler. The Agile Manifesto came out of (some of? all of? I forget) the signatories' experiences working on the Chrysler Comprehensive Compensation System, using the Extreme Programming methodology. How well did that work out?
From the Wikipedia article:
"The one-year delivery target was nearly achieved..."
"A few months after this first launch, the project's customer representative—a key role in the Extreme Programming methodology—quit due to burnout and stress, and couldn't be replaced."
"The plan was to roll out the system to different payroll 'populations' in stages, but C3 never managed to make another release despite two more years' development."
"Frank Gerhardt, a manager at the company, announced to the XP conference in 2000 that DaimlerChrysler had de facto banned XP after shutting down C3..."
I've worked long enough to have seen it all. The mistake hardware people make when reasoning about software projects is that it is like hardware projects. It's why most hardware companies are pretty terrible at software. It just doesn't fit in their world.
Most engineering disciplines spend a lot of time for creating designs and blue prints before proceeding with the implementation phase. The mistake is assuming that software requires that too. It requires a little bit of thinking for sure. But fundamentally, the ultimate blueprint for software is executable. Your blueprint isn't finished until it runs. The process of producing the working blueprint is the whole point of a software project. You produce an executable specification in some programming language. The process of translating that to a working system (aka. compiling) is automated and does not require any human input typically. So, you do that very often starting on day 0 of the project. Until you are satisfied with the results. And why stop there? Many software products continue after their first release.
If you are building a nuclear plant (or any similarly large civil engineering project), producing the blueprints takes many years. In fact, nuclear plants are infamous for lengthy delays during that phase. Almost like a big software project. Unexpected things happen. Which then require changes to the blue prints. And until you have those, you're haggling with authorities and other stakeholders over permits, requirements, safety, and what not. It's fundamentally an iterative process that doesn't respect deadlines. Like software. Of course building the thing is also not without challenges but there's typically no room for a lot of improvisation during that phase. So, it's relatively easy and more predictable.
You can't really plan a hardware projects until after you have the blueprints. Hardware is just like software in that sense. The difference is that with software you are done once you have those. Planning the deployment (or customer implementation as it is sometimes referred to) is relatively straightforward: take the thing, install it, and run it.
I don't think I really agree with that characterization. To take PCB design as an example, you could just as easily say that once you have your blueprints (i.e. manufacturing files) then you're done. I see two big differences between SW/HW. The first is that software has a free and fast iteration loop whereas that same feedback loop would cost 2+ weeks and $10k on a PCB design. This makes it so that upfront design and analysis is more reasonable than try it and see. The second is that SW team structures appear to be much more coupled and interdependent. Multiple hardware designers can work for months on the same project without having to communicate much with anyone whereas this seems rare for SW teams. This causes the interdependencies between personnel to be highly predictable even if the work itself is not predictable.
If you incorporated feedback along the way, then you did not do Waterfall. Which is good, because Waterfall is insane and brainless. I've witnessed a 10-figure Waterfall mistake (I was brought in near the tail end, it was years late, over budget, and delivered the wrong thing). If they'd incorporated feedback along the way they wouldn't have fucked up so badly.
This is what I mean by only listening to polemical takes on waterfall. Do you think that anyone advocating for waterfall would actually claim that you never incorporate feedback? That's why you have prototype phases
Yes. I've witnessed it several times. Some people are truly moronic. But there's no point in arguing for Waterfall if you actually use your brain and don't do Waterfall.
EDIT: You didn't do Waterfall, just to be clear. You used prototypes and feedback along the way which Waterfall explicitly excludes. So why do you insist you participated in Waterfall projects?
In order to advocate for Waterfall instead of any Agile methodology during the last decade or so one has to be overwhelmingly blindfolded by the religion. And these kind of folk tend to be quite strict in their faith. As in: "we had enough time to discuss things and think it through and now it's time to stick to the decisions we made".
And yes, I have witnesses just that quite a few times.
I know a software company that strictly uses classic Waterfall even to this day. A minor feature that should typically iterate in production for like 3 or 4 times and be done takes them about a year to implement and still never deliver what the customer wanted. And I'm not exaggerating. (The company prints free money from another line of business).
I've seen supposedly "agile" companies (in reality: Cargo cult agile, jira-driven micromanagement and long plans. Much waterfall) that failed to incorporate feedback. So yes, it could be.
do you think its possible that we all said 'yep, our planning process is faultless, all we have to do is wait 18 months and it'll come out exactly as we conceived'. how blind we all must have been
I've seen it more than once, but I'm not saying it's common. But people saying "We do Waterfall" are saying "We're brainless, insane, or only working on trivial systems (in which case any process would work)." Once you start incorporating feedback loops, you're not doing Waterfall.
Sadly, though, it is in fact a real thing. I've seen several projects fail spectacularly. One of the best failures was "only" a year late. They also learned not to go 6 months into development without integration (believing their plan was sound) and skipping testing for that entire time. It was either brainless or insane, and it took a lot of threats from customers for the PM to finally learn that, maybe, a plan made a year or more out and committed to wholeheartedly is stupid.
> do you think its possible that we all said 'yep, our planning process is faultless, all we have to do is wait 18 months and it'll come out exactly as we conceived'.
Yes. I know from direct experience.
> how blind we all must have been
Yep. The industry really was that dumb. Much of it still is.
There's a long list of things we do by default today that I had to fight tooth and nail for in 2012, and twice as many in 2002.
On the one hand I try to remind myself of this when I start feeling like nothing is going right on the project. We are less dumb, and I'm applying the same(?) energy to things that matter less.
On the other hand, telling people for 15 years that we should be doing X before people start reliably doing it is fucking exhausting. If I thought about it, I probably avoid reading about things that are too 'new' today because I'll only get to use them in my dreams.
Sometimes places that use a lot of contracting start doing this brand of waterfall in house as well.
People in startup culture have never actually seen what waterfall can be like. Waterfall as first 'proposed' was meant to be a strawman used to show how not to do things and some managers thought it sounded pretty good.
No one said, "Lets do waterfall". What happened was people were just working, with weekly meetings which didn't pry too hard into what engineers did. Then there was a massive fuck-up. Something took too long, was too buggy, too slow, and some customer was pissed and left.
Then the finger pointing happened.
Ops said, "Our servers aren't slow or missconfigured, QA should have done load tests before approving it. "
QA said, "The code we got was shit and buggy, it was all we could do to get all the breaking changes found before ship date!"
Devs said, "Product kept changing the design!"
Product said, "Sales sold something ill-defined and over promised!"
Sales said, "Everything we do isn't sellable! We needed these specs to compete! The reason the customers left was that Devs took too long to develop! Product couldn't write requirements! QA didn't test!".
So a solution was drawn. Product would get sales to get their sales-engineers to pre-qualify opportunities and sign off on a pre-qualification spec. Product would accept that, and once accepting it they would own it.
Product would take the Pre-Qualification doc, and produce a detailed requirements doc. Including timing estimates for interactions. They would present that to eng, and eng would sign off on it. Once they sign off, they wouldn't allow any changes to it. Product and Eng would need to complete a change request for every change to the requirements.
Eng would take the requirements and produce a detailed design doc. Then they would setup a time estimate that they would be held to. They would produce a machine spec sheet they would give to Ops to get the hardware ready. Once they complete eng, they would present the completed code to QA.
QA would have a test plan based on requirements only after they would accept the code from Devs. Once QA accepted it, they had a timeline to complete testing. When they have reached a low bug count, (Having a high number of bugs initially should have been caught by the pre-QA Test plan), they would sign off on the quality doc and hand code to Ops.
Ops would deploy the code, if the product didn't perform, it would be assumed at this point its the hardware config and thus ops issue.
All of this would be designed that the team wouldn't be "at fault" if they got the next team to accept the code, and every team would make sure everything was perfect before they accepted it. To keep deadlines, minimum times for doc creation and acceptance would be created. The only variable would be dev/qa time.
The process would be fool-proof, guaranteed that nothing could be produced that would cause another fuckup. Typically the first major project to go through this process would be the last. Either the company died before it would be completed, or the process destroyed itself in such a massive flame/backstabbing fest that even the CEO would get fired.
In my previous job, they began using scrum agile to develop hardware. I don't know if it ever worked because I left. But when I was there, it was more stressful than before agile scrum.
I've been through a couple attempted transitions to agile for hardware. There are some agile elements that I think could work in hardware development but the people pushing the changes have always been too stubborn to recognize when SW agile methods just don't make sense for HW.
All the Agile that I've seen is best suited to something like an outsourced Web/app agency that works for enterprise clients, and bills based on time.
Just iterate, with uninterruptible chunks of billable time, and keep getting customer feedback on each concrete visual thing you can show them.
If it's inefficient, and might miss big-picture needs/opportunities/coherence, but you're making revenue on that inefficiency, promptly and recurring.
And the customer's many stakeholders are probably happier than if you'd tried to do project management. They don't have to think hard upfront, nor coordinate too much, nor commit to risky decisions. They just have a meeting every week or two, to approve/reject/bikeshed.
Unfortunately, sometimes this methodology is applied to projects for which it's not suited as well, such as for a company that needs a solid product and business model. (Not VC-growth-oriented startup that hires a ton of people, for whom sprint tasks are a "proof of work" productivity, which is a relief from the organization's hard coordination problems.)
And I get the impression that a lot of people in tech right now have never been on a team that exposed them to project management other than creating and doing individual sprint tasks.
Having done entire startups with release-early, release often approaches, I disagree. Customer feedback, used well, increases efficiency because you can test customer and product hypotheses, allowing you to stop working on the things that aren't worth investing in.
Is it true that you can be incoherent like this? Sure. But people doing waterfall approaches can also build incoherent projects divorced from actual needs. But it's perfectly possible to build a solid product and a good business model releasing early and often.
What's fun is that even though this is highly useful for that type of relationship, the firm I used to work for would tout these practices and claim that's how they believed clients would best be served; and then as soon as we set foot on the ground, they'd just capitulate to whatever worst practices were on the ground, out of 'pragmatics'.
Nö amount of arguments can deny that the vast majority of Agile nowadays is just a blatant MLM scheme. Some people ripped off other people selling them BS "Scrum Master" certifications and then promptly proceeded into roping management into hiring them, so that more scrum matters and agile-related jobs are created.
In practice, I'm still to find a single developer that likes agile or scrum, and doesn't have to battle against the constant nagging and interruptions.
Agile is all about "letting developers choose what to work on" but in practice that's a delusional fantasy, everyone has people or customers above them that do not understand the slightest thing about how coding works and that want to prioritize some things over others.
For most people scrum is associated with shady figures with nonsensical diplomas that try to sell snake oil to their managers, and that's a hard reality any fan of Agile has to confront with.
I’m pretty sure most of the agile hate comes from developers who have never worked in a real agile company/project.
As many comments here already mention the majority of companies are just cargo culting.
I’ve worked on a few agile projects and it was great. It just doesn’t work in a factory like manner, that’s the problem, it can’t really be ‘defined’. You have to get people who know what they are doing and don’t require handholding (helping each other is fine of course) and you have to give them an environment where they can work with flexibility and trust.
I agree; I've had one recent job that was agile, and it was done pretty well. Neither management nor customers were overbearing, if things didn't get done on time we'd talk about why the estimate was so wrong, if things took longer than expected we'd just push them to the next sprint, etc. Overall, the whole thing was just a useful way of keeping people on-track, and aware of what their coworkers were doing, and organizing and prioritizing work tasks in a flexible manner, and providing a process to keep things that way. The stuff with Fibonacci numbers was a bit silly but it was fine; it's just a rough way of estimating the difficulty of a task (which is different from the time needed, because this varies by developer skill). It was also good for making sure that new code got reviewed and tested properly before being merged.
Of course, it doesn't work well if you use it in a cargo-cult manner. It needs to be applied intelligently to what your team is doing.
I can introduce you to multiple developers who love agile, including me.
Believe it or not, if you asked me before I joined my current team, I would say that agile is corporate BS and I detest it, but turns out I was wrong. You can do whatever you want and call it whatever you want (scrum/agile/kanban/etc), it doesn't make it agile.
There is nothing in agile stopping you from producing comprehensive designs or specs.
Every project I take on these days has a pre-production phase. This phase is 1-3 sprints. It involves speccing sophisticated systems, designing UX and several spikes to evaluate technology. These all have outputs in the form of documents. Even the spikes should produce a short document outlining the technology recomendations and the reasons for the recommendation.
Similarly, in a project with a more research orientation, we perform spikes which involve building various prototypes and documenting their feasibility.
None of this conflicts with agile principals nor creates a situation in which engineers and designers are pressured to forsake, "building it right".
In fact the only differences between this approach and "waterfall" is that there is frequent accountability and there is protection for the development team because they are still choosing what comes into a sprint.
I generally find that whatever form the various criticisms of agile take over the years, there are generally two impulses really at work underneath them. Either management is trying to undermine the protections agile and scrum create for the dev team by violating the "only pigs talk" principal or it's developers trying to avoid regular accountability so they can build their castles in the air.
To be 100% clear, I'm not even saying that approach never works. There are phenomenal developers who disappear for weeks and come back with magic. But often enough, developers who cannot deliver try to perform the same act and their incompetence sinks a project. The development team cannot collectively afford the risk to find out which kind of developer you might be and therefore you owe them accountability. We all owe each other accountability.
> There is nothing in agile stopping you from producing comprehensive designs or specs.
Agile manifesto, 2nd point "Working software over comprehensive documentation".
It certainly doesn't stop you but if you focus on writing comprehensive documentation before working software, it is not Agile, it is something else.
As for Agile offering "protection", maybe I misunderstand your point but for me, Agile is the opposite of protection. I think that if there is one thing that defines Agile, it is that the customer is very close the the team, so it can be, well, agile. The customer has to be there to see how things are going and adjust course as necessary, unlike the waterfall style where a formal contract protects both the team and customer.
Note that I don't consider "waterfall" bad, and "Agile" good. It is just different approaches for different situations. If the customer wants an "hands off" approach, or if you write certified software, then I believe that "waterfall" style approaches are superior most of the times.
OP's opening statement was a little off. What OP went on to describe was not "comprehensive documentation" in the Agile sense but merely "documenting progress and learnings as to not repeat mistakes". The "comprehensive documentation" in Agile is trying to avoid the "We have delivered exactly what you asked for even it if is no longer what you need."
Regarding documentation, my team tends to choose parts of a project to document in detail before the project begins. These are usually around requirements and dependencies that have a low likelihood of changing. But throughout the project, the product is the gospel and the docs evolve towards that until delivery.
I suppose if you did a pre-planning phase like ours but comprehensively documented your entire product and then delivered exactly to those docs 8 months later without input, you'd not be doing agile for sure.
When I say protection, I don't mean protection from the customer. The scrum process protects the team by requiring the ICs on the dev team to choose the user stories or work items which will come into the sprint. This prevents management from committing the team to work they themselves do not think can be accomplished. This meeting should leave a written record which is shield but also the acccountability for the dev team.
> Agile manifesto, 2nd point "Working software over comprehensive documentation".
Which is funny given that the father of TDD, Kent Beck, is one of the Agile Manifesto signatories. What does TDD give you before working software? Comprehensive documentation.
It's not comprehensive documentation, it's a process of software development.
Start with the most simple test first, make it green via the most simple implementation, refactor. (Red/Green/Refactor).
Repeat until you can not adjust your tests/specs and implementation anymore.
Doing TDD you'll not create comprehensive documentation upfront but will switch your testing- and implementation-hat every few minutes.
Not at first, but by the time your software is working, you will have comprehensive documentation about it.
> Doing TDD you'll not create comprehensive documentation upfront
Where did the idea of 'upfront' come from? Having comprehensive documentation before you have working software doesn't mean having comprehensive documentation before you have any software.
TDD doesn't give you documentation, it gives you tests. Typically automatic unit tests, which is code.
Tests serve a different purpose. They are not for humans to read, they are to make sure your software works ("working software"). They are an important component of Agile because it makes it easier to change your code without breaking things.
Now, I am not big into TDD, but for me, it is an approach that is fully compatible with the Agile manifesto.
> TDD doesn't give you documentation, it gives you tests
Test is the action that is carried out from the documentation, possibly by a human, but more likely by a machine. We are talking about the document that describes what the program is intended to do, which is then followed to carry out the tests.
> They are not for humans to read
I should hope your documentation is there for humans to read. If not, even small modifications required as business needs change would become a complete nightmare. The tests you need to perform, and thereby the documentation that needs to be followed, will not remain static through the entire lifetime of the application.
waterfall is a strawman. any pre-agile organization that signed off on a two-year plan and showed up two years later saying 'where's my thing?' is just as much of an idiot as they would be today.
yes, making plans that you indend to continuously adapt to the situation at hand doesn't conflict with agile. but that still leaves one wondering what agile really brought to the table. time boxing everything at a fixed 2 week interval?
> waterfall is a strawman. any pre-agile organization that signed off on a two-year plan and showed up two years later saying 'where's my thing?' is just as much of an idiot as they would be today.
It's not a strawman. Those organisations are very real.
> yes, making plans that you indend to continuously adapt to the situation at hand doesn't conflict with agile. but that still leaves one wondering what agile really brought to the table. time boxing everything at a fixed 2 week interval?
Agile emphasises that you need to deliver something working to the customer, every 2 weeks. Without that it's very easy to call something code-complete or tested but not released or whatever, and you get further and further behind where you think you are because none of the stuff you thought you'd finished actually works.
The us vs them mentality between Agile and Waterfall that coaches often preach is itself a violation of the agile principle of people over process. You should identify what process best fits your product/customer/team, then use that process. Some products require lots of upfront requirement elicitation before you can even identify the appropriate solution-- and that's ok. Imagine prioritizing a working software in a sprint over figuring out what your customer needs.
I recently observed from the outside a project that took upwards of two years of requirement elicitation, IP contracting, complex distribution of ownership between multiple competing companies, multiple OS's, complex hardware platform design revisions to support the intended software functionality. Not a single line of code was written for two years. Was this state of the art engineering? YES. Did they operate under the principles of the agile manifesto? YES.
I do mostly agree. I'm speaking in broad terms here. Most effective software developmwnt for moat of software development history has probably run off some bespoke to the organisation hybrid of something resembling waterfall in some cases and resembling agile in others.
Your last paragraph makes me wonder: are there two sets of people, one of people who perform better under Agile, and one of people who don’t? We acknowledge the above is true for, e.g., work from home, but have many ever taken the time to consider the same might be true for development process?
I’ve worked with (and supervised) developers who I’m fine with sending off into their corner to do what they think is best for weeks at a time. I’m confident they’ll come back with something great, more often than not. I’ve also seen developers who I’m not comfortable with sending off on their own for even a day in between daily standups.
I wonder if there’s a process that allows to reap the spoils of both types of developers. Naturally, I’d rather have a team of the first type and think hiring those is the best way, but inevitably, some of the second type make their way in.
What this normally looks like is a project leader that syncs with each developer as needed and aggregates that into the project management overview with no/minimal large stand-ups or syncs. If two developers should adjust or be in contact than the leader facilitates that. etc.
The agile framework can still be followed by the project leaders, but the developers (who don't like micromanaging) can be shielded from any rigidness.
And there are also adaptable people that make it work under most or any system, and another type of people who find the failure cases for almost any type of system.
A spike is when you do exploratory work on a concept/ticket/whatever to get a sense for its feasibility, utility, and how long it will take. It often goes hand-in-hand with TDD but it doesn't have to.
Basically you try and get as much of the feature implemented as quickly as possible. Forget tests, forget well-named identifiers (just call 'em foo1, foo2, foo3 if you're stuck), forget breaking other part of the app, just get enough of the feature working as quickly as possible that you can identify pain points. If you get enough of it working it can even be used as a prototype for the customer to play with to see if they actually want it.
Of course, when _and if_ you decide to actually implement the feature, you gotta throw away all your spike work or you're going to have a bad time. It is of course perfectly acceptable to keep it around on another branch for reference.
Absolutely, that's a good way to put it. All feature spikes should have tickets defining what it's trying to validate.
From a TDD standpoint, I also use the word "spike" to describe any quick experimental throw-away work done on a small slice of functionality before writing the first test. These should not have tickets as that would be a bit over-the-top (and you don't always know if you're going to do them).
I think it's happening. Capital One got rid of the 'Agile' division. Capital One follows leads from FAANG type companies and in turn, there are some other banks that follow (and marvel) what Capital One does.
They will just glom on to some other cargo cult. It was Six Sigma and the Capability Maturity Model and PMPs as far as the eye could see before Agile, Inc. figured out how to package and sell itself via SAFe and certifications.
Companies like Capital One will inevitably be seduced by the next great idea and will avoid hiring real leadership at the product and team level at all costs.
Decision-makers got to their lofty corporate positions because of how well they play the game will do everything in their power to make sure the game continues.
Not meant to defend (or bash) "agile" stuff, but isn't banking industry quite weird when it comes to modern software development? I'm simply not sure they can be a good example, given how generally awkward and legacy-ridden the whole industry is.
Capital One is still doing agile, they just fired the people that have that as their whole job. The managers and engineers now are in-charge of doing agile duties.
I have learnt that it is not the methodology that matters most, but the people carrying it out.
Second to the people carrying it out (the set of individuals) what matters next are the relationships between those people.
Methodologies like "agile" are not completely stupid, but are mostly useless. It is the job of a team manager to manage their team effectively. A rule book has its uses, (not totally useless) but if a manager follows it slavishly, or in the face of resistance by their team, it is really bad.
There is no rule book for good management. There is good and bad management, but the difference does not come down to a written set of rules.
So, getting finally to my point, the age of "cargo cults" must die.
I'd say the problem is a bit simpler than this: ability.
When you're part of a firm that's staffed by those with middling abilities, using concepts like Agile Development like a blunt instrument is typical.
The reason isn't a complex one. Simply, that's the best they can do.
In high performance firms, which are attracting the best and brightest talent, you have a lot less problems with this. They easily come up with taylored managerial solutions to managerial problems, perhaps inspired by ideas from various sources including Agile, rather than picking up a popular methodology they know of like a cudgel and beating everyone over the head with it.
This is why discussions around this topic aren't that fruitful. The only fruitful course of action is to strive to get employment at more competent firms with smarter managers.
I.e. work yourself up and out of it rather than bemoaning it.
I would have argued against this perspective a few years ago, but given my current situation, you are right. The ability to comprehend and manage complexity faster than others create it is key. The more help you have the better. Work with smart people who think about what they are doing.
This is a long article that has some history in it, but I don’t think it makes a point. Who cares if Agile came out of Lean? It didn’t, but they are closely aligned.
Agile is all over the place so it’s kind of hard to cargo cult agile. Maybe author is confusing it with scrum or whatever. And since agile is just a bunch of principles, there are also lots of dumbasses claiming to do agile when they are really just chaos.
The agile manifesto was a set of stated relative values that the authors felt lead to positive results when a development team bought in. It wasn't a description of any methodology or process. In fact I have never been able to tell how agile methodologies were related to the manifesto in any way. Software and manufacturing methodologies can be good as best practices or guidelines, but when they require documentation for following steps, and have a methodology for changing the methodology, they lose their value. If you have a well intentioned group of team players that understand the values and guidelines and buy in, then the formality is a waste of time, and if you don't have those people nothing can save you. Process gets used as a shield for accountability by those people not a tool to correct their performance. Accountability (not blaming) is simply a way to adjust the power given to well intentioned team players based on the results of their decisions and actions. It is not a punishment, even though it does hurt.
I really think that the problem with Agile isn't really Agile as such. It's that Agile, at least everywhere that I've seen it done, is used as a mechanism by management to squeeze developers harder rather than to be a better method of development.
In other words, it exists to serve management, not product development. That's why it ends up as a means to treat programmers like machines.
That's not really Agile's fault. As long as management has the last word about things, any methodology will end up the same way.
Covering the entire lean and agile history without touching on the work of Alistair Cockburn is missing some depth. He was on the list of Agile Manifesto signers and gave us Crystal and the Seven Properties.
More than a thousand citations is evidence that Alistair's work was novel and established consensus:
See Chapter 2. Many of these methods were practiced by my team in Boston in the late 90's without the jargon. The results set the stage for how we'd work years later.
I can’t even comment on generic “Agile” because it is so vague as to be meaningless.
Many teams I have worked with who have been successful work out their own process but slap “Kanban” on it as a label. It may or may not be Kanban, but by calling it Kanban, it is “acceptable” by TPTB as acceptably checking the agile box for management.
This is totally legit. Kanban is nothing more than tracking work by using tickets on a board organized by priority and status. It does not entail any Agile ceremony at all. I don't think it even requires that you track all the work on the board.
It's a very lightweight system that most can benefit from.
Cults gonna cult, and with special names like scrum master and rituals like standup, well ... cults have a way of fragmenting, as per the old Emo Phillips joke. Agile, in its flexibility, lacks what might best be called "orthodoxy." It lacks punishments in accord with the most common failings.
"He hath spoken the forbidden word 'deadline,' and we must pelt him with marshmallows." I'm joking about that, but only sort of. Agile has entirely too many ways it can be taken advantage of by bad and/or incompetent actors to be all that forgiving. I've seen so many sins: the stakeholders who will never meet or communicate with you, the deadline given only to have seven months elapse before they look at the product, the demand to see the prototype in the first week with all of its features (leading me to ask questions like "How fast does ten percent of a Porsche go?" and "If the prototype is feature complete, how is it a prototype?") Standups with a lot of clucking from the chickens, standups where the wrong people were present.
It isn't even a "process," it might better be defined as a failed attempt at a meta-process. It lacks the equivalent of unit tests and commandments graven on stone, so pretty much anything ends up going and Agile stops having its one original virtual: kicking the ball back to the stakeholders constantly so that the developers cannot be seen as the hold-up.
Really, that was the original magic of Agile, just keep punting it back to them so it is that much harder to blame programmers. This failed pretty early on.
As I grew more disenchanted with my latest cult exposure, for one project I finally built in a very specific "Did they even look at it?" framework, much to my boss' eventual dismay. I could tell who looked, when, how much of it, everything. He was very into Resume-Driven Development and blaming anyone else, so me batting it back on the absentee stakeholders didn't go over well ... but that was the original methodology of Agile.
And the defense is always the same: you didn't pray hard enough, you didn't pray right.
One thing I don't see very often in connection with Agile is the reasons pre-Agile project management approaches failed: Replanning a resource-leveled critical path analysis takes way too much time and effort for little benefit in the context of most kinds of software development projects.
That, in turn, points toward incremental assignment of people to tasks as a key difference. There are of course many other differences, but exactly zero Scrum certification training courses even come close to mentioning stuff like this. "Cargo cult" is the inevitable outcome of not knowing anything about why one should do Agile for software development projects.
Saying something failed is awkward as there are plenty of old software projects pre-agile that delivered amazing results, on time, within budget, at extremely high levels of quality.
Yes, and who else is annoyed by random Japanese words being used to say the thing English but make it sound exotic and sophisticated? From the article: “Jidoka: the harmonization of people and machines” (jidoka/自動化 which literally means automation), or random TED talks about Muda, Ikigai, or Mottainai (sprinkle in Konmari for good measure). People here in JP think, what’s so special about these words? Kanban in the end just means signboard.
I get that adding memorable words to things sometimes gets the point across extremely well, but all those other times, just say the thing you would like to say, heh.
Sometimes words have inflections, implicit, or multiple meanings in different contexts. The point of using a word from another language is to emphasize the meanings that it evokes beyond the literal, and always inadequate, translations. The simplest example I know of in Japanese is "hai", which literally translated means "yes", but はい doesn't necessarily signal agreement. Take the english word, "okay". Depending on the context and how it's said, it can mean simple acknowledgement or something closer to WTF.
Of course people who think the literal translation is the meaning will miss the connotations.
> Of course people who think the literal translation is the meaning will miss the connotations.
Isn’t that exactly the problem, though? In my mind, these loan words from other languages create the illusion that they were clear and unambiguous. In reality, however, people still often enough talk cross purposes, but as they now use exotic language, they often won’t even notice. (And when they do, they start a flamewar about who has the right definition, which rarely leads anywhere useful.)
One other downside is that all that made-up wording makes it unnecessarily difficult for job starters to find their bearings.
If you organise your team’s tasks with post-its on a whiteboard, why not just call that “task board”, plain and simple? That might not imply all the particularities of the process, but there are many examples from all areas of life where terms in themselves don’t convey perfectly comprehensive meaning, and yet, people manage to deal with that.
Then everyone would see "task board" and assign whatever meaning or implication they already have with that term, probably missing the point of why this system is not the same as the task board they learned about in elementary school or whatever. Different people would assign different understandings, and none of them would be the one intended by the system.
> unnecessarily difficult for job starters to find their bearings.
"unnecessarily" is a value judgment. You're assuming that using familiar words automatically leads to common understanding, that people will notice if they have different understandings, and there won't be flamewars about the meaning of common words. What's worse, the effort to explain to new people the system, or the results from everyone assuming "task board" means whatever they think it means and running off in different and possibly conflicting directions?
Words mean things. Loan words enrich language when they lead to shared understanding.
It will not end. If anything it'll get worse. It's become a tracking mechanism which management loves. There's also a gazillion agencies out there that will push for it as that daily meeting makes it look like there's constant progress, even if most people BS their way thru half the meetings.
Sturgeon's Law rephrased "90% of all work processes are crap but labelled with whatever is popular"
With "X is the problem", X is always a valid pointer to that 90% regardless of the actual merits or demerits of X itself. Changing fashions update the name, but the value is always the same...
Everything that is needed to develop working software in terms of project management is described in Frederick Brooks "The Mythical Man-Month".
Design software. Do incremental development. Half of the code should be automatic tests (doesn't have to be unit tests, that are often waste of time, functional test is typically more useful).
Unlike many, if not most, "Agile" proponents Brooks actually took part in creation of a really robust and badass piece of software, IBM's OS/360.
In addition to corporate agiles like Scrum of SAF (SAF is a true abomination, all the most stupid ideas put into one bucket, but expensive SAF certificate expires after a year and has to be repurchased, so they make a lot of money) there are also anecdotal agiles, like Basecamp "Shape Up" that probably can work if you have a lot of time, mature fashionable product that sells itself anyway and you don't have to answer to annoying customers.
All of this is mostly buzzword filled bullshit. Waste of time.
I'm intrigued by the referenced article's rallying cry
> It’s time to create a new operational standard.
What does that look like?
I do find agile to be tiresome and overbearing at times, but its operational model seems to make sense. At this point it has a an inescapable gravity. Can we think of a new operational model? I would love to try one out if I could just learn of ANYTHING else (please).
I’ve recently found myself thinking a lot about the coercibility of ideologies as an intrinsic property of the ideology. Agile software development is certainly as much and ideology as it is a methodology, and it seems particularly susceptible to coercion. I know people who love it and who hate it, and they almost all have one thing in common: the all disagree about what it actually is.
Personally, I think it’s clear and I am a huge fan of the espoused principles, but I feel like those same principles are constantly misinterpreted by people. Of course, maybe it’s me.
Or maybe the Agile Manifesto is simply broad enough and vague enough that everyone interprets it the way they want
I'm not a fan of Agile for precisely the reason you give. If the principles are constantly misinterpreted, it exists as name without an anchor.
I have encountered advocates of Agile who say that other people only say they use Agile but don't really use Agile. They are keen to point at successes an say "look They're using Agile" and failures as people who said they were using Agile but weren't really.
You can't point to the foundational document because advocates will claim that Agile has evolved since then. It is unfalsifiable in the sense that no mater what critique you have, it can be avoided by using the shifting definition.
The article here talks about Cargo Cult Agile, but the people doing that Agile don't believe it to be a Cargo Cult phenomenon. Who is the Author to claim that their view is the ground truth? Declaring that there are large groups using the name without the principles is the akin saying "My principles are less popular than you might think"
It has gotten to the point that I don't believe the name is meaningful unless it as a Tribal identifier. To be frank I would like the name to go away completely. The principles should be able to stand on their own merit, people should be free to take the things that work for them. There is no need to Proper Nounify ideas other than to show you are talking about the same thing. For Agile, that ship has sailed.
Instead of just bitching and moaning somebody senior on the team needs to aggressively push back with productive feedback and compromise. You can't just say stop all of the process. You are invalidating the Scrum Master/TPM's existent and will just meet fierce resistance.
Ask them to let you run the the standup the way you want to run it one day as an experiment. Ask them what metrics and reports are being shared so you understand why this is a hill they will die on. Ask them to participate as peers during the ceremonies so they understand where you're coming from.
I would never go back into a agile/scrum organized environment where people talk about stoey points, the meaning and contents of a retro or an issue with the title of a ticket. If i only think about it i think its pure hell to work in it.
Instead we work in self organized teams. Where we have defined our own rules. Our own way or working. Ten times more productive and enjoyable.
> Instead of keeping the assembly line running no matter what and then fixing things at the end (aka “always expected to be producing and delivering, incentivized to cut corners in order to get it done”), the expectation is instead to stop-the-line when problems are encountered so as not to leak quality issues to the next stage.
What is this supposed to mean in practice? Working outside of sprints?
It reminds me of what someone online said about ITIL (and if you haven't had the pleasure, consider yourself lucky):
It's like sex in high school. Everyone says they know all about it, everyone says they're really good at it, and everyone says they're doing loads of it all the time.
No-one knows much about it. No-one is any good at it. And absolutely no-one is doing any of it, at all.
The author is going of his way to explain what Agile is NOT. Instead, it would have been great if he could have defined his version of Agile. Beyond the Agile manifesto, Agile is nothing.
Scrum and daily stands up are not Agile, they are interpretations of the Agile Manifesto.
In my experience talent will succeed in any system and lack of talent will fail regardless of the system. Engineers will never design a system that accurately detects who is doing a good job vs not. It's not in their self interest to do so at scale.
Bit late to the party... but I don't think it even can be called cargo cult.
Cargo culting means imitating motions without understanding what they are supposed to achieve or why.
In case of Agile, even the motions are typically not being repeated. The basic tenets of Agile are moving responsibility for improving efficiency and quality to the teams and individual contributors with the goal of speeding up improvements. Why? Because in an infinite game whoever is improving faster is correlated with better chance to stay in the game for longer.
If you looked at the "motions" of truly Agile teams you would see that the management allows the team an ability to define how they work and achieve their performance and the team owns the process and is spending a significant portion of their effort towards improving it.
If "Agile" (to distinguish from actual Agile) is cargo culting, it is imitating other organisations which are imitating other organisations and so on, until both the motions and the reasons are completely lost in translation.
In "Agile", you will see some kind of advisor to the manager or the manager instituting "Agile" process by prescribing how teams are supposed to work with typically unspecified goal of making the organisations better (but no measurements to confirm the improvement). As a next level of perversion, they can even introduce Scaled Agile Framework which when implemented typically gives absolutely no room for teams to tailor their process to their needs and requires the team to get management approval for a smallest deviations. I have many times fought and lost battles to introduce small changes to Jira configuration.
I think managers are both uninformed about what Agile is but also even if they were informed, not mature enough to let go of their compulsion to determine how their organisation is supposed to run. I think this insecurity and immaturity is what is driving "Agile" deployments.
In my work with underperforming teams I try to undo as much of the "Agile" damage and distill it to the basics: agreement with the stakeholders about what Agile is and what it is, reducing ceremony to the only thing that really matters which is retrospective and working with the team to instil sense of ownership for whatever they are doing.
That's a really good article. It gets so boring to see every week another "Agile must end" cargo cult from software developers who lack both maturity and experience to see their own flawed argumentation and who couldn't be either bothered to get a really deep understanding about software development methodologies instead of looking only for pieces to reinforce their own believes.
Reportedly it took 2774 attempts for Edison's people to figure out the lightbulb. So probably more like agile and less like Waterfall. Unless 2773 failed attempts were all part of the grand plan.
There is something delicious about the main criticism of an article critical of agile being that it doesn't understand agile and was doing it wrong.
I do think some of the points here are solid, but skip responding to the snark of the other post. That was literally the weakest part of it and generally not worth addressing.
Agile as a commercial enterprise underwent the unsurprising journey of consultants selling a revolution (if you can't integrate into existing structures, change the structure) to selling the manager's magic pill of convincing salaried individuals to dream about making someone else successful and rich.
The core principles of Agile as an idea drew heavily from cybernetics - systems dependent on discovery and adaptation benefit from effective introspection.
The modern consultant agile business is about making the stakeholders that pay the bills happy. SAFe is one such agile. So don't be surprised about what agile is today. It was always meant to be just that. The people that started it made money and the people selling it now keep making money.
If you want a management system that actually works for an information dependent and/or producing system, nurture an information flow that enables rapid adaptation through introspection and exchange. Except this goes against the core interests of the salaried managerial class in any mature organization.
Good news for founders - disruption is faster and easier in a start-up.
Good luck with that, the whole enterprise space is full of companies whose business is to sell Agile development, with the whole package that belongs to it.
This is what managers care about, regardless of what is being discussed on the engineering floor about what Agile is supposed to be like in practice.
Methodologies have to start saying something about what is being built. Games, phone apps, embedded systems, etc, are all very different and we should put the specifics of each one into the guidelines of the methodology used to build them.
I’ve seen multiple posts on HN titled along the lines of “the age of x must end” in the last couple of days. Contents are always pretty bland with no new concepts or ideas.
I’m not the only one assuming this is some sort of chatgpt or similar experiment am I?
Agile as typically implemented is a set of processes developed by web consulting companies to deal with difficult clients.
If you're a web consulting company dealing with a difficult client it's probably a great way to make sure the project actually gets delivered and you actually get paid.
If you're doing something different then you might find the processes more or less useful depending on your domain.
What the hell is Agile? I started professional software development 13 years ago and ever since I have been hearing and listening about Agile but I never found out what it is. Can someone explain?
Agile at its core is a reducto-ad-minimum of project management, with a focus on talking to your customer first, then implementing features.
Originally, Agile was a manifesto:
> We are uncovering better ways of developing
> software by doing it and helping others do it.
> Through this work we have come to value:
>
> Individuals and interactions over processes and tools
> Working software over comprehensive documentation
> Customer collaboration over contract negotiation
> Responding to change over following a plan
> That is, while there is value in the items on
> the right, we value the items on the left more.
The idea being that teams that talk to each other are better than teams focused purely on "doing the motions" of their process, software that works is better than well documented software that doesn't, knowing what your customer needs rather than nitpicking over the details of contracts is more effective, and accepting that things will change and working accordingly produces a better product than something developed exactly to plan.
The core implementation of an Agile system is one where you have regular requirements meetings, build a product that can always be shipped (for some definition of "ship"), and value a stable product over glamorous feature creep. This manifests itself as two core loops: One where small, incremental changes are made in the smaller loop, and one where larger changes are discussed and planned over time, but subject to change.
Today, Agile has morphed to consume a bunch of aspects from Scrum, Extreme Programming, Lean, and Kanban. It's to the point where most people don't know the core tenants of Agile, say they're doing Agile, and then pass on this mutated version of Agile along to others who adapt it.
1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
What if the software requires a lot more work before it becomes valuable? What if continuous effort isn't sustainable or there aren't many features required? What if this just leads to software development for its own sake?
2. Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.
What if change is so constant that you never ship a single working feature? Or every time you change, it breaks the user experience? Or you simply can't keep pace with how much of the app is changing and are stuck in tech debt hell?
3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
What if the software is running in a car? A plane? A bank? A pacemaker? A power plant?
On and on it goes. Stupid overgeneralizing proclamations that are trivially debunked, yet passed off as some kind of grand scheme to solve all that ails making software products.
You know what software is? Materials. Materials are used in our everyday lives, from textiles to plastics, structural steel to paper. You build a machine to grow the cotton, then a machine to harvest the cotton, then a machine to spin it, then a machine to weave it, then a machine to dye it, then a machine to cut it into a pattern, then a machine to stitch a logo into it, then a machine to make the plastic bag it's packaged in, then a machine to ship it, and so on.
All of those materials and machines are created by people, and then used by people. They break down, they need maintenance, they get replaced, and they get used. The significant difference is, most of them aren't made so only Gardening Machine A can be used with Weaving Machine B to go into Dying Machine C and be used on Cutting Machine D, etc.
Software is written in a little box, and it's very fragile, like the works in a watch. It's not durable and universal. It's delicate and tiny and intricate, and made to fit. We spend all our time fretting and niggling and poking and prodding and teasing the software, and wonder why we can't ever estimate how long it takes to make something. But it's because we keep making new watch springs, each one slightly different than the last one. Of course there's no consistency; we're not making things consistent!
The software industry is like an industry of teenage art school students trying to build hospitals. Quite expensive and complicated, but made worse by a whole bunch of "creatives" constantly changing things to fit some custom ad-hoc sculpture sitting on a platform of marbles. The people who are smart enough to make it won't do it in a way that's repeatable; the people who have the money refuse to contribute their work to some sort of general industry-wide commons of resources; and the people who organize them just want to get anything done, so who cares about rigor or process or efficiency or uniformity, just shove some shit out the door so I can collect a paycheck.
At least cargo cults deliver a plane every so often.
> What if the software requires a lot more work before it becomes valuable?
Then you haven't broken the problem into small enough pieces. You're not trying to deliver a finished product, only something that can allow vested interests to gain a better picture of what is being built, enabling them to provide feedback early.
If you are building ChatGPT and deliver a simple speaker amplifier that will be used to allow the humanoid robot to be able to provide sound, someone can say "That's neat, but how about a text field in a web browser instead?" If you wait years until the entire humanoid robot is working, you've wasted all that time before getting the feedback that all you really needed was a text field in a web browser.
> What if the software is running in a car? A plane? A bank? A pacemaker? A power plant?
What if? If you are working on developing software for a car, you don't think you won't have a car to install it on to allow the vested parties to see it? Or, at very least, some kind of car simulator that achieves the same effect?
>Then you haven't broken the problem into small enough pieces. You're not trying to deliver a finished product, only something that can allow vested interests to gain a better picture of what is being built, enabling them to provide feedback early.
This is called roadmap. Client provide feedback to roadmap. There can be few iterations. Interests of multiple clients are taken into consideration. Based on roadmap products are build.
I worked in companies that made telecom software. The kind that manages both business processes and operational aspects. There are no "smaller pieces" that can be delivered to show process. There are also no people who will care about it on side of telecoms, because they know what will be delivered. Those systems are upgraded once, maybe twice a year after very long verification processes. They influence how things are working for hundreds of millions of people. Sometimes stuff that life can depend on, like 911 calls for example.
I also worked in multiple companies that made telecom network hardware and various network security appliances. It's same story. Clients know roadmap. Roadmap implemented. If this is new product, maybe a few key clients will get some alpha/beta hardware to start working on integration into their systems. Besides this, people on client side are busy with their day to day work and don't have time or interest to provide continuous feedback, because they know scope of product to be delivered.
Sometimes roadmap has fine adjustment or shift in priorities. Yeap. Yet, everything else that I wrote remains. No incremental deliveries, interaction with end users, etc.
Agile is extremely popular for a reason. Whatever developers think it promises a way to wrangle complexity (breaking things down), adapt on the fly better than waterfall, and provide estimates so the business can plan better.
Even if your company does it horribly wrong management is going to want those three things still. They won’t agree to take it away.
You have to offer them something with the same (or more) benefits that works better.
And “trust us we know what we’re doing” doesn’t tend to cut it.
So I think we’re stuck with it for a long time, for better or worse.
You're right. Agile was never for the developers, it's for the business. Developers complaining about Agile is like kids complaining about their homework - it's to be expected.
That said I've seen enough Agile projects flounder and fail to know that it's no silver bullet. Like homework, there is little indication that it actually fulfills its purpose.
Whatever snake oil comes next is only going to gain adoption if it promises even more control for the business.
Compared to the previous process, often just waterfall plus total guesses, I actually see it as a real improvement.
But people have to buy-in/trust it and do a half decent job. In my experience that’s not really what happens.
Even half-assed agile (if that) at one place I’ve been was a real improvement, though not as much as it could have been. There was a clear difference in the teams that failed/refused to embrace it to those who tried.
Agreed. But the old system of “well this is really big so 9 months to two years maybe? We’ll be over there trying to finish it” isn’t nearly as appealing. How far along are we? “I don’t know 27%?”
But “here’s 20 steps averaging X points so it will take Y long and we can easily track progress” that agile promises is much more appealing. It may not speed up good teams. Or bad teams. It may slow things down some.
A lot of the activities done in agile or scrum or whatever processes are potentially useful tools in specific situations. But people don't want to understand the history and reasoning behind the tools nor do they want to apply them judiciously as appropriate.
They want a one size fits all process that produces results, if not optimally, at least consistently. And whatever Agile flavor of the month is popular is just that, it's good enough. And the benefit is that you get this fun mind trick to deflect blame ("I don't know what happened we followed the process. People don't fail processes do.") or ("I'll tell you what went wrong we didn't follow the xzy process. If we had just agiled a little harder this project would have shipped on time").