Agile started to die in my division the moment we had to go to some buggy web page and fill out our progress.
Agile became just another vehicle for micromanagement. Our scrums became apologies against what we said we'd accomplish. Meetings were held by managers, outside the group, to discuss concerns about meeting our goals.
There were levels of permissions on the task list; the team couldn't edit tasks, and had to get permission to make changes. Pretty soon we weren't deciding what to do and how long it would take, all that was being done for us, by managers.
Scrum became another set of manacles, and something to be avoided at all cost.
The more and more I work in Agile the more I realize that the way many companies use it it is just an excuse to do micromanagement and attempt to treat everyone as an interchangeable resource/cog. Now this doesn't mean agile is bad, it just means for whatever reason management in many of my experiences has twisted it into a tool that seemingly leads to the following:
- Change direction every sprint to the point where an application has been morphed so many times into something new that it makes it a heroic task to actually engineer something that ends up performant without needing a complete rewrite
- Takes the pressure of the business/management folks to project, plan or foresee anything because they believe they can change direction at any point in time
- Cut down projects into such fine grained tasks that it takes the fun out of the work
Im sure in those cases agile was done wrong for whatever reason or another but consequently as a result of my experiences I've stopped putting up with it when I can. I'm definitely ignorant about project management but Ive found within a small team just talking and coordinating like normal human beings about how your going to work together and what your going to do works pretty well. I guess I'd call my approach the "no project manager" technique.
I used to work at a company that absolutely loved to do a variation on your first point. When faced with a multiple choice question, the answer would end up being "all of the above," but framed in such a way that "oh, we're merely keeping our options open." When that story comes up for implementation, what do you know but apparently MajorCustomer has heard all about how we're offering support for every computer ever made by human hands and is thrilled! They want to know when they can expect a demo!
Agile can also be a great way to uncover people who think they are "separate but more equal" too, when they decide that the concept of a sprint whose content is mutually agreed upon and doesn't change mid-stream is just a quaint little detail that doesn't apply to Important People.
A process that doesn't self-correct is not a good process. A process that doesn't contain mechanisms which naturally guide teams back on track is not a good process. A process that tends to be followed incorrectly and misused to a degree such that it becomes a hindrance (even relative to no process at all), even in the hands of teams operating earnestly and with good intentions, is not a good process. A process that succeeds only when used by the best teams filled with the best developers is not a good process (such teams will succeed regardless).
Agile is not a good process. It may contain elements which are useful, but on balance the history of agile has been a history of abuse and of use as an excuse for old fashioned micro-management. The infatuation with agile has set back the practice of software engineering in industry by a disheartening degree.
The future is not "do whatever you want" nor is it "revert to the bad, old waterfall days" as agile zealots try to scare everyone into believing, but the sooner we get past the religion of "agile" the better.
It has actually been pretty demoralizing to me participating in agile techniques as a professional. Ive been in SCRUM meetings where everyone would stand in a circle and the project manager would bring in a timer and time everyone when they started speaking. If you went over 2 minutes the PM would immediately cut you off and move to the next person. On other occasions Ive had PMs ask me (in an accusatory manner) why it has taken me 4 hours to do a task I estimated at 3 hours. Keep in mind that these experiences were at a relatively small startup not a mega-corp. I don't know of any other creative, well-paid, educated profession where this would be even close to tolerated and I think its a trajedy that this sort of stuff has become common place. Unfortunately for the company the way I and other coworkers "self-corrected" the process is by moving on to another place where that doesn't occur - ended up being a great solution for me.
A few of the agile "standups" I've been involved in have been run well, at a recent job I had the opposite problem to what you describe. For several months we had a team that was about a dozen folks, and a daily standup that typically clocked in at 45 minutes to an hour and a half, every day.
It boggles my mind how any manager can see 60 dev-hours a week get pissed away and fail to see the importance of it.
This pretty much exactly describes the experiences that our various teams have had. We've tried all manner of software tools over the years (Jira, since we use that for bug tracking already, Jira + Greenhopper, Agile Zen, Pivotal Tracker) and we pretty much always end up coming back to having cards on a board. Every tool ends up feeling too restrictive, too big brother, and too TPS-report-coversheet. The point starts to become "get the right data into the tool" instead of "use the tool to help us do our work."
We generally end up putting everything in Jira anyway for historical and statistical purposes, as well as for visibility when people are working remotely, and it simply ends up as someone's job to make sure Jira is in sync with the physical board every so often.
Sorry that that was your experience with AgileZen. Organizing work is a surprisingly difficult problem, and one for which we're very interested in providing a decent solution. I'd love to hear about your experiences and how we can do better. My email is (same username) at gmail.com, if you're interested. :)
Whenever I think about process and tool changes, in addition to thinking about what I want to work better, I think about what I don't want to change. If something is working well and you don't strive to keep it working while monkeying with how the "process" works, you will surely break it.
I saw an entire team self-destruct while building a product because management (including the inexperienced dev lead) kept wanting just a little more clarity into what was going on. Tweak after tweak increasing process, planning, and reporting at the expense of getting stuff done until the debs (myself included) couldn't take it any more.
If you don't try very hard to keep what is working, you will lose it. Often, what is working is more important than what is not.
Whether it's a board or a spreadsheet the point is that it's just a crutch, what works is facilitating communication, breaking work down into manageable chunks, and facilitating a conversation with the whole team on every aspect of work.
Very true. Even making the digital physical (say, printing out an email and putting on a stand on your desk) can be blown off.
I can certainly buy crappy tools screwing up a process, though - much of my job is making existing tools less crappy and less disruptive to processes (or replacing them).
The answer is probably to use simple tools - whatever gets the board or what you need in a convenient form without taking up much time. Stuff that doesn't have to be polished and that you determinedly don't spend much time tweaking and enhancing. I don't want to use pen and paper and a physical timer by my desk, but I don't need a "Pomodoro Technique(tm) Management System" when I can just use org-mode and a little timer script.
Wow. This is such a great post, and sums up exactly why every implementation of "Agile" I've seen at previous employers have been completely dysfunctional.
So there's an engineering team and they have a project manager. The post-its and standups and storyboards seem a little weird at first to the engineering team, but they go along with it. It's rarely perfect in the first iteration but they can see how doing this makes them more productive, and it beats getting a bunch of confused requirements from confused business owners, or working off cumbersome and inflexible product requirement documents. And it doesn't ask them to change much in the ways they're already communicating.
The problem is the project manager needs a way to provide "remote views" that the OP mentioned. The CEO needs to know where the implementation hours are being spent. The finance team wants to know which projects they can capitalize. And so forth. It is part of the project manager's responsibility to provide this.
This is where tools can be great. Enter in some data, enter in the stories and tasks and points and hours and projects and generate a dozen charts and graphs with the push of a button. Cool.
The problem is if the project manager doesn't manage this themselves, if they don't do the tedious task of entering in this information themselves, then it's going to be a mess. Because like the OP said, engineers aren't going to update the tool. They want to do work and if the work status needs to be updated to reflect their progress, that should just automatically happen. If they have to manage the updating, then you immediately have a point of failure because the engineer is never going to do this. Why? Because it's pointless. He's working on projects. He's cranking out code. He already has a dozen windows open on his desktop at all times to text editors, terminal shells, documentation on the web, documentation in PDFs. He already communicates to his peers via IM, email, in person, using version control, or even moving post-its on a storyboard. He is already passively indicating his status a dozen times a day. Why does he need to actively manage his status in some tool that has zero value to him or anyone he works with?
But the project manager implores, "just update your hours, guys, it only takes a little bit of time each day. We really want to maintain an accurate burndown chart."
If only it were so simple. Because beyond being completely inconvenient, the tool doesn't even use the same units of measure that he does! "Hours per story?" Does that count the hours brainstorming the solution? The hours actually typing in code? The hours working with the sysadmins to deploy the code? The hours (well, hopefully minutes) spent thinking about the project while taking a dump? If the story is closely related to another one that he basically worked on both at the same time, should he combine his hours and divide by two? And burndown chart? Does the burndown chart mean anything to him? Does it help or improve his life in anyway? We thought this project was going to take a week, but when we started getting into the implementation we realized there's huge hurdle X and now it's going to take two weeks. There's your damn burndown chart.
So the engineer hears, "just wax and shave your balls with vaseline, guys, it only takes a little bit of time each day." Because waxing and shaving his balls seems just as pointless as updating the tool. And even if it only takes a few minutes to wax and shave your balls, it's awkward enough that you will get really frustrated really quickly if this needs to become part of your workflow.
If the project manager wants to the use the tool because of the convenience and power of generating reporting charts and graphs to external teams, then good for them. But if they're going to ask the engineers to update it themselves, then they should be prepared for that to be as successful as if they asked them to shave their balls, because that is literally what they are doing.
First implementation of agile/scrum I worked with was a bunch of printed cards nailed to a board where in the daily meetings the PM used a pen to take hours out of it, or add more if we decided it was going to take more. After the meeting he took the cards to his computer and wrote some kind of report (no idea the format). Second company about the same except PM didn't do the report after but still post its on the wall. And it all worked great, then I got to the third company and no board, just tickets in Team Foundation the programmers had to update every day (heck, they even asked us to update various times a day). We had a daily meeting but not much else.
Truth be told, I usually just forget about the hours update, whenever I closed a ticket I just looked at the estimated time that was there (which wasn't decided by me) and put that as working hours. I was bugged everyday to keep my hours up to date because it was very important some higher up had a general idea on how things were going...
Here is the thing, first two places started using agile/scrum by reading online and books, third one payed a consultant more than 25K to implement this system.
You are absolutely right that manually updating hours is a huge chore and not surprisingly it is universally despised. Making estimates beforehand in scrum is also problematic because it doesn't involve the time spent in previous phases as you pointed out. Kanban, however, solves the problems inherent in scrum, and virtually all digital kanban solutions automatically track the time for you in each phase of your workflow.
So, it is possible to measure productivity relatively painlessly in kanban as it limits work in progress (discourages multitasking, reveals bottleneceks in your workflow). Kanban also takes idle time into account (ex: time spent waiting approval from the management or another dept). See http://flow.io/how-kanban-can-help-you-measure-productivity.... for a brief overview of kanban's benefits.
Disclaimer: I am the author of flow.io, a lean project management app based on kanban.
I think younger generations are getting more and more used to a digital-only workflow. I see also a shift in working habits from nine-to-five-office-work to asynchronical world-distributed work, which demands a such a workflow.
One of the things I really like about scrum is that it forces people to interact together. That builds team trust and solidarity. Those can be built in other ways, but it is much harder. If you want a great team, you need to build it somehow.
I managed a team in the Philippines remotely from the U.S. At the same time the rest of the firm was in NYC while I lived in Chicago. The remote experience was not exactly a positive one. Although remote teams and workers are a bigger part of business life today, I would avoid it when possible. It's very hard to get everyone on the same tempo of work and it is really easy to have a miscommunication.
I also had a nightmare managing programmers in the Philippines, 75% from my incompetency, 25% from other factors. Having my guys w/in a few feet is invaluable to me now.
And also agree with Daniel, there is a powerful simplicity in post-it notes. Automated tools on the other hand never improve your process. At best, at best they automate it and everything that is currently broken about it.
Can’t remember where I read it, but I recall a story about automating air traffic control, and one of the problems they encountered was that there is a lot of information being conveyed by watching people grab the little strips of information and move them around.
The state of the system is only one dimension, who’s interested in what and what they are doing is another. The claim was that humans are optimized for watching other people work, and most office automation hides all that:
You never see Snively grab the Horrocks file and open it on his desk. Anecdotally, I have seen benefits from physical cards: People might see someone grab a card and speak up “Hey, are you looking at Thermoregulating Widgets? I was talking to Phred and he said...”
This information is theoretically available when someone clicks “start” on a task, but somehow it is not as visceral as watching someone grab or move a note card.
I think there are pros and cons. On one hand, it can make communication a pain. On the other hand, it gives you access to a wide variety of people you might not otherwise be able to work with. With the right communication tools (especially video conferencing) and the finances to fly people in for meetings from time to time, it can be worthwhile.
You need both an extremely smooth running company and extremely motivated, well-organized, disciplined and communicative developers to pull it off.
One small problem: the number of developers who think they can handle working remotely in a team is much greater than those that actually can.
One major problem: none of these factors are constants. They are variables, and if they change in the middle of your project, none of the many things you could do to get a physically co-located team or company back on track will work.
In fact, unlike a team you see every day, it may take you a lot longer to even notice something is off. People so underestimate the importance of body language in human co-operation. The way someone walks into a room can sometimes tell you way more than 2 hours on a Skype call.
Pivotal Tracker [1] was amazing for my team. My team was spread all over the world, and it was hard for us to even schedule a stand-up for everyone because of time differences and other conflicts, so having a site that we could all get to was crucial.
Even when it got to a point in the project where it was just me tracking my own tasks, Pivotal worked better for me than post-it notes could possibly have. Among other things, I ALWAYS had access to them where I needed them -- at my computer.
I love PT and it's amazing for task management and reporting on a day-to-day basis, however it's not good at managing the big picture where I think cards on the wall is really good for exactly the reasons Daniel mentions. For remote use I don't think there will be any good software solution until 20' high-res touch screens are readily available :) In the meantime I use Writeboard.
I am a big fan of plain text documents. Where I work we have a tool that collects requirements and then they can all be exported into Word. The problem is that the tool takes these one and two line sentences and produces a document out of them. In between each blurb there is about 3-5 lines of redundant boiler plate about who wrote the two lines of text and when. It is almost unreadable in that form. Things like keeping a consistent verb tense/voice go out the window.
When I come on to a new project I like to see a well written document that has an introduction and several well written pages. I can read that and get the rest from the code.
The issue is that we don't need expensive tools to write text files.
While I agree with the author generally, I should say the one online tool I've found that has really helped me, particularly when I'm doing remote development with other people, is workflowy.
It's just a hierarchical list of stuff and you can mark items as being done. The radical simplicity (and slick UI) stays out of my way, and yet provides a centralized list of stuff to do for the group I'm working with. It's kind of like a shared emacs org-mode on the web.
My main concern is that they will add more features to it.
Agile became just another vehicle for micromanagement. Our scrums became apologies against what we said we'd accomplish. Meetings were held by managers, outside the group, to discuss concerns about meeting our goals.
There were levels of permissions on the task list; the team couldn't edit tasks, and had to get permission to make changes. Pretty soon we weren't deciding what to do and how long it would take, all that was being done for us, by managers.
Scrum became another set of manacles, and something to be avoided at all cost.