In software engineering you generally have a complex dependency graph to complete something that’s largely unknowable and each edge is a random variable with a high variability skewing right. If you take the joint probability that any specific and precise date is in fact accurate, it’s effectively zero and has a long tail in the “later” direction.
However management often confuses precision for accuracy. In fact, not just management - but humans do. A precisely articulated anything, no matter how much it’s caveated, is taken as a fact. It’s precise. You can measure it. To a human that feels accurate. We feel disappointed when it’s not accurate. Even when we were warned.
This then gets into the discussion of how to estimate better. Holding the construction above to be true for any effort of any complexity that’s futile.
The way I lead software is I keep a tight view of “could we do better now” and “are we doing the right next thing,” and if we learn we made a mistake we suck it up, fix it if we can, and then keep rolling forward. I give out precise estimates when asked and don’t tell anyone on the team. When we fail to meet those estimates I tell my story of precision and accuracy and get back to work. At some point things work well enough that the next step isn’t worth it given how long it’s taken and we are done. Seems to work well, yields good products, executes fast, and as long as I have a thick skin for being grilled on my inaccurate yet precise estimates its less stressful than the alternatives I’ve seen.
Recently I had the notion of making a variant of MS Project where the Gantt charts can have range estimates, conditionals, and general probabilistic features.
Then simply use Monte Carlo to run a few hundred thousand iterations to determine not just the estimate range (with probability curves!), but also things such as "which step is more critical to timely delivery?" and "who can't go on holidays during critical periods, and what are those periods?"
Etc...
Turns out that there are about a dozen such tools already, and people do use them for estimating complex projects. Think ITER and the like.
Fundamentally, it all goes back to what you said: You simply move forward every day and stop when it's "good enough". All the estimation in the world won't change the path you take. At most it'll decide if you embark on the journey, or not at all.
“Fundamentally, it all goes back to what you said: You simply move forward every day and stop when it's "good enough". All the estimation in the world won't change the path you take. At most it'll decide if you embark on the journey, or not at all.”
I often wish management would worry more about productivity vs estimates. In my company the only thing that comes from project management is the desire for better estimates. Nobody seems to really care about improving processes and staffing.
> I often wish management would worry more about productivity vs estimates.
A thousand times this. I just tried to explain to a manager overseeing developers that the minimum time for the "inner loop" of edit-build-debug is super important to productivity.
He just didn't get it, and was fine with adding multi-minute delays to that loop on his team just to avoid speaking to another manager once.
> Turns out that there are about a dozen such tools already, and people do use them for estimating complex projects. Think ITER and the like.
Can you link some? Some year ago, I spent many hours searching for exactly that, started a bunch of threads on social media, and in the end found nothing useful at all - except for one lead I accidentally discovered near the end: GERT, as in "Graphical Evaluation and Review Technique", or "let's take PERT and add proper directed graphs, and notions of conditionals and probabilities".
Unfortunately, I stumbled on that at the very tail end of my search, and had no time to dig deeper - but I assume if the software industry knew or cared about this, I'd have seen it mentioned earlier in any of the dozens of "project management" tools and startups I've looked at.
I created one of these tools when I interned at Crystal Ball (before it was acquired by Oracle. On the on hand, it really was as slick as you'd hope - like, since we used Project for the calculation engine, and Project integrates with Outlook's calendar, you can see that if Alice is the only one that can complete a task, that task can't be done while she's on vacation, etc.
On the other hand, I never had a chance to see it used for real (honestly, I'm not sure it ever was), but I'm not sure it would help. The results are hugely sensitive to the distributions, and the whole point is that even the distributions are difficult to estimate - you just don't know what you don't know.
Yeah I was developing in the thought works crowd in the SF 90s, and agile was a wonderful insight into how to best create high velocity and high quality, with high morale, software. I went into quant trading for a long time which by its nature is agile so missed out on the process adaptation of agile. The problem came when folks added story points and burn down charts - this invited the process people in the door to make things more predictable, which invited the management in to consume that desired predictability. My belief is software, for whatever reason, suffers from an uncertainty paradox. The act of measuring development impedes and slows development.
This is such a great précis of estimation, which I try to avoid at all costs but which people insist on me making. And like you I keep them from my team.
The only real problem is if, regardless of having a thick skin, you work for people who have little experience in the field and value planning over actual delivery. In my experience you are then well screwed.
I agree. The only solution that I have found for this problem is to take advantage of their ignorance and tell them things like "we need 2 weeks to do a hardening Sprint to protect from zero days" then you can do whatever you need to do those weeks and also upgrade a few packages.
I was explaining the project estimation process to my 8 year old (whip smart) daughter and told her about the "Estimate then add 10%" heuristic and she immediately recognized the flaw and said, yeah then add 10% to that, then 10% to that, etc...
The reality is, estimates are a business function not an engineering function.
Once you realize that, then you change your approach to: Never take on cold start projects that are business critical
Do this and you'll never have to estimate work that isn't already in some form of progress!
“Once you realize that, then you change your approach to: Never take on cold start projects that are business critical”
Unfortunately these are my favorite projects even if it’s a pain to tell management that there is no way to give them meaningful estimates. They pretty much have to agree to invest some money and then see what comes out of it.
> However management often confuses precision for accuracy. In fact, not just management - but humans do. A precisely articulated anything, no matter how much it’s caveated, is taken as a fact. It’s precise. You can measure it. To a human that feels accurate. We feel disappointed when it’s not accurate. Even when we were warned.
> However management often confuses precision for accuracy
One of the best tools for estimation, I've discovered, is not an estimation technique at all, but a communication technique:
Build error bars into your estimates when you deliver them. My preferred way is to give 30/60/90 estimates: I'm 30% confident it'll be done in 20 hours, 60% confident it'll be done in 45 hours, and 90% confident it'll be done in 60 hours. Deliver estimates thusly to management and they can determine how safe they want to be. If they say "great, we'll plan on 45 hours," remind them that there's nearly even odds you don't hit that. If they complain that the range is too wide ("how can it be 20-60 hours?! I need more confidence!") then you can use that as justification for exercises which increase confidence (prototyping, more detailed planning, research around any unknown systems, etc).
The exact format doesn't matter - maybe you prefer to give ranges or standard deviations or what-have-you - the important thing is to make it impossible for someone receiving your estimate to ignore the uncertainty in the numbers you give.
The US Navy developed program evaluation and review technique (PERT) way back in 1958 specifically to deal with that issue. It isn't perfect but it handles complex dependency chains and skewed variability just fine.
However, PERT hasn't been widely used in software. For most programs the improvement in estimation accuracy doesn't justify the management overhead. Better to do incremental delivery and reduce scope as needed to fit the schedule.
>However management often confuses precision for accuracy. In fact, not just management - but humans do.
And further down the path of madness, you have humans who confuse _lack_ of precision for accuracy, e.g. astrologers who prefer to base their theories and predictions on the movement of a single planet because taking other ones into account "creates confusion".
However management often confuses precision for accuracy. In fact, not just management - but humans do. A precisely articulated anything, no matter how much it’s caveated, is taken as a fact. It’s precise. You can measure it. To a human that feels accurate. We feel disappointed when it’s not accurate. Even when we were warned.
This then gets into the discussion of how to estimate better. Holding the construction above to be true for any effort of any complexity that’s futile.
The way I lead software is I keep a tight view of “could we do better now” and “are we doing the right next thing,” and if we learn we made a mistake we suck it up, fix it if we can, and then keep rolling forward. I give out precise estimates when asked and don’t tell anyone on the team. When we fail to meet those estimates I tell my story of precision and accuracy and get back to work. At some point things work well enough that the next step isn’t worth it given how long it’s taken and we are done. Seems to work well, yields good products, executes fast, and as long as I have a thick skin for being grilled on my inaccurate yet precise estimates its less stressful than the alternatives I’ve seen.