It's not fake research. It's actually quite an established science in the 24 years I've been doing it.
Take your first guess, double it, double it again if the stakeholder is a poser, add 20% per developer less experience than you, subtract 10% for the features you're going to essentially copy paste, add 15% for sick leave (browsing HN) and then double it for every question you have that are unresolved and divide it by the room temperature multiplied by the amount of people with mechanical keyboards.
That gives you roughly the right estimate for any job, until the next sprint.
The real problem is not with estimation but with the management denial.
I was asked to estimate a project I owned as a “green” lad. I offered 6 months with a real detailed plan and up to a year with some extras. Based on past rates we had etc. I even did some statistical modeling. They disagreed. They decided to alter the approach, that the principal engineer had suggested and “it would take 2-3 months.”
Two years later I learned they were still working to finish it.
Another example: they wanted a Byzantine Fault Tolerance rewrite along with the networking layer. I said we need 6 months. They said we know how it should be done in 1-2 months. I finished my stuff and moved on — they picked couple of people to work on the rest that had said 1-2 months. It took them a year to reach a point they could test it.
Now why did I say 6 above? Because that is a project I have done multiple times, I can do everything in my sleep and the delay beyond all other fundamentals is random interruptions. Did that matter? No.
When top head works with wishful thinking estimates are ways to find scapegoats; nothing more.
Your first mistake was misunderstanding the relationship with management.
Time estimations are bargaining, you ask for something and then get a counter offer from management, they don't care that the counter offer from them is delusional. The bargaining is the point, the less time you bargain for the more stressful the development can become. Don't estimate what it should take, estimate what it could take and bargain from there.
I think it mostly correlates with how technical PMs are.
People that were fairly technical prior to deciding to move to PM roles, those you can actually explain the context and reason with.
PMs that are more biased towards Business/Marketing are big on C Level networking etc., those are the ones that feel they are doing a good job if they try to bleed a stone with deadlines. They are adversarial like GP mentions and not someone you can reason with.
The other part is that management/PMs usually care about deadlines, not work effort (duration in man-hours, story points, or whatever).
Allow me to indulge in a fictional narrative:
You are handed "high level" (a.k.a rooted in the fuzzy non-technical alternate reality that is product management) requirements resulting in known unknowns and unknown unknowns. You're junior and naive and you put real tech-borne effort numbers in front of each task; or you're experienced and you put O(t) numbers on the first ones and random but high numbers on the second ones.
Anyway, the estimates end up landing past the desired deadline, management mutters TLAs such as "OKR" and whatnot, you'll have to justify why it doesn't fit, so you communicate work effort.
You are summoned into a retroplanning meeting to cull out stuff based on work effort so that it fits management's expected deadline, and try to pretend you're doing agile when it's all waterfall Gantt in disguise.
Project starts. Implicitly man-hours become wall-clock-hours, irrespective of other tasks such as CI/tooling/dependency breakage, code hygiene, or customer support escalations; your effort estimates became t{project start}+effort == deadline. You are now held accountable for the latter based on you communicating the former.
Time is now the scarcest of resources, you do your best, conjure magic, work extra hours, take shortcuts, and make it to the deadline, sacrificing body and soul. In a cloudy spreadsheet checkboxes have been checked, management is happy. Deployment is smooth but soon corner case bugs and escalations start trickling in. Management is either oblivious or not so happy after all.
Time for a new quarter to start. You know issues borne out of past quarter will rear their ugly head, and there's much tech debt to repay before interests quickly accrue towards tech bankruptcy. You're junior and swear to yourself to not make the same mistake, and you start inflating numbers; you're experienced and you realise the whole number dance is a sham as they don't matter anyway, but you still are required to do the dance because that's how the game is played, so you turn Scotty and just throw outrageously big numbers up in the air and just make sure the deadline is not too ludicrous, still hanging to the hope hoping that someone, somewhere, has seen the light of reason and built a company around it, and maybe you could go to these greener pastures; you're very experienced and you realise the only winning move is not to play, so you quit (not the job, the _industry_) and move to a remote plot of land with literal green pastures and a small cabin mountainside to raise sheep, or cut wood, or whatever, where the smallest time unit is the measurement error of eyeballing the zenith-relative angular position of a giant fireball 8.3 light-minutes away.
The tell is that as soon as they don’t like how things are going they flip from calling them estimates to commitments.
Also if someone’s estimates aren’t perfect, that’s something you can use to justify withholding salary. The game is fixed. It’s just a good deal less fixed for us than for the average person.
Well your responsibilities warrant a 20% raise but since your estimates are bad and your <bullshit other reason> was not 5 out of five, we’re giving you a 6% raise. Better luck next year right?
>>>. The real problem is not with estimation but with the management denial.
^^ This is the truth. All estimation processes in any company are just ceremony. Management already has a deadline fixed. They will make you agree to their deadlines, even though whatever process stipulated by them says otherwise.
The real problem is saying "yes we can do it all in that timeline" instead of saying "That's fine, but we won't get to everything. We will work with you to prioritise the most important things to get done in that time."
I know it's painful, but just hold that line. It's not lying or negotiating or padding; in fact it may be the only true statement you can say.
If a civil engineer were building a bridge, and couldn't do it safely in the budget and timeline, they'd say so, and the project manager would nod and diligently write that down. We have a more subservient relationship with project managers in software because we choose to.
You have to get them to take that perspective. What happens is "we want all this stuff how long is it going to take" and then they fight that number down while not answering any clarifying questions. So you have to sumo wrestle a bunch of assholes twice your weight into even framing the context correctly before the conversation you recommend can even take place.
> They will make you agree to their deadlines, even though whatever process stipulated by them says otherwise.
I'm not sure how. I mean, if I know it takes X time to do something, what would make me to commit to Y<<X time when I know it will fail and will be made my fault? If they want to record it as a deadline, so be it, but if it's an impossible deadline I'd make every effort to be on record that I said it from the start that it's impossible.
Not EVERY company. Just the ones that don't understand how estimation works... and unfortunately in software, that's still a very very common case. In more established kinds of engineering it works differently... which gives me hope that eventually our industry will understand too.
Management denial is not universal. It can't be - companies that have realistic management should out-compete companies where management is politics and unreality.
The problem is that management unreality can take a decade or several to destroy a place. But when you see that kind of management, know that that's the road they're on. That's your cue to start looking for somewhere with management that at least tries to live in reality.
I hope you are right. I have never been in or seen a workplace where management is not playing the 'get the order at (nearly) all costs - let others handle the problems' game. Again, I hope I am wrong. But these type of companies seem to outcompete all others. The only unicorn in a decade in my geographical area, is a company that plays this game very very well in the IT sector. As far as I have seen, they play a variation called 'the customer takes the problems (and the full financial impact). Due to building a good set of core competencies, that the competition mismanaged over years, they can get away with this strategy.
You’re forgetting that in companies with the management properties you say are bad, those properties may cause actual value to be delivered faster, even if it’s at long term cost. That value can accrue, gain interest, and that company can be more successful than the one with the management properties you say are bad.
I think you’re conflating success of a company with the pleasure of its employees. These 2 factors are usually (not always) opposing not correlated.
I don't disagree with you but I think people also need to understand that management often pick the team that provides lower estimates because team members will have to work harder to deliver it.
Management might not be so dumb. If someone said 2 months and you say 6 months, they might go with the person who said 2 months but will accept 5 months secretly.
The 2 month version experiences scaling issues at a 20% rollout, or takes 20s for a database query before returning a page.
In 6 months you need a better scaling version while the current version is creating cascading database failures; it's got terabytes of denormalized data that is out of sync and a 2nd team can't migrate the data in 12 months.
In 12 months, you find 5 other teams copied the 2 month solution.
In 2 years, you have 0 devs left that can solve the problem and a 1000% grown dev team that is great at the 2 month solution.
You ask the teams why they don't use the scalable solution and they say, "do we have any examples of that, the unscalable way is our convention"
In this vein, my manager doesn’t appreciate when they ask for an estimate and I say “If I knew how to accurately estimate software projects I would be making millions as a consultant. But I don’t so I’m here”
No my assumption isn’t that consultants make good estimates. It is that nobody in our entire industry knows how to accurately estimate how long a software project will take, so I would be a highly sought-after rare commodity in our industry.
Many, many moons ago I asked my mentor how to estimate. He told me that you should figure out what it feels like, then double it and then take it to the next higher time unit.
"hmm, adding that button probably takes about an hour"... 2 days is about right.
Almost 30 years later I still find this to be true. Truer than I like.
You don’t want them interacting with you too often for one hour tasks. You want to train them to lower resolutions slices. And you want to round up small ones to do refactoring as well.
Eventually they get a sense of just right. It’s never perfect but they are estimating before they even ask. Whic if we’re honest, they’ve done anyway and they’re hoping we will agree, absolving them of the responsibility.
Not that I've used this extensively, but months would likely increase to quarters.
And if you're estimating something to the lengths of months, you're already into project management territory size, rather than broken down to development/delivery sizes... The amount of unknown unknowns and other uncertainty certainly warrant happily estimating years length, surely.
I've found if you give realistic estimates, some contracting firm comes in, offers to do it in half the time, grabs the project and then takes 2x longer. Finding stakeholders competent enough to understand this pattern is important. If not, you might as well throw out any estimate and also take longer.
Yes, but how many story points is that keyle? Please be sure to use Fibonacci sequence numbers. If you don't give the ticket story points, we can't track the velocity of the sprint, and all our planning will fall apart.
I know you like to complain alot about how your cubicle roommate doesn't understand basic control structures and the difference between floating point and integers, but he, unlike you, always adds story points to the tickets. That's why he's on fast track to lead senior developer. He knows what's important. I'm trying to help you learn what's important too, but you've got to want to be helped.
Don't forget to multiply by the square root of 2 (management overhead coefficient). Also note that this is an irrational number so it has a subtle jab at management built-in.
I can't tell you are serious or satire. If you are serious that kind of estimate is as good as useless because the projects most likely won't be profitable. Can you imagine what will a property developer think if he is informed that his project is going to take at least twice as long to complete?
> Can you imagine what will a property developer think if he is informed that his project is going to take at least twice as long to complete?
Property developers aren't doing R&D jobs. Most of software development is a multiplayer life sciences R&D, because even if you're not developing new technology, the combination of your chosen stack, target market, stakeholder personalities and management policies, makes your project a one-off job in an unexplored environment.
My conclusion is there are 2 kinds of engineer: good ones and shit ones.
Good ones just need to be told what to do and be left to get on with it. They value autonomy, and any attempt to micromanage them with agile bullshit drastically demotivates them.
Shit ones are shit. They need a good one to help them get through their tickets, by asking what's taking so long and showing them better ways of approaching the problem. They need more handholding and don't yet find ticket-sizing a soul-sucking waste of time.
Shit ones might become good ones over time, or they might not.
The problem is that scrum masters are either former shit engineers or non- technical, so cling dearly to the belief they add some value to the process by acting like a parent (which only benefits the shit ones) or sizing things which benefits no one as it takes too long and is too inaccurate to be useful. Worse, it just demotivates good engineers.
Instead, teams need a mix of good and shit engineers, quick standups whose purpose is for the good engineers to realise shit ones need help, and very high level rough estimates at the project level of "this'll probably take us about 3 months". Then get out of their way but be there if they have a blocker.
Good engineers will tell you they need help, they don't need a frigging question every morning as if they're 5.
As someone who pulled part-time scrum master duties in a seed stage startup, the problem is that engineers, even good ones, tend to be not-so-great communicators. Having someone to grease the wheels in stand ups and planning meetings decreases the time it takes for issues to surface. The other half of the job is cleaning the backlog and keeping the product owner from derailing development. It’s more about waste reduction than adding value.
Granted this was just my very limited experience, and we didn’t spend much time on estimating and standups consisted of a scheduled daily Slack ping instead of a meeting since we had major timezone offsets. People over process and all that.
Not only that. Agile processes in general are supposed to protect everyone from sinking a tonne of effort into successfully building the wrong thing. It doesn't matter if people aren't going as fast as they otherwise might, if the counterfactual has them going in completely the wrong direction.
Scrum itself is more specifically designed to protect the team from management interference. Engineers might think it's a waste of time preventing them from getting things done, but in its absence (where it's truly needed) what would actually be happening is constant micromanagement, context switching, and death-march deadline pressure. In its originally-intended form it's not an arbitrary collection of hand-holding niceties, it's a self-defence measure.
That would be a really interesting HN feature. You can upvote once, and it's free. But if you really feel strongly, you can upvote more, but each additional upvote costs you a karma point.
I feel like the core problem here is one that can never be resolved, because it's the result of two valid but opposing forces.
On one hand you have business stakeholders, who can see the business reality - money doesn't grow on trees and runways are very real and scary, so without deadlines companies fundamentally cannot function.
On the other hand you have engineers, who either explicitly or implicitly understand the other truth - that reliably estimating a project requires knowing all the steps involved, and if you know all the steps involved, then you'd have already automated it.
When the problem can never be resolved, you redefine the problem until you find a version of the problem with a good solution. So if the problem is that software is hard to estimate, which is hard, we redefine the problem to something like “I want to be able to ship something people want to buy, before we run out of money” or “I want to ship a game in time for Christmas” or “I want to reduce operating costs so we don’t drown in debt.”
Incidentally, the process of writing a software program is often the same process—redefining the problem until you find a problem with a good software solution, rather than finding a hard problem and throwing software at it until you die from exhaustion.
I think a lot about the effect that ad-driven business models and open source must have had on the propensity to spend money on software products. Both in terms of pricing and in terms of build vs. buy. Mostly because I’m always musing about software developers’ hard earned reputation as the cheapest customer demographic on Earth.
I think about it too, sometimes. You know all the jokes/anecdotes about Microsoft and Bill Gates specifically calling Linux users and open source devs "communists" and accusing them of destroying the software industry? To the extent they actually said this, they were absolutely right - open source culture is very much responsible for destroying the viability of a simple and honest "pay money in exchange for value" business model in software. This of course didn't make software free - the market switched to monetizing it with worse, end-user-abusing models, like SaaS subscriptions and "free, but with ads".
FWIW, I'm still enjoying, using and supplying open source, especially that as a power user, I can get a better deal on the whole thing. However, I think we need to recognize the unintended consequences this has, and that in a sense, we're ourselves responsible for the shitty and abusive industry we work in and live with.
"money doesn't grow on trees and runways are very real and scary, so without deadlines companies fundamentally cannot function."
Runways being real doesn't mean you need deadlines. Deadlines are useful to motivate the unmotivated.
The better path to dealing with shrinking runways is early and continuous delivery of valuable software. This is facilitated by making sure the team is aligned and onboard with the solution (for motivation) then giving them the environment, support, and trust to get the job done.
This is further enhanced by self-organized teams since the best architectures, requirements, and designs emerge from self-organizing teams.
With that emergence, the team will deliver valuable and working software at regular intervals (measured by weeks or months at most, preferring the short timescale) and the stakeholders will determine when that valuable software is valuable enough for customers to pay for.
This early and continuous delivery is the primary measure of progress. It also facilitates a welcoming of changes to the requirements at any stage (as the focus is delivering value, not getting a roadmap done).
Roadmaps and deadlines will often get in the way of this system of efficiency in software delivery precisely because the estimation game gives team permission to use all the time negotiated in the estimation, which is already padded enough to ensure an "under promise and over delivery."
Interesting, though not sure I follow. My reasoning is that the ultimate deadline is the “we run out of money and go out of business” point, from which all other points follow. So a project may be organized differently, but under the hood it’s all the same - get more money before you lose all the money.
Say you have a half-sized team in order for the remaining people to get twice as much time. People working on the project will, on average, be more familiar with the problem. That's not a frequent answer, since most companies are more time-sensitive than budget-sensitive. But that's a good illustration of how budget and timeline are separate things.
That being said, my initial message wasn't about that, but about the idea that plenty of organizational methods abandon tight deadlines in favor of early feedback.
That usually works better when power is decentralized, since the higher you report to, the less people have time to actually look at what you produce (and therefore, the more they rely on low-involvement management methods such as setting deadlines). On the other hand, the more decision power is decentralized to the people actually using the software, the more early feedback will make setting deadlines redundant.
This is only a hard deadline if you have to wait until the end of the project before any money happens.
This is the problem with the word "project" and, more generally, with formal project management: it attempts to divide the timeline into "Before Delivery" and "After Delivery" and often it doesn't care what the world looks like other than as a bag of deliverables.
That's not the only way to do it, of course, but it's the mental trap you set for yourself by thinking about things as Projects with a capital P.
The solution is communication and having, at best, management which knows engineering and the potential hazards of planning.
If both sides agree that any estimation is a wild guess made with many assumptions, then you can go from there. Of course that means planning is difficult, but knowing that your plans need adaption is far superior to be in denial about it.
My instinct is that if you can put numbers and predictions based on real world estimations, then you are estimating simple things.
For example, you can estimate how long it will drive from A to B. Google maps does a brilliant job of this, showing the busy sections. Often I get there +/- a minute from what is estimated over a 60 minute journey.
But estimating how long someone will take to do a project, where the project by definition is something new in the context of the people doing it. If it wasn't then the code would already exist, and the estimate will be "how long to deploy that thing you already have using CI".
So in software you are always doing something new. We are building Sydney Opera Houses, not prefab housing estates.
Sorry, this made me laugh. I know there are software engineers which build clean and polished beautiful mega structures. I _aspire_ to that. Yet that is a niche market.
In reality, most devs are making unique family Vilas at best.
I wish I could ever say I feel like I contributed to an equivalent of 'Sydney Opera House'.
I might be mistaken but Sydney Opera House is brought here (as it often is) in context of project management as a heavily underestimated project (took 15 years to build while planned for 4, cost ~15x of original estimate) - not in the context how detailed and greatly architected it is.
There are deep analysis from the project management perspective but one point was that initial estimates were based on historical assumptions while some elements were novel and couldn’t be estimated with those techniques.
With software it’s similar. We have some general feel how things should work but often there’s “this” novel part that no one did before and it’s wishful thinking, leading to budget and timeline spillover.
If you ever had to guess-estimate something you never saw and it turned out to be overly optimistic - that’s similar case to Sydney Opera House (which I recommend to read about as there are many analyses of the case from various project-related points)
It also leaves out the political machinations that a conservative government was elected after the project had begun under a centre-left government and they continually interfered in the development until the original architect left the project in disgust.
After which the internals were completed by a set of "government architects" that did as about a good a job as you'd expect.
Not to mention that the original design had its own issues, including not actually being able to support proper opera theatrical performances due to the size and structure of the internals.
I am reminded of software like the “Snake” malware. Not exactly “ostentasteful” like the Sydney Opera House but grand and intricate with a specific, kinda-but-not-exactly unique purpose.
(I’m sure there are other examples but my mind has an information security bias and I learned about “Snake” recently.)
Not the bespoke CMS I work on for Bob’s Bait Shop.
Most of us our building prefab housing, but the challenge is there's infinite ways to put the modules together, and unlike real housing no physics to reduce the problem set of possbilities.
everyone works on stuff that's more complicated than it looks from the outside, and many other fields are able to estimate the amount of work it takes to do something. why cant software?
example: in 2006 china formalized plans to build a totally new high speed train system that spans the entire country by 2020, a totally new level of scale that had never been done before. and the plan worked out even a bit ahead of schedule. how did that planning happen? dont tell me that was simpler than your crud app
Funny fact. When I was a relatively new programmer, we used to be told we had to become better at estimating, like architects are on construction projects.
Then it became common knowledge that architects are as bad as we are at estimating costs. And people no longer use that comparison.
The fields that are actually able to estimate work well tend to have one big advantage. They are estimating how to do standardized tasks that are similar to ones that they have done many times before. Ones that don't have that advantage, are as bad as we are.
A train system is embarrassingly parallelized. It’s the equivalent of saying, we want to build 50 data centers in 3 years. It’s a lot, but it’s a lot of independent things that connect together at the end.
A falacy here is there are commercial just-crud apps. There are not. If it is just crud then just rails scaffold it, and estimate day 1 day.
We could be better and more like the subset of real infrastructure projects that are on time and budget. There is probably a subset of software that is too. You need to a great waterfall. Probably allow prototypes that can be thrown away and have a decent budget for the design phase that will shore up the design and therefore the estimate. And have developers on one project.
Hire project managers that are on their 15th ERP build and making a similar one for you. And use last decades tech so there are less uncertainties (probably not a bad idea for many things - gimme that winforms app without all the slower SPA spinners!)
Even with all this what software lacks is classic Engineering (long degree, good concepts taught, respectable education over leetcode practice and so on). Problem is California will piss all over your business with move fast attitude unless you are boring enough to be off their radar.
This is exactly my problem with software dev. There are no standards, no expectations, hardly any regulation. Employers grab fresh graduates and throw them right in the deep end and hardly anyone looks at their work or teaches them anything.
In Norway if you want to be a carpenter or plumber or electrician you have to be an apprentice for two years after your education and then do a test to prove you've learned to do your work properly.
Meanwhile in our field nobody even agrees what properly is, most of us are just here for the money and are planning to job hop in a year so they don't give a crap. Docs? Nah. Unit testing? Nah. Clean code? Nah. That only matters if you give a shit.
Government and regulatory issues apply to engineering whether it's software engineering, building engineering, or train engineering.
(In software, intra company estimates can easily be blocked with political "do it with only the current existing infrastructure or conventions" or my favorite, "now prove it in a test environment, but I'll give you 0 resources for a test environment", and government / business related - ie the cost of security breach, GDPR compliance, payment compliance, Twitter,Google Maps, or reddit making api access and pricing changes)
We can almost always say "it can't possibly take less than X". In fact, that X is often what gets given as the estimate, which is... not good, when you think about it.
We can also often say "it is very unlikely to take more than Y". Between X and Y is the range of possible values for a real estimate.
The problem comes when X and Y aren't just far apart, they're orders of magnitude apart. If you go back to your stakeholders with that range, what you should be able to hope for is a reasoned discussion not about how Y is too big (which it almost certainly will be) but on how to reduce the size of the range.
That involves having more information about either the problem or the solution, so you (or they) are either off into more analysis, or you're into solution design. When you have a design in terms of subparts similar to things that have been built before, you've got something that's less Sydney Opera House and more Sydney Opera House LEGO Creator Expert Set 10234.
So yes, at the specific point in time where someone asks you "how long to do this thing that's never been done before" there's almost certainly not enough information to give a response that bears relation to reality, but that should be the start of the conversation, not the end.
Yes but only in waterfall do you need to accept that issue. In small “a” agile you can now go back to the business and suggest alternatives. (The Opera House was not built exactly to the original plans, for example). Of course that doesn’t work if you need to follow OKR
dogma etc. and stick to the original plan.
Which plans? There weren't any, just rough sketches, plus the requirements were constantly changing. They also started construction before the planning was finished.
Most commercial software projects that I’ve worked on start with a horrific RFP process and legacy integrations that make otherwise simple projects into bespoke nightmares.
The whole process of commercial software procurement and project assurance is a mess.
And then you get consulting firms and body shops involved…
Also, to get better estimates you need to sometimes do some [research] work. Without it ykur guesses are even wilder but with it you have to ack that you've spent time on it, evenif you never get to do the actual work.
Sigh, this was the basis of "Iterative Development" that came along with the "Rational Unified Process" and was the precursor to the current fad of epics/stories/sprints/t-shirts/Fibonacci that has polluted current programming.
People won't follow it because it means reviewing their assumptions and re-planning, but there are always external entities that will not or can not do so (other departments, finance, customers, marketing, sales, etc etc).
As with many development methodologies, it probably has a formal version that I'm not understanding.
But in my head, it's always been (1) identify areas of risk, (2) sprint to a prototype that decreases or quantifies that risk, (3) evaluate whether you've reduced risk sufficiently, then either (4a) proceed to the next risk or (4b) do another prototype sprint on this risk, and finally after all pockets of risk have been reduced (5) begin building your finished product.
Believe it comes from state of the art development for military procurement, where "What is even possible?" is a valid question.
But it always resonated with software planning for me, by making it more about risk reduction rather than guessing.
Estimation was mostly guesswork. There’s a theory that work expands to fill the time you give it and I found that to be largely true.
Other agencies would quote 3 months for projects we’d complete in 6 weeks. Even then I was taking engineers 3 week estimates and doubling them for buffer.
We didn’t do anything special other than commit individuals to a single project, or two at most.
We would of course add additional resources if necessary to hit deadlines and then adjust future estimates as needed.
After a few years estimation felt more instinctual than scientific.
Similar point. I’ve been running an agency for 14 years. All that time we’ve tracked hours for every project whether T&M or fixed bid.
Now when a project comes in for bid I always estimate it based on experience but then validate it against one of 250 projects we’ve done that seems “similar” in scope. Since doing that we’ve only been off significantly when we misunderstood the original scope.
Tracking hours is a pain in the ass and we do it mostly in Jira tied to tasks but the data we get is invaluable.
> Tracking hours is a pain in the ass and we do it mostly in Jira tied to tasks but the data we get is invaluable.
The important point here is to actually feed that information back to the people that originally did the estimates and also to the people that are (to them) "pointlessly" clicking on the "start work", "stop work" buttons on their JIRA tickets.
Most of the time, this sort of timesheet stuff is fed to an accountant somewhere that turns it into magical "earned value" based on inaccurate project schedules that are always 99% complete but with a vertical S curve.
Yeah we mostly don’t do the start/stop thing with timers. For the most part we just have folks guesstimate the hours spent when the close or reassign a ticket. Hour or half hour precision is good enough.
We don’t hide hours and use them as a tool for raising flags as part of our weekly or bi-weekly reviews with team leads and part of our client reporting. If a lead sees someone spending too much time on a task it raises a flag on whether the task is unexpectedly large, impacting schedule and/or scope or if someone is potentially slacking off or having some other personal issues etc.
We are primarily remote and, like most agencies, will backfill sometimes with contractors. We’ve been hosed a few times with people sandbagging hours so making all this data public and reviewed has been a great way to hold everyone accountable.
I suspect that experience is a major factor as well. You somehow knew to double those estimates. At the agencies I've worked at over the years, less experienced individuals seemed to consistently underestimate the time it would take to do things and then complete those tasks haphazardly/late/bugged. I went freelance and haven't had any trouble providing accurate estimations.
I would guess a repeatable/known type of service or project helps a lot as well. Most of my engagements are relatively different from each other, and I'm doing different things all the time. I've worked places which were offering a narrower set of functionality, with limited customizations, and that's generally much easier to estimate. Also much easier to have estimates in which you've said 'no' to things altogether, vs spending time researching/testing/trialing stuff that's new to you.
To the point of providing flat rate pricing at my job. We realized spending an hour or two getting a sign off on an SOW or spending four hours on a discovery call didn't change the time it took to complete in 90% of cases. I can just price my stuff all the same because for the most part, EDI mapping is EDI mapping. If we run into something weird. Yeah. But thats once or twice per quarter.
Meanwhile our ecommerce integration product pricing is all over the place, because there is just so much variation. I am hoping to move that to base pricing for the product + time and materials for consulting.
In a sense my EDI team is busier, less stressed, but less gratified. The web team is always getting a curve ball and its certainly a younger man's game. Or at least someone who's willing to roll with the punches, but the payoff is sweet
Well for me it certainly helps that I usually have clear requirements up-front. Designers for instance take a lot of the guesswork out of what I need to do for website work. For other things, like debugging legacy code, I just tell clients that I need time to diagnose the issue before I can give an estimate.
At least when I was younger I was so perplexed by the kind of co-workers that have apparently chosen the profession of working with estimates from other people but continually work without a "confidence level" associated with each estimate.
("two weeks") means such a different thing in reality from ("two weeks", "confidence of 50%").
It's a tuple, it's always been a tuple, it's just downright silly to accept an estimate without the second half of the information. Like what diviniation is going on in that spreadsheet to communicate to others when it doesn't include half the meaningful information to begin with.
In this case why not just divide the estimate by the confidence to get the "true estimate" e.g in your example it's actually a 4 week. If you provide a tuple the confidence will eventually be conveniently dropped in some slide deck then the "2 weeks" will be socialised as the commitment.
The estimate-confidence relation is not linear. In my experience it follows some kind of S-curve, with 0% confidence approaching impossibly short times and 100% approaching infinity (like a logistic function).
I think software projects are best estimated as gamma distributions. To express that as confidence levels, you use the cumulative distribution function of the gamma, which are kind of S shaped, but stretched to the right.
You might imagine I disagree having written that flippant comment last night. On the contrary I've been on both sides at length over my career. While I find the tuple information somewhat useful (if I'm forced to provide semi-informed-guesses around timelines), I ultimately deep down agree with you wholeheartedly. The utility is moreso in having the conversation about why the confidence is low, which many methodologies have their own words and processes for.
This is great, thank you. Another approach is to do forecasting based on historical throughput (eg tasks/day). Generally using "story points" or other guesstimates turn out to be redundant if you have actual data (actual project by mostly the same team).
It takes into account stuff like n% of issues "blowing up" into subtasks, and some issues taking longer.
I read a great article on the subject which I've unfortunately misplaced - but the general gist is touched on in these:
"Forecasting software project’s completion date through Monte Carlo Simulation"
In general I'd like to be able to just point a tool at N "completed" GitHub projects, and a fresh one - and get a forecast that adjusts as issues are completed and added (possibly filtering on issue tags in completed and new project) - but I'm not aware of anything out of the box for that.
Thanks for mentioning Screenful. Just wanted to comment on your remark:
"In general I'd like to be able to just point a tool at N "completed" GitHub projects, and a fresh one - and get a forecast that adjusts as issues are completed and added (possibly filtering on issue tags in completed and new project)"
That's exactly how our Forecasting chart works. You can import any number of repositories and projects (classic or new) and get a forecast for the remaining work. You can learn more about that chart from this guide: https://screenful.com/how-to/how-to-read-the-forecasting-cha...
Thank you for that link, you sum up the idea that sometimes counting total tasks works fine quite well:
> Will this work if my tasks aren’t all the same size?
> The short answer is yes, your work items don’t have to be the same size for this approach to work. The chances are that If you take 100 tasks in the past and compare those to a sample of 100 tasks in your current backlog, they will be the same size on average. Also, they are likely to be the same size on average regardless of how you measure (e.g. in story points or by some other estimate) or simply by task count.
The datasets are not even data in the first place.
If you want to understand estimations and projects, you have to study them. This is a social science project. It's not a math game. It's not a problem that you can solve by collecting oversimplified numbers from dubious sources based on incoherent definitions of the units of measurement.
If somebody reports that they spent x days developing a component, what does that mean? What were they doing during those days? How do we know what they were doing? We don't know unless a researcher was right there in the room taking notes in real-time. There are so many confounding variables involved, even if your data collection is perfect, that analysis of the data is a process fraught with opportunities for accidental overfitting or researcher bias.
The one bit of project estimation research that I actually believed was that project estimates are closely affected by timeline desires of major finders/"stakeholders".
Anecdata:
I once asked a VP of a major telecom why the project deadline was Sept 30 and not Oct 1 or Sept 28, and he replied "because my quarterly bonus depends on it"
Interesting that most of the top comments here seem to be about the day to day process of estimation itself, when the linked article is about how poorly the research on estimation has been done to date. Did any of you actually read the article?
Glad I am not the only one who noticed this, thought I was reading the wrong thread. Software engineering research is something I was interested in a long time ago, so hoped for a better, more on-topic discussion.
A bad estimate is better than no estimates at all. There are plenty of empirical studies that prove that. Some computer scientists have a hard time dealing with non mathematical proofs. They have a huge blind spot for things like empirical case studies, qualitative studies, etc. Those are tools that people in other fields use when mathematical models fall short. There are not a lot of useful mathematical models you can use to model software development. So, understanding that software development is an inherently social process kind of points you in the direction of where to look for alternatives to that.
Humans are bad at estimating stuff. Software engineers doubly so. But even a bad estimate gives you some handle on how easy or hard something is to do. And you can break things down to improve the quality of the estimates. And then the rest is just using statistics to model the margins of error. Economists are very good at working with this sort of stuff. I always recommend people look at Don Reinertsen's work on lean development. He's using a lot of economical reasoning and notions such as cost of delay, amount of work in progress, and indeed cost estimates. Those being bad is less of an issue if you know they are only off by 2x vs. 10x. A 2x margin of error gives you some handle on the situation. A 10x uncertainty margin means you need to use some tricks to improve the quality. There are ways to do that. 2x is not great but is better than ignoring estimates.
> But even a bad estimate gives you some handle on how easy or hard something is to do
Can’t agree with this at all. A bad estimate is by definition bad. How does it give you a handle on anything if it’s unreliable? It’s like saying bad medicine is better than no medicine.
And when you talk about breaking an estimating problem into smaller estimates, you’re now firmly in the domain of actually doing the work. An economist can’t take a software problem and decompose it into tractable solutions. Even if they could - good luck getting the customer to pay for that.
That’s why I say estimating is an economic problem. The cost of accurate estimates asymptotically approaches the cost of the implementation. Nobody is willing to pay for accurate estimates, and anyway doing this doesn’t make sense.
Unfortunately there are very few levers to pull in terms of software planning. Time based project management is way over emphasised. This is why approaches like lean (MVP, CD) and agile exist in the first place.
That's the problem. You have an opinion but no empirical evidence and you think your opinion is pretty good. There's plenty of material out there that is a bit more well reasoned that disagrees with your opinion. For example Don Reinertsen's books.
Most Agile estimates are pretty bad. But that's OK because it's still better than just winging it. One of the realities of working with real companies is that they have things like dead lines. Programmers don't like them. The don't believe in them. They are bad at delivering stuff on them. But they still exist. The reason agile/lean/scrum/etc. exist is to beat some sense into programmers completely and utterly failing to deliver anything on a predictable schedule. Bad estimates are the foundation of these processes. Where bad is better than no estimates and the rest of the process is about mitigating the consequences of the estimates being low quality.
> That's the problem. You have an opinion but no empirical evidence and you think your opinion is pretty good. There's plenty of material out there that is a bit more well reasoned that disagrees with your opinion. For example Don Reinertsen's books.
Wait - there is a large body of work that is better reasoned than a quick post dashed I off in haste before dinner?
> One of the realities of working with real companies is that they have things like dead lines
Using condescending terms like "real companies" is unnecessary. You know nothing at all about me or how I've come to my conclusions; you just make arrogant assumptions that you know better about this than I do. I'm open to being wrong and learning new things, but do you really think you have the Silver Bullet?
My experience over a large number of projects suggests that the whole premise that most commercial software project problems can be solved using technical, mathematical or scientific means is wrong. My opinion - which aligns with the results of large-scale studies such as the Standish Group CHAOS reports, the PMI Pulse reports and more - is that the problems with commercial software projects appear to be almost entirely social and organisational. Projects are too ambitious, too complex, and are executed poorly by inexperienced people. No amount of time spent on estimation is going to fix that.
I am unfamiliar with Reinertsen's work, but I think it's quite interesting that economics is not exactly a science either.
A bad estimate gives you a handle on how easy or hard your engineers think that something is. Experience shows that this has little correlation with how easy or hard something really is.
That's a good example of a qualitative statement that would be a lot more credible with a lot of empirical evidence backing that up. "Experience shows" is not enough here. Raises more questions than that it solves anything here really.
Basically uncertainty is the key concept here. And cost, complexity, and time estimates are ways to get a grip on such uncertainty. Unlike software engineers, economists and sociologists are used to reasoning about and modeling uncertain situations. Going all hand wavy and winging it is typically not what they advertise.
And I'd argue that their bad estimate includes their knowledge of how difficult the task is. Unless you're completely broken, you're not going to under-estimate something wildly if you are unsure of what you need to do or have no idea how to do it.
Sure, I can give estimates that don’t underestimate and account for eventualities. It’s just that my error bars are large enough that in that case I have to estimate several months for tasks that I believe should take a week.
Yea, effort estimation is a waste of time much like building a piece by piece project plan of anything.
The key is workflows, like how buildings used to be built. You just keep working on it - no stupid schedule that fails immediately once one little detail to follow goes wrong. Building accurate plans never works and it takes ridiculous amounts of time.
I've started to use the Log-Normal distribution for estimation. Because I think of development like planning a travel time, the train will either arrive in time or the will be a random amount of interruptions/errors that add up. But there won't be as much stuff that speeds the train up (wins back some time).
Then I try to communicate the distribution in words or show it, which most often has a hard time to be accepted, because managers expect a normal distribution mean.
I let several people estimate large scale features and keep their variance to feed it into the log normal. Often I use a reference project, let the use smaller numbers (but not Fibonacci numbers [1]) to estimate and calculate it back to work days.
[1] I personally think the Planning Poker numbers are a lay man's way to force a Log-Normal distribution in the estimation
> Agile techniques are widely used these days, and datasets from the 1990s are not applicable. What datasets do researchers use to build Agile effort estimation models?
In an undergraduate software engineering course I took we learned an effort estimation methodology that assumed a particular way of working (similar to Agile), but also a particular software architecture. The problem was that students then had to adopt such an architecture, even though it wasn't the best fit to the problem at hand, just so that they could complete the effort estimate. Are there methods that are independent of the software architecture?
I once developed an „AI“ trying to predict when JIRA tickets in our startup get resolved. It was mainly a system based on k-nearest neighbor regression, predicting the closing date based on the average of k similar tickets (we had more than 1000 closed tickets already). It worked reasonable well. One of the most important features was the person creating the ticket. Some people wrote tickets really well and the tickets often got resolved in 48h. Others just wrote one cryptic sentence and nothing happened with these tickets for weeks.
But you need a lot of high quality data for that. Since that time I realized that writing wellformed tickets and also changing their state on time is really important for such tools. Modern tools actually don‘t use KNN, but some of them use Monte-Carlo-Simulation to estimate work:
Reminds me of my early days as a rookie people manager. I was asked to estimate a project for a team that I had no idea about their abilities. I was very hesitant to estimate since I had no data to begin with. My manager asked me to throw a number and see if there is a push back!!!
I soon made a decision not to be a manager responsible for guess work based estimation and moved back to an IC role where I estimate effort for work assigned to me.
I remember when I was a kid, being on a family vacation, driving up I-15 in Idaho. At one particular place we topped over a hill, and the road was straight from there to an overpass across the road way down in the distance. I wondered how my dad could aim so well that he'd go under that overpass all the way down there.
Now I know that he didn't aim that carefully. Instead, he kept steering.
Estimates are estimates. Sure, try to get them approximately right. (By the way, you can't get better if you don't close the loop. Look at estimates vs. reality on your own estimates in the past.) But even good estimates will only be somewhat close to right. You need management that will steer all along the road from estimate to reality. You also have to give them the feedback information so that they can steer.
Your estimates shouldn't be off by a factor of 10. "Off by a factor of two" isn't great, but it's not horrible - though they shouldn't always be off by a factor of two in the same direction. If management understands that that's what they're getting, and they know how to work with that, fine. If they think they're getting much more precision than that, that's a management problem.
(This is an oversimplification. For some kinds of work, you or your team or company has done that kind of work many times before, and you can easily be more accurate than a factor of two. Other times you need a research project just to identify the parts involved, then another research project to discover whether you can solve those parts, then yet another research project to be able to estimate how long those parts will take. Then you can give an estimate, and it still won't be a very precise one.)
I got better at software project estimation the longer I spent in the dev industry. But I still knew I was using back-of-brain knowledge and probably wouldn't benefit from bringing too much of it into my consciousness. I think it's like kicking a goal from a free kick in soccer. Conscious deliberation seems to be a distraction at best.
To estimate how much time something takes on average you need to repeat and measure it many times.
For instance if you had a clear unambiguous spec of what the the software must do, you could give it to 10 different teams and measure how long each took.
If you then modify the spec even slightly you could make it impossible to create such a program no matter how long you work on it.
But every project has its own different spec. We don't know how long it will take until we try to do it at least once.
Researches might study how long it takes on average to implement a given specific spec by different teams. But they can not estimate how much it would take to implement any arbitrary spec because - nobody has every tried to implement that spec before. And studying that tells you very little about how much time and effort is needed to implement any other arbitrary spec.
Thats true. For system complex enough you will spend more time learning system than modifying it. Thats why all developers love to write new code and mostly hate to suport existing, working, legacy code.
> If you have a month to plan a project in detail, spend it on four one-week iterations developing while you improve your estimates. If you have a week to plan a project, hold five one-day iterations. Feedback cycles give you information and the experience to make accurate estimates.
- Kent Beck, Extreme Programming Explained: Embrace Change (The XP Series)
Estimates are always portrayed as 'business reality' vs. 'software engineering preferences' - in my experience using estimates many times actually impedes businesses from measuring other metrics that are just as valuable or (imo) significantly more valuable than estimates. Those are a) 'full' lead-times for features, the coding bit is only a small bit in the whole process and estimating just this bit doesn't really tell the business much about how much effort, time and money it really takes to release a feature and b) value-created for the user by a new feature. Those are harder to do, but usually worth it
What mostly worked for me was two simple things: splitting the big tasks and constantly evaluating evaluations. With more smaller evaluations negative and positive errors anihilate each other. With adjusting the evaluations you can reach the a point where you don't have errors on only one side.
Why do software people think they're special? Like it's somehow expected that we can predict everything because we're computer geniuses? We would have accurate estimates if only we "scienced" better?
If you want to get very good at estimation, do it for a very long time. Then you will see how your estimates would be very accurate if all the other things in the world didn't happen. Just let go of the idea that you will ever have control over this aspect of life.
One of the big reasons that software estimation doesn't work well is because due to the virtual nature of software it is way too easy to move the goal post in subtle or not subtle ways.
In "real world" projects such as civil engineering everything is a lot more stable. You don't start with the intention of building a bridge and decide to turn it into an airport halfway through the project.
In software projects one way or another people do frequently keep changing things so the parameters are always changing from under your feet.
In the "real world," estimates are also terrible. Even for something as simple and well-defined as repaving an airport runway, research shows the actual cost exceeds estimated cost by a median factor of 1.25.https://ocw.mit.edu/courses/ids-333-risk-and-decision-analys...
Have you ever heard anyone ever say their kitchen renovation project came in under budget? Humans are just doomed to never be good at estimating, in any context, outside the simplest of tasks.
Software has unique challenges that make it inherently difficult, but the bigger problem is all the incessant micromanagement, tooling like Jira that enables management's worst behaviors, and lack of any union representation or solidarity among developers. There's an infestation of libertarians that think they're better than everyone else, that think they can meet these impossible standards. They're too naive to realize that if they did make a miracle happen, management would just move the goalposts again and again, until you burn out, your wife leaves you, and you decide to start farming goats or whatever. We all need to refuse to play no-win games collectively. No estimates is the only answer. They can't fire us all.
Collective action is probably a pipe dream, though. With that in mind, I think half the issues with this career would go away overnight if a Russian ransomware gang erased Jira and Atlassian out of existence. Those grifters are selling the corporate equivalent of fentanyl laced with benzos to middle managers. Jira is poison.
I can give you some kind of reasonable estimate for precisely one kind of work: stuff for which I have good domain knowledge, unfettered access to a small set of stakeholders who are in unambiguous agreement on the need, that is delivered on a technology platform that I know well, using people with whom I've worked in the past, and that has no dependencies on other teams.
This has happened approximately three times in my career.
Think about it for five minutes. Whatever number you come up with, multiply by 5. Estimate done. The only problem is that management won’t want to listen.
If your manager says "I need an estimate" and you can't tell your manager "no, that's fake research/not worth my time", does it really matter what is/isn't "fake research"?
> Effort estimation is an important component of any project, software or otherwise.
The most “important component”, what does that even mean? Do they mean task? Isn’t the most important thing to deliver results? Which could mean working software or aborting bad projects
I might be in a 1% of engineers that believe in estimation.
Can you make a step plan for some deliverable? No? Then, sorry, it's not that planning is not possible, it is that you are simply not experienced enough. That's fine. There is something called skills, and you can go get them.
Have you made a plan? Great, then is there is some task in your plan that you can't estimate? In rare cases that might be inherently true, but in the majority of cases if you are senior enough any given task you're asked to do at Company A is going to be something similar to some task you've done in your prior experience and you can collect personal data on that.
A plan with per task estimates and you have an estimate for a deliverable. The next piece, and this is crucial, is you aim at and execute rigorously to the plan -- prioritizing as you go. If you do any "padding" for insurance, you DO NOT add arbitrary constants or double etc to your estimates. You add extra scope, ie tasks delivering real value, to your plan, but scope that can be cut, if needed, as part of your rigorous execution, en route, if you find in iterations that you're slipping.
I tend to disagree. Estimate this more complex situation: Integrate vendor B’s API where you don’t have prior experience with vendor B’s product let alone their API, and where our system is built specifically around vendor A’s API and workflow. This requires backend and frontend (customer and admin UI) changes given that everything is tied to vendor A’s API currently.
Have you ever done this before? How long has it taken in the past?
What are the key first steps? Investigate the API and product right? How long has that taken when you've done it before? Your first deliverable is a plan of it's that unknown.
Now you've investigated you know more about what needs to be done. You can compare it to previous times you've done similar things and estimate how long it should take.
Your basic approach is sound. Problem is the existence of unknown unknowns, those that only appear when you reach a later stage where these unknown unknowns decide to present themselves. Often causing an unknown amount of unknown effort requires from an unknown team member or third party. This invalidates earlier timeframes, and in worst case even the entire strategy.
How does one put a proper estimate on this? Any answer is but a wild guess.
(This is where PMs provide added value by realigning business and product teams around the new situation.)
The more experience you have the more you're used to these things happening. I don't have to know that a specific thing might go wrong but I'm used to there being some random bullshit (weird CI, library happens to be deprecated, version inconsistencies, data turns out to have a fatal flaw, licensing, whatever).
You can't plan for everything though, and you're right this is where a good PM helps.
But then again this is all just estimates. The better you can estimate the better you can plan.
> Can you make a step plan for some deliverable? No? Then, sorry, it's not that planning is not possible, it is that you are simply not experienced enough.
Perhaps you work in a simple domain? This would be infeasible at my current job, and that's including the best-in-the-industry people with 20 years experience.
What’s your domain? Are you working on a mission critical system? Are you trying to solve an algorithmic problem that is novel enough to warrant a publication after you solve it? If not, I doubt estimating is innately infeasible. If your best-in-industry people can’t do it it’s because they probably haven’t valued it enough in their career to develop the particular skill set.
I have some experience with healthcare software and the core of it is pretty simple. The 2 biggest sources of unknowns however are quirks of the existing system and edge cases in human processes. Neither can be accurately estimated for because complex systems are never documented in enough detail and even humans who have been using them for decades overlook and misremember things.
I’m sure this is the case in any complex enough business system as well.
The second time you are doing exactly the same thing you will be a lot faster, and the third time you will write a script to automate it. If others are doing the same thing repeatedly across the industry, automated tools will appear.
It follows that the majority of a developers time (at any level of experience) will be spent doing things they haven't done before.
the map becomes the territory here. if you have enough information to accurately estimate a project then you've already done the bulk of the work, and all you have left to do is type. The reason estimates are hard is because the work surprises you. You find out there are problems you didn't expect that must be solved to meet the stated goals. This happens all the time, it always happens on anything non trivial, which is why you have to add time to estimates beyond your initial assumptions. What experience teaches you is that this is going to happen, and it even gives you an inkling of the sorts of ways it might happen, and even better it makes it easier for you to resolve these issues when they do happen because they tend to be similar to other problems you've encountered before, but what it can't tell you is which problems you'll run into, becuase you won't run into them, until you do.
> if you have enough information to accurately estimate a project then you've already done the bulk of the work
This is false. I’ve learnt 10s of new UX frameworks in my career and it takes about 20 hours from scratch to become proficient enough in one to design a new complex widget in it (say) — but knowing that doesn’t suddenly absolve me from spending those 20 hours.
Go look at the SysMLv2 spec. I want you to implement me a parser for it in Python. The resulting AST must respect the relations between fields defined in the standard. How long will it take you? It's just implementing what's written in a pdf, shouldn't be hard to break down and estimate. I'll wait.
Take your first guess, double it, double it again if the stakeholder is a poser, add 20% per developer less experience than you, subtract 10% for the features you're going to essentially copy paste, add 15% for sick leave (browsing HN) and then double it for every question you have that are unresolved and divide it by the room temperature multiplied by the amount of people with mechanical keyboards.
That gives you roughly the right estimate for any job, until the next sprint.