> Figuring out how to do the feature is usually part of the 'planning' and 'grooming' activities which are 'invisible work' with timeboxed team meeting slots allocated to it, even though a well-thought out implementation can save tons of time.
Those time-boxed meetings aren't where you come up with the well-thought out implementation plan. They're where the work is prioritized, the well-thought out implementation plan is communicated with the team, and the work size is estimated.
> Just plunging in and doing it means that a large uncertainty exists as to how much effort said feature wil take. This can either lead to time overruns or padded estimates (or both)
At the start of a project where you're going something different from what you have done before (and with software, if you're not doing something different... why aren't you just using the software you've already written to do the job? ;-), and needs/circumstances change faster than you can complete projects anyway, so unless you can predict those changes (in which case, you have the wrong job ;-), even if the task was without any novelty, there is intrinsically a lot of uncertainty.
Recognizing that the uncertainty is intrinsic, the agile approach to manage it is to break down the work into lots of smaller chunks with well defined checkpoints. This allows for many opportunities for course corrections in response to changing needs/circumstances and/or learnings about the size of the work. It's basically just a standard pid controller dealing with a noisy system.
You're right though: another approach is to spend a great deal more time up front making a detailed estimate of the work... although that doesn't control for uncertainty about changing needs/circumstances, which will often change during the time you make your detailed estimate. ;-) There's also the matter then of estimating how much time it will take to estimate the work... since until that part is complete you would have increasing uncertainty. Now you can estimate how much effort the estimate will take to complete. That's a smaller task that will take less time to complete, so there's less uncertainty about changing needs/circumstances while you're coming up with an estimate of effort for the estimate. Also, if you estimate how much effort it will take for you to estimate the effort to make the estimate. Once you observe how far off the mark you are estimating how much effort it will take for you to estimate the effort to make the estimate, you'll have more information to calibrate your estimates with, so you can use that to have a more accurate estimate of the effort to make the estimate...
Those time-boxed meetings aren't where you come up with the well-thought out implementation plan. They're where the work is prioritized, the well-thought out implementation plan is communicated with the team, and the work size is estimated.
> Just plunging in and doing it means that a large uncertainty exists as to how much effort said feature wil take. This can either lead to time overruns or padded estimates (or both)
At the start of a project where you're going something different from what you have done before (and with software, if you're not doing something different... why aren't you just using the software you've already written to do the job? ;-), and needs/circumstances change faster than you can complete projects anyway, so unless you can predict those changes (in which case, you have the wrong job ;-), even if the task was without any novelty, there is intrinsically a lot of uncertainty.
Recognizing that the uncertainty is intrinsic, the agile approach to manage it is to break down the work into lots of smaller chunks with well defined checkpoints. This allows for many opportunities for course corrections in response to changing needs/circumstances and/or learnings about the size of the work. It's basically just a standard pid controller dealing with a noisy system.
You're right though: another approach is to spend a great deal more time up front making a detailed estimate of the work... although that doesn't control for uncertainty about changing needs/circumstances, which will often change during the time you make your detailed estimate. ;-) There's also the matter then of estimating how much time it will take to estimate the work... since until that part is complete you would have increasing uncertainty. Now you can estimate how much effort the estimate will take to complete. That's a smaller task that will take less time to complete, so there's less uncertainty about changing needs/circumstances while you're coming up with an estimate of effort for the estimate. Also, if you estimate how much effort it will take for you to estimate the effort to make the estimate. Once you observe how far off the mark you are estimating how much effort it will take for you to estimate the effort to make the estimate, you'll have more information to calibrate your estimates with, so you can use that to have a more accurate estimate of the effort to make the estimate...
Yup, it kind of ends up as the same thing.