I get the impression that the process and practices of estimation in software are coming under fire at the moment. There are some people who claim that estimating is simply a form of waste; I first heard this from Karl Scotland who has blogged about his thinking and who I hope will comment further on this subject. This seems to appeal to developers who find estimation difficult and teams that struggle with accuracy of estimation. Certainly there is a view that, because estimation is difficult, inaccurate, can lead to false expectations and dashed hopes that it just shouldn’t be done (echo’s of Kent Beck’s “Doctor, Doctor it hurts when I do this”, “Don’t do that then” justification for many of the XP practices – something I buy into wholeheartedly). But I think we’re in danger of losing some key information, interactions and learning in the quest for ever greater short-term efficiency and flow.
First off, let’s agree that there are some situations in which estimation really isn’t useful: when you have no deadlines but just need to deliver as quickly as possible, when you are spiking new ideas and approaches, when you’re fixing defects or making bunches of minor enhancements. At Singletrack we mainly use a limited work in progress system because we spend a lot of time working without deadlines and spiking new stuff (and, very occasionally, fixing the odd defect or two ). This works well but also highlights to me some of what we miss when we eschew estimation.
The Process of Estimation
The original XP planning game had few ‘rules’: review what was delivered last iteration, compare estimates and actuals, calculate load factor and capacity for next iteration. Next, the customer articulates what they want by telling a story and the development team ask questions. The dev team estimate and identify levels of complexity or uncertainty and the customer asks questions. The customer sets the priority and the development team makes a commitment. Job done, usually in a couple of hours. As @mhsutton tweeted:
Estimation is not about the number that pops out. It is about exploring effort and discovering that you don’t know stuff.
I’ll come back to the bit about the number in a minute but the I couldn’t agree more about the exploration of effort and discovering that you don’t know stuff. Let’s think about what’s going on in a planning game:
- Requirements are being articulated for the first time. The development team is able to ask questions of the customer and gain direct and indirect knowledge of what’s needed.
- The development team needs to think about how to implement. Working as a whole they start to think abut design, about the unknowns (both known and unknown ), about the dependencies and risks.
- The customer gets to see this and, armed with this info and the estimates can set their priorities accordingly.
- The development team get to challenge theirs and the customer’s assumptions.
- Any uncertainty or complexity is out in the open; when something goes wrong the customer is fore-warned; no need to pad estimates or add in fudge factors (if you have a clued-up customer).
- The whole team is working together towards agreeing what will be delivered
When we work in LWIP mode, I find many of these interactions are still needed but we miss having a single point where the team as a whole gathers. So sometimes the conversations are had with part of the team absent or else are stretched out over a number of chats/emails/calls. Its not clear to me that this is more efficient and I feel that the opportunity to make plans as one team and agree to them together loses some of the ‘play to win’ spirit that XP promotes.
The Outputs of Estimation
For the past 12 years I’ve run businesses where the development estimates are one of the direct inputs into pricing what we deliver to customers. From a business point of view those numbers (actually we use a simple Small/Medium/Large scheme) are business intelligence and our ability to estimate well is part of our business advantage.
But even in a project those numbers have importance. Assigning business value or priority to a requirement is often as much an inexact process as estimating is. In my experience its very common for a customer to be convinced that a requirement is high-priority until they hear what they wont get because its such a large piece of work, or to suddenly get excited about a feature that can be delivered quickly and with little impact on what else they need. They are also very useful in release planning: making high-level statements about what can be delivered in the medium and long term. I must admit that I’ve yet to work with a non-startup business that doesn’t need to have some level of confidence in what can be delivered in three, six, nine and twelve months time, regardless of their commitment to lean or agile. Things like external commitments, regulatory controls and business budgets just keep getting in the way.
Lastly I think those numbers are a metric that is made visible just like number of tests, stories delivered or whatever the teams uses to help it reflect on the project. For example: lots of Large estimates could meant that there is some detail missing in terms of how things are going to be built or what the customer has asked for. Nothing wrong with that but perhaps a chance to run a couple of spikes or get some early reviews to help tie things down a bit. Perhaps lots of Small estimates mean this iteration is ‘just work’. Again, nothing wrong with that but perhaps an opportunity to do some Gold Card type tasks to avoid developers suffering from the relentless treadmill of delivery.
The Learning from Estimation
I can’t help but feel that the progression from the original Planning Game with its use of load factor/capacity and estimates vs. actuals, to velocity, to story points, to WIP has lost something very important: a key opportunity for us, as professionals, to reflect and learn about what we do. If we estimate something as a Small and it turns out to be a Medium we get the chance to ask “what happened there?”. Maybe it was something we could have avoided, maybe it is something we wouldn’t do again, maybe it was an assumption we shouldn’t have made, or maybe just shit happened. Whatever, everyone makes mistakes and the best thing we can do is learn from them.
I was lucky enough to work with a team whose core membership was stable for a period of several years. Towards the end of that time we could estimate fairly large projects with a great deal of accuracy. We could make commitments for nine months hence without any loss of agility; when priorities changed and new requirements surfaced we could quickly and accurately reflect back to the customer what the impact would be and then incorporate whatever they wanted to change. We only got to this stage by mis-estimating many times and reflecting as a team on what had gone wrong.
In other words, estimation is hard and can sometimes be impossible. But it is a learnable skill and when you can do it and do it well it can be a great aid to agility and efficiency.