What an estimate actually is

I was asked to put a sprint estimation framework in front of a team that had been shipping for years without one. Not because the estimates were bad. Because there were none, and the work still had to be scoped and sold and sequenced, so somebody was producing numbers informally and nobody could say what the numbers meant.

Writing the thing down forced a question I had been stepping around since my accounting years, when I produced estimates of a different kind and treated them as obviously well defined. What is an estimate a statement about.

Not duration

The default answer is that it predicts how long something will take. That answer survives about a week of contact with a real team.

Watch two people give the same figure. One has opened the system, found the three places the change has to land, and noticed that one of them is shared with something else. The other has read the title of the ticket and thought about it for eleven seconds. Both say three days. Both are being honest. One of those threes will hold and one will not, and the number contains no trace of the difference.

So the number is not the claim. The claim is underneath it, and it is a claim about how much has been looked at. An estimate says: I have investigated this to a particular depth, and conditional on that depth, here is my figure. Strip the first clause and the second is uninterpretable.

Which explains something I had watched for years without understanding. When two people argue about an estimate, they almost never resolve it by arguing about the number, and they usually do resolve it the moment somebody opens the thing and looks. The disagreement was never arithmetic. One person had done more investigation than the other and the scale gave them no way to say so.

What the framework actually had to do

Once I had that, the design changed. I had started out trying to make the scale consistent, which is the obvious goal and mostly a waste of effort. Points are not comparable across people and never become so. Two engineers who both score honestly will still score differently, forever, and calibration exercises buy you a month of agreement before it drifts back.

What was worth building was the part that made the investigation visible. What did you look at. What did you not look at, and what would change if you did. Where is the thing you are assuming rather than knowing. Answer those and the estimate becomes readable by somebody who was not there, which is the only property that mattered for a team whose scoping had to survive being turned into a statement of work.

The part I still get wrong

Two things.

The first is that people optimise the investigation to feel safe rather than to answer the question. Given a framework that rewards having looked, an engineer will look for a comfortable amount of time at the part they already understand. That is not dishonesty, it is what anybody does under mild pressure, and I did not see it coming because I had designed for the honest failure and not for the ordinary one.

The second is that I expected less arguing. There was not less arguing. There was different arguing, earlier, about what needed to be understood before a number was possible. I have come to believe that is the whole benefit, and I still could not have sold it to anyone in advance by describing it that way.

The most useful sentence I ended up using is not on any form. It is asking what somebody would have to open to be sure, and then noticing that the answer is usually twenty minutes of work and it never gets done, because twenty minutes before an estimate feels expensive in a way that three wrong days does not.