Deciding an error is too small

My first job was assessing audit reports for a federal agency in Ottawa, against the standards and the internal control policy, and testing whether the key controls over financial reporting did what they claimed to do. It was the first serious thing anybody had let me near, and I spent the first months slightly appalled by one particular idea.

The idea is materiality. Before you begin, you set a threshold. Below it, an error is known to exist, is not corrected, and does not stop anything. It is written down in advance. It is not a compromise anybody is embarrassed about. It is the method.

Naming the errors you will not fix

I have come to think this is one of the more honest things any profession does.

Every field has errors it does not fix. The difference is whether it says so. Most working environments claim, at least officially, that mistakes get corrected, and then quietly leave a great many of them alone. The selection is made by whoever is tired, or by whichever one surfaced on a bad afternoon, or by who found it and how loudly they are willing to talk about it. That is a threshold too. It is just an undeclared one, applied unevenly, and nobody can audit it because nobody will admit it exists.

Writing the threshold down does two things. It makes the omission reviewable, because you can argue about the number, and it moves the decision away from the person under pressure at the moment of discovery.

Who sets it, which is the uncomfortable part

The threshold is not derived from the truth. It is derived from the reader.

It is a judgement about the point at which a difference would change a decision somebody makes on the basis of the statements. Which is a strange and sensible idea: correctness defined relative to consequence rather than to reality. And it means the honest answer to “is this right” is never yes. It is “right enough for what you are about to do with it”, which is the answer anyone who has prepared numbers professionally gives when they are being careful, and the answer nobody wants.

The uncomfortable part is that the threshold is set by people who will be judged against it. That is not a scandal, it is reviewed and argued over and there are conventions. But there is no way to remove the structural fact that the size of the mistakes you are allowed to leave is set partly by the people who would otherwise have to fix them.

The mistake I made with it

I treated the threshold as a budget.

If small errors were formally acceptable, then effort spent below the line was waste, and I was proud of not wasting it. That is a reading of materiality that sounds efficient and is wrong, and it took somebody more experienced than me about one sentence to dismantle it.

The threshold applies to what you find. It does not apply to how you look. You cannot know the size of an error before you have found it, so a search calibrated to skip small things is a search calibrated to skip things of unknown size. The economy is only available at the end.

I still catch the instinct. It arrives disguised as judgement about where to spend time, which is a real skill, and the two are almost indistinguishable from the inside.

Where it does not travel

I carried this into delivery work and it does not transfer cleanly, because outside a set of financial statements errors do not rank by size in a way that means anything.

A figure on a report can be wrong by a rounding difference and be catastrophic if the difference is in the direction of the number rather than its magnitude. A defect can be trivial in every dimension except that it sits on the one screen a sponsor opens. There is no scale that orders those against each other, and I have watched people, including me, reach for a size argument out of habit and find that it does not apply to the thing in front of them.

What survives the move is narrower than I expected. Not the threshold. Just the habit of saying out loud which errors I have decided not to chase, so that the decision belongs to somebody other than my own fatigue.

The ones I left are the ones I remember.