You Optimised for the Metric. Then the Metric Moved

There is a rule economists call Goodhart’s law: when a measure becomes a target, it stops being a good measure.

Everyone nods at this and almost nobody plans for it, because the version in the textbook is the wrong one to worry about. The textbook version is gaming: people cheat the number, you catch them, you fix the incentive.

The version that actually kills companies is the opposite. Nobody cheats. Your people optimise the metric honestly, for years, with real skill, building tools and processes and expertise around it. Then the thing being measured changes, and every bit of that accumulated capability points in a direction nobody wants anymore.

An industry that lived through exactly this

US healthcare built an entire sector around one measure, and the whole arc is documented because a federal agency audits it and publishes what it finds.

Insurers covering Medicare patients get paid more for covering sicker people. That design is deliberate and sensible, because without it insurers would quietly avoid anyone expensive to treat. How sick a patient is gets established through diagnosis codes drawn from clinical notes.

So the measure that mattered became clear and countable: how many supportable diagnoses can we find that were missed?

Everything got built for that question. Software that reads charts and surfaces candidates. Review workflows tuned for throughput. Vendor contracts priced on volume found. Staff whose entire professional expertise was finding things other people had overlooked. Fifteen years of genuine, honest optimisation against a clearly stated target, by people doing exactly what they were asked to do well.

Then the regulator changed the question. Not how many diagnoses did you find, but how many of the ones you submitted hold up when somebody reads the record underneath them.

What that transition cost

The US Office of Inspector General published its detection method in December 2023: a toolkit with the selection logic, including the database queries it uses to flag diagnoses at high risk of lacking support. The test was public and anyone could run it on themselves.

When the agency ran it, roughly 70 percent of those codes were not supported by the associated medical records, with some categories above 90 percent. One national audit sampled 97 records carrying an acute stroke diagnosis. Not one was supported as an acute stroke by the documentation behind it.

The detail that makes the point: in 68 of those 97 cases the patient had genuinely had a stroke. It was recorded in their history, accurately. The submitted diagnosis described it as currently happening.

Nobody invented a patient. Nobody invented a condition. The systems did precisely what fifteen years of optimisation had trained them to do, and the measure had moved underneath them.

Why the capability itself becomes the trap

This is the part worth sitting with if you are building anything.

The software in that market predicts. Given a record, estimate which diagnoses probably apply, rank them by confidence, hand a human the list. Genuinely excellent at finding things. Structurally incapable of the new job, because a ranked list of candidates contains no entry that means “this one is wrong, take it out.” You can score something low. You cannot mark it incorrect. The output type does not include that answer.

So the better an organisation had got at the old measure, the more locked in it was. Its tooling produced one kind of answer. Its contracts paid for one kind of result. Its best people had spent a decade becoming excellent at one half of a problem that now had two halves.

Competence became the obstacle. That is the Goodhart failure nobody prepares for, because it does not look like a failure while it is happening. It looks like getting better at your job.

How to spot it before the measure moves

The useful question is not whether your metric is good. It is whether it is still attached to the thing it was standing in for.

Every metric starts as a proxy. Diagnoses found was a proxy for documenting patient illness accurately. Handle time is a proxy for efficient service. Pipeline activity is a proxy for revenue that will arrive. Pageviews are a proxy for something people found worth reading.

Proxies drift from goals slowly, and the drift is invisible because the number keeps improving. Nobody raises an alarm about a metric going up. By the time the gap is wide enough to see, you have usually built several years of infrastructure on top of it.

Three questions that surface the drift early.

• What is this number a proxy for, and could I measure the actual thing directly even if it is harder or slower? If nobody has tried in years, nobody knows how far apart they have moved.
• What would have to become true for this metric to stop mattering? If you cannot name a scenario, that means you have not thought about it, not that no scenario exists.
• Can my system produce the opposite answer? This is the structural one. If your tooling only pushes in one direction, a changed target cannot be met by trying harder. The ceiling is in the architecture, not in the effort.

The uncomfortable organisational part

There is a people dimension to this that strategy decks tend to skip.

When a measure has defined success for years, it stops being a metric and becomes an identity. We are the firm that finds what others miss. That story recruited people, won contracts and justified investment. It was true.

Changing the measure means telling capable people that the thing they are best at is no longer the thing that matters most. That is a hard conversation and it is usually deferred, which is how organisations end up responding to a changed target by doing more of what already stopped working. Not denial exactly. Momentum, and a reluctance to devalue expertise that was earned honestly.

The part that transfers

What forced the reckoning in that market was an external check against a published standard. It is formally called a radv audit in risk adjustment, and the organisations that came through it in reasonable shape were the ones that ran the test on themselves while doing so was still voluntary, rebuilt their review layer so it could return a negative result, and accepted that the old capability was now one half of the job rather than the whole of it.

Thirty months sat between the regulator publishing its method and enforcing against it. That is an unusually generous window, and most of the market spent it carrying on.

Your industry almost certainly will not publish its test in advance. Which makes the cheap version of this exercise, writing down what your numbers are actually proxies for and checking whether they still line up, worth doing in a quiet quarter rather than a loud one.