In its Q1 2022 filing, Unity Software told the SEC that a large customer had fed bad data into its ad-optimization product, and that the resulting drop in effectiveness would hit the business by roughly $110 million that year. Nothing crashed. No dashboard threw an error. The optimization engine kept running, kept making decisions, and kept getting quietly worse at its one job because the data it was learning from had gone bad upstream. By the time it showed up as a number executives could see, it was already a nine-figure problem.
That's the pattern behind the expensive errors: an impossible number gets challenged in the first meeting it appears in. A plausible one gets used, then used again, then built into next quarter's plan. A duplicate entity, a denominator that quietly changed, a formula that excludes one category it used to include, a risk measure built from last month's version of an input, none of these produce a number that looks wrong. They produce a number that looks fine.
Plausibility is not validation
A dashboard sitting inside last quarter's range has passed a genuinely weak test: it looks familiar. Familiar isn't the same as correct. It hasn't demonstrated that the underlying population is complete, that the joins preserved the intended grain, that the definitions haven't quietly shifted, or that the source data is current enough to support the decision someone's about to make on top of it.
The remedy has to sit upstream of the dashboard, and it has to be specific rather than a vague call for "more QA." Test uniqueness wherever one record is supposed to represent one entity. Reconcile row counts across every join instead of trusting the last one that got checked. Watch distributions and expected arrival times, not just final totals. Compare metric definitions against the actual decision they're meant to support, because a metric that's technically correct for last year's question can be silently wrong for this year's. The test isn't whether the number looks normal. It's whether you can walk the exact path that produced it and explain every step.
A better incident question
When a number turns out to be wrong, the instinct is to ask how it got through. That's the wrong first question, because it usually ends in blame. The better one is: which control, specifically, would have caught this before it became a decision someone acted on? That question produces something reusable. "How did this get through" just produces an apology.
An obvious error interrupts a meeting. A plausible error can steer a quarter.