You might come across a dashboard claiming something like "+52% vs Market Avg". It looks impressive. Before trusting a number like that, it's worth understanding exactly what's being compared — and where this kind of math quietly goes wrong.
1. The Common Mistake
Here's how the trap works: take a model's precision — the share of its "yes" calls that turn out correct — and stack it directly against an industry-wide accuracy figure, which measures something different: the share of every prediction, yes and no, that turns out correct.
Precision and accuracy answer different questions, over different denominators. A model can post a high precision score while only weighing in on a narrow slice of matches — a different question than the one an accuracy baseline is answering. Treating the gap between the two as a real "edge" overstates performance.
"A percentage is only meaningful next to another percentage that measures the same thing, on the same sample, with the same denominator."
That distinction is easy to skip past. Once two different metrics get compared as if they were one, the resulting "lift" isn't measuring what it claims to measure.
2. Why the Math Doesn't Hold Up
Here's a worked example of the mistake, so you can recognize it elsewhere:
That "+52%" looks like a real performance lift, but it isn't one. It's the result of subtracting an accuracy figure from a precision figure and dividing by the accuracy figure — two metrics that were never comparable to begin with. A relative-percentage "edge" built from unlike metrics will always overstate performance, no matter what the underlying numbers are.
3. Precision vs Accuracy: What Each One Measures
The two terms get used interchangeably in casual conversation, but they measure different things:
- Accuracy: Of every prediction made — every "yes" and every "no" — what share turned out correct?
- Precision: Of only the predictions that said "yes," what share turned out correct? It says nothing about how many "yes" calls were made, or how the "no" cases were handled.
- Why it matters: A model can raise its precision by only calling the matches it's most confident about, while its accuracy across the full set of matches tells a different story.
Before comparing any two percentages, ask three questions:
- Same task? Precision and accuracy answer different questions — don't compare them directly.
- Same sample? A figure from one dataset, league, or time window isn't directly comparable to another.
- Same denominator? "Correct out of all predictions" and "correct out of predictions we chose to make" are not the same base.
4. How to Compare Percentages Fairly
A fair comparison uses the same metric, measured the same way, on directly comparable samples. Here's the difference between a misleading comparison and an honest one:
Comparing Unlike Metrics
Taking a precision figure (a "yes"-only measurement) and subtracting an industry accuracy figure (an all-predictions measurement), then dividing by the accuracy figure. The result looks like a performance lift. It isn't — it's an artifact of mixing two different measurements.
Comparing Like-for-Like
Comparing accuracy against an accuracy baseline, on the same task and the same kind of sample. That's the only version of this comparison that means what it claims to mean.
"If a comparison would fall apart when you swap in the other metric's definition, it wasn't a fair comparison to begin with."
The Bottom Line
The next time you see a badge like "+52% vs Market Avg," check what's actually being compared:
- Same metric: Precision compared to precision, accuracy compared to accuracy — never mixed.
- Same sample: The same task, timeframe, and population on both sides of the comparison.
- Same denominator: "Out of what" needs to match on both sides, or the percentages aren't measuring the same thing.
A number that skips these checks isn't an edge. It's a rounding trick.
Want the Real Numbers?
See our published accuracy stats, measured the same way every time.
View Live Stats