I wish someone taught me this in my first year as a PM
It would’ve saved years of chasing the wrong goals and wasting my team’s time:
“Choosing the right metric is more important than choosing the right feature.”
Here are 4 metrics mistakes even billion-dollar companies have made — and what to do instead, with insights from Ron Kohavi:
1. Vanity Metrics
They look good. Until they don’t.
A social platform he worked with kept showing rising page views…
While revenue quietly declined.
The dashboard looked great. The business? Not so much.
Always track active usage tied to user value, not surface-level vanity.
2. Insensitive Metrics
They move too slowly to be useful.
At Microsoft, Ronny Kohavi’s team tried using LTV in experiments —
but saw zero significant movement for over 9 months.
The problem is:
You can’t build momentum on data that’s stuck in the future.
So, use proxy metrics that respond faster but still reflect long-term value.
3. Lagging Indicators
They confirm success after it’s too late to act.
At a subscription company, churn finally spiked…
but by then, 30% of impacted users were already gone.
Great for storytelling but let’s be honest, it’s useless for decision-making.
You can solve it by pairing lagging indicators with predictive signals.
(Things you can act on now.)
4. Misaligned Incentives
They push teams in the wrong direction.
One media outlet optimized for clicks — and everything was looking good until it wasn’t.
They watched their trust drop as clickbait headlines took over.
The metric had worked.
They might had “more MRR”.
But the product suffered in the long run.
It’s cliché but true:
Use metrics that align user value with business success.
Because Here’s the Real Cost of Bad Metrics
- 80% of team energy wasted optimizing what doesn’t matter
- Companies with mature metrics see 3–4× stronger alignment between experiments and outcomes
- High-performing teams run more tests but measure fewer, better things
Before you trust any metric, ask:
- Can it detect meaningful change faster?
- Does it map to real user or business value?
- Is it sensitive enough for experimentation?
- Can my team interpret and act on it?
- Does it balance short-term momentum and long-term goals?
If the answer is no, it’s not a metric worth using.
If you liked this, you’ll love the deep dive.
