You’re not data-driven unless you measure everything

Most product teams claim to be data-driven.

But ask them one simple question:

“What percentage of your product actually gets measured?”

And suddenly, the room goes quiet.

A new breed of companies — OpenAI, Notion, Meta — are doing something different.

They don’t treat experimentation as a checkbox. They’ve made it a core part of how product gets built.

After 15 years in product, I can confidently say:

The most efficient product teams don’t just run experiments. They build systems that answer two critical questions:

1. Did it move the metric we care about?
2. Did it unintentionally break or degrade anything else?

These teams value learning velocity > shipping velocity.

And that’s what separates great product orgs from the rest.

The Cost of Getting It Wrong

Most companies still get experimentation wrong. Here’s how:

• They treat experimentation like QA
• They only test when they’re unsure
• They never catch slow-burn losses
• They celebrate shipping…