Why OpenAI and Notion Always Win: The Hidden Power Behind Their Velocity

3 min read4 days ago

In the world of product and innovation, speed is often celebrated. The faster you ship, the better — right?

But here’s the uncomfortable truth: velocity without insight is just motion.

Some of the most admired companies today — like OpenAI and Notion — aren’t just moving fast. They’re moving with purpose. Behind their velocity is a powerful engine most teams underestimate: scaled experimentation.

Let’s unpack how the best teams in the world build cultures where experimentation isn’t a phase — it’s the default.

What Elite Teams Do Differently

Most companies test. A/B here, feature flag there. But it’s inconsistent. It’s reactive.

Now compare that to what happens at OpenAI or Notion.

When OpenAI integrated Statsig, they ran more experiments in two months than some companies do in an entire year.

Notion? Over 80% of their product launches are measured experiments. Even features they believe in get tested. Because belief isn’t enough. They want to know.

This is what separates elite teams:

They don’t just build.
They don’t just ship.
They measure everything — and learn faster than the rest.

It’s not about having conviction. It’s about having evidence.

The 3 Levels of Experimentation Maturity

Every team wants to test more. But most don’t realize there’s a maturity curve to this. Here’s how it breaks down:

Level 1: 10–20% of launches are experiments

You’re testing the obvious things. Maybe a landing page variant or an onboarding tweak. It’s a start — but you’re still mostly relying on gut.

Level 2: 30–50% of launches are experiments

You’ve built some process around testing. Maybe you have a few dashboards and someone from data helping out. But testing is still inconsistent, and teams often skip it when under pressure.

Level 3: ~100% of launches are experiments

This is where companies like OpenAI and Notion operate. Every launch — big or small — is measured. The organization doesn’t guess. It learns at scale.

As Vineeth Madhusadhanan, PM at Statsig, says:

“Velocity doesn’t matter if you’re only running experiments on 20% of your output.”

If you’re only measuring 20%, you’re flying blind for the other 80%.

The 3 Foundations of Scalable Experimentation

It’s one thing to say “we want to test everything.” It’s another to make it possible. To do it right, elite teams build around three pillars:

1. Easy Data Lookup

Past learnings should be accessible in seconds — not buried in Slack threads or trapped in someone’s memory. A good experiment is worthless if no one can find it later.

2. Trust & Consistency

Experimentation isn’t just about numbers. It’s about integrity. That means:

No cherry-picking results.
No massaging metrics post-launch.
A standardized way of running tests, so teams don’t debate methodology every time.

3. Experiment Velocity

You shouldn’t need a PhD or wait three weeks for the data team to weigh in. At top-performing companies, everyone in the product org can ship and measure. Experimentation is democratized.

How the Mental Model Has Evolved

Here’s how the thinking around experimentation has changed over the past decade:

2010: “Should we test?”
2015: “What should we test?”
2020: “How do we test everything?”
2025: “How do we build systems that remember what we’ve learned?”

It’s no longer just about testing more. It’s about learning faster — and forgetting less.

Institutional memory is the next frontier. The companies that win tomorrow aren’t just the ones who move fast. They’re the ones who can look back, reference a decade of experiments, and say with confidence: We’ve seen this before. Here’s what worked.

If You’re Trying to Build This Muscle…

Start small — but start deliberately.
Don’t just test to check a box. Test to learn.
Make it easy. Make it trustworthy. Make it part of your team’s daily rhythm.

If you’re serious about building an experimentation culture, I put together a full breakdown of how the best teams structure it here.

Build smarter not just faster.

If this resonated with you, follow for more insights on building great products and high-performance teams.