Ashley Stirrup

92% of experiments at Airbnb failed to move the metrics they were designed to improve. Ron Kohavi shared this stat from his time as VP & Technical Fellow at Airbnb — and it's been explicitly approved for sharing. His team ran 250 experiments. Only 20 were positive. His reaction? This is exactly how it's supposed to work. Here's the thing most teams get backwards: → If 90%+ of your experiments "succeed," you're not taking real risks → A failed experiment costs 2 weeks of learning → A failed feature — shipped without testing — costs months, headcount, and user trust Kohavi's summary slide says it plainly: "Triple your experiment rate and you triple your success (and failure) rate. Fail fast and often to succeed." "The less data, the stronger the opinions." Most teams have the opposite problem from Airbnb. They're not running enough experiments. They're running decisions through the HiPPO — the Highest Paid Person's Opinion — and measuring success as "% of plan delivered," not actual customer impact. The goal isn't a high success rate. The goal is learning fast enough to compound your wins. The teams that win at experimentation aren't the ones who fear failure. They're the ones who've made failure cheap. At GrowthBook, that's what we build: infrastructure that makes experiments fast, trustworthy, and low-cost — so you can run 10x more of them. How many experiments does your team run per month? Drop a number 👇

Only one snapshot so far — the engagement-over-time curve appears once the daily scrape has captured this post at least twice.