Post categories
How every post is categorized for analysis: one category per post, plus scroll-stopping (fun, curiosity-creating image), ★ influencer engaged (a recognized expert liked/commented), and 💸 boosted. Changes you make here are protected from re-classification and flow straight into Insights and the exec summary.
I’m happy to share that I’m starting a new position as Chief Marketing Officer at GrowthBook! GrowthBook is an open source experimentation and feature flag company. They help thousands of companies, including DropBox, Sony and the LA Time…
Social / culturetext/only
We once had a design team spend months redoing our onboarding flow.
They did everything right. User research. UX expert interviews. Detailed personas. They were absolutely certain this new flow was the future of the product.
Then we teste…
Past experiencetext/image💸 Boosted
The GrowthBook team just got back from Hawaii!
We're a team spread across the globe, and a few times a year, we close the distance. This week was one of those moments: a chance to think big, work through hard problems together, and just be…
Social / culturetext/image
Your metrics are green. Your customers are leaving.
Ron Kohavi calls these "watermelon metrics." Green on the outside. Red on the inside.
Here's how it happens.
A team picks metrics that feel right: pages visited, features used, support …
Thought leadershiptext/image
AI just killed the two biggest excuses for not experimenting.
"We don't have enough engineers."
"We don't have enough traffic."
Kameron Tanseli and the Fyxer's growth team are just 4 people. Last year they ran 360 experiments — more than …
Customer storytext/image💸 Boosted
Optimization to Enshitification: Dark Patterns in A/B Testing
A post allegedly written by a software engineer at a meal delivery company went viral recently. It described an A/B test on a “Priority delivery” fee, and not by making priority…
Thought leadershiptext/image💸 Boosted
The most dangerous animal in your organization isn't a bad idea. It's a senior leader with a strong opinion and no data to challenge it.
Ron Kohavi has a name for this: the HiPPO — the Highest Paid Person's Opinion. He and Avinash Kaushik …
Thought leadershiptext/image💸 Boosted
I know it sounds controversial, but I advocate for burning your backlog at the end of every cycle.
The first time you do it, you’ll feel a massive spike of anxiety because you’re "throwing away" hours of work and planning.
But that feelin…
Thought leadershiptext/videoscroll-stopping: n/a (video)
Lukas Vermeer 🃏 talked today at Experimentation Island about how simple stories can reshape how teams “think” and “talk” about complex concepts. This kind of story telling can be a great culture changer for any leader.
- “Bear sign” becam…
Thought leadershiptext/image
Three years into building our own experimentation platform at my last company, we found a bug in the statistical engine.
The question that followed was not a small one. Did we just invalidate every product decision we had made for the pas…
Past experiencetext/image
A year ago, most engineers wanted to see every line an AI wrote before it went near their codebase. They read the diffs. They were skeptical because they didn't trust it.
Today, a lot of those same engineers have not written much code by …
Thought leadershiptext/image
Have you seen how Khan Academy created an AI tutor? This edtech innovator is once again at the forefront of learning. And they're using AI-driven evaluation and A/B testing to ensure learning quality, student outcomes, and responsible AI be…
Marketing event / contenttext/image
Getting numbers is easy. Getting numbers you can trust is hard.
We hosted Ron Kohavi (co-author of Trustworthy Online Controlled Experiments, ex-Amazon, Microsoft, Airbnb) and Luke Sonnet, our head of experimentation, to pull apart a real …
Thought leadershiptext/image
If your experimentation stack is Statsig, the last few months have probably been confusing.
When OpenAI acquired Statsig, I saw many teams read it as good news. The assumption was that being owned by OpenAI would turn it into the most adva…
Industry commentarytext/only
I’m very excited to share that I’ve joined GrowthBook as a Developer Advocate!
This next chapter feels incredibly aligned: open source, experimentation, feature flags, developer education, community, and the kind of product storytelling th…
Social / culturetext/only
🚀After 426 pull requests and 351,114 lines of code, GrowthBook 4.4 is here!
This is our largest release to date and designed to help teams shipping products with AI to move even faster while maintaining control across the entire experiment…
Product functionalitytext/image
In baseball terms, the industry standard for any new product success is batting 200- and hitting home runs is extremely rare.
Most PMs hate admitting that. We are paid to be experts, yet the data says we are wrong 80% of the time.
The goa…
Thought leadershiptext/videoscroll-stopping: n/a (video)
How do you grow from $1M to $35M ARR in a year? Not by running the usual handful of tests per quarter. Fyxer ran 514 experiments and utilized GrowthBook and Claude Code to make it possible.
Each growth engineer owned a full loop, from idea…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
We thought Chess.com users wanted to review their losses.
80% were reviewing their wins.
For years, Chess.com's Game Review feature opened with a list of your blunders and misses. Then Nafis Shaikh, Director of Product Management at Chess…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
Congratulations to all of the Experimentation Culture Awards finalists. Everyone had such great stories. Special shout-out to the winners across the 4 categories and their team leads:
Decision Intelligence: Skyscanner, Joan Capdevila Puj…
Othertext/image
OpenAI bought Statsig for $1.1B in stock. Before that, Datadog swallowed Eppo.
For a CTO, this is more than "industry news." It calls into question your whole tooling infrastructure.
We've all seen this movie before. When you integrate a…
Industry commentarytext/image
Gave a talk on synthetic user A/B testing at MeasureCamp Berlin on Saturday. There was plenty of healthy skepticism about whether AI-simulated users can tell you anything real, which is exactly the right question to ask. Big thanks to Chris…
Marketing event / contenttext/image
This is exactly why Jeremy and I founded GrowthBook. Experimentation is hard but your experimentation platform need not be. We’ve focused so much on making it easy to plug GrowthBook into your existing tech stack and scale with your growth.…
Customer storytext/only
A Microsoft team once presented this finding internally about Office 365: "Users who use our feature are half as likely to churn."
The implication was obvious — the feature was working. Ship more of it. Double down.
There was just one pro…
Thought leadershiptext/image
The GrowthBook 4.2 webinar is this Thursday at 10am Pacific. Join Luke Sonnet, Ryan Feigenbaum, and Ashley Stirrup as they demo the latest features, including our new Product Analytics features.
Release 4.2 marks our final release of a ban…
Marketing event / contenttext/only
Stop worrying about losing tests.
Will Guyeskey (GoPro): do not fear a loser, do not fear a flat result.
What you should absolutely worry about is designing the test so it delivers impactful learnings regardless of the outcome.
Does your te…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
Great session on “How to scale your experimentation program” session at Experimentation Island
Jenni Bruckman from YETI reframed experimentation and digital product in a way that was refreshingly human:
- Velocity is value. Not shipping …
Marketing event / contenttext/image
The AI worked. Users quit anyway.
Ashraf Karim (ServiceNow) on the biggest learning from launching an agentic skill: people abandoned it and got frustrated.
The technology was not the problem. Performance and latency were.
A job that took h…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
Judging a product team solely on their ability to ship is a bad metric that rewards activity over actual progress.
I’ve seen teams hit every single deadline while their core business KPIs remained completely flat because they made no actua…
Thought leadershiptext/videoscroll-stopping: n/a (video)
My head of Experimentation named our product development framework after Jon Hamm. I chose not to ask questions.
What it actually does is stop teams from building things nobody wants.
Most product teams I see spec out the entire feature…
Thought leadershiptext/image
I just watched a terrific presentation by Kelli H. at Khan Academy at Experimentation Island yesterday. They are doing amazing things with GrowthBook to A/B test their AI tutor — and the results are remarkable.
Khan Academy's Khanmigo is a…
Customer storytext/image
Many "winning" A/B tests aren't actually wins. They're results that cross a threshold without being properly validated.
On July 16, Ron Kohavi and Luke Sonnet break down what separates a trustworthy experiment result from a false positive,…
Marketing event / contenttext/image
Be on the lookout across the San Francisco Bay Area today! If you're looking to make the switch, check out our buy-out offer and migration tools at https://lnkd.in/gM4FVNnE
Othertext/image
In many companies, the projects we work on are the ones backed by the person who speaks the loudest, or who is the highest paid.
We call this "ego-driven development." It is the fastest way to waste engineering hours on features that don't…
Thought leadershiptext/videoscroll-stopping: n/a (video)
Teams have no shortage of ideas, but implementing them is where teams hit roadblocks.
Sometimes the bottleneck is the company’s technical framework. Other times, it is limited engineering capacity or a lengthy approval process. Whatever …
Product functionalitytext/videoscroll-stopping: n/a (video)
92% of experiments at Airbnb failed to move the metrics they were designed to improve.
Ron Kohavi shared this stat from his time as VP & Technical Fellow at Airbnb — and it's been explicitly approved for sharing.
His team ran 250 experime…
Thought leadershiptext/image
You never know where your next home run idea will come from.
That's not a platitude. It's a data problem.
At Bing, an idea sat in the backlog for months. PMs had rated it: "meh." Too many higher priorities.
One engineer built it anyway. …
Thought leadershiptext/document
The most important transition for a product team is learning how to be ruthless with ideas without making it personal.
I’ve seen too many build features that we had evidence were not going to work because the team was more focused on prote…
Thought leadershiptext/videoscroll-stopping: n/a (video)
Day 1 of Product-Led Alliance Summit kicks off today in San Francisco.
Just in time for the GrowthBook 5.1 launch! Most of what's in it came from the conversation we keep having at booths like this one: you run the test, you get a result, …
Marketing event / contenttext/image
Exciting news to share... we hit 🎉 7,000 GitHub stars. We're humbled to reach this milestone. It confirms our belief that the future of product development is open, transparent, and data-driven.
Thank you to the 7,000+ stargazers who ackn…
Othertext/image
Cloudflare down? If your feature management or experimentation platform just brought you down, then it’s time to check out GrowthBook.
Engineers hate it when 3rd party libraries are in the critical rendering path, so we make it easy to re…
Industry commentarytext/image
Khan Academy gave their AI tutor a calculator to improve math accuracy. It worked, but it made responses painfully slow for students.
So they ran five sequential A/B tests:
removing the calculator (math errors doubled),
switching to GP…
Marketing event / contenttext/link
Product development is forever changed, and the pace of the change is incredible. We've seen this with our own product, too.
Engineering teams are using agentic coding tools such as Claude Code and Cursor to build features, manage flags, …
Product functionalitytext/only
Fanatics, UPS, and DoorDash all prove the same thing about testing.
The new edition of The Experimentation Edge newsletter rounds up my conversations with the experimentation leaders at all five: Medha U., David Massey, MFA, and Ilya Izrai…
Podcast (Experimentation Edge)text/link
I spent years seeing product decisions based on gut feel.
At education.com, we would spend months building a feature, ship it, and then wait to see if the numbers moved. The ideas came from industry trends, from whatever the loudest voice…
Past experiencetext/only
Most Engineers I know are supervising agents now, not writing the code.
Features that took weeks take hours. The volume of what ships to production has gone up significantly.
The fundamental question of product development remains unsolv…
Thought leadershiptext/image💸 Boosted
Amazon just held a mandatory meeting after vibe-coded changes caused multiple major outages.
This story is going to repeat itself across a lot of engineering orgs.
AI coding tools are genuinely impressive. I use them every day.
Here's …
Industry commentarytext/videoscroll-stopping: n/a (video)
How does Fanatics generate hundreds of millions in growth from experimentation every year?
Medha U., VP of Growth and Experimentation at Fanatics, shared the playbook on The Experimentation Edge. Over 10 years they went from 10 experiments…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
What makes an experimentation program an engine for long-term growth?
We’re excited to host Ron Kohavi, best-selling author of Trustworthy Online Controlled Experiments, in conversation with GrowthBook’s Luke Sonnet .
They’ll share real…
Marketing event / contenttext/only💸 Boosted
There was a point in my career where I traveled roughly 75% of the year. All to speak and attend developer conferences.
It was a lot but I did love it. I still love connecting with developers in person. I love the conversations that happen…
Marketing event / contenttext/image
GrowthBook 5.0 launch week: Today, agents 🙌
Companies don’t win with development speed alone. They make their products better after every ship.
GrowthBook 5.0 helps teams build, ship, and improve their product at any scale. We’ve opened…
Product functionalitytext/image
We spent three years building our own experimentation platform at education.com. Then we found a bug in the stats package.
The immediate question was not a small one: did we just invalidate every product decision we made over the last thre…
Past experiencetext/videoscroll-stopping: n/a (video)
At Ancestry, their goal is to make the past come to life. They used AI to do exactly that, and the results were incredible.
30%+ lift in CTR, 100%+ increase in storytelling success rate, and a 500% increase in AI story views.
It shifted h…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
Calling e-commerce experimentation companies!
Get free expert guidance from Ron Kohavi, Lukas Vermeer 🃏, and Jakub Linowski while shipping winning features that could grow your revenue by up to 7%.
They'll help you design, execute, and an…
Marketing event / contenttext/image
A team of four. 541 experiments in a year. A lot of them lost.
Losing is expected. What experimentation is really about is shipping, iterating, and learning. As Fyxer's Ben Fanning puts it, when the cost of shipping is low, you optimize f…
Marketing event / contenttext/videoscroll-stopping: n/a (video)
I asked five experimentation leaders the same two questions. Their answers were almost identical.
On The Experimentation Edge, I sat down with the people running experimentation at JPMorganChase, Twitch, The Home Depot, Squarespace, and Si…
Podcast (Experimentation Edge)text/link
The whole team GrowthBook is so excited to see Khan Academy win this award. I really think the work they did on labeling questions to their AI powered tutor is cutting edge. It enabled them to measure cognitive engagement and thereby incr…
Othertext/only
Most companies think choosing an experimentation vendor comes down to feature lists and UI.
That's the wrong frame.
When you deeply integrate a proprietary platform, the switching cost starts accumulating from day one. Your experiment h…
Thought leadershiptext/image💸 Boosted
The best experimentation teams celebrate the tests that fail.
Why? Because a win confirms what you already believed. A failure shows you where to improve, and that's where the real learning lies.
I asked Andrew Willingham about this. This…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
15 value-driven experiments beat 40 button-color ones.
That's the shift Craig Kistler made at Signet Jewelers, the parent company behind Kay Jewelers, Jared Jewelers, and Zales Jewelers.
His team used to run 40 to 50 tests a quarter. Butt…
Podcast (Experimentation Edge)text/carousel
Accuracy measures the test set. It doesn't measure trust.
I had Mayank Agarwal (Director of AI Products at RingCentral) on The Experimentation Edge this week. The metric he said most teams skip is acceptance rate (ie. how much of the AI's …
Podcast (Experimentation Edge)text/carousel
I've watched design teams spend months on an interface that looked great, and was based on research, and yet moved zero metrics when it shipped.
Their response was to deny the data, and want to not A/B test things anymore.
Wrong direction.…
Thought leadershiptext/videoscroll-stopping: n/a (video)
I interviewed five experimentation leaders in one week and compiled the most important findings into this article.
The Walt Disney Company, The Home Depot, Fin, Expedia Group, and Stitch Fix.
Every single one pushed back on calling tests …
Podcast (Experimentation Edge)text/link
This is kind of feedback makes working at GrowthBook really special. How many experiments is your company running? Running 5X - 10X more experiments can mean millions for many businesses. Thats why we work so hard to make it easy to run …
Customer storytext/only
The biggest experiment wins usually aren't in the product at all.
Nick Beyler from Stitch Fix noticed the same pattern across every example his teams shared.
He said the most impactful learnings rarely came from a bug in a feature. They c…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
If you work inside an experimentation program, you have probably noticed that leadership speaks a different language than you do.
You are talking about tests, quality scores, and learnings. They are looking at company KPIs. They will nod …
Thought leadershiptext/image
Khan Academy was one of the first companies to develop with the OpenAI models. This NYT story covers how they built Khanmigo, an AI tutor, for learning, not just answers.
https://lnkd.in/gaBsmpbj
If you want to know more about the role ex…
Marketing event / contenttext/link
I have been a software engineer long enough to have the grey hairs to show for it.
Monolith to microservices and back. Waterfall to Agile to Kanban. None of it compares to what I am watching now.
Engineers on my team are running six to …
Thought leadershiptext/videoscroll-stopping: n/a (video)
What Fyxer, Ancestry, and Chess.com shows that all you need is 25%...
Three companies at completely different stages have showed your intuition loses roughly three times out of four.
Kameron T. at Fyxer ran 360 experiments with a team of …
Podcast (Experimentation Edge)text/link
Can a single Product Analytics dashboard uncover a $400K conversion problem?
Join us for a live demo of GrowthBook Product Analytics and Experimentation Platform. You'll see how one company diagnosed a conversion problem from a custom dash…
Marketing event / contenttext/image
Most product teams use evals to figure out if their AI tools or products are working correctly. And quite often, those evals can lie to you.
Evals show you how your models perform under preset conditions, usually based on real user behavio…
Thought leadershiptext/videoscroll-stopping: n/a (video)
AI coding tools are genuinely impressive. And this is literally the worst they will ever be.
They allow us to build products faster than ever. Engineers are now supervising agents instead of writing code. The volume of what ships to produc…
Thought leadershiptext/video💸 Boostedscroll-stopping: n/a (video)
Most engineering teams I talk to are still shipping every feature directly to production.
The code hits the infrastructure. The feature is live. Every user is affected immediately.
When something breaks, and it will break, the only move i…
Thought leadershiptext/videoscroll-stopping: n/a (video)
GrowthBook 4.2 is here!
We’re excited to announce the beta version of GrowthBook Product Analytics.
For the first time, our customers will be able to analyze product performance, deploy new features and measure their performance all within…
Product functionalitytext/image
Building your own experimentation stack is a fun engineering challenge until the day it isn't.
When I was CTO at education.com, we evaluated every tool on the market. Most could only answer about half our data questions, when we really us…
Past experiencetext/image💸 Boosted
Your A/B test won, but is it actually trustworthy? 🤔
Ron Kohavi has audited thousands of experiments at Microsoft and Airbnb. A lot of the "winners" turn out to be wrong: bad power calculations, misread p-values, sample ratio mismatch nob…
Marketing event / contenttext/only
Is MCP the magic wand I’ve been waiting for?
Because “please clean up my stale feature flags” is something I’ve wanted to say and have somebody else deal with for a while now.
In this video, I put GrowthBook’s MCP server to work in Claude…
Product demotext/videoscroll-stopping: n/a (video)
Artificial intelligence helps you feel artificially intelligent.
It is almost as if it was named perfectly. If you are not a real expert in a field, you might look at what it produced and think it is genius.
Then someone who is actually g…
Thought leadershiptext/videoscroll-stopping: n/a (video)
Not every experiment that looks perfect on a whiteboard works in the real world.
Kameron Tanseli, Head of Growth Engineering at @fyxerofficial, learned this the hard way.
Fyxer has a scheduling feature — similar to Calendly. Kameron hypot…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
What if your next #Angular app started with a single prompt? 🤯
In this demo, Alyssa Nicoll shows how to build a production-ready onboarding workflow using Kendo UI, MCP & Cursor. 🙌
From multi-step forms to validation and persistence, A…
Product demotext/videoscroll-stopping: n/a (video)
Ask any product manager what they hate most about their job, and the answer is rarely the work itself.
It is the meetings where opinion fights opinion and whoever is loudest wins.
Before experimentation, we shipped products based on vib…
Thought leadershiptext/only
Startup life is mostly reactive.
You're in the task, in the Slack thread, in the meeting.
The bigger priorities don't disappear. They just get buried under the urgent stuff until you create space to find them again.
For me, I find that s…
Social / culturetext/videoscroll-stopping: n/a (video)
A simple UI change was worth $120 million to Bing.
The problem is, it's impossible to know which change will deliver what impact.
That's where A/B testing comes in.
Rigorous A/B testing is crucial for uncovering potential and ensuring re…
Marketing event / contenttext/videoscroll-stopping: n/a (video)
Killing a feature you spent months (or years) building is one of the hardest calls in product.
The team put in the work. It was on roadmap for months. Someone built a complex spec, and pushed for prioritization. Sitting in the room when t…
Thought leadershiptext/videoscroll-stopping: n/a (video)
Most companies ship the first version of the product they build. And the chance that's the best version is basically zero.
It's just really unlikely that the first version is the one that lands best with your user. So it's important to ite…
Thought leadershiptext/videoscroll-stopping: n/a (video)
DoorDash saved thousands of subscriptions per week.
What was that experiment?
Ilya Izrailevsky showed canceling DashPass subscribers exactly how much they already saved.
Customers had no idea what they got from DashPass.
One data point …
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
One very large financial company, a name you would recognize, told me they were doing experimentation well. They were running three tests a quarter.
The next conversation I had was with a leader running 60,000 tests at a time. Both told m…
Thought leadershiptext/videoscroll-stopping: n/a (video)
Most experimentation stories focus on wins. Breeze also focused on avoiding costly mistakes.
They brought experimentation and measurement into a single system, stopped reconciling data across tools, and doubled testing capacity without add…
Customer storytext/carousel
Heading to Experimentation island (#Ei2026)? We are! This isn’t just another conference. It’s an all-inclusive learning and networking event on St. Simons Island, GA, on February 24-26.
Registration opens Wednesday, but scroll down to ski…
Marketing event / contenttext/image
Even the best product people on Earth have a surprisingly low batting average.
In baseball, a .300 hitter is a Hall of Famer. In product, you are lucky if three out of ten ideas actually move the needle.
Realizing this is deeply humbling.…
Thought leadershiptext/image
Many teams aspire to scale their experimentation program to learn fast and iterate constantly. Fyxer ran 541 experiments last year, more than two every working day.
That pace only works because AI does most of the grunt work, the research …
Marketing event / contenttext/only
Day 1 of The AI Conference starts today. Find us at booth 206 to see how teams ship AI features safely and know whether they worked.
We're drawing a winner for a pair of AirPods Max 2 at 3:45pm. Stop by the booth to enter.
Marketing event / contenttext/carousel
The average engineer stays at a company about two years.
I don't blame anyone for that. Moving is often the fastest way to raise your salary, and I would do the same. But if you are running the company, that number should worry you.
Eve…
Thought leadershiptext/link
I interviewed three experimentation leaders from three industries that could not be more different. I compiled the sharpest findings into this edition.
Here are the industries:
- Corporate governance
- Content cloud
- Real-time ad auctions…
Podcast (Experimentation Edge)text/link
Most product teams spec out the entire feature before they've shipped a single line of code.
The real question nobody is asking first: what indication do you have that this product is actually worth building?
We use a framework called HAM…
Thought leadershiptext/video💸 Boostedscroll-stopping: n/a (video)
Most teams talk about experimentation. Treatwell made it their default.
Instead of relying on assumptions or pre/post analysis, their marketplace teams built a system for running experiments continuously and confidently.
Here’s what chang…
Customer storytext/carousel
When was the last time you saw innovation from a public company?
Honestly, from my experience, it's pretty rare.
And that feels worth raising given the recent news of an experimentation platform being sold off.
It draws into question wha…
Industry commentarytext/videoscroll-stopping: n/a (video)
How do you keep every one pointed at the same North Star?
This week’s Experimentation Edge newsletter brings together Rich Liebling at Learneo, Whitney Perez at Realtor.com, and Mi Tian at Zalando.
Rich tests the opposite of the hypothesi…
Podcast (Experimentation Edge)text/link
We don't have product managers at GrowthBook. That is deliberate...
One reason is ownership. The usual setup puts a layer between the people building the product and the people using it. Someone gathers requirements, translates them, and …
Thought leadershiptext/image
Most people think A/B testing is for button colors and SaaS onboarding flows.
TodayTix is using it to figure out what goes on a Broadway poster.
50+ experiments later, they’ve built an entirely new design rulebook showing up from the Toda…
Customer storytext/carousel
When we rebuilt product analytics for 5.0, we started with a question I could not shake. What does anyone actually want from a dashboard?
My honest answer is that almost nobody wants to stare at a graph. A graph is good for spotting a tre…
Product functionalitytext/image
🚨 NEWSLETTER RELEASE 🚨
This edition breaks down how each team built a program where even the losing tests pay for themselves.
Three teams all stopped asking "did we win?" and started asking "what did we learn?"
Luis Trindade at FARFETCH…
Podcast (Experimentation Edge)text/link
How Disney goes from no idea what to test → 110 ranked ideas
Crystal Ammari Iorio runs experimentation at The Walt Disney Company, and her ideation workshop is the cleanest system I've seen for turning "we should test more" into an actual …
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
It's really hard when you really believed in feature, you managed to get it prioritized, you shipped it to production, and then it fails.
If you don't have the right culture around separating a lost test from a bad idea, that becomes a rea…
Thought leadershiptext/videoscroll-stopping: n/a (video)
I talk to a lot of teams who are proud of how much they experiment.
A company you would recognize once told me they were doing it well.
When I asked what that meant, the number was three tests a quarter. Three!
Their product team ship…
Thought leadershiptext/only
Ready for Day 2 of The AI Conference? We are! 🎉
We’re talking with teams shipping AI features faster than ever, but very few feel confident about what happens after they deploy.
If that sounds familiar, stop by the booth to learn how feat…
Marketing event / contenttext/carousel
A couple of years ago, the first AI tool I used was Copilot. We saw 20 to 30 percent improvement in engineering productivity. Real, but incremental.
Now with Claude Code and similar tools, we are managing multiple agents simultaneously. T…
Thought leadershiptext/image
I do not love the word governance. It is boring. Nobody gets excited buying a governance system, and I build one.
But the boring word covers the thing that decides whether an experimentation program survives its own growth.
When a compa…
Product functionalitytext/image
There may be a goldmine of growth hidden in your last experiment.
Most teams read a test as a single number "+0.9, decent, ship it" and move on. But that one average can quietly bury the most valuable thing the test found.
So I'm kicking …
Data sciencetext/document
I once sat in a meeting that ran for an hour debating which shade of green to use for a button.
Nobody could agree, and nobody had data to break the deadlock.
At some point I realised that in the time we had been arguing, I could have b…
Past experiencetext/image
Attention everyone!
The Experimentation Edge newsletter is getting an upgrade, and it's moving to my channel here on LinkedIn.
The format is changing too. Instead of one episode per issue, each article will now pull together a few episode…
Podcast (Experimentation Edge)text/only
There's a difference between a bad result and a bad experiment.
James A. Falzone, Director of Product at Kargo, put it plainly on our show. If you're never failing, are you really trying anything new?
His team hit results that sent them b…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
A 5-pixel change almost cost LinkedIn $10M a year.
A designer adjusted the padding on the top ad banner. No test. Just pushed it live.
It reduced the visibility of one of LinkedIn's highest-performing ad slots. Click-through rate dropped …
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
Governance is important. (At least llama thinks so 😉)
Governance is also a word that can make an entire room quietly check its email.
So naturally, I made a ridiculous llama video about it.
The actual point underneath the llama-shaped cha…
Product functionalitytext/videoscroll-stopping: n/a (video)
Ron Kohavi and Luke Sonnet are hosting a discussion tomorrow about how to decide what to do after you have results in your A/B test. It's free and open to all - hope you can join!
Marketing event / contenttext/only
Many "data-driven" companies are practicing sophisticated guessing.
They have the dashboards. They have the scorecards. The data is there, but it's not actionable. They still can't tell you which feature actually moved revenue, or where t…
Thought leadershiptext/only
All of AI right now is entirely reactive.
You have to know what to ask before it can help you, which is usually the hard part. We think the future of product development and experimentation is more proactive than that. That's what we're bu…
Thought leadershiptext/videoscroll-stopping: n/a (video)
For years I chased my team for experiment write-ups after the fact.
It never worked, and eventually I understood why.
You ask someone to document a test at the exact moment they have the least reason to care. The results are in, that was…
Thought leadershiptext/link
We ran an internal contest at GrowthBook a while back. The setup was simple. Anyone on the team could build an experiment on our own site and see what happened.
I expected our engineers to run away with it. What actually surprised me was …
Product functionalitytext/image
AI has made it ridiculously easy to build and ship features. The problem is that AI doesn't actually solve any of the problems of product development.
It hasn't told you whether that feature is going to work, or whether it did work.
So th…
Thought leadershiptext/videoscroll-stopping: n/a (video)
If you use 3rd party vendor, you are betting your infrastructure on someone else's board meeting.
The recent wave of experimentation acquisitions is a warning.
When a closed-source vendor gets bought, your tooling is now at the whims of t…
Industry commentarytext/videoscroll-stopping: n/a (video)
Win rates across the industry sit around 20%, and I do not think AI is going to improve on that.
A lot of teams have quietly started using AI to examine experiment ideas, or even as synthetic users to simulate experiment results.
On the …
Thought leadershiptext/image
Building new features and not tracking the results?? It's time for an experimentation platform that helps you learn from every new feature.
Using Statsig? Avoid vendor lock-in and go with the open source leader.
Othertext/only
It's Twyman's Law: any figure that looks interesting or surprising is usually wrong.
Here is a perfect example.
A peer-reviewed paper found that rounded corners on buttons outperform square ones by 55%. Designers took note. The reasoning …
Thought leadershiptext/image
How many experiments did you not run this week because setup felt like too much work or costs were too high?
That question sits behind GrowthBook 5.0’s improvements to experiments. There’s no lack of ambition in experimentation teams. Ther…
Product functionalitytext/image
Teams routinely misunderstand averages in experimentation and what those averages are hiding.
I see product teams treat the average effect as the decision. Variant B beat A,
p-value valid, so the product was shipped.
An average is a su…
Data sciencetext/videoscroll-stopping: n/a (video)
If you are an engineering leader at a growth-stage company currently maintaining your own experimentation platform, this is the Dropbox story you should read before your next build-versus-buy review.
Dropbox runs 3 billion feature evaluat…
Customer storytext/image
Congrats to three GrowthBook customers nominated for this year’s Experimentation Culture Awards! 🎉
These teams are setting the standard for modern experimentation programs:
• Khan Academy improved their genAI-powered tutor, increasing ne…
Othertext/link
4 engineers. 360 experiments. One year.
That's Fyxer's growth engineering team, which helped the AI email assistant scale from $1M to $35M ARR last year.
How does a 4-person team run that volume? AI compresses the whole experimentation lo…
Podcast (Experimentation Edge)text/link
I cannot look at a hundred experiments and tell you what they have in common. Nobody can.
That is not a knock on anyone. A hundred experiments is just past the size a human brain holds at once.
For a long time the answer to this was doc…
Product functionalitytext/image
We shipped GrowthBook 4.3 today. It includes a geeky feature called post-stratification which basically helps A/B testers get to trustworthy (statistically significant) experiment results 20% faster. That means companies can ship their wi…
Product functionalitytext/only
A bug in our code accidentally set our paywall to zero.
Revenue jumped 30%.
Usually when you find a bug, you fix it and move on. The number goes back to where it was. Nobody asks questions. Life continues.
We had the data. So instead o…
Past experiencetext/image
PMs ship experiment winners too early.
A metric moves 3%, the test gets called, and the PM moves to the next item on the roadmap. On paper, it looks like a win. In practice, a lot of learning just got left on the table.
When I dig into …
Thought leadershiptext/videoscroll-stopping: n/a (video)
People with smaller palms live longer.
That's a real correlation. Statistically significant. Backed by data.
The good news is, you don't need to bandage your hands!
The real variable is gender. Women have smaller palms on average and liv…
Thought leadershiptext/image
David Massey, MFA just shared his secret success at UPS.
Everyone thought his test was crazy and it would just make customers angry, but he did it anyway.
He stripped every distraction out of the checkout flow once a customer hits shippin…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
AI usage is making A/B testing even more important. In this GrowthBook Office Hours - We’ll cover GrowthBook MCP Server Setup and show you how you can use AI to analyze past experiments and design new ones. Make sure to attend it on Janua…
Marketing event / contenttext/only
Shipping on flat can be so tempting!
"The project is strategic!" or "It's important to the CEO!"
Truly data-driven organizations have the humility to accept that they will test many losers and keep exploring.
Check out the replay. A lin…
Thought leadershiptext/only
How did The Social Hub (B Corp™) cut experimentation costs by 82% while scaling their testing program?
Rudger de Groot of mintminds AG recently shared his success with us in helping The Social Hub expand their experimentation program with…
Customer storytext/image
Every experiment you run produces a piece of knowledge about your users.
Run a few hundred, and you have built something genuinely valuable: a record of what works and what does not for your specific product.
Here is the problem...
A…
Thought leadershiptext/only
Engineer's who ship products without measuring them are making a big mistake.
If you aren't running controlled experiments, you have no way to prove if a 15% latency spike or a churn increase was caused by your latest updates. There are so…
Thought leadershiptext/videoscroll-stopping: n/a (video)
I saw the Amazon report. Four major outages in one week from AI-assisted code changes. They held a mandatory meeting.
Most engineering orgs will not get a warning that visible.
AI coding tools are genuinely impressive. I use them every …
Industry commentarytext/image
You can run a thousand experiments, but there's one big question your business has to understand...
Pedro Tabacof, Principal ML Scientist at Fin, thinks that's the real risk as AI enters the workflow.
AI is genuinely good at the low-level…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
Don’t confuse shipping with winning.
Microsoft figured this out in 2014. For decades, they’d handed out “Ship It” awards. Engineers loved them. Teams competed for them.
Then Satya Nadella canceled them. November 7, 2014. His first year as…
Thought leadershiptext/image
Beyond the Front End Field Notes — Day 5
"I think what I’m trying to say is: it feels really good to see people mean it."
Today’s field note is less about a new concept and more about a culture I’m getting to see up close here at GrowthBo…
Social / culturetext/image
When I talk to product teams lately, a strange thing keeps coming up.
A large share of their traffic is no longer human. One company found roughly 80 percent of its documentation views were from agents, not people.
Think about that if y…
Thought leadershiptext/image
It's entirely possible to run a test, see it win, ship the feature, and still damage your product.
The metric moved. The experiment passed. Yet later you discover cancellation rate went up. Customer support volume went up. You just weren't…
Thought leadershiptext/videoscroll-stopping: n/a (video)
At education.com, we ran thousands of experiments. Roughly a third won. The ones that lost taught us more than the ones that won, and it took me longer than I would like to admit to see the value in those tests.
A losing experiment tells …
Past experiencetext/image
We made it to AI Engineer World's Fair, we’re here at booth S26.
Here's what we're showing all week: how to ship AI features fast without the 2AM rollback panic. Every release goes out behind a feature flag, ramps on a schedule, and pulls …
Marketing event / contenttext/image
CS degree enrollment is dropping, and I do not blame the students.
If I were eighteen right now, watching what AI is doing to software engineering, I am not sure I would bet four years and a tuition bill on a career that might not exist i…
Industry commentarytext/videoscroll-stopping: n/a (video)
Angular v21 is packed with goodness — Signals-powered forms, the brand-new Angular ARIA primitives, tooling improvements, and a roadmap that honestly has me excited for what’s coming next.
I pulled all my favorite v21 features into a quick…
Othertext/link
Congratulations to Khan Academy, Kelli H., Tushar Udeshi and the entire Khanmigo team. Khan Academy was named a Learning Organization Finalist for this year's Experimentation Culture Awards!
Their case study covers how they built an exper…
Othertext/image
B2C: "It's black, it fits my apartment, it makes hot water for my tea. I love it."
B2B: "We need to supply a factory with water and serve a thousand people across multiple offices."
Fabian Hans, Founder at COGNITEER, uses that contrast to…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
How do you A/B test something that has 300 million monthly players constantly clicking?
The new edition of The Experimentation Edge newsletter rounds up my conversations with Shan Huang, data scientist on Supercell's central experimentatio…
Podcast (Experimentation Edge)text/link
JobLeads grew its testing velocity 10x in under 2 years. Clover has been well established for years.
So how does a growing company and a mature company look the same when it comes to testing?
The new edition of The Experimentation Edge ne…
Podcast (Experimentation Edge)text/link
Running hundreds of experiments at a time is the dream of many teams, but it makes patterns harder to spot. Past a certain point, almost no one is reading and analyzing every result.
With Learnings in GrowthBook, you can use AI to find pa…
Product demotext/videoscroll-stopping: n/a (video)
On the latest episode of The Experimentation Edge, Vijay L. made the case for how to improve your product, here's what he said...
Your win rate will never be close to 100%, so the goal isn't avoiding failure.
It's failing fast, extracting…
Podcast (Experimentation Edge)text/image
Nobody should bench press 300 pounds their first day in the gym.
The reason is obvious. Yet I see companies do it with experimentation all the time. They decide to run thousands of tests, across the whole organization, before anyone has bu…
Thought leadershiptext/videoscroll-stopping: n/a (video)
Most engineering teams using feature flags treat them as circuit breakers. Something goes wrong in production, flip the flag, turn it off.
That is genuinely valuable, and most engineering teams think about flags exactly this way.
What I…
Thought leadershiptext/videoscroll-stopping: n/a (video)
Everyone celebrates the winning test, but the losing ones are where the real money hides.
I sat down with Geoffrey Bell, who has run experiments at Lowe's Companies, Inc., Microsoft, and now Ford Credit, and we talked about why his failed …
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
Our GrowthBook 5.1 Office Hours recording is up, here is my recap version but check out the playlist for the whole enchilada!
Every segment is the engineer who built the feature walking through it at their own pace. No clock, no "we'll get…
Product functionalitytext/videoscroll-stopping: n/a (video)
The return you report from experimentation is higher than the return you actually got.
This is the part nobody enjoys hearing, and it is worth sitting with, because most programs are sending the wrong number upward every quarter.
You run …
Data sciencetext/image
5.0 Launch Week Day 3️⃣: Product Analytics GA
If you're already running experiments in GrowthBook, you've done the hard part. Your warehouse is connected and your metrics are defined.
Now those same definitions power dashboards, funnels,…
Product functionalitytext/image
GrowthBook 4.3 is here! It’s time for faster experiments & deeper insights.
We're excited to announce post-stratification for experiment analysis. Combined with CUPED, this variance reduction technique lets you run experiments 20%+ faster.…
Product functionalitytext/carousel
We had a dashboard for every metric at my last company. What we did not have was a way to answer the question that followed every spike: what actually caused this.
There is a certain catharsis that comes from looking at a graph. The numbe…
Past experiencetext/image
That's a wrap on Product-Led Summit.
Teams are shipping faster than ever with agents writing the code, but few have changed how they measure what’s actually working.
We’re here to change that, if we didn’t get to meet with you, DM us and …
Marketing event / contenttext/carousel
Building a culture of experimentation is about creating a culture of humility, that learns together and shares wins together.
Here is one small example from @grahammcnicoll about how GrowthBook helps: GrowthBook auto-generates a slide de…
Product functionalitytext/videoscroll-stopping: n/a (video)
I spoke to an engineering manager last week whose team still tracks sprint progress in story points.
In an AI-assisted world, story points matter less than ever. They were never that useful to begin with.
They are a proxy metric standin…
Thought leadershiptext/videoscroll-stopping: n/a (video)
From last week's podcast episodes I learned a common lesson:
A result only makes sense once you understand the customer's journey
The new edition of The Experimentation Edge newsletter rounds up my conversations with Joe Yevoli of Charlie …
Podcast (Experimentation Edge)text/link
Token costs are exploding across our industry.
Every week there is another story about cost overruns, and the natural reaction from leadership is to start handing out token budgets.
I actually believe you should give your teams almost i…
Past experiencetext/only
Two things slow down every experiment: Waiting for significance and then figuring out what the results actually mean. GrowthBook 4.3 tackles both:
👉 Post-stratification gets you to answers faster
👉 New drilldowns help you understand what…
Marketing event / contenttext/only
Most product teams I talk to know how to run an experiment. They have the tooling, a primary metric, and a process for calling a result.
What I see less of is a clear answer to: what are we not measuring?
A primary metric tells you whet…
Thought leadershiptext/image
From my experience, experimentation programs don't get killed off in a meeting because people decide one day that they don't believe in A/B testing...
It's almost always that die from starvation.
Teams just stop running tests and leaders…
Thought leadershiptext/videoscroll-stopping: n/a (video)
Experimentation unlocks learning you can't get any other way. These learnings often surface as good or bad surprises. Either way, it's important to act fast and use that learning to inform new features and new experiments. It can unlock …
Thought leadershiptext/only
I used the angular MCP server to validate that I was using their suggested best practices in my project yesterday. I feel like this is a really cool use of AI. I can get so overwhelmed by going through a checklist, whereas AI is very method…
Othertext/link
Token costs are exploding across our industry. Here's how we approach it.
I definitely believe you should give your teams almost infinite token budgets, because token usage correlates very highly to productivity. Or at least it should.
Yo…
Thought leadershiptext/videoscroll-stopping: n/a (video)
The bottleneck used to be shipping. Now it's knowing what to keep.
AI is putting more code out the door faster than most teams can evaluate it. More changes in production means more variables, more surface area, and less certainty about wh…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
If your team already has feature flags wired into production, you can run painted door tests on every speculative feature on the roadmap in minutes. Most of you have not realized how easy it can be.
Most teams I talk to have heard of a pa…
Thought leadershiptext/image
Uncovering new understandings of user behavior can unlock new sales or huge savings opportunities. In our 4.2 webinar, we'll show how product analytics can help you identify new opportunities and then experimentation will prove whether you…
Marketing event / contenttext/only
Feature flag governance can sound like adding more rules, more approvals, and more chances for someone to say “not yet.”
That is not the goal.
As AI increases the speed and volume of software delivery, teams need better ways to catch mist…
Product functionalitytext/image
Software teams get measured on delivery speed. It's an easy thing to track, it fits neatly into a sprint review, and it makes the leaders feel like things are moving.
The problem is that you can move very fast in the wrong direction.
I …
Thought leadershiptext/image
What if your AI coding tool could run your entire experiment workflow?
Luke Sonnet (Head of Experimentation at GrowthBook) walks through the end-to-end experiment lifecycle using Claude Code and GrowthBook.
Watch him execute every step w…
Product demotext/videoscroll-stopping: n/a (video)
I have sat in spec reviews where the Jira tickets were written, the designs were finished, the user flows were mapped end to end, and the full spec sheet described a product from beginning to end. Nobody in the room had asked the only quest…
Thought leadershiptext/image
Andrew Willingham's career advice for product managers: stop being right.
Stop walking in with the answer.
Stop saying "here's what we'll build."
Start saying "here's my plan for testing, and the confidence level I'll bet on."
He calls …
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
Faster experiments, deeper insights. GrowthBook 4.3 has arrived! 🎉
This release brings major upgrades to your experimentation workflow, including Post-Stratification for more precise results and powerful new Metric Drilldowns. Plus, we've …
Product functionalitytext/document
If your product team has lots of data, but still can't see where the biggest opportunities are to grow your business - This webinar is for you!
Less than 30% of all new features deliver the expected business results. Worse yet, 30% or mor…
Marketing event / contenttext/only
This is an excellent case study on the power of experimentation! Kameron T. and team are doing an impressive job of using AI to iterate faster. More importantly, they aren't just throwing features at the wall to see what sticks. They are…
Customer storytext/only
A mistake where over 1,000,000 people could see is terrifying...
So how do you avoid a catastrophe even if a mistake does happen?
The new edition of The Experimentation Edge newsletter features Vijay L., a lead product manager running exp…
Podcast (Experimentation Edge)text/link
Join us for a live demo and deep dive into the latest release from GrowthBook. We've added Product Analytics to our Feature Management and Experimentation Platform. A single platform that connects to all your data means new insights with le…
Marketing event / contenttext/only
We make it stupidly easy to run experiments. Whether you're a back-end engineer, a front-end engineer, or even a product manager.
Run it, get the results, the analysis, and show the results of something you built.
I think people on your t…
Product functionalitytext/videoscroll-stopping: n/a (video)
$180 million in one year. Half the story is what they did not ship.
Gaurav Sethi (PayPal) on how his experimentation platform measured its 2025 impact:
Revenue from the winning experiments, plus the cost avoided by killing features that fai…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
A slide made it all the way to a VP all-hands at Microsoft.
It said: "New users who use our new feature are half as likely to churn compared to new users who don't use it in the next 30 days."
The conclusion: the feature reduces churn and…
Thought leadershiptext/image
GrowthBook 4.1: Holdouts, dashboards, and AI that's not only unbelievably useful but customizable, too. Scroll through for all the highlights included in this release!
Product functionalitytext/document
Hiring has always been the most critical part of building a company, and it takes a thoughtful process and a lot of time.
Hiring for engineers, in particular, has fundamentally changed in the last year.
You may notice the effects now w…
Thought leadershiptext/image
I see so many software teams launch massive features without actually defining what success looks like beforehand.
They pull up a BI tool after the fact and try to find a metric that moved in the right direction to justify the engineering …
Thought leadershiptext/videoscroll-stopping: n/a (video)
5.0 Launch Week Day 2️⃣: AI Visual Editor
Growth and marketing teams run into the same problem: plenty of ideas, not enough engineering time to test them all.
Our AI Visual Editor fixes that. Describe the change you want in plain English …
Product functionalitytext/image
I cannot look at 100 experiments and tell you what they have in common.
That is the problem an experimentation program hits as it scales.
The number of tests easily moves past what one person can hold in their head, so the conclusions and…
Product functionalitytext/videoscroll-stopping: n/a (video)
It's countdown time! 🚀 Join us this Thursday for a live walkthrough.
GrowthBook 4.3 is one of our biggest releases —
⌛️ New variance reduction to reach significance faster
📊 Metric drilldowns to actually understand your results
🐞 Feature …
Marketing event / contenttext/only
Khan Academy gave their AI tutor a calculator to improve math accuracy. It worked, but it made responses painfully slow for students.
So they ran five sequential A/B tests:
✅ Removed the calculator (math errors doubled)
✅ Switched to G…
Marketing event / contenttext/link
When an experiment fails, there should be a discussion about if the idea was bad or if it was just the implementation that was bad or even if it was a bad test. Sometimes this turns into an argument.
Sorting through all the possible ways t…
Thought leadershiptext/image
There's always been a tension between marketing teams and engineering teams.
Engineering doesn't really want to work on marketing projects, and marketing teams just have to struggle for resources.
It's something we've tried to solve in G…
Product functionalitytext/videoscroll-stopping: n/a (video)
Intuition is the most dangerous tool in gen AI. It could have cost Khan Academy their AI tutor's math accuracy.
Disabling their AI tutor's math verification tool seemed reasonable. Latency was a real problem and the model had gotten better…
Marketing event / contenttext/videoscroll-stopping: n/a (video)
Building a feature now costs almost nothing, which turns out to be the problem.
When you can build everything, doing everything stops being an advantage.
A Swiss Army knife confuses users, probably does nothing especially well, and is ful…
Thought leadershiptext/videoscroll-stopping: n/a (video)
Many organizations are optimizing perfectly—for the wrong thing. Dark patterns rarely start with bad intent. They emerge when success is narrowly defined, and learning is replaced by short-term optimization.
The risk isn’t just user trust.…
Thought leadershiptext/only
A lot of engineering teams have feature flags to control the release of features and act as a circuit breaker. If something goes wrong, they just turn it off.
The frustrating part is that the natural next question should be: okay, if it pa…
Thought leadershiptext/videoscroll-stopping: n/a (video)
I have watched a deploy take down products complete because there was no flag in place to stop it.
The team spent hours rolling back while users sat on a broken experience.
Most teams I talk to are still shipping features directly to pr…
Thought leadershiptext/image
The one part of experimentation AI won't take over for a long time.
Michael Mathews Jr (FanDuel) on what humans still own:
The hypothesis and the goal.
AI could do a decent job, but it's hard to hand an agent the full picture of the busines…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
Early access to GrowthBook's ✨NEW AI Visual Editor✨ launches Monday, June 22.
Cloud accounts (Free, Pro, and Enterprise) can join the early access list. This enables teams to:
• Empower non-technical users to create website variants in mi…
Product functionalitytext/image
Every experimentation leader eventually runs into the result nobody expected.
A counterintuitive result lands, the team does not like what it says, and the instinct kicks in. Blame the platform. Blame the data. Rerun it until it says some…
Thought leadershiptext/videoscroll-stopping: n/a (video)
Most experimentation programs start the same way. A few A/B tests.
Some promising results. Then the hard part begins: scaling it across teams.
Adding AI features and software takes scaling to a new level: rapid iterations, experiment vel…
Marketing event / contenttext/only
Fifteen research interviews said one thing. The concept test said another.
Michal Lenik (Grubhub) on a new merchant promotions experience:
The team expected merchants to want way more metrics and information.
Merchants chose the simpler opt…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
A bug in our codebase accidentally set our paywall to zero. As a side effect, revenue jumped 30%.
Usually when you find a bug, you fix it and move on.
Because we had the data, we were able to interrogate what was actually happening. That …
Past experiencetext/videoscroll-stopping: n/a (video)
We fed 80+ experiments into AI. We learned what users actually want—then built and tested a new feature in minutes!
GrowthBook (free) + GrowthBook MCP Server + Cursor IDE
Watch Ryan Feigenbaum, PhD on how GrowthBook's MCP Server transform…
Product functionalitytext/image
There's a level of excitement teams get when the product just works and they're not fighting their internal tooling all the time.
We see it a lot when companies bring in GrowthBook to replace a previous platform or a home-built one.
We've…
Product functionalitytext/videoscroll-stopping: n/a (video)
How to shift your PM mindset to think more like an executive.
That's the latest edition of the newsletter, featuring Andrew Willingham Head of People and Legal Products at Atlassian.
Andrew spent 11 years at Amazon learning that the exper…
Podcast (Experimentation Edge)text/link
New edition of The Experimentation Edge: Chess.com runs 400 experiments a year for a user base that spans people who don't know how pieces move and FIDE-rated competitors. Here's what that skill gap forces them to get right.
Nafis Shaikh (…
Podcast (Experimentation Edge)text/link
Most product teams I talk to struggle to analyze the true impact of any feature they ship, even though they have dashboards.
A graph tells you something moved. It does not tell you what caused it, or whether your last release had anything…
Thought leadershiptext/videoscroll-stopping: n/a (video)
It’s one thing to run an experiment.
It’s another to operationalize experimentation at scale.
Lingokids doubled experimentation velocity with GrowthBook and drove a 7% conversion lift, without release bottlenecks.
(Case study link in com…
Customer storytext/carousel
Excited to learn when to iterate and when to obliterate! We'll be raffling off 3 tickets to Ronny's next webinar to live attendees. So be sure to watch live!
Marketing event / contenttext/only
High activity doesn't mean high quality. Sometimes it means the opposite.
A user who refreshes five times, frantically changes their search terms, or keeps clicking around isn't highly engaged; they're stuck. That's frustration wearing the…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
It's not often that you can get world-class experimentation tools AND save 82% versus the competition!
Customer storytext/only
4.3 includes some great new features, like Daily Average Usage tracking, which is key for AI apps and measuring engagement. Post stratification will help you finish experiments 20% faster. Finally, feature flag evaluation diagnostics make…
Marketing event / contenttext/only
Friends don't let their product managers use bad reality distortion.
Steve Jobs had a famous reality distortion field — a rare gift for convincing people they could do things they didn't believe were possible. It gave us the iPhone and the…
Thought leadershiptext/image
If you could onlypick one for your experimentation program, which would you rather have?
1. High experiment velocity (swinging at everything)
2. Perfect statistical rigor (very low false positive)
Curious where you land--drop your pick a…
Thought leadershiptext/only
Running 300 million feature flag evaluations per second sounds like an engineering flex.
It's not. It's what it takes to actually know if your features work.
Ashley Stirrup asked Ilya Izrailevsky at DoorDash how they think about experimen…
Podcast (Experimentation Edge)text/link
Rudger de Groot and Mintminds have been working with Growthbook for 3 years. If you are going to Conversion Hotel (#CH2025), I highly recommend talking to Rudger to learn more about GrowthBook.
Marketing event / contenttext/only
Thank you to our Gold Sponsor, GrowthBook!
GrowthBook is the leading warehouse-native experimentation, feature flag, and product analytics platform for AI-native product teams. From the largest websites to the AI teams defining what’s nex…
Marketing event / contenttext/image
Amazon got a mandatory meeting. Most engineering orgs will get a quieter version of the same problem first.
Four major outages in a single week. Reporting on an internal memo described a pattern of unsafe practices: AI-generated code reac…
Industry commentarytext/image
This week on Angular Air 👀
Manfred Steyer joins us to explore agentic UIs in Angular — and he’s using Hashbrown to show what this looks like in a real application.
This isn’t just about chat interfaces — it’s about AI actively shaping the…
Othertext/link
A two-point lift in close rate sounds like a rounding error. At Ford Credit, it's the number that turned skeptics into advocates.
I sat down with Geoffrey Bell and we talked about how connecting an online test to an offline car sale change…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
If you have two ranking algorithms and not enough traffic to tell them apart, an A/B test won't settle it. You'll sit on an inconclusive result for weeks.
Interleaving is the way around that. Both algorithms feed into the same result set …
Marketing event / contenttext/only
Is 1 year to choose an experimentation tool overkill? Khan Academy says no.
Dr. Kelli Hill, Head of Data at Khan Academy, shared how they thought about it. After 2.5 years rebuilding their entire backend and porting a million lines of code…
Customer storytext/videoscroll-stopping: n/a (video)
Every booth at The AI Conference is going to tell you to move at AI speed. None of them will tell you whether the feature you shipped actually grew your business.
Shipping at AI speed is solved. Learning at AI speed is not. Flag the AI fea…
Marketing event / contenttext/image
I see experimentation leads run two or three tests a month and feel like they have figured this out. They haven't.
Running an experiment and running an experimentation program are two different jobs.
The first thing I see that breaks is…
Thought leadershiptext/videoscroll-stopping: n/a (video)
One of the things we tell a lot of our customers is that it's the number of at-bats that will ultimately lead to your success as an experimentation-driven company.
Hack weeks tie into that, because it encourages your team to think creative…
Thought leadershiptext/videoscroll-stopping: n/a (video)
Product teams obsess over whether they shipped on time. It's a reasonable thing to track. It's just not a very useful one.
I talk to engineering leaders every week who can tell me exactly how many story points their team hit last sprint. …
Thought leadershiptext/image
Anyone can run an A/B test. Typeform decided that wasn't enough.
On the latest Experimentation Edge, I talked with Aleks Bass, Chief Product & Technology Officer at Typeform, about what it actually takes to build experimentation culture.
…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
Most experimentation programs are confidently wrong - unless they have a plan.
The webinar "Designing Experiments for Long-Term Growth" with Ron Kohavi and Luke Sonnet was a hit yesterday.
My 4 takeaways:
Naive metrics can point you in …
Marketing event / contenttext/image
Marketing, product, and the newsroom at the Philadelphia Inquirer all collaboratively run experiments under one set of standards. That's rare. Most orgs get one team testing well and never get past it.
The hard part of scaling experimentat…
Marketing event / contenttext/image
One team runs roughly tests a month, the other ran 1,500 a year. Both stopped keeping score on wins, here's why...
The new edition of The Experimentation Edge newsletter rounds up my conversations with Will Guyeskey of GoPro and Wanli Lau …
Podcast (Experimentation Edge)text/link
Designers and engineers get attached to their work. It makes sense; you spent real effort on it.
When someone rewrites it or cuts it, it can feel like a reflection on you.
With AI, this is happening more often. I am regularly in the mid…
Thought leadershiptext/image
Huge congratulations to everyone recognized in the Experimentation Culture Awards! 🎉
It's great to see the experimentation community continue to grow, and to see so many people sharing what they've learned to help others build better produ…
Othertext/only
UPS's Journey Experience and Design Innovation (nicknamed J.E.D.I.) team generated $500M+ in incremental revenue from digital experimentation. Now the hardest problem is demand, not results.
David Massey, MFA didn't plan to build UPS's exp…
Podcast (Experimentation Edge)text/link
We're just going to go ahead and say it: GrowthBook is the most comprehensive open-source experimentation platform.
Now with #FerretDB support.
Because open source means ALL the way down.
Learn more: https://lnkd.in/gGhBJMSW
Product functionalitytext/link
I spoke to an engineering lead last week whose team still tracks progress in story points. In a world where AI is doing most of the execution, what does a story point even measure anymore?
Honestly, I am not sure they ever measured much t…
Thought leadershiptext/image
A Hacker News thread a while ago argued that A/B testing is unethical. I believe their framing was wrong, but I am curious what the product leaders on my feed think.
The objection was around being included in an A/B test without consent. …
Thought leadershiptext/image
A test without a specific falsifiable hypothesis is just a fishing expedition. Priya Singhee of Early Warning on The Experimentation Edge: the question is never "should we test" but "can you test this validly?" Listen to the full episode of…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
Think about how many layers sit between you and an actual user.
User research tells you what people want. Personas abstract real user segments. Dashboards average behavior. And, increasingly, evals where one model grades the output of ano…
Thought leadershiptext/only
"We decided it was a better use of our time to get teams to run experiments, as opposed to continuing to build a thing that would allow them to run experiments."
That's how Mikhail Popov, data scientist at the Wikimedia Foundation, explain…
Customer storytext/image
AI coding tools have made shipping features the easy part. The hard part is releasing safely and knowing you’re moving the right metrics.
GrowthBook handles that part. Every feature can ship behind a flag, roll back automatically if a gua…
Marketing event / contenttext/image
Small changes to a user interface can have a huge impact on conversion rates and revenue. But, only 2 out of 10 features have a positive impact. That's why A/B testing is so important.
Check out the full webinar with Ron Kohavi and Luk…
Marketing event / contenttext/only
This fantastic comic (xkcd 2899) is about which law?
Data sciencetext/image
A designer at LinkedIn removed 5 pixels from an ad banner. No big deal, right?
Revenue dropped by a million dollars in a month.
Makram Mansour, former experimentation leader at LinkedIn and Intuit and now Head of Marketplace at ID.me, mad…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
Khan Academy built one of the most thoughtful experimentation programs in education. Privacy-first, warehouse-native, and now extending to AI with Khanmigo.
Kelli H. breaks it all down live on April 16 with GrowthBook.
Case study + webin…
Marketing event / contenttext/carousel
AI makes shipping new features easy. But knowing _what_ to ship is still hard. If making better use of AI is on your new year's resolution list, don't miss our next GrowthBook Office Hours with Ryan Feigenbaum, PhD and Ashley Stirrup.
We'…
Marketing event / contenttext/only
Claude Code turned weeks of data analysis into a few hours.
Raunak Kumar at Fin (formerly Intercom), is clear about what that actually means.
On the show, the Senior Manager of GTM Analytics said the speed isn't the point. If the work is …
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
Amazon's team was confident in their data.
Then someone noticed Oprah had called Kindle her favorite thing.
Between the two test periods — Website A and Website B — Oprah had told her national TV audience that the Amazon Kindle was her new…
Thought leadershiptext/document
We're thrilled to have Kelli Hill, PhD from Khan Academy speaking at Experimentation Island.
Kelli's focus on building a thoughtful, responsible experimentation culture is what truly makes their story a must-hear.
It’s a compelling look a…
Marketing event / contenttext/only
When teams like marketing and product test in separate silos, they miss the experiments that only make sense together.
The The Philadelphia Inquirer fixed that. Marketing, product, and the newsroom now test under one set of standards, and …
Marketing event / contenttext/image
We gave the GrowthBook team a simple challenge:
Use the Visual Editor to redesign our homepage. Make it impressive, funny, completely over the top, or all three. 🏆
Elyse Fox won with a Jurassic Park homepage: a dinosaur getting rolled b…
Past experiencetext/carousel
Beyond the Front End Field Notes — Day 3
Yesterday I learned about ramp schedules.
A ramp schedule is a way to roll something out gradually instead of flipping the big scary launch switch all at once.
Instead of going from 0% to 100%, yo…
Product functionalitytext/image
As a company, we do things companies do:
• Run A/B tests (obviously)
• SEO
• And now, GEO - optimizing for visibility in AI searches like ChatGPT
What's GEO (Generative Engine Optimization)? We were wondering that too, so we did som…
Thought leadershiptext/link
A "winning" test can still be wrong.
If your team runs experiments at any scale, you’ll want to join our webinar on July 16.
Ron Kohavi and Luke Sonnet will dive into the experiment pitfalls that make results look better than they are: S…
Marketing event / contenttext/image
Most teams ship a feature and move on.
Kameron Tanseli at @fyxerofficial does the opposite.
His approach: ship the core experience, then immediately start running experiments to drive adoption.
Because here's the uncomfortable truth abou…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
This week we've been sharing everything included in our latest 4.1 release, which includes:
→ 150+ commits
→ 8 major features (holdouts, dashboards, AI, Vercel Native, Pre-computed dimensions, MCP update, FerretDB, Sanity)
→ 0 vendor lo…
Product functionalitytext/link
Your PM: "What happened with that experiment?"
You: Opens 17 tabs, 3 spreadsheets, and a Slack thread
Sound familiar? We fixed that.
Introducing Experiment Dashboards in GrowthBook 4.1:
✅ Create custom views for each experiment
✅ Highligh…
Product functionalitytext/image
What a lineup 🔥 Interleaving, agents, and a panel we’ve been looking forward to all month. Thanks to the Berlin Experimentation Meetup for having us!
See you at Zalando HQ on the 15th!
Marketing event / contenttext/only
Most teams kill a failed experiment and never look at it again.
Kameron Tanseli at @fyxerofficial retests them — and keeps winning.
Here's why it works:
Your product changes. Your customers change. The competitive environment changes.
A…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
Beyond the Front End Field Notes — Day 2
Yesterday I learned that a bandit is not a variant.
The variant is the thing users see.
The bandit is the way users are dynamically routed between variants based on performance signals.
A traditio…
Data sciencetext/image
MeasureCamp Chicago is a free unconference. Sessions are submitted the morning of, so the day covers whatever the room actually wants to talk about
We’ll be there September 19. Stop by our table to talk about running experiments on top of …
Marketing event / contenttext/image
Most experimentation programs stay stuck with one team, which caps how far impact can go.
The Philadelphia Inquirer built one that spans across the whole org. Marketing, product, and the newsroom all run tests under the same standards, an…
Marketing event / contenttext/image
At #JSNationUS and interested in feature flagging and experimentation (and, why wouldn't you be?). Come find Ryan to talk about it!
Marketing event / contenttext/only
Average tenure for product developers is about two years, which puts everything your team has learned on a timer.
The old joke is that Silicon Valley is one big company and people just move departments. It is funny because it is close to t…
Thought leadershiptext/image
Most product teams build features the expensive way.
Someone has the idea. The team specs the whole thing. Engineering builds it.
It ships. Then you find out whether users actually wanted it.
By the time you have the answer, you have a…
Thought leadershiptext/videoscroll-stopping: n/a (video)
The research said it couldn't miss: 50% returning customers, the same order every week, users multi-tabbing three delivery sites. So the team cut the ordering flow from 22 inputs to 5. Edd Saunders of JobLeads tells the story of the 'sure t…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
AI Evals tell you if the model works. A/B tests tell you if users care.
These answer different questions. Evals measure competence—can it do the job? A/B tests measure value—do users actually behave differently?
DoorDash saw a model that …
Thought leadershiptext/link
A small change to type ahead search drove double digit gains in add to cart.
Dorothy Crepin (MilliporeSigma) on the test:
The old autocomplete pushed a matching product and risked getting it wrong.
The new version suggests a search term ins…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
AI didn’t just change how we write code – it changed how we build products. 👷
Join Alyssa Nicoll, Ed Charbeneau and Kathryn Grayson Nanz for our very first Discord Developer Roundtable to unpack everything dev tools in the age of AI!
🗓️ W…
Marketing event / contenttext/image
When you deliver flowers and gifts across 9 countries, not everyone behaves the same way. Why should your testing assume the same?
Find out how the Floward data science team helped drive double-digit revenue growth through experimentation …
Customer storytext/carousel
The Home Depot does $150 billion a year as an enterprise.
The online org alone just passed $25 billion, and this has a lot to do with Kim Ting Li, Senior Manager of Experimentation at The Home Depot.
Most people have no idea the digital b…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
"If a test is safe enough to let AI run the whole thing, why run it at all?"
That line from Lina N., Head of Product Analytics at Squarespace, reframes the AI conversation.
The reflex is to hand AI the low-risk tests: write the brief, run…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
If your team is shipping AI and algorithmic changes without an A/B test, you're flying blind.
Makram Mansour, former experimentation leader at LinkedIn and Intuit and now Head of Marketplace at ID.me, learned this firsthand when an algorit…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
If your finance team is currently drafting a memo about capping engineering AI spend, I am going to save you a quarter of productivity loss before you send it.
Token costs are exploding, and the reflex from finance is predictable.
Ask th…
Thought leadershiptext/image
Engineers as the champions of experimentation? Rich Liebling of Learneo on the culture at Shop It To Me where every idea was judged on merit and a new hire's second merge request shipped their own A/B test. Listen to the full episode of The…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
Most companies run experiments in a silo.
The growth team knows what's working. Nobody else does.
When Kameron Tanseli joined @fyxerofficial, one of his first moves was to change that completely.
He created a public Slack channel — #topi…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
It’s been so fun getting to know the #React team and seeing the other side of web dev while working backstage @reactconf in Vegas 🥰
Marketing event / contenttext/image
If you have ever seen a kid use an iPad, you have watched a perfect UX.
A child seeing it literally for the first time can push and zoom and scroll. Apple nailed the interface so completely that someone with zero context just gets it.
I…
Thought leadershiptext/image
Khan Academy spent 3 years defining one metric. Because most AI teams have no idea what "good" output actually is.
Dr. Kelli Hill, Head of Data at Khan Academy, shared how her team built cognitive engagement as the measure behind every Kha…
Marketing event / contenttext/videoscroll-stopping: n/a (video)
Last year, Fyxer ran 541 experiments and grew from $1M to $35M in ARR. 🤯
They treated every product change as a hypothesis to validate. Accepted the failure rate. Found the winners. Moved the product in the right direction and cut the los…
Marketing event / contenttext/link
Your roadmap shows what will ship. It doesn’t show what will work.
We're at booth 22 at Product-Led Summit, September 22 and 23 in San Francisco. Come chat about knowing what works and correcting course fast ➡️ https://buff.ly/EsFEpQH
Marketing event / contenttext/image
After hundreds of experiments, what does your team actually know?
They probably don’t remember each variant that won. What they learned is that showing pricing earlier in the funnel lifts conversion or that simplified onboarding helps new …
Product functionalitytext/image
UPS added a required email field to their shipping checkout.
The outcome? Not good...
Conversion dropped so sharply they pulled the test after 24 hours, the shortest experiment the team has ever run.
Two years later, the international shi…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
One week until AI Engineer World's Fair.
We'll be at booth S26 all four days, June 29 to July 2 at Moscone West. Stop by to see feature flags and auto-rollbacks in action or just talk shop about shipping AI features safely and at scale.
W…
Marketing event / contenttext/image
This is a great way for any e-commerce experimenter to learn and work with industry leaders while helping to grow your business. Work on features that have won for other businesses and could grow your revenue by as much as 7%.
Interested?…
Marketing event / contenttext/only
Angular has had Template Forms and Reactive Forms for years.
Both work well… but neither is quite perfect.
When the Angular team introduced Signals, they realized form management needed to evolve too.
Enter Signal Forms.
🎥 Quick explana…
Othertext/link
Beyond the Front End Field Notes — Day 6
Today’s field note is about CUPED.
"Which leaves me wondering... should all experiments be cuped?? Why wouldn't you?"
CUPED reduces statistical noise by using relevant pre-experiment behavior as c…
Data sciencetext/image
This was such a fun episode I did with @edcharbeneau! I learned a lot about using planning mode when building out apps in @cursor_ai. Check out the full recording here https://lnkd.in/g48kV8bj 🫶
Othertext/videoscroll-stopping: n/a (video)
I love when my dev tools start talking to each other.
Not in a scary “the toaster has opinions” way. (If my air fryer had thoughts, I don't think we'd want those floating around.)
More like:
“Hey Cursor, can you create a feature flag for …
Product functionalitytext/image
Sanity offers a powerful content management solution that effortlessly integrates with GrowthBook, enabling you to conduct experiments directly from your content editor without altering any code. This approach allows you to benefit from fea…
Product functionalitytext/link
Ever wonder if your feature actually moved the needle, or if that metric bump was just seasonal variance?
Most teams struggle with this because they can't isolate the real impact of their changes.
That's where holdouts come in - and why …
Product functionalitytext/link
The strongest experimentation cultures I've seen are built on ceremonies.
Luis Trindade, Principal Product Manager of Experimentation at FARFETCH, walked me through his team's rhythm on The Experimentation Edge, and it's some of the best o…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
A C-suite exec walked in and said "we need video chat."
Amazon Fire had just launched theirs. The commercials were everywhere.
Crystal Ammari Iorio, Lead of Digital Product Optimization at The Walt Disney Company, shared how her team handl…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
One week out from Product-Led Summit, and we’re asking you to bring us your worst launch story. 🫣
You know the one. The dashboard moved, everyone had a theory, and nobody could prove a thing. We'll show you what it looks like when a PM ch…
Marketing event / contenttext/image
Most teams say they experiment. Few make it a daily habit.
Oda did it by ditching their hard-coded homegrown setup and plugging GrowthBook into the Snowflake stack they already had. No new infrastructure. No vendor lock-in. Just experiment…
Customer storytext/carousel
Experimentation drives ~8% of annual growth at Fanatics.
On a $3B business, that's hundreds of millions every year.
I asked Medha U., VP of Growth and Experimentation at Fanatics, how they got there. Her answer wasn't about tools. It was …
Podcast (Experimentation Edge)text/link
We'll be at hanging out at booth 206, talking feature flags, experimentation, and product analytics for teams shipping AI to production.
Come say hi 👋 and grab some swag 😎
Marketing event / contenttext/image
User research that once took three weeks now takes about two hours.
Daniel Layfield at Diligent has lived both versions.
Daniel described the old way at Codecademy: find a hundred users, email every one, spend weeks getting them on the pho…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
Most companies run experiments in a silo.
The growth team knows what's working. The product team doesn't. The founders are guessing.
Fyxer solved this with one Slack channel.
From day one, Kameron Tanseli created #topic-experiments — a p…
Customer storytext/videoscroll-stopping: n/a (video)
Most growth teams are waiting until they're "big enough" to experiment.
They're falling further behind every week.
Kameron Tanseli built his career on proving this wrong.
When he joined Fyxer, his instincts were calibrated to B2C health …
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
8+ out of 10 features your team ships will fail.
That's not pessimism — it's data. The median organization has an experiment success rate of around 10–15%.
Most product teams haven't fully accepted this. So they keep celebrating "code com…
Marketing event / contenttext/videoscroll-stopping: n/a (video)
Back to the Basics continues — and this week we’re tackling one of the most misunderstood topics in modern Angular:
SSR, hydration, and partial hydration.
I’m joined by Adam Plumer to break down what these terms actually mean, how they wo…
Othertext/link
Are you shipping AI features safely?
Most teams can ship fast now. Fewer can tell you what happens after the deploy, which features moved their metrics in the right direction, or which one tanked a metric last Tuesday.
That's what we'll b…
Marketing event / contenttext/image
A 300% attach rate, and the test still lost money. Whitney Perez of Realtor.com breaks down the checkout bundling experiment that taught her team to set secondary metrics and their thresholds before the victory lap. Listen to the full episo…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
GrowthBook 5.0 was one of our biggest releases ever!
Missed the launch? You can watch our Release Office Hours in the Product Hunt post for a tour of everything that’s new.
If you’re excited about where GrowthBook is heading with integrated…
Product functionalitytext/link
What does full-cycle reactivity in Angular actually look like?
Tomorrow on Angular Air, Manfred Steyer will walk us through Signal Forms, Signal Store, Resources, and the Mutation API — and how it all fits together in a reactive Angular arc…
Othertext/link
Yesterday's comic was, of course, in reference to Twyman's Law.
Applicable not only to experiment analysis, but also life. If it looks too good to be true, then it probably is 😅
Thought leadershiptext/carousel
📢 Streaming FRIDAY at 11am CT!
Join me and Alfredo Perez on Angular Air as we dive into Spec Driven Development with Claude + Cursor.
We’ll explore how writing tests first — with the help of an AI agent — can actually boost clarity and cod…
Marketing event / contenttext/link
Missed us at the The AI Conference? Here's the write-up behind the LLM traces handout we were giving out.
Your evals stop at deploy. They tell you a new model or prompt clears your quality bar. They can't tell you what it does to revenue, …
Product functionalitytext/link
5.0 Launch Week Day 2️⃣: AI Visual Editor
Growth and marketing teams run into the same problem: plenty of ideas, not enough engineering time to test them all.
Our AI Visual Editor fixes that. Describe the change you want in plain English …
Product functionalitytext/only
B2C testing is more mature and more rigorous. Anuradha Tempe, MBA, MS of Samsung on borrowing that discipline for B2B without losing the soul of a fast-paced business. Listen to the full episode of The Experimentation Edge at growthbook.io/…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
🔴 LIVE NOW — Back to the Basics kicks off!
We’re live with the incredible Deborah Kurata breaking down Signal Forms in a way that truly clicks.
If you’ve wanted Signals + Forms explained clearly and practically — this is your episode.
Jo…
Marketing event / contenttext/link
Unit tests have one job: tell you if something is broken. AI apps just made that job impossible.
When your application is non-deterministic, there's no pass or fail. There's only better or worse. And the only way to know which one you're l…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
Part two, round one of the Mad CSS bracket tournament just dropped this morning. Who says front end skills are dead?!?! This is the show I didn’t know I needed. https://lnkd.in/gTeeTKk5
Othertext/link
A growth loop that looked perfect on a whiteboard completely backfired.
Fyxer has a scheduling feature — similar to Calendly. When someone books a meeting, you get a confirmation.
Kameron Tanseli's hypothesis: send that booking confirmati…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
Fyxer retests experiments that failed last year.
And they keep winning.
This sounds counterintuitive. If a test lost, why run it again?
Because your product changes. Your customers change. Other experiments change the environment that ca…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
A prompt is a suggestion. A skill is a procedure.
When you type "create a feature flag" into Claude or Cursor, the agent can already call the GrowthBook API, but it doesn’t know your own procedures. New flags ship enabled at 100%. "Analyze…
Product functionalitytext/image
Looking for some A/B testing content to watch this weekend 👀
Ron Kohavi has compiled a list of some bangers on YouTube.
Of course, you could also check out our channel (like + subscribe): https://lnkd.in/gWW46Ddi
Thought leadershiptext/only
If you're serious about experimentation, there's no better place to learn, network, and share than the Test & Learn Community (TLC). And, if the sparkly costume in the image didn't give it away, they're also a really fun and welcoming group…
Marketing event / contenttext/only
A billion dollars.
That's JPMorganChase's estimated value from experimentation and innovation over the years.
And that number only counts the winners.
It doesn't include the losing tests that quietly stopped bad ideas from ever shipping.…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
Your experiments showed +5%, +3%, +7% wins. But did they actually add up to +15%?
Spoiler: Probably not. Airbnb found naive sums overstate impact by 2x.
New in GrowthBook: ✨ Holdouts ✨ the gold standard for measuring what all that shippin…
Product functionalitytext/link
A simple UI change that sat in Bing’s backlog for months was worth over $120M in annual revenue. Nobody saw it coming.
That's the paradox of experimentation: the median organization sees only 10% of experiments move the metrics they were d…
Thought leadershiptext/videoscroll-stopping: n/a (video)
Sometimes the p-value is never coming.
Wanli Lau (Twilio) on experimentation when your customer count is a fraction of a consumer company's:
Many tests are not driven to statistical significance. The team learns from the signals.
Good enoug…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
Again, for those in the back: We've added actually useful AI features to GrowthBook in 4.1.
🤖 Auto-summarize experiment results in plain English
📊 Generate SQL queries from natural language
💡 Improve experiment hypotheses with AI suggest…
Product functionalitytext/only
LIVE NOW 🔴
Talking agentic UIs in Angular with @ManfredSteyer — where AI starts driving the experience, not just responding.
👩💻 https://lnkd.in/gVDQJ_nE
📱 https://lnkd.in/gX79U3Dy
Shout out to briebug.com — helping us keep Angular Air …
Othertext/link
Why would Principal Financial Group A/B test on customers who don't exist?
The new edition of The Experimentation Edge newsletter rounds up my conversations with Erika Dunn, Assistant Director of Data Science at Principal Financial Group,…
Podcast (Experimentation Edge)text/link
It's almost showtime! Crossing my fingers for 3 of the finalists that are all using GrowthBook: Khan Academy, Fyxer and Wellhub. Positive thoughts everyone! :)
Othertext/only
LIVE NOW!
We’re live with Wagner Maciel from the Angular team talking Angular ARIA and what’s coming in v21.
Watch here:
📺 Desktop — https://lnkd.in/gkTwa9aa
📱 Mobile — https://lnkd.in/gYM2STxg
Marketing event / contenttext/link
You spent weeks building the feature.
Nobody used it.
This happens more often than most product teams admit. And the default response is to move on and build something else.
Kameron Tanseli at Fyxer does the opposite.
His approach: ship…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
POV: you’re not hitting your protein goals on your trip, so you just start eating pouch tuna in the hotel hallway, but you don’t have a fork so this knife will have to do….
Othertext/image
Sorry everyone!!! I messed up (me, Alyssa). Alfredo's episode will be Oct. 3rd. See you then! 🎥
Podcast (Experimentation Edge)text/only
GrowthBook is now native on Vercel Marketplace 🎯
What this means for your Next.js apps:
→ One-click installation
→ Unified billing (one less vendor to manage)
→ Edge-ready performance (we're talking microseconds)
→ Feature flags that actua…
Product functionalitytext/link
LIVE NOW!
Sander Elias is on Angular Air walking us through Signal Forms — dynamic + recursive structures, intuitive validation, and custom controls that actually feel good to build.
Come see what next-gen Angular forms look like in practi…
Othertext/link
Tomorrow! 🔥
We’re diving into Signal Forms with @esosanderelias — recursion, dynamic structures, validation magic, and custom controls.
Big thanks to our sponsor Progress / Kendo UI for supporting the Angular community.
👉 https://lnkd.in/…
Marketing event / contenttext/image
The best time to start managing your feature flag system was probably like two years ago 😬
The second best time is now.
Our Dev Advocate, Ryan Feigenbaum, PhD is sharing industry best practices for managing flags at scale in his talk, "Be…
Marketing event / contenttext/only
Not everything deserves an experiment. Ben Schein of Clover on The Experimentation Edge: uncertainty and downside decide testing depth, so payment authorization flows get ground-level testing while table stakes features get a monitored roll…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
Vibe experimentation is here.
Testing a new hero or layout used to mean a design request, a dev ticket, and a week of waiting. Now it's a prompt.
GrowthBook's Visual Editor uses AI to generate and launch new content in the browser.
Product functionalitytext/videoscroll-stopping: n/a (video)
We are thrilled to have Kelli H., senior director of data insights from Khan Academy, on our next webinar. Kelli will share Khan Academy's experiences as they grew and strengthened their A/B testing program. I'm particularly excited to he…
Marketing event / contenttext/only
Love this! Rudger de Groot makes a great case for Holdouts (part of our latest 4.1 release): It demonstrates the value of experimentation tout court. You have a number "nobody can dispute"
Product functionalitytext/only
Love the updates we released, yesterday. If you are into experimentation (or would like to be), GrowthBook is for sure worth checking out.
Product functionalitytext/only
Raj Mehta joined me on the Experimentation Edge Podcast this week. Its a great real-world example of how AI can have a huge impact on a more traditional business like Moxie Pest Control. Their use of AI to scale call coaching especially s…
Podcast (Experimentation Edge)text/only
Most ideas fail. Data shows the median organization has a success rate of only 10-15%. If you accept this, the probability of making errors is high, and therefore, shipping unfinished work is counterproductive.
Remember the reasons for fai…
Thought leadershiptext/videoscroll-stopping: n/a (video)
🔍 AI in Angular: We need your input!
The Angular team is gathering real-world use cases for building AI-enabled applications. Your perspective can help shape the future of Angular’s approach.
Check out the RFC, join the discussion, and add…
Othertext/link
Experimentation at scale is hard. TOOM made it systematic.
By replacing manual analysis with automated experiment decisions, they turned testing into infrastructure.
Full story in the comments.
#Experimentation
#ABTesting
#RetailTech
Customer storytext/carousel
Feature flags, experimentation, and product analytics usually live in three separate tools with three separate definitions of "conversion."
Our new 5-minute tour of GrowthBook shows what it looks like when they don't: add an SDK, wrap a fe…
Product demotext/image
Could not agree more! Sam was an incredible project manager. 🙌 Loved working with her at Pluralsight / Code School!
Social / culturetext/only
International Women’s Day always makes me pause and reflect on the women who have shaped my career.
For me, that person is Sara Faatz.
There isn’t one specific moment I can point to. It’s something she’s shown consistently, over and over …
Social / culturetext/only
One week until The AI Conference!
If you're going, come by booth 206. We'll be running live demos: wrap an AI feature in a flag, launch an experiment against your own warehouse data, and watch a guardrail metric roll a bad release back aut…
Marketing event / contenttext/image
This week on Angular Air, we’re welcoming Wagner Maciel from the Angular team at Google to talk about something HUGE landing in Angular v21 — Angular ARIA.
Angular is introducing a brand-new suite of ARIA-ready components (accordion, menu,…
Othertext/link
The problem with A/B tests? They end.
But what about the cumulative impact of 50 features shipped over 6 months?
Enter Holdouts: the experiment that never stops watching.
Here's how it works:
→ Keep 5% of users on the original experience…
Product functionalitytext/link
I love this Valentine's twist!! 💞 🤣
Social / culturetext/only
If the idea sounds weak in plain English, don't run the experiment.
Arun Bodapati at Twitch sees too many teams skip that gut check.
On the show, he pushed PMs, engineers, and data scientists to do the real work before hitting play.
Get …
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
Experiment analysis used to be restricted to data analysts and product managers with experiment knowledge. Now, with AI, basically everyone at Supercell can run and analyze experiments on their own. Shan Huang of Supercell on The Experiment…
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)
Just like last week, this week's Angular Air episode is adding a second simultaneous livestream! 🎉
We’ve heard from mobile viewers that the standard YouTube stream layout can be hard to watch on small screens — so we’re launching a mobile-…
Marketing event / contenttext/link
Your agent can write the feature in 30 seconds. Shipping it safely is the part that keeps you up at night.
At booth S26 we're handing the keyboard to Claude Code, Cursor, and Codex and watching them run the whole loop through the GrowthBoo…
Marketing event / contenttext/videoscroll-stopping: n/a (video)
Let’s kick off a new series I’ve been cooking up —
Back to the Basics. The Angular basics, that is.
And there is NO BETTER PERSON to launch this with than the magnanimous Deborah Kurata.
She’s breaking down Signal Forms at a level you’ll …
Othertext/link
One week until Snowflake World Tour NYC.
AI has made shipping new features easy. Today's winners are the companies that can learn the fastest.
GrowthBook helps your product builders run 5X more experiments - all from their AI coding assi…
Marketing event / contenttext/image
Last day at AI Engineer World's Fair, and the socks are going fast.
If we haven't met yet: GrowthBook is open source and warehouse native. You can read the entire stats engine on GitHub, and your experiment data never leaves your warehouse…
Marketing event / contenttext/image
What are we thankful for and not arguing about? Clear, statistically significant, verifiable data. Please join us this Thursday at 10am Pacific to see GrowthBook 4.2 live.
A single Product Analytics dashboard CAN uncover a $400K conversion…
Marketing event / contenttext/only
We’re live on Angular Air!
Join us now as Manfred Steyer unpacks full-cycle reactivity in Angular — Signal Forms, Signal Store, Resources, and the Mutation API.
Watch here → https://lnkd.in/g59hP7hC
Thanks to our sponsor, BrieBug, for help…
Marketing event / contenttext/link
🔴 LIVE NOW — Back to the Basics continues!
We’re diving into AI-assisted i18n workflows in Angular with Ady Ngom — exploring how to scale internationalization in a way that’s actually practical.
If you’ve ever struggled with i18n (or avoi…
Marketing event / contenttext/image
We’re at Snowflake World Tour NYC today!
If you’re exploring how to measure the impact of AI features or get more out of your experimentation, come talk to the GrowthBook team.
GrowthBook runs experiments directly in your Snowflake wareho…
Marketing event / contenttext/carousel
The web has always had this pendulum swing between client-side and server-side rendering.
Today we’re exploring where that pendulum is landing next with Adam Plumer from the Angular team at Google — one of the engineers working on Angular’…
Podcast (Experimentation Edge)text/link
🔴 We’re live now!
Come join me and Alfredo Perez as we explore Spec Driven Development — a workflow where your AI agent follows your tests (not the other way around).
✨ Featuring Claude + Cursor + Angular
🎥 https://lnkd.in/gB9xPiii
🙏 Hug…
Marketing event / contenttext/link
Most holdouts measure only shipped features. Ours measure everything—including failed experiments.
Because that's what your users actually experienced.
Technical breakdown → https://lnkd.in/gCv_T6_6
Data sciencetext/link
Most experimentation tools ask you to ship your data to them. If you already run Snowflake, that's a second copy of your data and a second definition of every metric.
GrowthBook is warehouse-native, so feature flags, experimentation, and p…
Marketing event / contenttext/image
7000 stars. Seems like just yesterday it was 6k. They grow up so fast 🥲
And, if you're looking for something to do on this Friday, check us out on GitHub and help us get our next thousand: https://lnkd.in/gdj7jBZe
Othertext/only
🔴 LIVE NOW — Back to the Basics continues!
We’re diving into SSR, hydration, and partial hydration with Adam Plumer — breaking down what Angular is actually doing under the hood and why it matters.
Join us:
👩💻 Desktop → https://lnkd.in/g…
Marketing event / contenttext/link
MeasureCamp Chicago is almost one week away! 🙌
The agenda doesn't exist yet. It gets built that morning by whoever's willing to write a topic on a card and stick it to the board. If you’ve been sitting on a topic, we can’t wait to hear it…
Marketing event / contenttext/image
Clearing our calendars for this one 📅
Go see Hampus at Berlin Experimentation Meetup on the 15th!
Marketing event / contenttext/only
I can’t speak for UX design, but as for Web Dev and DevRel, I’d choose this mess again.
Social / culturetext/only
Your best feature might be hiding in the wrong place.
Danielle Olean, Director of eCommerce at Box, shared a funnel insight on our show that applies to any product.
Say only a small number of users find a feature, but those who do use it …
Podcast (Experimentation Edge)text/videoscroll-stopping: n/a (video)