Posts

104 original posts (reposts excluded)

Source:All Ashley StirrupGrowthBookGraham McNicollAlyssa Nicoll
Sort:Most recentTop engagement
Channel:📰 Newsletter
SourcePostFormatTopicPublishedImpr.LikesCommentsShares
Graham McNicollNobody should bench press 300 pounds their first day in the gym. The reason is obvious. Ytext_videothought_leadership2026-10-14—1510
Graham McNicollAverage tenure for product developers is about two years, which puts everything your team text_imagethought_leadership2026-10-12—700
Graham McNicollGave a talk on synthetic user A/B testing at MeasureCamp Berlin on Saturday. There was pletext_imagemarketing_event2026-10-11—4050
Graham McNicollThe return you report from experimentation is higher than the return you actually got. Thtext_imagedata_science2026-10-092181300
Graham McNicollAll of AI right now is entirely reactive. You have to know what to ask before it can helptext_videothought_leadership2026-10-071951520
Graham McNicollWin rates across the industry sit around 20%, and I do not think AI is going to improve ontext_imagethought_leadership2026-10-053481330
Graham McNicollA year ago, most engineers wanted to see every line an AI wrote before it went near their text_imagethought_leadership2026-10-0213,83841133
Graham McNicollArtificial intelligence helps you feel artificially intelligent. It is almost as if it watext_videothought_leadership2026-09-303422020
Graham McNicollWhen an experiment fails, there should be a discussion about if the idea was bad or if it text_imagethought_leadership2026-09-28231910
Graham McNicollA couple of years ago, the first AI tool I used was Copilot. We saw 20 to 30 percent improtext_imagethought_leadership2026-09-254781620
Graham McNicollBuilding a feature now costs almost nothing, which turns out to be the problem. When you text_videothought_leadership2026-09-232941000
Graham McNicollHiring has always been the most critical part of building a company, and it takes a thoughtext_imagethought_leadership2026-09-18526920
Graham McNicollToken costs are exploding across our industry. Every week there is another story about ctext_onlypast_experience2026-09-175241030
Graham McNicollI cannot look at 100 experiments and tell you what they have in common. That is the probltext_videoproduct_functionality2026-09-15270920
Graham McNicollThink about how many layers sit between you and an actual user. User research tells you text_onlythought_leadership2026-09-11346710
Graham McNicollIf you work inside an experimentation program, you have probably noticed that leadership stext_imagethought_leadership2026-09-094112230
Graham McNicollMost product teams use evals to figure out if their AI tools or products are working corretext_videothought_leadership2026-09-084681940
Graham McNicollDesigners and engineers get attached to their work. It makes sense; you spent real effort text_imagethought_leadership2026-09-04428900
Graham McNicollThere's always been a tension between marketing teams and engineering teams. Engineeringtext_videoproduct_functionality2026-09-02267910
Graham McNicollEvery experiment you run produces a piece of knowledge about your users. Run a few hundrtext_onlythought_leadership2026-09-015081230
Graham McNicollWhen I talk to product teams lately, a strange thing keeps coming up. A large share of ttext_imagethought_leadership2026-08-283991500
Graham McNicollWhen was the last time you saw innovation from a public company? Honestly, from my experitext_videoindustry_commentary2026-08-275031621
Graham McNicollI talk to a lot of teams who are proud of how much they experiment. A company you would text_onlythought_leadership2026-08-259401620
Graham McNicollAI has made it ridiculously easy to build and ship features. The problem is that AI doesn'text_videothought_leadership2026-08-204681412
Graham McNicollI do not love the word governance. It is boring. Nobody gets excited buying a governance stext_imageproduct_functionality2026-08-194021620
Graham McNicollA lot of engineering teams have feature flags to control the release of features and act atext_videothought_leadership2026-08-18310910
Graham McNicollIt's really hard when you really believed in feature, you managed to get it prioritized, ytext_videothought_leadership2026-08-133331530
Graham McNicollI cannot look at a hundred experiments and tell you what they have in common. Nobody can. text_imageproduct_functionality2026-08-127251600
Graham McNicollFrom my experience, experimentation programs don't get killed off in a meeting because peotext_videothought_leadership2026-08-112991020
Graham McNicollWe make it stupidly easy to run experiments. Whether you're a back-end engineer, a front-etext_videoproduct_functionality2026-08-064471001
Graham McNicollThe average engineer stays at a company about two years. I don't blame anyone for that. text_linkthought_leadership2026-08-051,1992000
Graham McNicollToken costs are exploding across our industry. Here's how we approach it. I definitely betext_videothought_leadership2026-08-043231020
Graham McNicollWhen we rebuilt product analytics for 5.0, we started with a question I could not shake. Wtext_imageproduct_functionality2026-07-302881711
Graham McNicollOne of the things we tell a lot of our customers is that it's the number of at-bats that wtext_videothought_leadership2026-07-29342720
Graham McNicollWe don't have product managers at GrowthBook. That is deliberate... One reason is ownerstext_imagethought_leadership2026-07-286891630
Graham McNicollFor years I chased my team for experiment write-ups after the fact. It never worked, and text_linkthought_leadership2026-07-241,4531610
Graham McNicollWe ran an internal contest at GrowthBook a while back. The setup was simple. Anyone on thetext_imageproduct_functionality2026-07-237551700
Graham McNicollTeams have no shortage of ideas, but implementing them is where teams hit roadblocks. Stext_videoproduct_functionality2026-07-221,1802571
Graham McNicollIf you have ever seen a kid use an iPad, you have watched a perfect UX. A child seeing itext_imagethought_leadership2026-07-21974510
Graham McNicollMost companies ship the first version of the product they build. And the chance that's thetext_videothought_leadership2026-07-166541740
Graham McNicollThere's a level of excitement teams get when the product just works and they're not fightitext_videoproduct_functionality2026-07-15404820
Graham McNicollAsk any product manager what they hate most about their job, and the answer is rarely the text_onlythought_leadership2026-07-149882110
Graham McNicollMost product teams build features the expensive way. Someone has the idea. The team spectext_videothought_leadership2026-07-09291520
Graham McNicollIf your finance team is currently drafting a memo about capping engineering AI spend, I amtext_imagethought_leadership2026-07-08371420
Graham McNicollPMs ship experiment winners too early. A metric moves 3%, the test gets called, and the text_videothought_leadership2026-07-073511510
Graham McNicollCS degree enrollment is dropping, and I do not blame the students. If I were eighteen ritext_videoindustry_commentary2026-07-027881040
Graham McNicollIf your team already has feature flags wired into production, you can run painted door testext_imagethought_leadership2026-07-016951110
Graham McNicollOne very large financial company, a name you would recognize, told me they were doing expetext_videothought_leadership2026-06-307131731
Graham McNicollTeams routinely misunderstand averages in experimentation and what those averages are hiditext_videodata_science2026-06-256341321
Graham McNicollA Hacker News thread a while ago argued that A/B testing is unethical. I believe their fratext_imagethought_leadership2026-06-24567620
Graham McNicollI see experimentation leads run two or three tests a month and feel like they have figuredtext_videothought_leadership2026-06-23406810
Graham McNicollI have been a software engineer long enough to have the grey hairs to show for it. Monoltext_videothought_leadership2026-06-189692130
Graham McNicollIf you are an engineering leader at a growth-stage company currently maintaining your own text_imagecustomer_story2026-06-175221420
Graham McNicollKilling a feature you spent months (or years) building is one of the hardest calls in prodtext_videothought_leadership2026-06-164751551
Graham McNicollIf your experimentation stack is Statsig, the last few months have probably been confusingtext_onlyindustry_commentary2026-06-123,4034833
Graham McNicollEvery experimentation leader eventually runs into the result nobody expected. A counteritext_videothought_leadership2026-06-113671000
Graham McNicollI spoke to an engineering lead last week whose team still tracks progress in story points.text_imagethought_leadership2026-06-10474710
Graham McNicollMost engineering teams using feature flags treat them as circuit breakers. Something goes text_videothought_leadership2026-06-093331210
Graham McNicollProduct development is forever changed, and the pace of the change is incredible. We've setext_onlyproduct_functionality2026-06-086992730
Graham McNicollThree years into building our own experimentation platform at my last company, we found a text_imagepast_experience2026-06-053,6655351
Graham McNicollI spoke to an engineering manager last week whose team still tracks sprint progress in stotext_videothought_leadership2026-06-034691210
Graham McNicollMy head of Experimentation named our product development framework after Jon Hamm. I chosetext_imagethought_leadership2026-06-029673150
Graham McNicollWe had a dashboard for every metric at my last company. What we did not have was a way to text_imagepast_experience2026-05-29620940
Graham McNicollAmazon just held a mandatory meeting after vibe-coded changes caused multiple major outagetext_videoindustry_commentary2026-05-265972370
Graham McNicollI have watched a deploy take down products complete because there was no flag in place to text_imagethought_leadership2026-05-22478901
Graham McNicollMost product teams I talk to struggle to analyze the true impact of any feature they ship,text_videothought_leadership2026-05-20312900
Graham McNicollAt education.com, we ran thousands of experiments. Roughly a third won. The ones that losttext_imagepast_experience2026-05-193511410
Graham McNicollA bug in our code accidentally set our paywall to zero. Revenue jumped 30%. Usually whetext_imagepast_experience2026-05-151,4461510
Graham McNicollI saw the Amazon report. Four major outages in one week from AI-assisted code changes. Thetext_imageindustry_commentary2026-05-141,6881500
Graham McNicollMost engineering teams I talk to are still shipping every feature directly to production. text_videothought_leadership2026-05-131,3432021
Graham McNicollMost product teams I talk to know how to run an experiment. They have the tooling, a primatext_imagethought_leadership2026-05-129871200
Graham McNicollI have sat in spec reviews where the Jira tickets were written, the designs were finished,text_imagethought_leadership2026-05-081,0251010
Graham McNicollAmazon got a mandatory meeting. Most engineering orgs will get a quieter version of the satext_imageindustry_commentary2026-05-071,005810
Graham McNicollMost product teams spec out the entire feature before they've shipped a single line of codtext_videothought_leadership2026-05-062,2441820
Graham McNicollMost Engineers I know are supervising agents now, not writing the code. Features that tootext_imagethought_leadership2026-05-055,3392731
Graham McNicollA bug in our codebase accidentally set our paywall to zero. As a side effect, revenue jumptext_videopast_experience2026-05-011,0521000
Graham McNicollI spent years seeing product decisions based on gut feel. At education.com, we would spetext_onlypast_experience2026-04-301,8732640
Graham McNicollAI coding tools are genuinely impressive. And this is literally the worst they will ever btext_videothought_leadership2026-04-296,3232210
Graham McNicollI once sat in a meeting that ran for an hour debating which shade of green to use for a butext_imagepast_experience2026-04-281,6161250
Graham McNicollIf you could onlypick one for your experimentation program, which would you rather have? text_onlythought_leadership2026-04-241,676450
Graham McNicollWe spent three years building our own experimentation platform at education.com. Then we ftext_videopast_experience2026-04-231,7272502
Graham McNicollIf you use 3rd party vendor, you are betting your infrastructure on someone else's board mtext_videoindustry_commentary2026-04-221,1231430
Graham McNicollSoftware teams get measured on delivery speed. It's an easy thing to track, it fits neatlytext_imagethought_leadership2026-04-216571110
Graham McNicollIt's entirely possible to run a test, see it win, ship the feature, and still damage your text_videothought_leadership2026-04-179521320
Graham McNicollMany "data-driven" companies are practicing sophisticated guessing. They have the dashbotext_onlythought_leadership2026-04-169681430
Graham McNicollStartup life is mostly reactive. You're in the task, in the Slack thread, in the meeting. text_videosocial2026-04-159341930
Graham McNicollI see so many software teams launch massive features without actually defining what succestext_videothought_leadership2026-04-147411100
Graham McNicollBuilding your own experimentation stack is a fun engineering challenge until the day it istext_imagepast_experience2026-04-091,4052201
Graham McNicollEngineer's who ship products without measuring them are making a big mistake. If you arentext_videothought_leadership2026-04-086321500
Graham McNicollProduct teams obsess over whether they shipped on time. It's a reasonable thing to track. text_imagethought_leadership2026-04-07—810
Graham McNicollMost companies think choosing an experimentation vendor comes down to feature lists and UItext_imagethought_leadership2026-04-022,2672520
Graham McNicollI've watched design teams spend months on an interface that looked great, and was based ontext_videothought_leadership2026-04-016712221
Graham McNicollEven the best product people on Earth have a surprisingly low batting average. In basebaltext_imagethought_leadership2026-03-319641910
Graham McNicollIn many companies, the projects we work on are the ones backed by the person who speaks thtext_videothought_leadership2026-03-261,3972841
Graham McNicollThe most important transition for a product team is learning how to be ruthless with ideastext_videothought_leadership2026-03-241,3332840
Graham McNicollWe once had a design team spend months redoing our onboarding flow. They did everything rtext_imagepast_experience2026-03-2083,977161238
Graham McNicollJudging a product team solely on their ability to ship is a bad metric that rewards activitext_videothought_leadership2026-03-189243510
Graham McNicollOpenAI bought Statsig for $1.1B in stock. Before that, Datadog swallowed Eppo. For a CTO,text_imageindustry_commentary2026-03-162,4764321
Graham McNicollI know it sounds controversial, but I advocate for burning your backlog at the end of evertext_videothought_leadership2026-03-121,7955670
Graham McNicollIn baseball terms, the industry standard for any new product success is batting 200- and htext_videothought_leadership2026-03-101,4524910
Graham McNicollRon Kohavi and Luke Sonnet are hosting a discussion tomorrow about how to decide what to dtext_onlymarketing_event2026-03-028951610
Graham McNicollOptimization to Enshitification: Dark Patterns in A/B Testing A post allegedly written bytext_imagethought_leadership2026-01-1917,6115652
Graham McNicollThe GrowthBook 4.2 webinar is this Thursday at 10am Pacific. Join Luke Sonnet, Ryan Feigentext_onlymarketing_event2025-12-031,3114310
Graham McNicollThis is exactly why Jeremy and I founded GrowthBook. Experimentation is hard but your expetext_onlycustomer_story2025-11-242,5374023