Posts

89 original posts (reposts excluded), filtered

Source:All Ashley StirrupGrowthBookGraham McNicollAlyssa Nicoll
Sort:Most recentTop engagement
Channel:📰 Newsletter
Filters:Topic: thought_leadership ✕Clear all
SourcePostFormatTopicPublishedImpr.LikesCommentsShares
Graham McNicollNobody should bench press 300 pounds their first day in the gym. The reason is obvious. Ytext_videothought_leadership2026-10-14—1510
Graham McNicollAverage tenure for product developers is about two years, which puts everything your team text_imagethought_leadership2026-10-12—700
Graham McNicollAll of AI right now is entirely reactive. You have to know what to ask before it can helptext_videothought_leadership2026-10-071951520
Graham McNicollWin rates across the industry sit around 20%, and I do not think AI is going to improve ontext_imagethought_leadership2026-10-053481330
Graham McNicollA year ago, most engineers wanted to see every line an AI wrote before it went near their text_imagethought_leadership2026-10-0213,83841133
Graham McNicollArtificial intelligence helps you feel artificially intelligent. It is almost as if it watext_videothought_leadership2026-09-303422020
Graham McNicollWhen an experiment fails, there should be a discussion about if the idea was bad or if it text_imagethought_leadership2026-09-28231910
Graham McNicollA couple of years ago, the first AI tool I used was Copilot. We saw 20 to 30 percent improtext_imagethought_leadership2026-09-254781620
Graham McNicollBuilding a feature now costs almost nothing, which turns out to be the problem. When you text_videothought_leadership2026-09-232941000
Graham McNicollHiring has always been the most critical part of building a company, and it takes a thoughtext_imagethought_leadership2026-09-18526920
Graham McNicollThink about how many layers sit between you and an actual user. User research tells you text_onlythought_leadership2026-09-11346710
Graham McNicollIf you work inside an experimentation program, you have probably noticed that leadership stext_imagethought_leadership2026-09-094112230
Graham McNicollMost product teams use evals to figure out if their AI tools or products are working corretext_videothought_leadership2026-09-084681940
Graham McNicollDesigners and engineers get attached to their work. It makes sense; you spent real effort text_imagethought_leadership2026-09-04428900
Graham McNicollEvery experiment you run produces a piece of knowledge about your users. Run a few hundrtext_onlythought_leadership2026-09-015081230
Graham McNicollWhen I talk to product teams lately, a strange thing keeps coming up. A large share of ttext_imagethought_leadership2026-08-283991500
Graham McNicollI talk to a lot of teams who are proud of how much they experiment. A company you would text_onlythought_leadership2026-08-259401620
Graham McNicollAI has made it ridiculously easy to build and ship features. The problem is that AI doesn'text_videothought_leadership2026-08-204681412
Graham McNicollA lot of engineering teams have feature flags to control the release of features and act atext_videothought_leadership2026-08-18310910
Graham McNicollIt's really hard when you really believed in feature, you managed to get it prioritized, ytext_videothought_leadership2026-08-133331530
Graham McNicollFrom my experience, experimentation programs don't get killed off in a meeting because peotext_videothought_leadership2026-08-112991020
Graham McNicollThe average engineer stays at a company about two years. I don't blame anyone for that. text_linkthought_leadership2026-08-051,1992000
Graham McNicollToken costs are exploding across our industry. Here's how we approach it. I definitely betext_videothought_leadership2026-08-043231020
Graham McNicollOne of the things we tell a lot of our customers is that it's the number of at-bats that wtext_videothought_leadership2026-07-29342720
Graham McNicollWe don't have product managers at GrowthBook. That is deliberate... One reason is ownerstext_imagethought_leadership2026-07-286891630
Graham McNicollFor years I chased my team for experiment write-ups after the fact. It never worked, and text_linkthought_leadership2026-07-241,4531610
Graham McNicollIf you have ever seen a kid use an iPad, you have watched a perfect UX. A child seeing itext_imagethought_leadership2026-07-21974510
Ashley StirrupGetting numbers is easy. Getting numbers you can trust is hard. We hosted Ron Kohavi (co-text_imagethought_leadership2026-07-173,6464681
Graham McNicollMost companies ship the first version of the product they build. And the chance that's thetext_videothought_leadership2026-07-166541740
Graham McNicollAsk any product manager what they hate most about their job, and the answer is rarely the text_onlythought_leadership2026-07-149882110
Graham McNicollMost product teams build features the expensive way. Someone has the idea. The team spectext_videothought_leadership2026-07-09291520
Graham McNicollIf your finance team is currently drafting a memo about capping engineering AI spend, I amtext_imagethought_leadership2026-07-08371420
Graham McNicollPMs ship experiment winners too early. A metric moves 3%, the test gets called, and the text_videothought_leadership2026-07-073511510
Graham McNicollIf your team already has feature flags wired into production, you can run painted door testext_imagethought_leadership2026-07-016951110
Graham McNicollOne very large financial company, a name you would recognize, told me they were doing expetext_videothought_leadership2026-06-307131731
Ashley StirrupPeople with smaller palms live longer. That's a real correlation. Statistically significatext_imagethought_leadership2026-06-291,5291510
Graham McNicollA Hacker News thread a while ago argued that A/B testing is unethical. I believe their fratext_imagethought_leadership2026-06-24567620
Graham McNicollI see experimentation leads run two or three tests a month and feel like they have figuredtext_videothought_leadership2026-06-23406810
Graham McNicollI have been a software engineer long enough to have the grey hairs to show for it. Monoltext_videothought_leadership2026-06-189692130
Graham McNicollKilling a feature you spent months (or years) building is one of the hardest calls in prodtext_videothought_leadership2026-06-164751551
Graham McNicollEvery experimentation leader eventually runs into the result nobody expected. A counteritext_videothought_leadership2026-06-113671000
Graham McNicollI spoke to an engineering lead last week whose team still tracks progress in story points.text_imagethought_leadership2026-06-10474710
Graham McNicollMost engineering teams using feature flags treat them as circuit breakers. Something goes text_videothought_leadership2026-06-093331210
Graham McNicollI spoke to an engineering manager last week whose team still tracks sprint progress in stotext_videothought_leadership2026-06-034691210
Graham McNicollMy head of Experimentation named our product development framework after Jon Hamm. I chosetext_imagethought_leadership2026-06-029673150
Ashley StirrupDon’t confuse shipping with winning. Microsoft figured this out in 2014. For decades, thetext_imagethought_leadership2026-05-267761401
Graham McNicollI have watched a deploy take down products complete because there was no flag in place to text_imagethought_leadership2026-05-22478901
Graham McNicollMost product teams I talk to struggle to analyze the true impact of any feature they ship,text_videothought_leadership2026-05-20312900
Graham McNicollMost engineering teams I talk to are still shipping every feature directly to production. text_videothought_leadership2026-05-131,3432021
Graham McNicollMost product teams I talk to know how to run an experiment. They have the tooling, a primatext_imagethought_leadership2026-05-129871200
Graham McNicollI have sat in spec reviews where the Jira tickets were written, the designs were finished,text_imagethought_leadership2026-05-081,0251010
Graham McNicollMost product teams spec out the entire feature before they've shipped a single line of codtext_videothought_leadership2026-05-062,2441820
Graham McNicollMost Engineers I know are supervising agents now, not writing the code. Features that tootext_imagethought_leadership2026-05-055,3392731
Graham McNicollAI coding tools are genuinely impressive. And this is literally the worst they will ever btext_videothought_leadership2026-04-296,3232210
Graham McNicollIf you could onlypick one for your experimentation program, which would you rather have? text_onlythought_leadership2026-04-241,676450
Graham McNicollSoftware teams get measured on delivery speed. It's an easy thing to track, it fits neatlytext_imagethought_leadership2026-04-216571110
Graham McNicollIt's entirely possible to run a test, see it win, ship the feature, and still damage your text_videothought_leadership2026-04-179521320
Graham McNicollMany "data-driven" companies are practicing sophisticated guessing. They have the dashbotext_onlythought_leadership2026-04-169681430
Graham McNicollI see so many software teams launch massive features without actually defining what succestext_videothought_leadership2026-04-147411100
GrowthBookA simple UI change that sat in Bing’s backlog for months was worth over $120M in annual retext_videothought_leadership2026-04-13409400
Graham McNicollEngineer's who ship products without measuring them are making a big mistake. If you arentext_videothought_leadership2026-04-086321500
Graham McNicollProduct teams obsess over whether they shipped on time. It's a reasonable thing to track. text_imagethought_leadership2026-04-07—810
Graham McNicollMost companies think choosing an experimentation vendor comes down to feature lists and UItext_imagethought_leadership2026-04-022,2672520
Graham McNicollI've watched design teams spend months on an interface that looked great, and was based ontext_videothought_leadership2026-04-016712221
Graham McNicollEven the best product people on Earth have a surprisingly low batting average. In basebaltext_imagethought_leadership2026-03-319641910
Graham McNicollIn many companies, the projects we work on are the ones backed by the person who speaks thtext_videothought_leadership2026-03-261,3972841
Graham McNicollThe most important transition for a product team is learning how to be ruthless with ideastext_videothought_leadership2026-03-241,3332840
Ashley StirrupYour metrics are green. Your customers are leaving. Ron Kohavi calls these "watermelon metext_imagethought_leadership2026-03-194,75796144
Graham McNicollJudging a product team solely on their ability to ship is a bad metric that rewards activitext_videothought_leadership2026-03-189243510
Ashley StirrupFriends don't let their product managers use bad reality distortion. Steve Jobs had a famtext_imagethought_leadership2026-03-161,218810
Graham McNicollI know it sounds controversial, but I advocate for burning your backlog at the end of evertext_videothought_leadership2026-03-121,7955670
Ashley StirrupMost ideas fail. Data shows the median organization has a success rate of only 10-15%. If text_videothought_leadership2026-03-1174200
Graham McNicollIn baseball terms, the industry standard for any new product success is batting 200- and htext_videothought_leadership2026-03-101,4524910
Ashley StirrupIt's Twyman's Law: any figure that looks interesting or surprising is usually wrong. Heretext_imagethought_leadership2026-03-061,3931510
GrowthBookShipping on flat can be so tempting! "The project is strategic!" or "It's important to thtext_onlythought_leadership2026-03-047871410
Ashley StirrupA slide made it all the way to a VP all-hands at Microsoft. It said: "New users who use otext_imagethought_leadership2026-03-03565920
Ashley StirrupAmazon's team was confident in their data. Then someone noticed Oprah had called Kindle hetext_documentthought_leadership2026-03-02617710
Ashley StirrupA Microsoft team once presented this finding internally about Office 365: "Users who use otext_imagethought_leadership2026-02-272,9344410
Ashley StirrupLukas Vermeer 🃏 talked today at Experimentation Island about how simple stories can reshatext_imagethought_leadership2026-02-262,7145840
Ashley StirrupThe most dangerous animal in your organization isn't a bad idea. It's a senior leader withtext_imagethought_leadership2026-02-2622,3556262
Ashley Stirrup92% of experiments at Airbnb failed to move the metrics they were designed to improve. Rotext_imagethought_leadership2026-02-243,2652832
Ashley StirrupYou never know where your next home run idea will come from. That's not a platitude. It'stext_documentthought_leadership2026-02-233,5583030
Ashley StirrupExperimentation unlocks learning you can't get any other way. These learnings often surfatext_onlythought_leadership2026-02-202451110
GrowthBookAI Evals tell you if the model works. A/B tests tell you if users care. These answer difftext_linkthought_leadership2026-01-27540601
GrowthBookMany organizations are optimizing perfectly—for the wrong thing. Dark patterns rarely startext_onlythought_leadership2026-01-204481000
Graham McNicollOptimization to Enshitification: Dark Patterns in A/B Testing A post allegedly written bytext_imagethought_leadership2026-01-1917,6115652
GrowthBookYesterday's comic was, of course, in reference to Twyman's Law. Applicable not only to etext_carouselthought_leadership2025-09-17322400
GrowthBookAs a company, we do things companies do: • Run A/B tests (obviously) • SEO • And notext_linkthought_leadership2025-09-15487700
GrowthBookLooking for some A/B testing content to watch this weekend 👀 Ron Kohavi has compiled a text_onlythought_leadership2025-09-12496400