Alyssa Nicoll

Beyond the Front End Field Notes — Day 6 Today’s field note is about CUPED. "Which leaves me wondering... should all experiments be cuped?? Why wouldn't you?" CUPED reduces statistical noise by using relevant pre-experiment behavior as context. When that historical behavior is predictive, it can help an experiment reach the same level of certainty with less traffic. Say you are testing a new checkout page and measuring revenue. Some customers regularly spend $500. Others regularly spend $20. By chance, one variation might receive a few more big spenders. Without CUPED, that natural difference creates noise. You may wonder: did the new checkout increase revenue, or did that group simply contain customers who already spend more? CUPED looks at each customer’s behavior before the experiment and uses it as a baseline. Conceptually, it asks: “Knowing what this person normally does, did their behavior change more than we would have expected?” Which leaves me wondering... should all experiments be cuped?? Why wouldn't you? https://lnkd.in/gGeMuSj6