Ashley Stirrup

It's Twyman's Law: any figure that looks interesting or surprising is usually wrong. Here is a perfect example. A peer-reviewed paper found that rounded corners on buttons outperform square ones by 55%. Designers took note. The reasoning made sense. Softer edges, reduced visual friction, friendlier feel. There was one problem. The study had 474 users on one side and 445 on the other. Less than 500 people per group. Small sample sizes do not just make results noisier. They make the results that survive statistical significance look bigger than they really are. If your experiment is underpowered and you still get a significant result, that result almost certainly exaggerates the true effect. So Ron Kohavi, Lukas Vermeer 🃏, Jakub Linowski, Joachim Furuseth, Ravikiran Rajagopal, Andrey Andreev, Majed Dodin ran the replication. SeaWorld Orlando and Co-op Norway ran the tests with roughly 2,000 times the original sample size. Results: None were statistically significant. The biggest run came in slightly negative. Rounded buttons do not outperform square ones by 55% or by any margin that a real business should act on. Before you ship a change based on a study, check the sample size. Hat tip to the team for running the replication nobody else bothered to. Has your team ever acted on a result that turned out to be too small to trust?

Only one snapshot so far — the engagement-over-time curve appears once the daily scrape has captured this post at least twice.