Graham McNicoll

Many "data-driven" companies are practicing sophisticated guessing. They have the dashboards. They have the scorecards. The data is there, but it's not actionable. They still can't tell you which feature actually moved revenue, or where to invest next. My background is in physics. In physics, if you give a number without error bars, it's incomplete. You don't have a real measurement without understanding the uncertainty. You have a point estimate with no sense of whether to trust it. I see the same thing in product teams every week. A PM squinting at a before-and-after graph, estimating the impact of a new feature by eyeballing the curve. Unless you can isolate the change through a controlled experiment, you are guessing, just with better-looking charts. The best way to measure the impact of what you've built is to run a controlled experiment. Without that isolation, you don't have an insight. At best you have a correlation dressed up as one. To truly understand what's happening, you also need to be able to explore the results you're seeing. We make the SQL visible in GrowthBook for this exact reason. When teams start arguing about whether a number is real, or spot something odd in the results, they need a way to interrogate it. Too many experimentation platforms are black boxes. Trust decays faster than it builds. One result that doesn't pass the sniff test can kill a data team's credibility for months. Transparency is a prerequisite for trust. Audit the data, examine the queries, and the insights will follow. Does your team trust the numbers? Do you have any way to audit the results?

Only one snapshot so far — the engagement-over-time curve appears once the daily scrape has captured this post at least twice.