Graham McNicoll

Token costs are exploding across our industry. Every week there is another story about cost overruns, and the natural reaction from leadership is to start handing out token budgets. I actually believe you should give your teams almost infinite token budgets. Here is why. Token usage correlates very highly with productivity (or at least it should). So a token budget is really a productivity budget. You would never ask your team to work less hard on something. That just wouldn't come out of a manager's mouth. But yet here we are, constraining engineers on tokens. Now, there are absolutely times when tokens get spent inefficiently. So here is what we do at GrowthBook. Every week we sit down as engineers to share best practices and re-examine which models are best for which purpose. Things change so fast; a weekly cadence seems about right. Sometimes we find that a new model isn't great at a certain type of task, so we use another one; or a tool version has improved significantly, and we should try it, etc. Some AI tools are amazing at some things, yet poor at others. Knowing the difference lets you spend your tokens efficiently. Capping the budget just brings your output down. So give your team the tokens, learn and share best practices, and improve your productivity.