Economics
Token
Definition
A token is the unit of text a model reads and writes — roughly three quarters of a word — and the unit in which almost all AI usage is priced.
Tokens are the meter on the machine. Because both the input and the output are billed, verbose prompts and chatty responses cost real money at volume, in a way that is easy to overlook until the monthly invoice arrives. Translating token pricing into cost per decision is the step that turns an abstract rate card into a number an operating committee can act on, rather than a technical detail buried in an engineering budget.
From rate card to unit economics
Nobody budgets in tokens, and presenting a token price to an operating committee communicates almost nothing about whether a workflow is economically sound. Convert to cost per submission, per claim, per loan file — then compare it to the labour cost of the same step and to the value of the decision itself, because that comparison is the one a business leader can actually use to decide whether the investment makes sense at the volumes the operation actually runs.
Retries and agents multiply the meter
An agent that reasons over several steps, calls tools, and retries on failure can consume ten to fifty times the tokens of a single call for the same visible outcome, because each internal step, tool call, and retry attempt adds its own token cost even though the end user only sees one final answer. Business cases built on the token cost of a single simple call routinely understate the real cost of an agentic workflow by an order of magnitude or more.
Token efficiency is a design choice, not just a model choice
Prompt structure, the amount of retrieved context included, and how verbosely the model is instructed to respond all affect token consumption independently of which model is chosen. Two teams using the identical model can see very different unit costs purely because of how carefully one of them designed its prompts and retrieval to avoid unnecessary token spend, which means cost control is partly an engineering discipline that has nothing to do with negotiating a better rate from the vendor.
In practice
- Report cost per decision, never cost per million tokens, to the business
- Instrument token spend per workflow so a regression is visible in days
- Include retries and tool calls in the cost model, not just the happy path
- Treat prompt and retrieval design as a token-efficiency lever, not just an accuracy one
