Token burning: when token bills explode

Tokens change budgeting

A classic SaaS license is predictable. A token-based AI bill follows user behavior, agent behavior and code behavior. If an assistant summarizes long documents, if an agent loops, if a coding tool retries calls, or if an API key is poorly protected, consumption can rise quickly.

The difficulty is that the cost is not always visible to the user. An answer feels instant and immaterial, but it relies on inference, context, sometimes embeddings, document searches and intermediate calls. Token-based pricing therefore shifts part of the risk to real usage. The more teams adopt AI, the more the bill depends on behaviors that are difficult to predict without instrumentation.

Public examples show the risk

Specialized press reported that Accenture asked some employees to reduce non-essential AI usage amid rapidly rising token spend. Recent coverage also cites companies such as Uber and Microsoft putting guardrails on some AI development tools. An extreme case attributed to an unnamed enterprise described a reported 500 million dollar Claude bill in one month after insufficient limits.

Agents intensify the phenomenon. A user may think they asked a single question, while the system performed several searches, reformulations, tool calls and checks. This invisible chain can be useful, but it makes consumption harder to anticipate. Without limits and observability, the company discovers cost after the fact, once the bill already exists.

These examples are market signals. The issue is not that AI is bad; the issue is that an unbounded variable cost model can surprise even mature organizations.

Why the surprise happens

The mechanism is cumulative: long prompts increase input tokens, long answers increase output tokens, agents repeat steps that remain invisible to the user, and coding or RAG tools multiply calls in the background. Finance teams often discover the real level of spend only after the usage has already happened.

The surprise also comes from success. A useful experiment attracts more users, then more documents, then more workflows. What was a test becomes a habit. If the architecture has no quotas, adapted models, cache or routing rules, each new use case increases dependence on variable billing.

Why OPA is a response

OPA reduces the risk by moving recurring workloads onto private AI infrastructure. Cost becomes tied to known server capacity instead of an open-ended token meter. Internal assistants, RAG, business workflows and some agentic workloads can run locally with quotas, logs and visibility.

This approach treats AI as infrastructure rather than simple API consumption. Models can be chosen according to the task, embeddings shared, documents indexed locally and usage measured. Cost becomes connected to an investment and a known capacity, rather than to consumption that silently increases with every prompt.

Sources: ITPro on Accenture token spend, Yahoo Finance on the reported Claude bill, GAP on runaway token costs.

Tom Cheniaux - rephrased using AI