The AI Cost Black Box
Why most companies have no idea what their AI spend is actually buying.
Ask a company what its cloud bill was last month and someone pulls the figure in a minute, split by team, service, and environment. Ask what its AI spend actually bought, which features, which teams, which of those made money, and you’ll usually get a lump sum and a shrug. The spend arrived faster than any instrument to watch it, and it’s still climbing: Gartner expects worldwide AI spending to grow 47% in 2026. So most AI deployments run on a fuel gauge with one number on it: bigger than last month. You can’t optimize what you can’t see, and right now most companies can’t see much.
The most useful way to organize the response is a three-part framework, borrowing from the FinOps Foundation’s work on AI cost. The order matters, because it’s a maturity ladder, not three dashboards you stand up at once. Visibility, then economics, then value. Each pillar answers a harder question than the last, and skipping ahead doesn’t work: you can’t tie cost to revenue before you can explain what’s driving cost, and you can’t explain the drivers before you can see the total.
Pillar One: See the Spend
The first pillar is basic tracking. Total AI spend, average daily cost, and enough anomaly detection to catch a runaway job before it becomes a five-figure surprise on the invoice. The metrics are unglamorous and most companies still don’t have them.
Its real job is less technical than it looks. Getting a single credible spend number onto an executive’s screen is what turns AI costs from an abstraction into something with an owner. The baseline it establishes is the thing every later optimization gets measured against. Until the number is visible to someone with the authority to ask why it moved, nobody is accountable for the answer, and the spend just compounds quietly in the background.
Pillar Two: Explain the Spend
Once you can see the total, you need to know what’s moving it, and this is where the metrics get operational. Cost per token, input-to-output ratios, cache-hit ratios, and one worth giving a name: call it token-to-spend drift. When the bill climbs, there are two possible causes, and they call for opposite responses. Either you’re consuming more tokens, or your traffic has drifted toward more expensive models. Drift separates the two.
The distinction is the whole value of the pillar because the symptom is identical and the right response is not. If cost is up because volume is up, you’re looking at either healthy adoption or a Jevons runaway, and the question is whether the consumption is producing anything. If cost is up because requests that used to run on a cheap model migrated to a frontier one, the answer is routing, not throttling. Same rising line, two different problems, and without the metric you’re guessing which one you have and applying the wrong fix half the time.
Pillar Three: Steer the Spend
The third pillar is the hard one and the point of the whole exercise. Cost per use case, and inference cost measured against the revenue each capability actually drives. This is where you find out which AI features pay for themselves and which are burning capital inside a demo everyone admires.
A support agent that deflects a thousand tickets a week has an inference cost you can set directly against the salaries and the churn it saves. A generative feature that ships to everyone and delights nobody has an inference cost with no revenue on the other side of the ledger, and no dashboard of token counts will tell you that; only cost-per-value will. This pillar changes what AI spend even is. Managed at the visibility level, it’s a cost center to be minimized. Managed at the value level, it’s capital allocation: you fund the capabilities that earn and starve the ones that don’t, and “spend less on AI” stops being the goal because it was never the right one.
Automate It: The Control Plane
A framework you run by hand, in spreadsheets and monthly exports and a standing review, is where every organization should start, because doing it manually is how you learn what’s worth measuring. Once it works, you automate it into an internal platform, the control plane.
The market already sells part of this as the AI gateway, a proxy that fronts every provider behind one API and tracks tokens, spend, and routing; Portkey, TrueFoundry, Kong, LiteLLM, and Cloudflare all ship one. A control plane is broader. It unifies data across providers, because almost nobody is single-vendor anymore, and puts spend, token economics, GPU fleet utilization, and business value on one surface. No off-the-shelf product covers all of that yet, which is why the teams that want it build their own. It’s the instrument panel the three pillars imply once you’re tired of assembling them by hand every month. The pillars tell you what to measure; the control plane is what measures it continuously instead of in arrears.
Build It at Agent Speed
Here’s the part specific to this moment. The teams that build these control planes build them with AI assistance, fast, and the speed isn’t a vanity metric. It’s the requirement because the thing being governed doesn’t wait for a normal release cycle.
Internal AI adoption doesn’t grow on a smooth curve. It lurches. Roll a coding agent out to the engineering org on a Tuesday and token usage can multiply by Friday in a pattern no capacity plan saw coming, because the people adopting it are precisely the people who build fast and share what works. A governance platform that takes two quarters to ship is obsolete the week it lands, since the adoption it was designed to watch has already outgrown it. The only development pace that keeps up with agent-speed adoption is agent-speed development. You end up building the meter with the same tools that are blowing up the bill, and that symmetry is the point rather than an irony: the org that can deploy agents fast enough to explode its own token usage is the same org that can build the instrument to watch the explosion, if it decides to.
Structural Viability Is the Real Target
The goal here was never a smaller bill. It’s a deployment that stays viable as it scales, which is a different and harder thing. Visibility so you can see the spend, economics so you can explain it, value so you can steer it, and a control plane quick enough to keep up with how violently the usage moves. Take those in order and AI stops being a line item that surprises you and becomes a system you actually run.
The companies that treat all of this as a reporting chore they’ll get to later will keep meeting their own AI economics for the first time on the invoice after the decisions are made and the architecture is set. The ones that build the instrument panel while the spend is still small get to steer, and steering early, while the numbers are small enough to move, is the whole advantage. The bill is coming either way. Whether you’re reading it or writing it is the choice.