The AI Cost & Governance Glossary

Token

The unit AI models read and write, and the unit you're billed in. A token is a chunk of text — often a word, a piece of a word, or a punctuation mark. Models don't see letters or words; they see tokens.

Input tokens (prompt tokens)

The tokens you send into the model — your prompt, the instructions, any documents or history you include. You pay for these whether or not the answer is any good.

Output tokens (completion tokens)

The tokens the model generates back to you. Output tokens almost always cost more per token than input tokens — often 3–5× more — because generating text is the expensive part.

Reasoning tokens

Extra output tokens some models generate as internal "thinking" before they produce a final answer. You're billed for them, but you usually never see them — they're the model working through the problem.

Cached tokens

Input tokens the provider recognizes from a previous request and serves at a steep discount instead of full price. Distinct from a full cache hit — here the model still runs, it just doesn't re-charge full rate for the repeated part of your prompt.

Context window

The maximum number of tokens a model can consider at once — input plus output combined. It's the model's working memory for a single call. Go over it and the request fails or silently drops the oldest content.

Model tier

The rough class of a model by capability and price. Providers offer a range — small/fast/cheap up to large/slow/expensive — and the price gap between tiers is often 10–30×.

Cost per 1K tokens

The standard way pricing is quoted: dollars per 1,000 tokens, listed separately for input and output. Providers increasingly quote per million tokens, but the per-1K figure is still the mental unit for estimating a call.

Billable tokensTokenTotal

The tokens you actually charge a customer for — as opposed to the tokens you merely consumed. When you resell AI, billable tokens are the input and output tokens attributed to a specific customer of record, rated at your price (your provider cost plus your margin) rather than the raw provider cost.

Prompt caching

A provider feature that stores the repeated front portion of your prompt so you don't pay full price to send it again. You mark the stable part (instructions, examples, a document); the provider bills it at a discount on subsequent calls. See cached tokens.

Semantic caching

Serving a stored answer when a new request means the same thing as a previous one — even if the wording is different — so the model never runs and the provider is never billed. Unlike prompt caching (a provider discount), a semantic cache hit costs nothing at the provider.

Model routing

Automatically sending each request to the cheapest model that can handle it, instead of defaulting everything to one expensive model. The routing decision happens before the call, based on the task.

Provider arbitrage

Choosing among providers (or hosts) that serve comparable models at different prices, and sending traffic to the cheaper one. The same open model is often available from several vendors at meaningfully different rates.

Batch API

A cheaper, slower processing mode where you submit many requests at once and accept results within a delay window (often up to 24 hours) in exchange for a large discount, commonly around 50%.

Prompt compression

Shrinking a prompt to fewer tokens without losing what the model needs — trimming boilerplate, deduplicating context, summarizing history — so you pay for less input.

Cost attribution

Tying each dollar of AI spend to who spent it and why — the person, team, project, or agent responsible — rather than seeing one lump sum on a provider invoice. Attribution is the difference between "we spent $40,000 on AI" and "the research team's document pipeline spent $40,000."

Chargeback

Billing each internal team for the AI spend it actually caused, moving the cost onto that team's budget. It makes teams feel their own usage, which is the fastest way to curb waste.

Showback

The lighter cousin of chargeback: showing each team what it spent without moving the money. It creates visibility and accountability without the friction of internal billing.

Cost per successful task

Spend divided by successful outcomes, not by calls or tokens. It's the metric that reveals whether AI is actually efficient, because a cheap call that fails and gets retried three times is not cheap.

Unit economics

The cost of AI to serve one unit of your business — one customer, one transaction, one support ticket — so you can tell whether a feature makes or loses money as it scales.

Run-rate

Your current spend projected forward over a full period, usually annualized, to answer "if today's pace held, what would this cost us for the year?"

GL allocation

Mapping AI spend to the right lines in your general ledger — the accounting system of record — so it lands in the correct cost center, department, or project code instead of one catch-all "software" bucket.

Invoice reconciliation

Matching your own records of what was spent against the provider's invoice, line by line, to confirm they agree — and to catch discrepancies before you pay.

FinOps

The discipline of managing cloud (and now AI) spend as a shared, ongoing practice across engineering, finance, and product — bringing accountability to variable, usage-based costs. The FinOps Foundation maintains the standard framework.

Usage-based billingTokenTotal

Charging customers in proportion to what they actually consume — tokens, calls, or compute — instead of a flat fee, seat count, or user-count proxy. It aligns price with cost: because AI cost scales with tokens, usage-based billing is the only pricing that keeps a reseller's margin stable as customers grow.

Revenue leakageTokenTotal

Revenue a business could bill for but doesn't — because of a pricing proxy, an un-billed overage, a dropped metering event, or a missed reconciliation. For AI resellers it's the gap between usage-based cost and proxy-based price; industry estimates put it at 3–9% of revenue for usage-based software.

Metering accuracyTokenTotal

How completely and correctly a system counts the usage it bills. A meter is accurate when every unit of consumption is captured exactly once — idempotently, under load, attributed to the right customer. The maxim: metering accuracy is revenue accuracy — every dropped or double-counted event is money mis-billed.

Credit burndownTokenTotal

Drawing down a customer's prepaid balance as they consume, and recognizing that revenue on burn rather than at purchase. Prepaid credits are deferred revenue when sold; each metered unit of usage converts a slice of that liability into recognized revenue — which is why accurate metering is also an accounting control, not just a billing one.

Virtual AI KeyTokenTotal

A governed stand-in for a raw provider API key: a key you issue per team, person, project, or agent that carries its own budget, its own limits, and full attribution — while the real provider key stays hidden. The concept mirrors the virtual credit card, where each card has its own limit and owner. (In TokenTotal these are issued as tk_live_… keys.)

Binding keyTokenTotal

A second factor attached to a virtual key so the key string alone isn't enough to spend. A token that leaks without its binding key is a dead key — exfiltrating the string buys an attacker nothing.

Budget cap (hard cap)TokenTotal

A spending limit enforced before a call is sent, not an alert fired after. When the cap is reached, the next call is refused — no spend occurs. A hard cap is a circuit breaker; an alert is a smoke detector that may or may not wake anyone.

BYOK (Bring Your Own Key)

An arrangement where you keep your own provider account and credentials, and the governing layer uses your keys on your behalf rather than reselling access. Your provider relationship, spend, and data path stay yours.

PII pre-flightTokenTotal

A check that scans a prompt for personal or sensitive data before it reaches the model provider, and blocks or redacts it if found. "Fail-closed" means a fault stops the call rather than letting it through — the safe default when a check can't complete.

Anomaly detectionTokenTotal

Watching spend or usage for sudden deviations from the normal pattern and flagging them the instant they happen, so a spike is caught on the request path rather than in next month's report.

Append-only auditTokenTotal

A record that can be added to but never edited or deleted, so the history of what happened is tamper-evident. It's the difference between a log an insider can quietly rewrite and one an auditor will accept as evidence.

AI agent

An AI system that doesn't just answer once but works toward a goal over multiple steps — planning, calling tools, reacting to results, and looping until done. Because an agent can make many calls per task, it can spend far more than a single prompt, and it spends without a human watching each step.

Subagent

A smaller agent spawned by a parent agent to handle a piece of the work. Subagents let one task fan out into many, which is powerful — and a spend-attribution challenge, because their cost needs to roll back up to the agent that launched them.

MCP (Model Context Protocol)

An open standard for how AI agents connect to external tools and data sources in a consistent way — a common "plug" so any compatible agent can use any compatible tool. It's how modern agents reach beyond the model to your systems.

Agent identity

Giving an autonomous agent its own distinct identity — its own key, budget, and record — rather than having it act under a human's credentials. Without it, an agent's spend and actions are indistinguishable from the person who launched it.

Tool call

A single action an agent takes to use an external capability — query a database, search the web, run a function — as opposed to just generating text. Each tool call is a step where the agent reaches outside the model.

AI gateway

A single point that all AI traffic passes through on its way to the providers — the place where keys are resolved, budgets are checked, calls are logged, and policies are enforced. It's the choke point that makes control possible, because everything flows through it.

Spend control planeTokenTotal

The layer that governs AI spend across an organization: it issues virtual keys, enforces budgets before calls, attributes every dollar, and produces the audit record. "Observability" tells you what you spent; a control plane decides whether you spend it — the enforcement happens before the call, not after. (This is TokenTotal's category.)

SOC 2

A widely used security-and-controls audit standard (from the AICPA) that assesses how well a service organization protects customer data across areas like security, availability, and confidentiality. A SOC 2 report is often the first thing an enterprise buyer asks for.

ISO 27001

An international standard for an information security management system (ISMS) — a structured, certifiable program for managing security risk across an organization. Where SOC 2 is common in the US, ISO 27001 is the global benchmark.

ISO 42001

The first international standard for an AI management system (AIMS) — a certifiable framework specifically for governing how an organization builds and runs AI responsibly, including risk, oversight, and accountability. It's the AI-specific counterpart to ISO 27001.

NIST AI RMF

The AI Risk Management Framework from the US National Institute of Standards and Technology — a voluntary, widely referenced guide for identifying and managing AI risks across a system's life cycle. Not a certification, but a common language for AI governance.

Audit trail

The connected, time-ordered record of who did what, when, and under what authorization — the evidence you produce when someone asks "what happened?" For AI spend, a good audit trail records intent (the budget authorized) alongside execution (what actually ran).

ASC 606 (for AI usage)TokenTotal

The revenue-recognition standard that requires you to recognize revenue as a performance obligation is satisfied. For usage-based AI, that means recognizing revenue as tokens are consumed — not when a contract is signed or credits are sold. Prepaid credits are deferred revenue drawn down on burn; a spreadsheet can't defend that treatment to an auditor.