What a control plane is, and what it is not
The State of FinOps 2026 report, published by the FinOps Foundation in February 2026 on responses from 1,192 practitioners representing more than $83 billion in annual cloud spend, contains one number that reframes the whole discipline: 98% of respondents now manage AI spend, up from 31% two years earlier. That is not gradual adoption, it is a category appearing from nothing. The same survey reports that 73% of AI projects still overrun their budget, and that the single most-requested capability among practitioners is granular monitoring of AI spend broken down by tokens, LLM requests and GPU utilisation. Read those three findings together and the picture is unambiguous: almost everyone is now responsible for this cost, most of them cannot hold it, and what they are asking for first is visibility at the level of the individual call.
Visibility is the right first ask and the wrong place to stop. A dashboard that shows you spent four times your budget over a completed billing period is a very expensive way to learn something a finance email would have told you for free. The component that actually changes the outcome is the one that can decline the call before it is made — and that is what a control plane is. Define it narrowly, because the term is being stretched by marketing in both directions.
A control plane is not a gateway. A gateway routes, retries, caches, load-balances and terminates connections. It is an excellent place to install an enforcement hook, and most teams should put theirs there, but a gateway on its own usually knows only the calling API key. It cannot tell you that this particular request belongs to a reconciliation run for a specific customer, started by a specific person, with £4 of its £12 ceiling already spent. The finest-grained thing a gateway can block is a key, and blocking a key means blocking an entire application.
A control plane is not an observability tool. Tracing, token accounting and per-feature cost attribution are genuinely useful and you will want all three, but they describe the past. By the time a trace exists, the money has gone. Observability answers "what happened"; a control plane answers "may this happen", and the two questions need different data structures and different latency budgets.
A control plane is not a permission system. Deciding which tools an agent may call, and with what arguments, is a separate and equally important discipline — it is covered properly in designing the permission boundary around agent tools, and this article deliberately does not repeat it. Permissions answer "what may this agent touch". A control plane answers "how much may this agent consume, and what happens when it reaches the end of that". An agent can be perfectly permissioned and still bankrupt you.
What is left, once you have subtracted all three, is a small and specific thing: the layer that can say no and be believed. It holds the authoritative answer to whether a given unit of spend is permitted right now, for this principal, against this grant, and it returns a decision the caller is contractually obliged to honour. Everything in the rest of this article exists to make that sentence implementable.
The commonest architecture failure here is building the control plane as advice rather than as enforcement. If the calling code can ignore the decision — because the check is a helper function developers may or may not call, or because a failed check logs a warning and proceeds — then you have built an expensive linting tool. The decision must sit on the only path to the model provider, and the credential that reaches the provider must not be obtainable any other way.
This layer is being commercialised quickly, which is the clearest signal that the problem is real and widely felt. Between April and September 2026, venture investors put $435 million into 12 financings for companies selling enterprise AI agent security and governance. Established vendors are moving too: Boomi announced its Agent Control Plane on 2 September 2026, describing centralised visibility across agents and tools, identity and rate limits, curbs on token overruns, and the ability to hold high-risk transactional actions for human approval, deployable into public cloud, a customer's own VPC or on-premises. That feature list is worth reading not as a product recommendation but as a specification: it is a competent summary of what this layer has to do, arrived at independently by people selling it. The question for your team is whether you buy that or build it, and the answer often turns on how deeply it needs to reach into your existing identity and billing systems. If you have not yet made the broader build-or-buy call for your agent stack, the decision ladder between a managed agent runtime and your own harness is the right place to start; the rest of this guide assumes you have landed on building.
One more piece of context, because it explains why this is worth a fortnight of engineering. On IDC and Lenovo's numbers, the large majority of enterprises that start an agent initiative never get it into production — 88% of them. Gartner's outlook is similarly unsparing: it expects the majority of a projected 40%-plus of agentic projects abandoned before the end of 2027 to fail on runaway cost and absent risk controls rather than on anything to do with model quality. That is a forecast rather than an observation, so treat it as a directional signal. Even so, the causes it names are exactly what a control plane addresses. Escalating cost is what a ceiling stops. Inadequate risk control is what an audit trail answers. A team that can show a finance director a per-feature cost curve and a hard cap is a team whose pilot survives the budget review.
The unit of control is agent identity
Every ungovernable agent bill I have seen traces back to the same root cause: the application has one API key, and everything the application does is that key. The provider's own dashboard can only tell you what that key spent. Your own logs can tell you a little more if someone thought to add a tag. Nobody can tell you which customer, which feature or which person caused the spike, because at the moment of spending there was no subject to attribute it to.
The fix is conceptually old and worth stating plainly: give every agent run a principal. A principal is an identity that can hold a credential, own a budget, and appear in an audit record. It is the same idea as a service account, with the difference that agent runs are numerous, short-lived and hierarchical, so you need the idea to be cheap enough to instantiate thousands of times a day.
There are three scopes worth distinguishing, and most production systems need all three at once.
Run-scoped identity is minted when a unit of work begins and dies when it ends. It is the right subject for a per-run ceiling — "no single contract review may cost more than £3" — and the right subject for a trace. Its credential should be short-lived and audience-restricted, so that a leaked one is worth nothing an hour later. This is the scope teams skip, and it is the one that does the most work, because it is the only level at which you can express a bound on a single unit of business value.
Agent-scoped identity is stable across runs and belongs to a deployed agent definition: the reconciliation agent, the support triage agent, the document extraction agent. It is the right subject for rate limits, for the set of models an agent may use, and for the per-feature unit economics that finance will eventually ask for. When someone asks what the triage agent costs per month, this is the identity that answers.
Tenant-scoped identity is the customer, the business unit or the cost centre. It is the right subject for the monthly commercial ceiling, for the cross-charge, and for the conversation that starts "this customer is now unprofitable". For a UK software vendor selling into the public sector, or an Indian firm running a GCC engagement for an overseas parent, the tenant is usually also the unit that appears on an invoice, which is precisely why the ledger underneath it needs to be defensible.
The scopes nest. A run belongs to an agent, which belongs to a tenant, and a reservation made at the run level should consume the run's grant, the agent's grant and the tenant's grant simultaneously. Get that nesting explicit in the data model and a great many awkward questions answer themselves; leave it implicit and you will spend a quarter writing reconciliation scripts.
Do not attempt to reconstruct principals after the fact from log tags. Tags are advisory, they drift, they get forgotten on the code path someone added in a hurry, and they are absent exactly on the runs you most want to investigate. The principal has to exist before the first call, because it is the thing that holds the credential the first call needs.
The FinOps survey noted one practice that is worth copying directly: organisations are increasingly issuing developers personal token budgets treated as a managed resource, in the same way they might be issued a cloud sandbox with a cap. It works because it converts an abstract shared cost into a personal, visible allocation, and because a developer who has hit their own ceiling files a ticket rather than quietly burning the team's. If you are introducing principals anyway, developer principals are the least contentious place to start.
The data model: principals, grants and an append-only ledger
Here is the schema. It is written for PostgreSQL because that is what most teams already run, and the shape transfers to anything with transactions. Four concepts: principals, who can spend; budget_grants, how much they may spend and over what window; usage_ledger, an append-only record of every reservation and settlement; and enforcement_decisions, a record of what the control plane told each caller and why.
-- 1. Principals: the subject a budget attaches to.
CREATE TABLE principals (
principal_id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
scope text NOT NULL CHECK (scope IN ('tenant', 'agent', 'run')),
parent_id uuid REFERENCES principals (principal_id),
tenant_id uuid NOT NULL,
agent_slug text,
acting_for text, -- human user id, or NULL for scheduled work
status text NOT NULL DEFAULT 'active'
CHECK (status IN ('active', 'suspended', 'closed')),
created_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT non_tenant_needs_parent
CHECK (scope = 'tenant' OR parent_id IS NOT NULL)
);
CREATE INDEX principals_tenant_scope ON principals (tenant_id, scope);
CREATE INDEX principals_parent ON principals (parent_id);
-- 2. Budget grants: an allowance, in micro-units of currency, for a window.
CREATE TABLE budget_grants (
grant_id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
principal_id uuid NOT NULL REFERENCES principals (principal_id),
window_start timestamptz NOT NULL,
window_end timestamptz NOT NULL,
limit_micros bigint NOT NULL CHECK (limit_micros > 0),
currency char(3) NOT NULL,
on_exhaustion text NOT NULL DEFAULT 'block'
CHECK (on_exhaustion IN ('block','degrade','park','warn_only')),
policy_version text NOT NULL,
created_at timestamptz NOT NULL DEFAULT now(),
CHECK (window_end > window_start)
);
CREATE UNIQUE INDEX budget_grants_window
ON budget_grants (principal_id, window_start, window_end);
-- 3. Usage ledger: append-only. Every row is a fact that happened.
-- reserve = +estimate (a hold placed before the call)
-- settle = +/-delta (correction to true cost after the call)
-- release = -estimate (the hold given back; call never happened)
CREATE TABLE usage_ledger (
entry_id bigserial PRIMARY KEY,
principal_id uuid NOT NULL REFERENCES principals (principal_id),
grant_id uuid NOT NULL REFERENCES budget_grants (grant_id),
reservation_id uuid NOT NULL,
entry_type text NOT NULL CHECK (entry_type IN ('reserve','settle','release')),
delta_micros bigint NOT NULL,
model_id text,
input_tokens integer,
output_tokens integer,
decision_id uuid,
recorded_at timestamptz NOT NULL DEFAULT now()
);
-- A reservation is opened once and closed once. These two partial unique
-- indexes are what make retries idempotent instead of double-charging.
CREATE UNIQUE INDEX ledger_open_once ON usage_ledger (reservation_id)
WHERE entry_type = 'reserve';
CREATE UNIQUE INDEX ledger_close_once ON usage_ledger (reservation_id)
WHERE entry_type IN ('settle', 'release');
CREATE INDEX ledger_by_grant ON usage_ledger (grant_id, recorded_at);
-- Append-only, enforced by the database rather than by convention.
CREATE RULE usage_ledger_no_update AS ON UPDATE TO usage_ledger DO INSTEAD NOTHING;
CREATE RULE usage_ledger_no_delete AS ON DELETE TO usage_ledger DO INSTEAD NOTHING;
REVOKE UPDATE, DELETE ON usage_ledger FROM app_rw;
-- 4. Enforcement decisions: what we told the caller, and why.
CREATE TABLE enforcement_decisions (
decision_id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
principal_id uuid NOT NULL REFERENCES principals (principal_id),
grant_id uuid REFERENCES budget_grants (grant_id),
action text NOT NULL CHECK (action IN ('allow','block','degrade','park')),
reason_code text NOT NULL,
estimate_micros bigint,
available_micros bigint,
policy_version text NOT NULL,
decided_at timestamptz NOT NULL DEFAULT now()
);
CREATE INDEX decisions_by_principal ON enforcement_decisions (principal_id, decided_at);
-- 5. A cached balance per grant. The ledger is the truth; this is the row we
-- lock to serialise concurrent reservations, and it is reconcilable.
CREATE TABLE grant_balances (
grant_id uuid PRIMARY KEY REFERENCES budget_grants (grant_id),
committed_micros bigint NOT NULL DEFAULT 0,
updated_at timestamptz NOT NULL DEFAULT now()
);
CREATE FUNCTION init_grant_balance() RETURNS trigger LANGUAGE plpgsql AS $fn$
BEGIN
INSERT INTO grant_balances (grant_id) VALUES (NEW.grant_id);
RETURN NEW;
END;
$fn$;
CREATE TRIGGER budget_grants_balance AFTER INSERT ON budget_grants
FOR EACH ROW EXECUTE FUNCTION init_grant_balance();
-- Atomic reservation. The row lock on grant_balances is what makes this safe
-- under concurrency: two callers cannot both read the same free balance.
-- Returns remaining micros if the hold was placed, NULL if it was refused.
CREATE FUNCTION reserve_budget(
p_grant uuid,
p_reservation uuid,
p_estimate bigint,
p_model text
) RETURNS bigint LANGUAGE plpgsql AS $fn$
DECLARE
v_limit bigint;
v_committed bigint;
v_principal uuid;
BEGIN
SELECT g.limit_micros, g.principal_id, b.committed_micros
INTO v_limit, v_principal, v_committed
FROM budget_grants g
JOIN grant_balances b ON b.grant_id = g.grant_id
WHERE g.grant_id = p_grant
AND now() BETWEEN g.window_start AND g.window_end
FOR UPDATE OF b;
IF NOT FOUND OR v_committed + p_estimate > v_limit THEN
RETURN NULL;
END IF;
INSERT INTO usage_ledger (principal_id, grant_id, reservation_id,
entry_type, delta_micros, model_id)
VALUES (v_principal, p_grant, p_reservation, 'reserve', p_estimate, p_model);
UPDATE grant_balances
SET committed_micros = committed_micros + p_estimate, updated_at = now()
WHERE grant_id = p_grant;
RETURN v_limit - (v_committed + p_estimate);
END;
$fn$;
-- Nightly reconciliation: the cache must always equal the ledger.
SELECT b.grant_id, b.committed_micros, COALESCE(SUM(l.delta_micros), 0) AS ledger_micros
FROM grant_balances b
LEFT JOIN usage_ledger l ON l.grant_id = b.grant_id
GROUP BY b.grant_id, b.committed_micros
HAVING b.committed_micros <> COALESCE(SUM(l.delta_micros), 0);
Two design choices in there deserve an explanation, because they are the ones people argue about.
Why an append-only ledger rather than a mutable counter? A counter — a single spent_micros column you increment — is smaller, faster and completely adequate right up to the first time somebody asks a question about the past. Then it fails at everything. It cannot tell you what a run cost, only what a month cost. It cannot be reconciled against a provider invoice, because it has no line items. It cannot survive a bug, because when a miscount is discovered there is no history to replay and no way to distinguish the wrong number from the right one. And it cannot answer an auditor, because a number that can be overwritten is not evidence. The ledger costs you a table scan and a cached balance row; in exchange, every number your system has ever produced is derivable from primary facts and every correction is itself a fact. That trade is not close.
Why store currency in integer micro-units? Because floating-point money is a bug waiting for a scale at which it matters, and token pricing produces very small per-call amounts that accumulate over millions of calls. Micro-units — a millionth of a rupee or a penny — give you exact integer arithmetic with enough resolution that a single cheap call still registers as a non-zero amount. Do the conversion once, at the point where you price tokens, and never let a float near the ledger.
Store model_id, input_tokens and output_tokens on every settlement even though you have already converted them to money. Prices change, and when a provider adjusts a rate you will want to reprice history to compare like with like. Keeping the raw token counts means the repricing is a query; keeping only the amount means it is impossible. The same columns make the ledger the natural input to the cost model described in forecasting your LLM bill before launch, which is a far better forecast when it is fed by your own measured distribution rather than by a guess.
Enforcement: reserve first, reconcile after
This is the crux of the whole design, and it is where most home-grown attempts quietly fail. The awkward fact is this: you cannot know what a model call costs until after you have made it. Input tokens you can count in advance. Output tokens you cannot, because the model decides how much to write. So a naive ceiling check — "have we spent less than the limit? then proceed" — cannot bound the call it is about to authorise. It can only bound the calls that came before.
The resolution is borrowed from payments, where the same problem was solved decades ago at the petrol pump. You do not know what the customer will pump, so you authorise a pessimistic hold, let them pump, then capture the true amount and release the difference. Applied to model calls: reserve an upper-bound estimate before the call, then settle to the true cost when the response completes.
The estimate is not a guess if you construct it properly. Input tokens are countable. The output is bounded by the max_tokens you are about to pass, and you are passing one whether you think about it or not. Price both at the model's published rate and the reservation is a genuine ceiling on that call. It will usually be pessimistic — most completions come in well under max_tokens — and the settlement gives the surplus straight back.
| Strategy | How it works | Worst-case overrun | Cost to build | Use when |
|---|---|---|---|---|
| Post-hoc accounting only | Log usage after each call; alert when a threshold is crossed | Unbounded — limited only by how fast you can notice | Low | Never as the only control; fine as a reporting layer on top |
| Check-before-call on committed spend | Read spend so far; proceed if under the limit | One full call per concurrent worker, past the ceiling | Low | Single-threaded batch jobs with small, uniform calls |
| Pre-flight reservation plus reconcile | Hold a priced upper bound, call, settle to true cost | Zero above the ceiling; temporarily pessimistic below it | Moderate | The default for anything with concurrency or real money |
| Pre-paid credit deduction | Principal holds a credit balance debited at reservation | Zero, and self-service top-up is a product feature | Moderate to high | Multi-tenant SaaS where end customers buy usage directly |
| Gateway rate limit only | Requests per minute per key, enforced at the proxy | Bounds speed, not spend; one key can still be a whole app | Very low | Alongside a ceiling, never instead of one |
Here is the pattern in Python, wrapped so that application code cannot forget any part of it. The important structural detail is the try/finally: a run that crashes, times out or is killed mid-flight must release its hold, or your ceilings will drift downwards over weeks until a healthy system starts refusing work it can afford.
import uuid
import contextlib
from dataclasses import dataclass
MICROS = 1_000_000 # micro-units of currency per unit
@dataclass(frozen=True)
class Price:
input_per_mtok: float # currency units per 1M input tokens
output_per_mtok: float
@dataclass(frozen=True)
class Usage:
input_tokens: int
output_tokens: int
# Loaded from config and versioned, never hard-coded in application code.
PRICES: dict[str, Price] = load_price_table()
def cost_micros(model: str, input_tokens: int, output_tokens: int) -> int:
p = PRICES[model]
units = (input_tokens * p.input_per_mtok
+ output_tokens * p.output_per_mtok) / 1_000_000
return round(units * MICROS)
class BudgetExceeded(Exception):
def __init__(self, decision_id: uuid.UUID, reason: str) -> None:
super().__init__(f"budget refused ({reason}); decision={decision_id}")
self.decision_id = decision_id
self.reason = reason
class ControlPlane:
"""Thin wrapper over the SQL functions. One connection pool, no ORM."""
def __init__(self, db):
self.db = db
def reserve(self, grant_id, principal_id, model, estimate_micros) -> uuid.UUID:
reservation_id = uuid.uuid4()
remaining = self.db.scalar(
"SELECT reserve_budget(%s, %s, %s, %s)",
(grant_id, reservation_id, estimate_micros, model),
)
if remaining is None:
decision_id = self.db.scalar(
"""INSERT INTO enforcement_decisions
(principal_id, grant_id, action, reason_code,
estimate_micros, policy_version)
VALUES (%s, %s, 'block', 'grant_exhausted', %s, %s)
RETURNING decision_id""",
(principal_id, grant_id, estimate_micros, POLICY_VERSION),
)
raise BudgetExceeded(decision_id, "grant_exhausted")
return reservation_id
def close(self, grant_id, reservation_id, estimate_micros, actual_micros):
"""Settle or release. Idempotent: ledger_close_once rejects a second
attempt, so a retried settlement cannot double-charge."""
delta = actual_micros - estimate_micros
entry = "release" if actual_micros == 0 else "settle"
self.db.execute(
"SELECT close_reservation(%s, %s, %s, %s)",
(grant_id, reservation_id, entry, delta),
)
@contextlib.contextmanager
def metered(cp: ControlPlane, grant_id, principal_id, model,
prompt_tokens: int, max_output_tokens: int):
"""Reserve a priced upper bound, yield, then reconcile — always."""
estimate = cost_micros(model, prompt_tokens, max_output_tokens)
reservation = cp.reserve(grant_id, principal_id, model, estimate)
box: dict[str, Usage | None] = {"usage": None}
try:
yield box
finally:
usage = box["usage"]
if usage is None:
# Crash, timeout or kill before any usage was recorded:
# give the whole hold back rather than stranding it.
cp.close(grant_id, reservation, estimate, 0)
else:
actual = cost_micros(model, usage.input_tokens, usage.output_tokens)
cp.close(grant_id, reservation, estimate, actual)
def run_completion(cp, client, grant_id, principal_id, model,
messages, max_output_tokens: int) -> str:
prompt_tokens = client.count_tokens(model=model, messages=messages)
with metered(cp, grant_id, principal_id, model,
prompt_tokens, max_output_tokens) as box:
parts: list[str] = []
try:
with client.messages.stream(
model=model, messages=messages, max_tokens=max_output_tokens
) as stream:
for text in stream.text_stream:
parts.append(text)
# Streaming APIs report usage only on the terminal event,
# so the authoritative counts exist only at this point.
final = stream.get_final_message()
box["usage"] = Usage(final.usage.input_tokens,
final.usage.output_tokens)
except Exception:
# A stream that dies half-way was still billed for what it
# produced. Charge the tokens we received rather than
# releasing the entire hold and under-counting the run.
box["usage"] = Usage(prompt_tokens,
approx_tokens("".join(parts)))
raise
return "".join(parts)
The four failure modes this pattern has to survive
Concurrency races. The classic bug is read-then-write: two workers both read a balance showing £5 free, both decide their £4 call fits, and the grant ends £3 over. Nothing in application code fixes this; the check and the deduction must be one atomic operation. In the schema above that is the FOR UPDATE OF b row lock inside reserve_budget, which serialises every reservation against the same grant. It costs you a lock hold measured in microseconds, and it is the single most important line in the file. If your control plane lives in Redis rather than Postgres, the equivalent is a Lua script, for the same reason — see the next section.
Retries that double-charge. Agent frameworks retry aggressively, and a retry is genuinely a new call that genuinely costs money, so each attempt should get its own reservation. What must not happen is the same reservation being settled twice because a settlement request timed out and was retried at the network layer. The ledger_close_once partial unique index makes the second settlement fail at the database rather than succeed silently, which converts a financial error into a caught exception.
Streaming responses. With a streaming API, usage metadata arrives on the terminal event, after every token has already been generated and billed. This has two consequences worth internalising. The reservation must be sized on max_tokens, because that is the only bound you have while the stream is open; and a stream that is abandoned half-way still cost you money, so the exception path has to charge for what was received rather than releasing the whole hold. The code above does both. Teams that get this wrong tend to discover it as a persistent gap between their internal numbers and the provider invoice.
Sub-agent recursion. This is the one that produces the genuinely frightening bills. A supervisor spawns five workers; each worker is issued a fresh run-scoped ceiling; the run as a whole spends five times its stated cap while every component stays within its own limit. Worse, a worker that can itself spawn workers turns a linear overrun into an exponential one. The rule is that a ceiling is a property of the run tree, not of a node: give the child its own principal so attribution survives, but make every descendant's reservation land against the root run's grant. Then add a fan-out cap and a depth cap next to the spend cap, because an agent that spawns children faster than it spends money will exhaust a connection pool long before it exhausts a budget. If you are choosing between orchestration shapes in the first place, the supervisor, swarm and fan-out patterns compared sets out how differently each of them consumes.
Every article here is written by a Verified Builder. Want your name on the next one?
AI Tech Connect lists AI engineers, founders and researchers across India and the UK — and the people hiring browse it to find them. Adding your profile is free.
Become a Verified Builder →Quotas that survive reality
A spend ceiling bounds total exposure. It says nothing about the rate at which that exposure accumulates, and rate is what turns a bug into an incident. A looping agent with a generous monthly grant can exhaust it between lunch and the end of the afternoon, and a ceiling that only fires at the end gives your alerting no time to matter. So you need a second control operating on a much shorter horizon, and the shape of that control matters.
| Algorithm | State per key | Burst behaviour | Boundary accuracy | Best for |
|---|---|---|---|---|
| Fixed window | One counter, one window id | Allows 2x the limit across a boundary | Poor — the classic edge spike | Coarse abuse limits where 2x is survivable |
| Sliding window log | One timestamp per event | Exact; no boundary artefact | Exact | Low-volume, high-value limits where memory is free |
| Sliding window counter | Two counters, weighted by overlap | Smooths the boundary; slightly approximate | Good | High-volume request-rate limits at the gateway |
| Token bucket | Token count plus last-refill timestamp | Deliberate, bounded burst up to capacity | Good | LLM traffic — cost varies per call, so charge by weight |
| Leaky bucket | Queue depth plus drain rate | No burst; smooths output completely | Good | Protecting a fragile downstream from any spike at all |
For agent traffic the token bucket is usually the right default, for a reason specific to this workload: not all calls are equal. A request-counting limiter treats a 200-token classification and a 180,000-token document analysis identically, which is precisely backwards. A token bucket lets you charge each call a cost proportional to its estimated token consumption, so the limiter naturally throttles expensive work harder than cheap work. It also permits a controlled burst up to capacity, which matters because real agent runs are bursty by nature and a limiter that forbids all bursting will make a correctly-behaving agent look broken.
The non-negotiable property is atomicity. A limiter implemented as read-compute-write from the client is not a limiter; under concurrency it is a suggestion. In Redis the answer is a Lua script, because Redis executes a script to completion without interleaving other commands. In Postgres the answer is a single statement or a locking function, as above. There is no third option that is both correct and simple.
-- token_bucket.lua — evaluated atomically by Redis.
-- KEYS[1] : bucket key, e.g. "quota:agent:triage:tokens"
-- ARGV[1] : capacity (max tokens the bucket can hold)
-- ARGV[2] : refill_per_second (steady-state allowance)
-- ARGV[3] : now_ms (caller's clock, ms since epoch)
-- ARGV[4] : cost (weight of this request)
-- ARGV[5] : ttl_seconds (idle expiry, >= capacity / refill)
-- Returns : { allowed, tokens_left, retry_after_ms }
local capacity = tonumber(ARGV[1])
local refill = tonumber(ARGV[2])
local now_ms = tonumber(ARGV[3])
local cost = tonumber(ARGV[4])
local ttl = tonumber(ARGV[5])
local state = redis.call('HMGET', KEYS[1], 'tokens', 'ts')
local tokens = tonumber(state[1])
local ts = tonumber(state[2])
if tokens == nil or ts == nil then
tokens = capacity
ts = now_ms
end
-- Refill for elapsed time, clamped so a clock skew backwards cannot
-- mint tokens and cannot strand the bucket either.
local elapsed = math.max(0, now_ms - ts) / 1000.0
tokens = math.min(capacity, tokens + elapsed * refill)
local allowed = 0
local retry_after_ms = 0
if tokens >= cost then
tokens = tokens - cost
allowed = 1
else
retry_after_ms = math.ceil(((cost - tokens) / refill) * 1000)
end
redis.call('HSET', KEYS[1], 'tokens', tokens, 'ts', math.max(ts, now_ms))
redis.call('EXPIRE', KEYS[1], ttl)
return { allowed, math.floor(tokens), retry_after_ms }
import time
class TokenBucket:
"""Atomic because the whole decision happens inside one Lua evaluation.
Never implement this as GET, compute, SET from the client."""
def __init__(self, redis_client, script_source: str):
self.redis = redis_client
self.script = redis_client.register_script(script_source)
def allow(self, key: str, cost: int, capacity: int,
refill_per_second: float) -> tuple[bool, int, int]:
ttl = int(capacity / refill_per_second) + 60
allowed, left, retry_ms = self.script(
keys=[key],
args=[capacity, refill_per_second,
int(time.time() * 1000), cost, ttl],
)
return bool(allowed), int(left), int(retry_ms)
# Three layers, checked in increasing order of cost to evaluate.
def admit(bucket, cp, req) -> None:
ok, _, retry_ms = bucket.allow(
f"quota:agent:{req.agent_slug}:tokens",
cost=req.estimated_tokens,
capacity=600_000, # one minute of burst headroom
refill_per_second=10_000, # 600k tokens/minute steady state
)
if not ok:
raise RateLimited(retry_after_ms=retry_ms)
# Per-run and per-tenant spend ceilings are checked by reserve_budget
# inside metered(); the bucket only bounds the rate of arrival.
Note the clock handling. Passing now_ms from the caller keeps the script deterministic and easy to test, but it means a client with a badly skewed clock could otherwise mint tokens; the math.max(0, ...) on elapsed time and the math.max(ts, now_ms) on the stored timestamp between them prevent a backwards jump from either creating free capacity or freezing the bucket. If you would rather not trust callers at all, use redis.call('TIME') inside the script instead and accept that the script is then non-deterministic in the replication sense.
What to do when a ceiling is hit mid-run
Blocking is the honest default and the wrong universal answer. A run that has already spent nine minutes and £2.40 gathering context, and then hits its ceiling on the final summarisation call, should not throw all of that away if a cheaper completion would finish the job acceptably. The decision belongs in policy — the on_exhaustion column on the grant — rather than scattered through application code.
| Situation | Response | What the caller sees | Why |
|---|---|---|---|
| Run ceiling exceeded on the first call | Fail hard | Typed error with a decision id | Nothing is lost; the estimate itself was out of budget |
| Run ceiling exceeded mid-run, work is salvageable | Degrade to a cheaper model tier | Result, flagged as degraded in the response envelope | Preserves sunk context; quality drop must be visible |
| Run ceiling exceeded, no acceptable cheaper tier | Park for approval | Pending state, run resumable for a bounded window | A human can authorise the overspend on the spot |
| Tenant monthly grant exhausted | Block, and notify the account owner | 402-style refusal with a top-up path | This is a commercial decision, not an engineering one |
| Per-minute token bucket empty | Retry with backoff | 429 with retry_after_ms from the bucket |
Transient by construction; the bucket refills |
| Spend rate is anomalous, not merely high | Suspend the principal and page | Hard refusal, run terminated | A loop or a compromised credential — not a budget event |
Make the degraded path a first-class, tested outcome rather than an emergency fallback. If a cheaper tier is ever going to serve a real user, it belongs in your evaluation suite at the same quality bar as the primary path — which is the argument made at length in setting a reliability target and building the harness that qualifies against it. A fallback nobody has measured is a quality regression you have scheduled for your worst day.
Approval gates are an integration point, not the product
When a run is parked — because it exceeded its ceiling, or because it is about to take a high-value transactional action — something has to hold it and something has to let a human decide. The temptation is to build that review experience into the control plane, because the control plane is where the pause happens. Resist it.
The control plane's job at this boundary is narrow and should stay narrow: durably record that the run is suspended, write an enforcement_decisions row with action = 'park', emit an event carrying the run identifier, the principal, the estimate, the remaining balance and enough context for a decision, and expose exactly two operations — approve with an optional grant top-up, and reject. That is the entire contract. Approvals must be idempotent, they must carry the approver's identity into the ledger, and the pending state must expire rather than accumulating parked runs nobody will ever action.
Everything else — queue design, routing, service levels, reviewer interfaces, what happens to a decision nobody makes — is a substantial domain in its own right, and it is covered in designing the human review queue behind agent escalation. Keeping the two components separate is what lets you change your review tooling without touching the thing that enforces your ceilings, and that separation will repay itself the first time you replace the review UI.
Making it auditable
The test to design against is specific: six months from now, someone who is not on your team must be able to reconstruct a decision without asking you. They will be a finance analyst querying an invoice line, an internal auditor sampling controls, or a customer's security reviewer during a renewal. If answering them requires an engineer to write a one-off query against application logs, you do not have an audit trail, you have an archaeology project.
Four things have to be preserved for the retention period. Who spent — the principal, and through acting_for the human or customer it was acting on behalf of. What was spent — a ledger entry per reservation and settlement, carrying model, token counts and amount, so a monthly provider invoice can be reconciled line by line against your own records rather than accepted on faith. What was decided — every enforcement decision including the allows, with the policy version that produced it, so that "was the control on in July?" is a query rather than a recollection. And who approved — the identity behind any parked run that was released, stored next to the ledger entries that followed it.
The regulatory framing differs by market, and it is worth being precise because both markets are routinely misdescribed. UK teams selling into financial services or the public sector are answering to existing sector regulators — the ICO on data protection, the FCA on operational resilience and outsourcing — rather than to any AI-specific statute: as of September 2026 the UK has no AI Act and no AI Bill before Parliament, and a vendor questionnaire will reflect that by asking sector-standard control questions rather than AI-specific ones. Indian teams are building towards the DPDP framework, whose obligations are phased with substantive compliance falling due on 13 May 2027. On cross-border transfer, DPDP operates a negative-list model: transfers are broadly permitted unless the government notifies a country as restricted, and none has been notified. It is not a localisation mandate, whatever a procurement template may assert. What both markets share is a preference for immutable records over reconstructed ones, and that is a design decision you make once, at the schema, not later under deadline.
The deployment question follows from the same buyers. Boomi's September 2026 announcement explicitly offered public cloud, customer VPC and on-premises deployment, and that is not a marketing flourish — it is the shape of the requirement. A UK local-authority supplier or an Indian BFSI vendor will often be told that the governance layer must sit inside the customer's own boundary even when the model calls leave it. Building your own control plane has the useful side effect that this is a configuration choice rather than a vendor negotiation.
"We built the ledger for the finance team and it turned into the product analytics we never got round to writing. Within a fortnight we could see that one workflow was costing more per run than the customer paid for it monthly. We would never have found that in a dashboard, because the dashboard was aggregated at exactly the level that hid it."
— Verified Builder · Bengaluru, IndiaThat observation generalises. Once every call is attributed to a principal and every principal is attached to an agent and a tenant, per-feature unit economics fall out of the same ledger for free. Cost per run, cost per successful outcome, cost per customer, the distribution rather than the mean — all of it is a GROUP BY over data you were already keeping for compliance. Teams that build cost attribution separately from governance end up maintaining two systems that disagree; teams that build one get the second for the price of a query. And the distribution is the part that matters, because agent cost is heavily long-tailed and a mean will reliably understate what your worst decile of runs is doing to you.
Rolling it out without stopping the team
The fastest way to kill a control plane is to switch it on. Hard ceilings introduced into a system nobody has measured will block legitimate work within the hour, engineers will route around the blockage, and the component will be quietly disabled by the end of the week. Introduce it the way you would introduce any other interception layer: observe first, then warn, then enforce softly, then enforce.
| Phase | Typical duration | What is switched on | Exit criteria |
|---|---|---|---|
| 1. Shadow | 2 to 3 weeks | Principals minted, ledger written, decisions recorded as allow regardless of balance |
Ledger reconciles to the provider invoice within 2%; every run has a principal |
| 2. Alerting | 2 weeks | Grants sized from observed p95 per run; breaches alert, nothing blocks | Fewer than 5 alerts per week that are not genuine anomalies |
| 3. Soft ceilings | 2 to 4 weeks | Run-scoped ceilings enforced with degrade or park; tenant ceilings still warn-only | Degraded-path quality accepted; parked runs actioned within the service level |
| 4. Hard ceilings | Ongoing | All scopes enforced; token buckets live; suspension on anomalous rate | Steady state — review grant sizing quarterly against the ledger |
Two details make the difference between a rollout that lands and one that stalls. Size the grants from your own shadow data rather than from intuition: take the p95 cost per run from phase one and set the ceiling meaningfully above it, because a ceiling set at the mean will block a quarter of legitimate work and destroy trust in the component on day one. And give developers a self-service view of their own principals and balances from phase two onward, since a limit you can see coming is a constraint and a limit that arrives without warning is an outage. That is the same logic behind issuing developers personal token budgets as a managed resource — the FinOps survey found organisations increasingly doing exactly that, and it works because the allocation is visible before it binds.
Start with the ledger. It is the piece that is hardest to retrofit, because every day you run without it is a day of history you cannot reconstruct, and it is the piece everything else reads from. The enforcement function is an afternoon once the schema exists. The token bucket is an afternoon. The approval integration is a week. The schema is the decision you will live with, and it is the one worth getting right before you write the first line of the wrapper. If you have a profile on this site and you have built one of these, put the ledger design on it — the people browsing Builder profiles are looking for exactly this kind of unglamorous infrastructure work, because it is the work that decides whether an agent pilot becomes a product.