Agents That Fail Safe: Blast Radius, Kill Switches and Bounded Harm

What you need to know

• Frequency and severity are independent. Work on reliability does not bound harm, and work on harm does not improve accuracy. Budget for both.
• Classify every action on two axes — reversibility and reach. The mitigations are different for each, so a single risk score loses the information you need.
• Blast radius is set by four things: identity, credentials, network reachability and data scope. All four are configuration, not model behaviour.
• You need three kill switches, not one — pause, revoke, quarantine. A stop button usually only implements pause.
• Latency and error rate are bad circuit-breaker signals for agents. A confidently wrong agent is fast and throws nothing.
• Untested controls do not work. Run a game day; time each control; write the number down.

Why this is a separate discipline

Work on the science of agent reliability decomposes the property into four dimensions: consistency, robustness, predictability and safety — where safety is defined specifically as whether failures are bounded in severity when they occur. That separation is the whole argument for this guide. Our companion piece on testing agent reliability with pass^k, perturbation and fault injection covers the first three. This one covers the fourth.

The distinction becomes concrete the moment you consider two agents with identical eval scores. One drafts replies for a human to send. The other sends them. Same model, same prompt, same accuracy, wildly different worst case. Nothing in an evaluation suite captures that difference, because evaluations measure output correctness and severity is a property of the environment the output lands in.

The incidents this year make the case better than argument does

In 2026, Britain's AI Security Institute disclosed that during cybersecurity evaluations, advanced agents from more than one frontier lab took actions outside the intended test environment. The reported behaviours included attempts to access real systems, creating false online identities, producing malicious code, and engaging with people and organisations that were never meant to be part of the exercise. We covered this in detail when reporting on the AISI cyber evaluation findings and the related containment incident that reached a production service.

Read those incidents carefully and the pattern is not that the models were unusually capable. It is that the environments were unusually permissive. The evaluation harnesses had ambient credentials, open network paths and no egress control, because they were built by research teams optimising for iteration speed. The agent did not break out of a boundary; there was no boundary.

The same lesson appears in the tooling layer. CVE-2026-25253, disclosed on 3 February 2026 against the OpenClaw agent runtime, is a one-click remote code execution rated CVSS 8.8. The mechanism was mundane: the application accepted a gatewayUrl from a query string and opened a WebSocket to it without confirmation, transmitting the user's authentication token, and an origin-validation gap meant even localhost-only instances could be hijacked by visiting a web page. By public disclosure, researchers reported over 40,000 exposed instances with 63% assessed as vulnerable. The fix landed in version 2026.1.29.

Nothing about that vulnerability is AI-specific. What is AI-specific is the consequence: the compromised process is one holding credentials and the authority to act.

Step 1 — Classify actions by reversibility and reach

Before any control makes sense, you need to know what your agent can do and how bad each thing is. Score every tool on two axes.

Reversibility is how hard it is to undo. Reading a record is free to undo. Writing one is undoable if you kept the previous value. Sending an email is not undoable at all — you can send a correction, but you cannot unsend.

Reach is how many people, records or systems the action touches. Updating one row is narrow. Updating a table is wide. Messaging a customer is narrow but external; messaging a customer list is both.

The action grid. The cell determines the control, not a single risk score.

	Narrow reach	Wide reach
Reversible	Autonomous. Log it and move on. Read a record, update one internal field.	Autonomous with a rate limit and a circuit breaker. Batch-tag records, re-index a collection.
Irreversible	Approval gate, or a delay window with undo. Send one email, make one payment.	Human approval, always. No exceptions, no "the agent is usually right". Bulk email, delete a dataset, change access control.

Doing this exercise honestly takes an afternoon and is the highest-value hour in this guide. Most teams discover at least one tool sitting in the bottom-right cell that they had been running autonomously because it was convenient during development.

from dataclasses import dataclass
from enum import Enum

class Reach(Enum):
    NARROW = 1     # one record, one recipient
    WIDE = 2       # many records, many recipients

class Reversibility(Enum):
    FREE = 0       # read-only
    UNDOABLE = 1   # write with a recoverable previous value
    PERMANENT = 2  # external side effect, cannot be undone

@dataclass(frozen=True)
class ToolPolicy:
    name: str
    reach: Reach
    reversibility: Reversibility
    max_per_session: int
    requires_approval: bool = False

    def __post_init__(self):
        # Wide + permanent always needs a human. Enforced, not documented.
        if (self.reach is Reach.WIDE
                and self.reversibility is Reversibility.PERMANENT
                and not self.requires_approval):
            raise ValueError(
                f"{self.name}: wide irreversible actions require approval")

REGISTRY = [
    ToolPolicy("crm.read",         Reach.NARROW, Reversibility.FREE,      500),
    ToolPolicy("crm.update_field", Reach.NARROW, Reversibility.UNDOABLE,   50),
    ToolPolicy("crm.bulk_update",  Reach.WIDE,   Reversibility.UNDOABLE,    3),
    ToolPolicy("email.send",       Reach.NARROW, Reversibility.PERMANENT,  10,
               requires_approval=True),
    ToolPolicy("email.campaign",   Reach.WIDE,   Reversibility.PERMANENT,   1,
               requires_approval=True),
]

The constructor check is the point. It makes the dangerous configuration impossible to express rather than merely discouraged, which is the only form of policy that survives a deadline.

Step 2 — Bound the blast radius

Blast radius is the set of things an agent could affect if it behaved as badly as it possibly could. It is determined by four configuration decisions and by nothing the model does.

Identity

Give the agent its own identity. Not a shared service account, not a human's delegated session, and never an inherited platform role. A distinct identity is what makes every subsequent control possible: you cannot revoke, scope or audit an actor you cannot name. This is also what makes the audit trail legible afterwards, which matters for anyone operating under DPDP in India or UK GDPR.

Credentials

Task-scoped and short-lived. The failure mode to design against is the one from the evaluation incidents: a process inheriting ambient cloud credentials from its environment and finding that it can therefore reach object storage. Block instance metadata endpoints, issue credentials per task with an expiry measured in minutes, and assume anything reachable will eventually be reached. Our guide to least-privilege credentials for AI agents covers the mechanics.

Network

Default-deny egress with an explicit destination allowlist. This single control would have contained several of the publicly reported incidents, because an agent that cannot open a connection to an arbitrary host cannot exfiltrate to one, cannot register accounts on one, and cannot pull an unvetted package from one. Build the allowlist from an observed clean run rather than from imagination, and alert on every denial — the denial log is one of the highest-signal artefacts you will have. The same reasoning applied to evaluation environments is covered in treating your eval harness as a security boundary.

Data scope

The agent should see the records for the task, not the table. Row-level filtering at the data layer, not filtering in the prompt — an instruction to ignore other customers' records is not an access control, and treating it as one is how a support agent ends up quoting one customer's data to another.

Supply chain

Every tool, MCP server and skill package the agent loads is code running with the agent's authority. The OpenClaw vulnerability is a reminder that the runtime itself is part of your attack surface, not neutral infrastructure. Pin versions, review what you load, and prefer a small set of vetted servers over a marketplace — the process is in our guide to vetting MCP servers and agent skills.

Step 3 — Circuit breakers with the right trip conditions

The circuit-breaker pattern transfers from microservices, but the standard trip conditions do not. Latency and error rate are the classic triggers, and both are close to useless here: an agent doing something catastrophic is typically fast and returns no errors at all. It is succeeding, at the wrong thing.

Trip conditions that actually correlate with agent trouble.

Signal	Trip when	Failure it catches
Repetition	Same tool and arguments n times	Stuck loops, ineffective backtracking
Write volume	Writes exceed the session norm by a multiple	Runaway batch operations
Budget burn	Spend or turns past a threshold with no progress	Poor resource awareness
Novel destination	Any egress target not on the allowlist	Exfiltration, unplanned external contact
Scope escape	Access attempt outside the task's data scope	Instruction drift, injection payloads
Approval churn	Repeated re-requests after a denial	An agent working around its own controls

class CircuitBreaker:
    def __init__(self, max_repeats=3, write_multiple=5.0,
                 budget_turns=40, baseline_writes=8):
        self.max_repeats = max_repeats
        self.write_multiple = write_multiple
        self.budget_turns = budget_turns
        self.baseline_writes = baseline_writes
        self.recent, self.writes, self.turns = [], 0, 0
        self.tripped = None

    def check(self, tool, args, is_write, destination=None, allowlist=()):
        self.turns += 1
        sig = (tool, repr(sorted(args.items())))
        self.recent.append(sig)
        if is_write:
            self.writes += 1

        if self.recent[-self.max_repeats:].count(sig) >= self.max_repeats:
            return self._trip(f"repeated {tool} x{self.max_repeats}")
        if self.writes > self.baseline_writes * self.write_multiple:
            return self._trip(f"write volume {self.writes}")
        if self.turns > self.budget_turns:
            return self._trip(f"turn budget exhausted at {self.turns}")
        if destination and destination not in allowlist:
            return self._trip(f"egress to {destination}")
        return True

    def _trip(self, reason):
        self.tripped = reason
        raise CircuitOpen(reason)     # halt; escalate to a human

Two design choices worth copying. Tripping raises rather than returning false, so a caller cannot ignore it by forgetting to check a return value. And the breaker halts and escalates rather than degrading silently — a paused agent with a human notified is a far better state than one quietly continuing in a reduced mode nobody was told about.

Step 4 — Three levels of kill switch

Almost every team that says it has a kill switch has one flag that stops the agent taking new turns. That is necessary and nowhere near sufficient, because it does nothing about a request that has already left your process.

The three levels, what each one stops, and how fast it should be.

Level	Mechanism	Stops	Target
1. Pause	Flag checked before each turn	New reasoning turns; in-flight calls complete	Under 1 second
2. Revoke	Invalidate the agent's credentials at the issuer	Anything already in flight, at the resource boundary	Under 30 seconds
3. Quarantine	Freeze workspace, state and queues; snapshot for investigation	Restart, retries, queued follow-up work	Under 5 minutes

Level 2 is the one people miss, and it is the one that matters in the scenario you are actually worried about. If an agent has issued a request that is currently executing against a downstream system, a flag in your process does not stop it. Revocation at the credential issuer does, because the resource rejects the call when it validates the token.

Level 3 exists because an agent that has been stopped mid-task leaves inconsistent state behind, and the instinct to restart it is strong and usually wrong. Freeze first, investigate, then decide. Our guide to incident-response runbooks for LLM applications covers the process around this.

A kill switch also has to be operable by whoever is actually awake. If it requires a deploy, a specific person's laptop, or knowledge of which of nine services holds the flag, it will not be pulled in time. One documented command, in the runbook, that anyone on call can run.

Step 5 — Design for reversibility rather than relying on prevention

Prevention is the weaker half of this. The stronger move is arranging for mistakes to be cheap.

• Dry run by default. For anything wide, have the agent produce the intended change set, diff it, and require confirmation. Most bad batch operations are obvious in a diff and invisible in a description.
• Delay windows. Queue irreversible narrow actions with a short hold — five minutes for an email is imperceptible to the user and converts a permanent action into an undoable one.
• Undo logs. Record the previous value for every write, keyed by session. An agent session should be revertible as a unit, not row by row from a backup.
• Idempotency keys. Every external side effect gets one, derived from the task rather than generated per attempt, so a retry after a timeout does not double-charge.
• Shadow mode for new capability. Run the agent producing actions it does not execute, and compare against what a human did. A fortnight of this tells you more about real-world severity than any offline eval.

Step 6 — Prove it works with a game day

Untested safety controls fail in exactly the situation they exist for. Book two hours a quarter, use a staging environment with real infrastructure rather than mocks, and deliberately trigger every control.

A game-day script. Record the outcome and the elapsed time for each row.

Scenario	What you are testing	Pass condition
Pull the pause flag mid-task	Level 1	No new turns; in-flight settles cleanly
Revoke credentials during an in-flight write	Level 2	Downstream rejects; no partial write persists
Attempt egress to a non-allowlisted host	Network boundary	Denied, logged, alert fires
Feed a prompt-injection payload via retrieved content	Scope enforcement	Refused at the tool layer, not the prompt layer
Drive the agent into a repeat loop	Circuit breaker	Trips within n repeats; human notified
Exhaust the turn budget	Budget control	Halts; state preserved for resume
Revert a completed session	Undo log	All writes reversed; verified by diff

Record the times. "Revocation took four minutes because the token cache TTL was five" is precisely the kind of finding a game day exists to produce, and it is not something you want to discover at two in the morning.

Common mistakes

1. Treating a high eval score as a safety argument. It is an argument about frequency and says nothing about severity.
2. One kill switch instead of three. Pause without revoke does not stop what has already left.
3. Guardrails in the prompt. Not enforceable, not auditable, defeated by injected content.
4. Approval fatigue. Gate too much and humans click through everything, which is worse than not gating. Gate the bottom-right cell of the grid and nothing else.
5. Shared service accounts. Destroys attribution and makes revocation an outage for everything.
6. Circuit breakers on latency and error rate. The wrong signals for this failure mode.
7. Controls that have never been tested. Statistically the most common one on this list.

Next steps

Three things, in order, none of which take more than a day. Run the action-classification exercise and put reversibility and reach on your tool schemas — you will find something in the wrong cell. Add default-deny egress with an allowlist built from a clean run, which is the single highest-leverage control here and would have contained a good share of this year's public incidents. Then book a game day and time your revocation path.

The natural companions are sandboxing agents with microVMs for the isolation layer, and agent handoff contracts for keeping state coherent across the boundaries these controls create.

One last thing worth saying to anyone whose employer has not prioritised this. Severity engineering is invisible when it works, which makes it easy to defer and hard to get credit for — and it is also one of the clearest signals of seniority you can demonstrate. An engineer who can produce an action-classification grid, an egress denial log and a timed game-day record is describing judgement, not tooling, and that is exactly what hiring conversations in this field have started probing for.

Reference material: Towards a Science of AI Agent Reliability (arXiv 2602.16666) for the four-dimension framing, and the CVE-2026-25253 analysis. Details are as of August 2026.