Context engineering for long-horizon AI agents

What you need to know

• Context engineering is not prompt engineering. Prompt engineering is how to ask. Context engineering is what the agent knows, sees and remembers at the moment of action — memory, policies, tool outputs, prior-step history and sub-agent visibility boundaries.
• The window is a budget, not a bucket. Effective context is smaller than the advertised window. Recall degrades before the hard limit. Spend tokens deliberately.
• Order matters. Put the most decision-relevant material first, right after the system prompt. Models attend most reliably to the start and the end.
• Memory is structured, not a dump. Short-term working context, long-term semantic and episodic memory, and periodic summarisation each have a job.
• Tool outputs are the silent killer. A raw 40k-token API response will rot a window faster than anything. Filter, truncate and clear.
• RAG, MCP and long context are complementary. Retrieval for breadth, MCP for governed delivery, long context for cohesion. Most enterprise teams combine all three.
• Compaction keeps long runs alive. Summarise the old, reinitiate with the summary plus the live working set, preserve open tasks verbatim.
• Four failure modes recur: poisoning, distraction, clash and rot. You catch them with tracing, provenance and mid-window evaluation.

If you have shipped a single-turn assistant, you have done prompt engineering: you tuned the wording, added a few examples, and the quality jumped. Then you built an agent that plans, calls tools, reads their results, calls more tools, and runs for an hour — and the careful prompt stopped being the thing that mattered. What mattered was everything else in the window by turn eighty: the stale tool output, the half-remembered constraint, the contradictory note a sub-agent left behind. That gap between a good prompt and a reliable agent is the territory of context engineering, and as of June 2026 it is the discipline most teams are underinvesting in. This guide is deliberately vendor-neutral on the ideas, but the worked examples use Claude (Anthropic) because that is our house model and its agent tooling is among the most explicit about these patterns.

What context engineering actually is

The cleanest definition: prompt engineering answers how to ask; context engineering answers what the agent knows, sees and remembers at the moment of action. The term was popularised by Andrej Karpathy in mid-2025 and formalised over the following year as agents moved from demos into long-running production work. It is the engineering of agent state — and that state is broader than most people assume. It includes the system prompt and persona; the memory the agent has loaded for this user and this task; the corporate constraints and policies it must obey; the outputs of every tool it has called so far; the history of its own prior steps and decisions; and, for multi-agent systems, the visibility boundaries that govern what each sub-agent can and cannot see.

Put differently: prompt engineering is one message; context engineering is the whole window, assembled fresh on every turn, evolving across a run that might span hundreds of turns. The prompt is a subset of the context. Getting the prompt perfect while ignoring everything else in the window is like writing a flawless opening sentence for a meeting where nobody briefed the attendees, half the documents on the table are out of date, and two people are working from contradictory instructions.

The context budget and ordering

The first mental shift is to stop thinking of the context window as a bucket you fill and start thinking of it as a budget you spend. Every token you add has a cost in money, in latency, and — most importantly — in the model's ability to find the signal. The advertised window size is a ceiling, not a target. As of June 2026, the practical guidance is to allocate the budget across categories and enforce the allocation in code rather than hoping it works out.

Here is an illustrative allocation for a coding agent on a model with a large window. The exact splits are a tuning exercise against your own workload; the discipline of having a budget is the load-bearing part.

Slot	What goes here	Budget	Ordering
System & policy	Persona, rules, corporate constraints, output contract	~5%	First — stable, cached
Decision-relevant context	The few facts and passages this step truly needs	~25%	Immediately after system prompt
Working set	Current task, files or records being acted on	~30%	Middle
Recent history	Last few turns and tool results, decision-relevant only	~20%	Toward the end
Headroom	Space for the model's own reasoning and output	~20%	Reserved, never filled

Two ordering rules follow from how transformers attend. First, the most decision-relevant information goes immediately after the system prompt — not buried in the middle, where attention is weakest. Second, keep the system prompt and any other stable preamble byte-for-byte identical across turns so prompt caching can take effect; a single changed character invalidates the cache. The pattern most teams converge on is a sliding window with prioritisation: hold a fixed budget for recent interactions, but rank what goes in by relevance rather than blindly keeping the literal last N turns. Here is a sketch of a context-assembly function that enforces a budget:

def assemble_context(task, memory, history, tool_results, token_budget):
    """Build the window deliberately, most decision-relevant first."""
    blocks = []

    # 1. Stable, cacheable preamble — keep byte-identical across turns
    blocks.append(Block("system", SYSTEM_PROMPT, cacheable=True))

    # 2. Most decision-relevant context, right after the system prompt
    relevant = rank_by_relevance(memory.semantic + retrieve(task.query), task)
    blocks.append(Block("context", relevant, budget=0.25 * token_budget))

    # 3. The working set the agent is acting on this step
    blocks.append(Block("working_set", task.active_items, budget=0.30 * token_budget))

    # 4. Recent history — decision-relevant turns, not the literal last N
    recent = select_recent(history, tool_results, keep_decisions=True)
    blocks.append(Block("history", recent, budget=0.20 * token_budget))

    window = pack(blocks, reserve_headroom=0.20 * token_budget)

    # If we blew the budget, compact the oldest history rather than truncating blindly
    if window.tokens > token_budget:
        window = compact_oldest(window, target=0.70 * token_budget)
    return window

Memory: short-term, long-term and summarisation

Memory is how context survives the moment. Short-term memory is the working context: the current task, recent turns, the scratchpad of intermediate reasoning. It lives in the window, it is cheap to read because it is already in the prompt, and it disappears when the run ends. Long-term memory is what persists across runs, and it is worth splitting by type. Semantic memory holds durable facts and preferences — "this user works in British English", "the production database is in Mumbai". Episodic memory holds specific past events — "on the last run, the deploy failed because the migration ran before the lint step". Procedural memory holds how-to knowledge for recurring tasks. We cover the storage architecture for all of this in depth in our companion guide on agent memory in production; here the focus is narrower, on how memory is selected into the window.

The selection rule is the same as the budget rule: load the few most decision-relevant memories, not everything you have. A common and effective approach is to retain decision-relevant recent interactions in full, and summarise or discard older context. Summarisation is the workhorse. Rather than carrying fifty turns of conversation verbatim, periodically distil them into a compact note that preserves the decisions, the open tasks and the unresolved errors, and drop the chatter. In one 2026 study of agent summarisation, an automated approach reported roughly 91.6% complete itemisation of the salient points while cutting token count and, the authors argued, improving overall context quality — though, as with any single-source figure, treat that number as indicative of the direction rather than a benchmark to quote as settled.

def summarise_history(turns, model):
    """Compact older turns into a durable note; preserve what the agent must not forget."""
    keep_verbatim = [t for t in turns if t.is_open_task or t.is_unresolved_error]
    to_compress   = [t for t in turns if t not in keep_verbatim]

    summary = model.complete(
        system="Summarise the conversation below. Preserve every decision made, "
               "every open task, every constraint stated, and every unresolved error. "
               "Drop pleasantries and superseded intermediate reasoning. Be high-fidelity.",
        input=render(to_compress),
    )
    archive(to_compress)                      # raw turns go to cold storage for audit
    return [Block("summary", summary)] + keep_verbatim

Tool-output management

The single fastest way to rot a context window is to feed raw tool outputs straight back into it. An agent that queries an API and dumps the full JSON response, or reads a file and pastes the whole thing, will fill the window with low-signal tokens within a handful of turns. Tool-output management is the unglamorous discipline that separates a demo from a production agent. Three moves, in order of impact:

• Filter at the source. Ask the tool for only the fields you need. A database query should select columns, not SELECT *. An API call should request the minimal projection. The cheapest token is the one you never retrieve.
• Truncate and reference. When a result is large but occasionally needed in full, keep a short summary plus an identifier in the window and store the full payload out of band, retrievable on demand. The agent sees "returned 412 rows, here are the 5 that matched your filter; full set at result_id=…" rather than 412 rows.
• Clear stale results. A tool output that mattered ten turns ago and is now superseded is pure distraction. Anthropic's Claude tooling exposes context-editing primitives that let an agent clear old tool results from the window automatically; in their reported testing, clearing stale tool calls in a long multi-step run cut token consumption dramatically while enabling workflows that would otherwise have exhausted the window. The principle holds whatever model you run: do not carry results you no longer need.

RAG vs MCP vs long context: choosing the delivery mechanism

Once you accept that the window is a budget, the question becomes: how does the right information get into it at the right moment? As of June 2026 there are three dominant mechanisms, and the most common mistake is treating them as rivals when they are complements. Retrieval-augmented generation (RAG) pulls a small set of relevant passages from a large corpus at query time — it is how you give an agent access to a body of knowledge far larger than any window. The Model Context Protocol (MCP) is a governed delivery channel: a controlled server that exposes tools, live data and resources to the agent with permissioning, structured schemas and an audit trail — it is how you deliver corporate data and actions under governance rather than pasting them in. A long-context model, meanwhile, lets you put whole documents in the window and reason over them together, which retrieval cannot do because retrieval fragments.

Mechanism	What it is	Strengths	Limits	Reach for it when
RAG (retrieval)	Embed a corpus; fetch the top passages per query	Scales to corpora far larger than any window; cheap per query; updatable	Fragments documents; misses cross-passage reasoning; quality bounded by chunking	Large, mostly static knowledge base; you need breadth, not whole-document cohesion
MCP (governed metadata)	A server delivering tools, live data and resources under permissioning	Governance, audit, fresh and structured data, controlled actions; least-privilege	You build and operate the server; latency of live calls; another moving part	Corporate or live data; you need access control, provenance and audit, not a paste
Long-context model	Put whole documents in the window and reason over them together	True cross-document cohesion; no chunking artefacts; simplest to build	Cost and latency scale with tokens; context rot past the reliable zone	The task genuinely needs whole documents reasoned over as one

The practical guidance is to layer them. Use MCP as the governed gateway through which corporate and live data reaches the agent at all; use RAG behind it to select the relevant slice of a large corpus; and use the model's long context to hold the working set cohesively once the relevant material has been narrowed. A decision sketch:

def choose_delivery(source):
    if source.is_live or source.needs_governance:
        # Corporate data, fresh data, or anything needing audit and permissioning
        return "mcp"
    if source.size_tokens > model.reliable_context and source.is_corpus:
        # Bigger than the reliable zone and made of independent passages
        return "rag"
    if source.needs_whole_document_reasoning:
        # Must be reasoned over as one coherent body
        return "long_context"
    return "long_context"  # small enough to just include

There is a dual-market wrinkle worth designing in. The choice is not only technical — it is about data residency and latency. For Indian users, keeping a retrieval index and any MCP server in an in-region cloud such as AWS Mumbai (ap-south-1) keeps personal data under the DPDP regime and trims round-trip latency; for UK users, London (eu-west-2) does the same under UK GDPR. A hosted long-context model call that ships whole documents to an endpoint in another region can be a cross-border transfer you need a basis for, whereas retrieving a small governed slice through an in-region MCP server keeps the compliance story simple. We cover the retrieval side of this in our production RAG and hybrid retrieval guide, and the governed-delivery side in building your first MCP server.

Sub-agent visibility boundaries

Multi-agent systems add a dimension that single agents do not have: deciding what each sub-agent is allowed to see. This is a context-engineering decision, and getting it wrong is a common source of both bloat and leakage. The orchestrator-and-workers pattern — a lead agent that decomposes a task and delegates slices to specialised sub-agents — works precisely because each sub-agent gets a clean, narrow context scoped to its slice, rather than inheriting the lead's entire bloated window. Our guide on agent design patterns covers the orchestration shapes; the context angle is the visibility boundary.

The default should be least-visibility: a sub-agent sees the task it was given, the resources it needs for that task, and nothing else. It does not inherit the lead agent's full conversation history, the other sub-agents' working sets, or memories irrelevant to its slice. There are two reasons. The first is quality: a sub-agent handed a focused 8k-token context will reason far more reliably than one handed the lead's rotting 200k-token window. The second is safety and governance: if a sub-agent calls an untrusted tool or processes untrusted content, you do not want it to have had visibility of every secret and constraint in the parent context. Pass a sub-agent a curated brief and let it return a curated result; do not let contexts bleed into each other by default.

The four failure modes and how to detect them

Long-horizon agents fail in characteristic ways that have, over 2025 and 2026, acquired names. Knowing the names is useful because each has a distinct cause and a distinct fix.

• Context poisoning. A wrong fact enters the window — from a hallucination, a bad tool result, or a malicious input — and persists, compounding into every later step that reads it. The agent confidently builds on a false premise. Fix: attach provenance to every fact, validate tool outputs before they enter the window, and make it cheap to trace a belief back to its source.
• Context distraction. Irrelevant tokens drown the signal. The window is full of stale tool outputs, repeated errors and tangential history, and the model loses the thread of the actual task. Fix: the tool-output management and budgeting above — filter, truncate, clear, and rank by relevance.
• Context clash. Two contradictory facts sit in the window at once — an old value and its correction, two sub-agents' incompatible conclusions — and the model picks the wrong one or oscillates. Fix: reconcile on write, prefer updates over appends for facts that change, and surface contradictions explicitly rather than letting both quietly coexist.
• Context rot. Recall simply drops as the window grows long, independent of any bad data. Information in the middle of a very long context is processed less reliably. Fix: keep decision-critical facts in the reliable zone, compact aggressively, and never lean on the assumption that a fact buried at token 400,000 will be recalled.

Detection is an observability problem. You cannot fix what you cannot see, so instrument the assembled prompt itself — log what was in the window at each decision, with provenance on each block — and build evaluation sets that probe mid-window recall and contradiction handling, not just the happy path. Our guide on agent observability with OpenTelemetry covers the tracing that makes these failures visible in production; without it, a poisoned context looks identical to a model that simply got the answer wrong, and you will debug the wrong thing for a week.

Putting it together: a long-horizon research agent

To make this concrete, picture a research agent serving analysts at a firm with desks in Bengaluru and London. The task: given a question, gather evidence from an internal corpus and live systems, reason over it, and produce a cited brief — a run that routinely spans dozens of tool calls and an hour of wall-clock time. Here is how the pieces compose.

The window is assembled on a budget on every turn: a stable, cached system prompt carrying the analyst's policies and the output contract; then the most decision-relevant retrieved passages and loaded memories; then the working set of evidence gathered so far; then a compacted summary of earlier turns with open questions preserved verbatim; and a fifth of the window held back as headroom. Live corporate data — portfolio positions, fresh filings — arrives through an in-region MCP server with permissioning and an audit log, scoped to Mumbai for Indian data and London for UK data. The broad internal corpus is reached through RAG, which narrows thousands of documents to the handful that matter. The model's long context then holds those documents together so the agent can reason across them rather than passage by passage.

When the agent spawns sub-agents to chase parallel sub-questions, each gets a narrow brief and returns a curated finding, never inheriting the lead's full window. Tool outputs are filtered at the source and cleared once superseded; a failed live call is summarised to its cause rather than dumped as a traceback. At roughly seventy percent window occupancy the agent compacts, distilling resolved threads into a schema-shaped note and archiving the raw turns. Every assembled prompt is traced with provenance, so when a reviewer asks "where did this claim come from?" the answer is one query away — and a poisoned fact can be caught before it compounds. The result the team reported was the one that matters: the agent sustained a long run without rotting, stayed within budget, and produced briefs whose every claim was traceable to a governed source. That is context engineering doing its job.

Where to go next

Context engineering is the discipline of deciding, deliberately and on every turn, what your agent knows, sees and remembers. Treat the window as a budget; put the most decision-relevant material first; structure memory rather than dumping it; manage tool outputs ruthlessly; layer RAG, MCP and long context rather than choosing one; scope sub-agents tightly; compact before you hit the wall; and instrument everything so the four failure modes are visible. If you are wiring this into a real system, the companion guides on agent memory in production, agent design patterns, production RAG and hybrid retrieval, building your first MCP server and agent observability cover the surrounding pieces. Get the context right and the agent stops being a clever demo — and starts being something you can trust to run for an hour unattended.