Markdown

Agent harness architecture¶

Scope: the runtime that wraps a foundation model and turns its token output into reliable multi-step action. The harness owns context management, tool selection, error recovery, state, and external memory; the model only proposes. Past a model-capability floor, this layer drives more of an agent's reliability than the weights do. The foundations it drives are the agent loop, tools, context and memory, and planning; the control it needs is the orchestration control plane.

Code and configs here are reference templates; pin versions and validate before relying on them.

flowchart TB
  MODEL["Foundation model<br/>(proposes: text + tool calls)"]
  subgraph HARNESS["Harness (decides and executes)"]
    CTX["Context manager<br/>(budget, compaction, recitation)"]
    REG["Tool registry + dispatch"]
    ERR["Error recovery<br/>(retries, replan)"]
    STATE["Durable state<br/>(filesystem / store)"]
    MEM["External memory"]
  end
  MODEL -->|"tool call"| REG
  REG -->|"observation"| CTX
  CTX -->|"assembled prompt"| MODEL
  REG --> ERR
  ERR -->|"replan signal"| MODEL
  REG --> STATE
  STATE --> MEM
  MEM --> CTX

Overview¶

An agent is a model plus the program that runs it in a loop. The model is a stateless next-token predictor; everything that makes it behave like an agent (remembering earlier steps, calling a tool and reading the result, recovering from a failed action, staying inside a budget) lives in the surrounding program. That program is the harness.

The load-bearing claim of harness engineering: once the underlying model clears a capability floor for a task, the harness explains more of the performance variance than the model choice. The same model swung between failure and success by its harness is the common experience of teams shipping long-horizon, tool-using agents; the qualification is that below the floor no harness rescues weak reasoning, and on pure single-shot reasoning the harness barely matters.¹ The implication for the agent loop is that engineering effort moves from prompt wording to the loop, the tools, and the state around it. Increasingly the harness is written and optimized as code rather than prose, a shift surveyed under the banner of code as the agent harness and pushed furthest by designs where the program, not the model, owns control flow (self-improving harnesses).⁷

Core knowledge¶

What a harness controls¶

A harness is defined by five responsibilities. Get these right and a mid-tier model runs reliably; get them wrong and a frontier model still loops, forgets, or burns the budget.¹

Context management. What goes into each model call, in what order, and what gets evicted when the window fills. The dominant cost and the dominant failure source.
Tool selection. Which tools exist, how they are described, and how the harness constrains which can be called when.
Error recovery. What happens when a tool call fails, times out, or returns garbage: retry, replan, or abort.
State management. How progress survives across turns and restarts so the agent does not redo or forget work.
External memory. Durable storage (files, a store) the agent reads and writes beyond the context window.

Convergent designs¶

Production harnesses built independently have converged on the same moves, which is the strongest evidence the responsibilities above are real rather than stylistic.

Minimal, sharp tool sets. A small set of general tools (read, write, edit, run a command, search) plus an extension mechanism beats a large catalogue of narrow ones. Collapsing an over-broad tool surface to a couple of primitives has repeatedly raised task accuracy while cutting tokens and latency, because every extra tool is another decision the model can get wrong.¹
The filesystem as state and memory. Treating a working directory as the agent's externalised memory, writing intermediate artifacts to files and keeping restoration paths (URLs, file paths) in context rather than full contents, gives unbounded, inspectable state that survives compaction.²
Context as the optimisation target. Stable prompt prefixes and append-only context keep the KV cache warm, which is the difference between cheap and expensive turns at the high input:output ratios agents run; compaction preserves pointers, and summarization is the last resort, not the first.²
A plan kept in view. Re-stating the task and the current to-do list at the end of the context counters the lost-in-the-middle effect, where models attend least to the middle of a long prompt.³

Grading a harness on what the model can see¶

The five responsibilities above say what a harness must do. A complementary rubric asks how much of the machinery is exposed to the model rather than only to the developer, and it is sharper for comparing candidates. NVIDIA's object-oriented agents paper proposes six axes: typed I/O (validated return values, not free text), pass by reference (live in-process objects, not serialized copies), code as action (real control flow, not one JSON call per turn), loop engineering (can the model author orchestration, or only you), object state (typed and held out of history eviction, rather than the transcript being the state), and harness APIs (can the model inspect and manage its own context blocks and event history).⁶

The discriminating rule is that a capability counts only when it is first-class in what the model sees. Developer-only versions score partial, and tracing dashboards, automatic compaction, and hidden callbacks do not count as harness APIs at all. Scored that way, the paper's survey of fourteen frameworks finds most systems adopting several axes, often behind a flag, and none exposing all six on one surface. Use the axes as a checklist; re-derive the cells, because the published scores were read at commits pinned in July 2026 by the authors of one of the entries. The implementation and a validated audit of its claims are in object-oriented agents.

Constraining the model with tool modes¶

The harness does not have to accept whatever the model emits. Decoding can be masked so that, per turn, tool calls are auto (model may call any tool or none), required (must call some tool), or specified (must call a named subset). Constraining the action space at the decoder removes a class of malformed or out-of-policy calls before they happen, which is cheaper and more reliable than catching them after.² This is the foundations-side complement to the policy gate in the control plane.

Harness components are not free¶

Adding a harness component can hurt. The governing rule from harness-ablation work: a component helps when it introduces a new, independent signal and regresses when it merely recycles the doer model's own signal. A verifier or candidate-selector that is the same model grading itself inherits the doer's blind spots and adds cost without lift; file-backed state and self-improvement loops that bring in outside information tend to help.⁴ The corollary is an audit discipline: ablate each component with the substrate held fixed (pinned model snapshot, pinned sandbox image, content-hashed eval set, seeded RNG, isolated egress), because otherwise a moving substrate masks which change actually moved the metric.⁴

There are two harnesses, and conflating them is the defect¶

A harness for training and a harness for production are different artifacts with opposite objectives, and most teams ship one and call it both.⁵ Training wants a maximal action space so the policy learns tool selection and error recovery; production wants a minimal, deny-by-default one. Training welcomes failures because they are optimizer signal; production suppresses them.

Dimension	Training harness	Production harness
Action space	Maximal, anything plausibly useful	Minimal, explicit allowlist
Tools	Raw and low-level	Wrapped, scoped, versioned, schema-validated
Failures	Welcome, they are signal	Suppressed, fail closed and alert
Network	Offline or recorded, adversarially perturbed	Live with strict egress policy
Guardrails	KL caps, reward shaping, curriculum gates	RBAC, scoped credentials, action gates
Verifier	Programmatic, scaled, deliberately noisy	Deterministic, human in the loop where critical
State	Forkable, snapshottable, replayable	Durable, per user, auditable
Success	Policy improves on held-out data	User task completes without incident

Two symmetric failures follow. Over-shackling in training: importing the production allowlist into rollouts "for safety" produces a policy that is well-behaved inside that allowlist and helpless outside it, because the guardrails did the tool selection and error recovery the model needed to learn. Under-fencing in production: shipping the wide-open training harness, then patching with post-hoc filters after the first prompt injection lands. The rule is that the harness should be widest where the model is trained and narrowest where it is deployed, and the gap between the two should be an audited artifact rather than an accident.⁵

An evaluation harness is the third artifact: it mirrors production closely enough to catch behavioural regressions, and it is graded on parity rather than on capability (evaluating agents).

Optimise on the cheapest surface, and design for removal¶

Three surfaces can absorb a fix, and they differ by an order of magnitude in cost to change: skills and prompts (text edits, hourly to daily, owned by product builders), the harness (code, daily to weekly, owned by research or applied engineering), and the model (post-training compute, quarterly, usually lab-side). Pushing work down to the cheapest surface that can hold it is the whole of "thin harness, fat skills".⁵

That matters because harness components dissolve as models improve, and the historical record is consistent: retrieval pipelines built as an external memory layer were absorbed by longer contexts and native retrieval; orchestrator-worker graphs that compensated for unreliable tool calling were absorbed by native interleaved tool use; wrappers around well-documented APIs were absorbed by models that read the specification directly. A harness engineered for the current model generation is an artifact of that generation.

The practical discipline is the removability test: for each component, ask how long it would take to delete when it becomes unnecessary. An hour means you hold an option. A week means you hold debt. Designs that keep the option are the ones built from low-level primitives with a thin, deliberately under-specified surface, including the extreme case where the agent writes its own missing helpers at runtime instead of consuming pre-built wrappers (self-improving harnesses).

One warning about the empirical case for any particular harness: harness quality is measured jointly with a model, and the pairing is not transitive. Published comparisons report a third-party harness beating a vendor's own on that vendor's flagship model while losing on another vendor's, and first-party harnesses showing much larger advantages on some models than others. Those specific figures come from secondary sources and are not independently verified here, but the pattern is robust enough to act on: a harness benchmark result is a statement about a harness-model pair, and it does not survive a model swap. The RAG paradigm scaling study measures the same effect from the other side, with three file-search harnesses scoring 86.3, 82.3, and 43.9 on identical questions, model, corpus, and judge.

Don't-miss checklist¶

Budget context deliberately: stable prefix, append-only history, compaction that keeps pointers, summarization last.
Keep the training harness and the production harness as separate, separately-reviewed artifacts; audit the gap.
Apply the removability test to every component; prefer designs you can delete in an hour.
Read every harness benchmark as a harness-model pair result, not as a property of the harness.
Keep the tool set small and general; justify every additional tool against the decision cost it adds.
Externalise state to the filesystem or a store; never rely on the context window as durable memory.
Make every tool failure a structured, replannable signal back to the model, not a silent drop.
Ablate harness components against a fixed substrate; delete any that recycle the model's own signal.
Keep a re-stated plan or to-do list in view to fight lost-in-the-middle.

Failure modes¶

Context rot. The window fills with stale tool output; the model loses the thread and repeats or contradicts earlier steps. Compaction and recitation are the fix.
Tool sprawl. Too many narrow tools; the model picks the wrong one or stalls choosing. Symptoms are low accuracy with high token use.
Cache-hostile prompts. Re-ordering or rewriting the prefix each turn evicts the KV cache and multiplies cost at agentic input:output ratios.
Silent error swallowing. A failed tool call returns nothing useful; the model proceeds on a false premise. Surface the error as an observation.
Same-model verification. A self-grading verifier recycles the doer's mistakes, adding latency and cost without catching anything.
State in the prompt. Progress kept only in context is lost on compaction or restart; the agent redoes or abandons work.
One harness for training and production. Either the policy never learns recovery because production guardrails did it, or production inherits a wide-open action space and gets patched with filters after the first incident.
Auto-optimised harness overfitting. Automatically tuned harnesses compensate for one task distribution and can underperform a generic harness once the distribution shifts, while adding opacity and train/production skew (automated harness optimization).
Unremovable scaffolding. A component that takes a week to delete outlives its usefulness by a model generation, and becomes the bottleneck for the next one.

Open questions & validation¶

Where the capability floor sits for a given task class, below which harness work does not pay off.
How much of a harness benchmark result transfers across a model swap; the pairing effect is well attested but its size is not characterised.
Whether the training/production harness gap can be measured directly, rather than inferred after a behavioural regression.
Which harness components survive a fixed-substrate ablation on the target workload, since the answer is task-dependent.
How to measure context quality directly rather than inferring it from end-task success.

References¶

Anthropic, Building effective agents: https://www.anthropic.com/research/building-effective-agents
Manus, Context engineering for AI agents: https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus
Liu et al., Lost in the Middle: How Language Models Use Long Contexts: https://arxiv.org/abs/2307.03172
Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models: https://arxiv.org/abs/2210.03629
SWE-bench (long-horizon coding agent benchmark): https://arxiv.org/abs/2310.06770
OSWorld (computer-use agent benchmark): https://arxiv.org/abs/2404.07972
NLAH (harness-component ablation against a fixed substrate): https://arxiv.org/abs/2603.25723
APEX-Agents (long-horizon agent benchmark): https://arxiv.org/abs/2601.14242
Model Context Protocol: https://modelcontextprotocol.io/
Code as Agent Harness (survey of code-as-harness): https://arxiv.org/abs/2605.18747
NVIDIA-labs OO Agents (the six model-facing capabilities and a fourteen-framework survey): https://arxiv.org/abs/2607.20709
Lee, Hidden Technical Debt of AI Systems: Agent Harness: https://leehanchung.github.io/blogs/2026/05/08/hidden-technical-debt-agent-harness/
Sutton, The Bitter Lesson: http://www.incompleteideas.net/IncIdeas/BitterLesson.html

The harness owns context management, tool selection, error recovery, state, and external memory; once a model clears a per-task capability floor, this layer explains more reliability variance than the model choice, most strongly for long-horizon tool-using tasks and least for single-shot reasoning. ↩↩↩
Manus, Context engineering: filesystem as externalised memory, stable KV-cache-friendly prefixes with append-only context, compaction that preserves restoration paths before summarization, and logit-masked tool modes (auto / required / specified). ↩↩↩
Liu et al. show models attend least to the middle of a long context, motivating end-of-context recitation of the task and plan. ↩
Harness-ablation finding: components that add an independent signal help; components that recycle the doer model's own signal (e.g. same-model verifiers and selectors) regress. Valid ablation requires a fixed substrate: pinned model snapshot and sandbox image, content-hashed eval set, seeded RNG, isolated egress. ↩↩
Lee, "Hidden Technical Debt of AI Systems: Agent Harness" (2026-05-08). Source of the training-versus-production harness table, the over-shackling and under-fencing failure pair, the three optimisation surfaces with their cost-to-change and iteration cadence, the removability test, and the dissolution pattern argued from Sutton's Bitter Lesson. The harness-model pairing figures it reports (first-party against third-party harness scores on three frontier models, and a memory-focused third-party harness beating a vendor harness on one model while losing on another) are secondary and were not verified against a primary benchmark here; only the qualitative non-transitivity is asserted on this page. ↩↩↩
NVIDIA-labs OO Agents (arXiv 2607.20709), section 5 and Table 7. Six model-facing capabilities scored across fourteen frameworks and harnesses at commits pinned 2026-07-07 to 2026-07-09, with a green score requiring the capability to be first-class in what the model sees. The scoring is by the authors of one of the entries; treat the axes as durable and the cells as a dated snapshot. ↩
Code as Agent Harness (arXiv 2605.18747), survey: the harness is increasingly code, organized into the harness interface (to reasoning and environment), harness mechanisms (planning and adaptive control), and multi-agent scaling over shared code artifacts. The optimization of that code is covered in self-improving harnesses. ↩