Self-improving harnesses¶
Scope: treating the harness as something to optimize rather than hand-tune. It covers searching and ablating harness components, harnesses that edit themselves, optimizing prompts and context instead of weights, and lifting agent control flow out of the prompt into an explicit program graph. This is the dynamic counterpart to the static harness architecture; it depends on fixed-substrate evaluation and on the controls in governing self-modifying agents.
Code and configs here are reference templates; pin versions and validate before relying on them.
flowchart TB
subgraph OPT["Harness optimization loop"]
PROP["Propose harness change<br/>(prompt, tool set, component)"] --> EVAL["Evaluate on a fixed substrate"]
EVAL --> KEEP{"Improves?"}
KEEP -->|"yes"| ADOPT["Adopt"]
KEEP -->|"no"| ROLL["Roll back"]
ADOPT --> PROP
ROLL --> PROP
end
FLAT["Implicit control flow in the prompt"] --> LIFT["Lift into a program"]
LIFT --> DAG["Explicit graph (DAG of LLM calls)"]Overview¶
The static view treats the harness as something an engineer writes once and tunes by hand (harness architecture). The frontier treats the harness as an object to optimize: search over its components, evolve its prompts, let it edit itself, or rewrite its control flow as an explicit program. The shared premise is that the harness, not only the model weights, is a lever on agent performance, so the harness deserves the same optimization machinery the model gets.1 A recent survey organizes this whole "code as agent harness" space into three layers: the harness interface that connects the agent to reasoning and environment, the harness mechanisms for planning and adaptive control, and multi-agent scaling over shared code artifacts.5
Core knowledge¶
The harness is often the bottleneck¶
When an agent fails, the cause is frequently the harness rather than the model. AutoHarness synthesizes a code harness automatically and reports that a large share of agent losses were illegal or invalid actions the harness failed to constrain, which reframes the harness as a policy over the action space rather than passive plumbing.1 In its evaluation a generator model iteratively refines a protective code wrapper from environment feedback until it eliminates illegal moves across 145 TextArena games, after which the wrapped smaller model outperforms a larger unwrapped one at lower cost.1 Constraining what the model is allowed to emit (the tool-mode idea in harness architecture) is the manual version of the same insight.
Searching and ablating the harness¶
If the harness is a lever, it can be searched. Meta-Harness performs end-to-end optimization of a model's harness with an agentic proposer that searches over the harness code itself, reading its source, its scores, and its execution traces, and reports gains on text classification, retrieval-augmented math reasoning, and agentic coding; NLAH ablates individual harness components to find which ones carry their weight.23 The governing rule from that ablation work is covered in harness architecture: a component earns its place only if it introduces an independent signal, and an ablation is valid only against a fixed substrate (see below).
Harnesses that improve themselves¶
The strongest form closes the loop: the harness proposes and applies its own changes. Self-Harness runs a three-stage loop, identifying a model's specific failure patterns, proposing targeted harness modifications, and validating each change by regression test before adopting it, and reports lifting one model's pass rate from 40.5% to 61.9%.4 This raises the question the static view never has to answer, which is whether the optimizer is safe; that question is the subject of governing self-modifying agents, not this page.
Harness-R1 moves the learning target from the resulting harness to a dedicated editing policy. A separate 9B engineer reads batches of frozen-target failures, emits validated Python hooks at four lifecycle positions, and receives online GRPO reward from same-batch target reruns. Its transductive reward and generated-code boundary make held-out promotion and external sandboxing part of deployment, not optional extensions.
Recursive harness self-improvement from pairwise history¶
Self-Harness validates each change by regression test, which needs a scored test suite. RHI (Recursive Harness Self-Improvement) targets the case where you do not have one, and it is worth separating into its own method because of what it optimizes and what it costs.
What it is. RHI represents the harness as a prompt-level specification of the agent loop (role and instruction, interface contract, interaction structure) and refines it with an LLM harness optimizer. Each iteration runs the agent once per task under the current harness, has an evaluator produce a pairwise judgment against the previous iteration's output, appends that judgment to a per-task history, and updates the harness conditioned on the accumulated history. One agent execution and one pairwise evaluation per iteration, no scalar reward, no gradients.12
Why the history matters. Because the harness space is discrete and textual, pairwise preferences do not define gradients; the accumulated history instead acts as what the authors call a momentum-semantic signal. The validated block below shows the statistical reason: a single noisy comparison recovers the correct direction only at its own reliability rate, while a majority over accumulated comparisons sharpens it, provided each comparison is better than chance. If your evaluator is biased rather than merely noisy, accumulation makes you confidently wrong faster, which is the failure mode to watch.
When to use it. When the harness is prompt-level and cheap to rewrite, when you have an evaluator that can compare two outputs but no absolute scorer, and when your update budget is a handful of iterations rather than hundreds. RHI's whole premise is that "continually updating provider-built scaffolds is costly and labor-intensive", so it optimizes user-constructed harnesses task-specifically instead.12
How to stop. The loop computes an improvement rate s_i, the fraction of tasks where iteration i beat iteration i-1, and breaks when s_i < epsilon. That single threshold is the entire convergence control, and the block below shows it stops within a few iterations on a harness that improves then saturates, with epsilon trading iterations against residual headroom.
What it buys. Across 30 synthetic ML research tasks spanning quantitative finance, robotics, and pharmacy, a few RHI iterations "substantially raise the performance ceiling of low-reasoning-effort agents, exceeding the corresponding maximum-reasoning-effort setting while reducing inference cost by up to 60%."12 That is the sentence to internalize, and it connects this page directly to reasoning-effort control: a better harness at low effort beat the same model at max effort. Crucially the authors attribute the gain to "improved task-specific context management through more effective inter-agent information flow rather than longer reasoning traces", and report output-token usage staying nearly constant across iterations. The harness is not winning by thinking longer; it is winning by moving the right information between components, which is the same lever agentic context management pulls at the tool level.
# rhi_loop.py — validated: RHI's improvement-rate stopping rule and why the
# pairwise HISTORY is accumulated rather than trusted one comparison at a time.
import numpy as np
def improvement_rate(wins: np.ndarray) -> float: # s_i in Algorithm 1
return float(wins.mean())
def rhi_run(true_gain, n_tasks, eps, judge_noise, max_iter, rng):
"""true_gain[i] = probability revision i is genuinely better; the judge
observes it through symmetric pairwise noise."""
history = []
for i in range(max_iter):
p = true_gain[min(i, len(true_gain) - 1)]
p_obs = (1 - judge_noise) * p + judge_noise * (1 - p)
s = improvement_rate(rng.random(n_tasks) < p_obs)
history.append(s)
if s < eps:
return i, history
return max_iter, history
rng = np.random.default_rng(3)
GAIN = np.array([0.92, 0.85, 0.70, 0.45, 0.20, 0.10, 0.05, 0.05]) # improves, then saturates
stops = [rhi_run(GAIN, 30, 0.5, 0.10, 8, rng)[0] for _ in range(400)]
assert 2 <= float(np.mean(stops)) <= 5 # converges in a FEW iterations
loose = [rhi_run(GAIN, 30, 0.9, 0.10, 8, rng)[0] for _ in range(400)]
tight = [rhi_run(GAIN, 30, 0.2, 0.10, 8, rng)[0] for _ in range(400)]
assert float(np.mean(loose)) < float(np.mean(stops)) < float(np.mean(tight))
assert abs(((1 - 0.5) * 0.92 + 0.5 * 0.08) - 0.5) < 1e-12 # a 50%-noise judge carries NO signal
assert abs(((1 - 0.5) * 0.10 + 0.5 * 0.90) - 0.5) < 1e-12 # ...whatever the truth is
def accumulated_direction(p, k, rng): # majority over k judgments
return (rng.random(k) < p).sum() > k / 2
P, TRIALS = 0.62, 20_000
one = np.mean([rng.random() < P for _ in range(TRIALS)])
acc5 = np.mean([accumulated_direction(P, 5, rng) for _ in range(TRIALS)])
acc9 = np.mean([accumulated_direction(P, 9, rng) for _ in range(TRIALS)])
assert abs(one - P) < 0.02 and acc5 > one and acc9 > acc5 and acc9 > 0.72
bad = np.mean([accumulated_direction(0.45, 9, rng) for _ in range(TRIALS)])
assert bad < 0.45 # a BIASED judge: accumulation makes you wrong faster, not righter
LOW_EFFORT_TOK, MAX_EFFORT_TOK = 42_000, 105_000 # the "up to 60%" cost shape
assert abs((1 - LOW_EFFORT_TOK / MAX_EFFORT_TOK) - 0.60) < 1e-12
tok_by_iter = np.array([42_000, 41_600, 42_400, 42_100, 41_900]) # traces do NOT get longer
assert tok_by_iter.std() / tok_by_iter.mean() < 0.01
print(f"stop iter eps=.5: {np.mean(stops):.2f} | direction 1/5/9: "
f"{one:.3f}/{acc5:.3f}/{acc9:.3f} | token CV: {tok_by_iter.std()/tok_by_iter.mean():.4f}")
Two design details that are easy to get wrong when reimplementing this. First, the evaluation prompt x_eval is given only to the evaluator, never to the harness optimizer, so the optimizer aligns with the evaluation criterion indirectly through accumulated preferences rather than by reading and gaming it. Second, RHI updates directly from H^(i) to H^(i+1) conditioned on the whole history, rather than restarting a search, which is what keeps it lightweight.
Across iterations RHI is also reported to induce task-dependent local harness components such as acceptance gates, failure fallback rules, and evidence-motivated recall triggers.12 Those are the same primitives a human harness engineer eventually adds by hand, which is a useful sanity signal that the optimizer is finding real structure rather than prompt noise.
Verification as the control signal: recursive self-improvement in deep research¶
AREX applies the recursive-improvement idea to deep research and swaps the control signal from preference to verification.13 Its premise is a discovery-verification asymmetry: finding an answer that jointly satisfies many constraints requires navigating a large, sparsely informative search space, whereas checking a proposed candidate decomposes into far simpler constraint-wise checks.
The consequence is architectural. AREX alternates an inner research loop (gather evidence, evaluate candidates, construct a provisional answer) with an outer self-improvement loop that audits the answer constraint by constraint, identifies which claims are supported and which remain unresolved or conflicting, and directs the next round of research at the gaps. Verified progress is preserved across rounds, and constraint-level belief estimates govern both whether to continue and when to stop. Verification stops being a ranking step and becomes "the transition between research rounds".
For this page the transferable idea is the third component: an autonomous context-update tool that compresses growing interaction history into a compact improvement state which preserves verified evidence and unresolved constraints, without relying on an external model. That is the same move as agentic context management, specialised so that what survives compression is exactly the audit state. AREX is trained through agentic mid-training and long-horizon RL and released as both a 4B dense and a 122B-A10B MoE variant, evaluated on BrowseComp, WideSearch, DeepSearchQA and HLE among others.
If your task has checkable constraints, prefer AREX's shape (verify, then target the unresolved) over RHI's (compare, then revise): a constraint-wise audit is a much richer signal than a single pairwise preference, and it tells you where to improve rather than only whether you did.
Optimizing prompts and context, not weights¶
Much of the gain lives in the prompt and the context, which can be optimized directly without touching the weights. GEPA evolves prompts reflectively, using the agent's own traces to propose edits, and ACE treats context engineering itself as the thing to optimize.78 These are cheaper than fine-tuning and operate on the part of the system the harness already owns.
GEPA is the most fully specified member of this family and has its own page, because two of its design choices generalize beyond prompts. It selects the candidate to mutate from an instance-wise Pareto front rather than from the current best, which is the concrete fix for the local-optimum trap this section keeps running into: a candidate survives only if it is the sole leader on at least one validation instance, so a specialist with the worst average score still receives mutation budget. And it gates the expensive full-validation evaluation behind a cheap minibatch check, which is what turns the search from thousands of rollouts into hundreds. The same page records where the method fails, including a merge variant that regresses below the unoptimized baseline on one benchmark.
Control flow as a program¶
A parallel move lifts control flow out of the implicit prompt transcript into an explicit program. Instead of hoping the model re-derives the plan each turn, the structure is expressed as code: a scheduler-theoretic framework lifts agent control flow into an explicit static graph, and Autellix treats agentic programs as a DAG of LLM calls to be scheduled.910 LLM-as-Code takes the paradigm to its conclusion: the program governs all control flow and the model is invoked only for the reasoning and generation steps, keeping deterministic looping and branching in code where they are reliable, which its authors show stabilizes long computer-use sequences.6 The program owns the control flow; the context becomes a call graph rather than a flat transcript. This is one instance of externalization, the broader pattern of moving state, control, and memory out of the model and into inspectable structure.11
The fixed-substrate requirement¶
Every claim of harness improvement is a measurement, and the measurement is only as trustworthy as its substrate. A searched or self-editing harness must be evaluated against a pinned model snapshot, a pinned sandbox image, a content-hashed eval set, and seeded randomness; otherwise the reported gain is drift, not progress (evaluation, harness architecture). A self-editing harness additionally needs the controls in governing self-modifying agents, the most important of which is that the optimizer must never be able to edit its own guardrails or success metric.
The discipline is small enough to state as code. This runnable block models the propose-evaluate-keep loop and asserts the two properties that make it trustworthy: on a fixed substrate the loop is monotone (it never adopts a regression), and a moving substrate breaks it (its accepted "gains" fail to transfer to a held-out pinned eval):
# harness_opt.py — validated: keep a harness change only if it beats the current one on a FIXED
# substrate; a moving substrate makes the gain fail to transfer. numpy only.
import numpy as np
def score(harness, substrate_seed): # pinned tasks + seeded order == fixed substrate
r = np.random.default_rng(substrate_seed)
tasks = r.random(300); picks = r.integers(0, len(harness), 300)
return float((harness[picks] > tasks).mean()) # pass a task if the picked capability beats it
def optimize(base, eval_seed, rounds=80, seed=0):
rng = np.random.default_rng(seed)
cur = base.copy(); cur_s = score(cur, eval_seed()); hist = [cur_s]
for _ in range(rounds):
cand = cur.copy(); i = rng.integers(len(cand))
cand[i] = np.clip(cand[i] + rng.normal(0, 0.15), 0, 1) # propose a harness tweak
if score(cand, eval_seed()) > cur_s: # adopt ONLY on measured improvement
cur = cand
cur_s = score(cur, eval_seed()); hist.append(cur_s)
return cur, hist
base = np.random.default_rng(1).random(24)
FIXED = 42
cur_fixed, hist = optimize(base, lambda: FIXED) # optimize on a pinned substrate
assert all(b <= a + 1e-9 for b, a in zip(hist, hist[1:])) # monotone: never regress on a fixed substrate
assert score(cur_fixed, FIXED) > score(base, FIXED) # a real, measured improvement
m = np.random.default_rng(7) # adversarial: a MOVING substrate each step
cur_moving, _ = optimize(base, lambda: int(m.integers(1_000_000)))
assert score(cur_fixed, FIXED) > score(cur_moving, FIXED) # moving-substrate gains do not transfer
print(f"held-out: base={score(base,FIXED):.3f} fixed={score(cur_fixed,FIXED):.3f} moving={score(cur_moving,FIXED):.3f}")
Don't-miss checklist¶
- Treat the harness (prompts, tools, components, control flow) as optimizable, not fixed.
- Optimize prompts and context first; it is cheaper than touching weights.
- Keep or drop each searched component by whether it adds an independent signal.
- Evaluate every harness change against a fixed substrate, or the gain is unmeasurable.
- Lift control flow into an explicit program when the plan must be reliable across turns.
- Put a self-editing harness under self-modifying-agent governance; never let it edit its own guardrails.
Failure modes¶
- Moving substrate. A model or data change is read as a harness improvement; the optimization chases noise.
- Same-signal components. A searched component recycles the doer model's own judgment and adds cost without lift.
- Self-improvement without guardrails. The optimizer edits its own success metric or policy and the loop diverges.
- Over-rigid graphs. Control flow lifted into a static DAG loses the flexibility a loop needs for genuinely open-ended tasks.
- Prompt-evolution overfitting. Reflective prompt edits overfit the evaluation traces and regress on held-out tasks.
Open questions & validation¶
- Which harness components are worth searching is task-dependent; validate on the target workload, not a generic benchmark.
- How far control flow should be lifted into a static graph versus left to the loop is unsettled and task-specific.
- Whether self-improving harnesses converge or drift over long horizons is an open question; gate them on held-out evaluation.
References¶
- Lee, Khattab, Finn et al., Meta-Harness (end-to-end harness optimization): https://arxiv.org/abs/2603.28052
- AutoHarness (synthesizing a code harness): https://arxiv.org/abs/2603.03329
- Self-Harness (harnesses that improve themselves): https://arxiv.org/abs/2606.09498
- LLM-as-Code (the program governs control flow; the model is a callee): https://arxiv.org/abs/2606.15874
- Code as Agent Harness (survey of code-as-harness): https://arxiv.org/abs/2605.18747
- NLAH (harness-component ablation): https://arxiv.org/abs/2603.25723
- GEPA: Reflective Prompt Evolution: https://arxiv.org/abs/2507.19457
- ACE: Agentic Context Engineering: https://arxiv.org/abs/2510.04618
- From Agent Loops to Structured Graphs: https://arxiv.org/abs/2604.11378
- Autellix (agentic programs as DAGs of LLM calls): https://arxiv.org/abs/2502.13965
- Externalization in LLM agents (survey): https://arxiv.org/abs/2604.08224
- RHI: Recursive Harness Self-Improvement: https://arxiv.org/abs/2607.15524
- AREX: Towards a Recursively Self-Improving Agent for Deep Research: https://arxiv.org/abs/2607.21461
Related: Harness-R1 learned runtime editing · GEPA reflective prompt evolution · Harness architecture · Harness Handbook behavior localization · Governing self-modifying agents · Evaluating agents · Orchestration & control plane · The agent loop · Autonomous experimentation loops · Evaluation integrity & anti-gaming · Automated harness optimization · Skill optimization · Loop engineering · Agentic systems · HarnessX
-
AutoHarness synthesizes a code harness automatically and finds that a large fraction of agent losses are illegal or invalid actions the harness failed to constrain, recasting the harness as a policy over the action space. ↩↩↩
-
Meta-Harness performs end-to-end optimization of a model's harness rather than the weights. ↩
-
NLAH ablates harness components to identify which carry their weight; valid ablation requires a fixed substrate. ↩
-
Self-Harness (arXiv 2606.09498): a three-stage self-improvement loop, identify a model's failure patterns, propose targeted harness modifications, and validate each by regression test before adopting; reported lifting a model's pass rate from 40.5% to 61.9%. ↩
-
Code as Agent Harness (arXiv 2605.18747), survey: organizes code-as-harness into the harness interface (to reasoning and environment), harness mechanisms (planning and adaptive control), and multi-agent scaling over shared code artifacts. ↩
-
LLM-as-Code (arXiv 2606.15874): invert the usual arrangement so the program governs all control flow and the model is invoked only for reasoning and generation; keeping deterministic looping and branching in code stabilizes long computer-use sequences. ↩
-
GEPA (arXiv 2507.19457, ICLR 2026 Oral) evolves prompts reflectively from the agent's own execution traces, selecting parents from an instance-wise Pareto front and gating full validation behind a minibatch check. Reported beating GRPO on the six-task aggregate with 24,000 GRPO rollouts per task; on IFBench it found its best prompt after 678 rollouts. See GEPA for the validated selection algorithm and the result audit. ↩
-
ACE treats context engineering as the optimization target, tuning what enters the context rather than the weights. ↩
-
A scheduler-theoretic framework lifts agent control flow from the implicit prompt context into an explicit static graph. ↩
-
Autellix treats agentic programs as a DAG of LLM calls to be scheduled. ↩
-
A survey of externalization frames agents as moving state, control, and memory out of the model into inspectable structure. ↩
-
RHI (arXiv 2607.15524): represents the harness as a prompt-level specification of the agent loop and refines it with an LLM optimizer conditioned on the accumulated pairwise-preference history over its own revisions; one agent execution and one pairwise evaluation per iteration, stopping when the improvement rate falls below a threshold. Across 30 synthetic ML research tasks in quantitative finance, robotics and pharmacy, a few iterations raised a low-reasoning-effort agent past the same model's maximum-reasoning-effort setting while cutting inference cost by up to 60%, with output-token usage nearly constant across iterations. ↩↩↩↩
-
AREX (arXiv 2607.21461): exploits the discovery-verification asymmetry in deep research, alternating an inner research loop with an outer loop that audits the provisional answer constraint-wise and targets the unresolved constraints, plus an autonomous context-update tool that compresses history into a compact improvement state without an external model. Released as 4B dense and 122B-A10B MoE variants. ↩