Markdown

Agent-as-a-Router (ACRouter)¶

Scope: an agentic router that picks, per coding task, which model in a heterogeneous pool should run it, and improves that choice online from verified execution outcomes rather than from a fixed classifier. Proposed in "Agent-as-a-Router: Agentic Model Routing for Coding Tasks" (Zhou et al., 2026), it formalises routing as a Context-Action-Feedback (C-A-F) loop driven by three modules (an Orchestrator, a Verifier, and a Memory) and evaluates it on CodeRouterBench by cumulative regret against a per-task oracle. This is the dynamic, feedback-driven end of LLM request routing: that page owns the static taxonomy (predictive vs cascading, decision signals, placement); this page owns the loop that keeps learning after deployment. It is the "frontier escape hatch" router referenced by own or rent a coding model.

The Python block below is executed and asserted (numpy plus stdlib). It models the C-A-F loop as an online, per-dimension routing problem and reproduces the paper's QUALITATIVE claims (static routing suffers near-linear cumulative regret, a feedback router drives it sublinear, task-dimension statistics beat a context-free router, a dominator removes any routing benefit, and a noisy verifier degrades the feedback router toward static). Its numbers are a toy, not the paper's benchmark values. The reference-loop template is unexecuted. Benchmark figures below are as reported by a single recent (2026) paper; on the out-of-distribution split, the paper's own results tables and its official repository's own recomputed baseline table diverge substantially on several per-router numbers (commonly 5-30+ points, with one gap the repository's own maintainer notes flag directly), so treat every out-of-distribution figure as approximate and verify against the source before you quote it.

What it is¶

A conventional LLM router treats dispatch as one-shot classification: read the prompt, predict which model wins, send it, and never look at what happened. ACRouter treats dispatch as a decision process that closes the loop. For each streamed task it runs one turn of a C-A-F cycle: c_i (Decide) -> a_i (Execute) -> f_i (Memorize) -> c_{i+1}.

Context c_i: the task prompt, optional metadata (a short description, a difficulty estimate, the language), and the Memory state accumulated from all prior tasks.
Action a_i: the index of the model chosen from the pool M, produced by the Orchestrator.
Feedback f_i: the Verifier-observed performance score for the chosen model on this task, plus the monetary cost computed from token consumption and list prices.

The feedback is written to Memory and folded back into the next task's context, so the router's information about "which model wins on which kind of task" grows during deployment rather than being frozen at training time. The paper's framing is that the bottleneck for routing is an information deficit, not a reasoning failure: a router that is simply told each model's per-dimension track record already routes far better than one asked to guess cold.

Three modules carry the loop:

Orchestrator (Decide). A small, cheap routing policy that combines a static per-dimension prior (which model has historically been best on tasks of this dimension), the nearest past tasks retrieved from Memory, and the task metadata, and emits the model index. The paper instantiates it as a sub-1B model fine-tuned on the benchmark's probing set, kept small so routing latency and cost stay a fraction of the generation it gates.
Verifier (Feedback). The trust root of the whole system. It turns a candidate output into a scalar performance score in [0,1] by aggregating execution-grounded signals: abstract-syntax-tree parsing, sandboxed execution (pass@1), prompt-embedded tests, and rule-based checks, with an LLM-as-judge for dimensions that cannot be executed. Because every downstream statistic is built from this score, a mis-specified Verifier silently poisons the Memory (see the adversarial case in the code and the failure modes below).
Memory (Memorize). An online vector store keyed by task embedding, whose value logs the chosen model, the observed score, the cost, and the verification trace. The Orchestrator queries it by cosine k-nearest-neighbours to recall how similar past tasks resolved; it is bounded (FIFO in the paper's implementation) so it stays fast and forgets stale entries.

Why use it¶

Complementary strengths, exploited online. Frontier coding models succeed on different subsets of tasks; no single model dominates. A router recovers that headroom, and a feedback router keeps recovering it as the traffic mix and the model pool drift, which a frozen classifier cannot.
Information, not cleverness, is the lever. The paper reports that simply augmenting a vanilla LLM router with per-dimension performance statistics yields a 15.3% relative gain and surpasses a heuristic router built on the same priors. The win comes from feeding the router grounded evidence, so it is cheap to obtain and does not require a larger router.
Cost is in the objective, not bolted on. The per-task reward trades performance against spend (a positive weight on the Verifier score, a small negative weight on cost), so the router optimises quality-per-dollar rather than raw quality. On out-of-distribution agentic tasks the paper reports ACRouter beating an always-use-the-strongest-model policy on both performance and cost efficiency.
Generalisation through adaptation. Static learners that fit the in-distribution split can collapse on shifted traffic; the paper reports four of its five trained baselines (LogReg, RouteLLM-BERT, TF-IDF+MLP, RouteLLM-MF) dropping below random on the out-of-distribution set, while the online loop keeps adapting.

When to use it (and when not)¶

Use it when:

The pool holds several genuinely complementary coding models and you can afford a routing turn per task.
You have a reliable, execution-grounded verifier (unit tests, a sandbox, compile-and-run) so feedback reflects real correctness rather than a noisy proxy.
Traffic streams and drifts: new repositories, new task types, models added or retired, so a one-shot classifier decays.
You already run a frontier escape hatch and want the routing decision to earn its keep rather than defaulting everything to the most expensive model.

Skip or defer it when:

One model dominates every task type in your pool. Routing then has no headroom to recover, and the online explorer only adds overhead (the boundary case in the code makes this exact: a trivial fixed router equals the oracle, and the learner loses to it on exploration cost).
You cannot verify outcomes cheaply and honestly. Without a trustworthy Verifier the loop learns from noise, and its advantage over a static router evaporates.
Traffic is tiny, uniform, or ultra-low-latency, where even a small router turn is not worth it. Reach first for the static routers on LLM request routing.

Architecture¶

The loop is the architecture: Context is assembled from priors, Memory recall, and metadata; the Orchestrator decides; the pool executes; the Verifier scores; Memory records; and the record re-enters the next Context. Note that this is inter-model routing (which endpoint serves the request), not the intra-model token-to-expert gating of MoE routing and load balancing.

flowchart LR
  subgraph CTX["Context c_i"]
    T["Task prompt + metadata<br/>(language, difficulty)"]
    PRIOR["Task-dimension prior<br/>(DimensionBest)"]
    MEMR["Memory recall<br/>(kNN of similar past tasks)"]
  end
  CTX -->|"Decide"| ORCH["Orchestrator<br/>(small routing policy)"]
  ORCH -->|"Action a_i: pick model"| POOL["Model pool M<br/>(8 frontier LLMs)"]
  POOL -->|"Execute"| OUT["Candidate output"]
  OUT --> VER["Verifier<br/>AST parse, sandbox pass@1,<br/>tests, judge -> score in [0,1]"]
  VER -->|"Feedback f_i: score + cost"| MEM["Memory<br/>(vector store:<br/>model, score, cost, trace)"]
  MEM -->|"Memorize"| MEMR
  MEM -.->|"aggregate updates"| PRIOR

The load-bearing invariant: the only edge that carries new information is Execute -> Verifier -> Memory. Everything else is bookkeeping around it. That is why the Verifier is the trust root and why the reward's cost term has to be in the loop from the start rather than applied as a post-hoc filter.

Modelling the loop: cumulative regret, executed¶

Cumulative regret is the metric the paper optimises: the summed gap, over the task stream, between the per-task oracle's reward and the router's. For a stream of N tasks, CumReg_N = sum_i ( V*(t_i) - r_i(a_i) ), where V* picks the best model for each task independently. A static classifier that is wrong on some task type keeps paying that gap every time the type recurs, so its regret grows roughly linearly; a feedback router that learns the per-dimension winner pays the gap only while exploring, so its regret grows sublinearly. This block reproduces that contrast as an online contextual-bandit problem (per-dimension UCB1, Auer et al. 2002), and it also reproduces the boundary and the honest failure mode.

# acrouter_regret.py -- core simulation, runnable (numpy + stdlib only).
# Models the C-A-F loop of Agent-as-a-Router as an online, contextual (per task-dimension)
# routing problem and reproduces the paper's QUALITATIVE claims:
#   - a STATIC router (fixed classification, no feedback) suffers ~linear cumulative regret;
#   - a FEEDBACK router that updates per-dimension estimates from execution outcomes
#     (ACRouter-style) drives cumulative regret SUBLINEAR (learns which model wins per dim);
#   - augmenting the router with TASK-DIMENSION statistics beats a context-free router
#     (the paper's "15.3% relative gain" idea; here we reproduce only the DIRECTION and sign);
#   - if one model dominates every dimension, all routers converge (no routing benefit);
#   - a MIS-SPECIFIED (noisy) verifier degrades the feedback router toward static.
# Numbers here are a TOY, not the paper's benchmark values. Regret is the expected-reward
# gap vs the per-task oracle: CumReg_N = sum_i ( mu[d_i,k*] - mu[d_i,a_i] ), k* = argmax_k mu[d_i,k].
import numpy as np


def make_world(D=8, K=6, seed=0, dominator=False):
    rng = np.random.default_rng(seed)
    mu = rng.uniform(0.30, 0.70, size=(D, K))
    if dominator:
        mu[:, 0] = 0.85                       # model 0 best on EVERY dimension -> routing cannot help
    else:
        for d in range(D):                    # complementary strengths: each dim favours a model
            mu[d, d % K] = 0.85
    return mu


def make_stream(mu, N=6000, seed=1):
    D, K = mu.shape
    rng = np.random.default_rng(seed)
    dims = rng.integers(0, D, size=N)                          # task-dimension of each streamed task
    R = (rng.random((N, K)) < mu[dims]).astype(np.int64)       # realised execution outcome per arm
    return dims, R


def per_task_regret(mu, dims, actions):
    best = mu.argmax(axis=1)[dims]                             # per-task oracle arm
    return mu[dims, best] - mu[dims, actions]                  # >= 0 by construction


def avg_perf(mu, dims, actions):
    return float(mu[dims, actions].mean())                     # expected performance of the policy


# --- routers -------------------------------------------------------------------
def oracle_actions(mu, dims):
    return mu.argmax(axis=1)[dims]


def probe_prior(mu, m=3, seed=7):
    """Fixed prior from a tiny probing sample (m draws per (dim,model)); never updated."""
    rng = np.random.default_rng(seed)
    return (rng.random((m,) + mu.shape) < mu).mean(axis=0)     # noisy estimate of mu


def static_dimensionbest(mu, dims, m=3):
    pick = probe_prior(mu, m=m).argmax(axis=1)                 # per-dimension fixed choice (heuristic prior)
    return pick[dims]


def static_contextfree(mu, dims, m=3):
    k = probe_prior(mu, m=m).mean(axis=0).argmax()            # single global-best arm, no dimension info
    return np.full(dims.shape, k, dtype=np.int64)


def feedback_router(mu, dims, R, contextual=True, noise=0.0, c=0.6, seed=3):
    """Online UCB over arms. contextual=True keeps stats PER task-dimension (ACRouter-style);
    contextual=False pools all tasks (context-free). noise = verifier mis-specification rate."""
    D, K = mu.shape
    N = len(dims)
    rng = np.random.default_rng(seed)
    G = D if contextual else 1
    s = np.zeros((G, K)); n = np.zeros((G, K)); t = np.zeros(G)
    actions = np.empty(N, dtype=np.int64)
    for i in range(N):
        g = dims[i] if contextual else 0
        t[g] += 1
        if (n[g] == 0).any():
            a = int(np.where(n[g] == 0)[0][0])                 # pull each arm once (init)
        else:
            a = int((s[g] / n[g] + c * np.sqrt(np.log(t[g] + 1) / n[g])).argmax())
        obs = R[i, a]
        if noise and rng.random() < noise:
            obs = int(rng.integers(0, 2))                      # mis-specified verifier: random bit
        s[g, a] += obs; n[g, a] += 1
        actions[i] = a
    return actions


def decile_avg(reg, lo, hi):
    N = len(reg)
    return float(reg[int(lo * N):int(hi * N)].mean())


# --- experiment ----------------------------------------------------------------
mu = make_world()
dims, R = make_stream(mu, N=6000)

acts = {
    "oracle":     oracle_actions(mu, dims),
    "static_cf":  static_contextfree(mu, dims),
    "static_db":  static_dimensionbest(mu, dims),
    "fb_ctx":     feedback_router(mu, dims, R, contextual=True),
    "fb_cf":      feedback_router(mu, dims, R, contextual=False),
}
reg = {k: per_task_regret(mu, dims, a) for k, a in acts.items()}
cum = {k: float(np.cumsum(r)[-1]) for k, r in reg.items()}

# 0) Sanity: per-task regret is non-negative; the per-task oracle has exactly zero regret.
assert all((r >= -1e-12).all() for r in reg.values())
assert cum["oracle"] == 0.0

# 0b) Equivalence to a slow, explicit reference over a slice (guards the vectorised regret).
best = mu.argmax(axis=1)
slow = sum(mu[dims[i], best[dims[i]]] - mu[dims[i], acts["fb_ctx"][i]] for i in range(500))
assert abs(slow - float(np.cumsum(reg["fb_ctx"])[499])) < 1e-9

# 1) Static routers grow ~linearly; the feedback router grows sublinearly. Compare the mean
#    per-task regret in the last decile vs the first decile.
first_last = {k: (decile_avg(reg[k], 0.0, 0.1), decile_avg(reg[k], 0.9, 1.0)) for k in reg}
sc_f, sc_l = first_last["static_cf"]
db_f, db_l = first_last["static_db"]
fb_f, fb_l = first_last["fb_ctx"]
assert 0.6 < sc_l / sc_f < 1.6, first_last["static_cf"]     # context-free static: no convergence (linear)
assert 0.5 < db_l / db_f < 1.7, first_last["static_db"]     # heuristic-prior static: still no convergence
assert fb_l / fb_f < 0.15, first_last["fb_ctx"]             # feedback: late regret collapses (sublinear)

# 2) The feedback router beats the best fixed heuristic, and lands far below the linear routers.
assert cum["fb_ctx"] < cum["static_db"] < cum["static_cf"]
assert cum["fb_ctx"] < 0.35 * cum["static_db"]              # online learning roughly a third of the heuristic

# 3) TASK-DIMENSION statistics help (the paper's 15.3% direction), both offline and online.
#    Relative AvgPerf gain of dimension-aware over context-free; we reproduce the SIGN, not 15.3%.
def rel_gain(better, base):
    return (avg_perf(mu, dims, acts[better]) - avg_perf(mu, dims, acts[base])) / avg_perf(mu, dims, acts[base])
gain_static = rel_gain("static_db", "static_cf")            # DimensionBest prior vs context-free prior
gain_online = rel_gain("fb_ctx", "fb_cf")                   # dimension-aware online vs context-free online
assert gain_static > 0.03                                  # a modest but real OFFLINE lift from dim priors
assert gain_online > 0.20                                  # the ONLINE lift from dim stats is far larger
assert cum["fb_ctx"] < cum["fb_cf"]                         # ... and they cut regret online too (7x here)

# 4) BOUNDARY: one model dominates every dimension -> routing cannot help; all routers converge
#    and the dimension statistics buy nothing.
mu_d = make_world(dominator=True)
dims_d, R_d = make_stream(mu_d, N=6000)
acts_d = {
    "static_cf": static_contextfree(mu_d, dims_d),
    "static_db": static_dimensionbest(mu_d, dims_d),
    "fb_ctx":    feedback_router(mu_d, dims_d, R_d, contextual=True),
}
cum_d = {k: float(np.cumsum(per_task_regret(mu_d, dims_d, a))[-1]) for k, a in acts_d.items()}
gain_d = (avg_perf(mu_d, dims_d, acts_d["static_db"]) - avg_perf(mu_d, dims_d, acts_d["static_cf"])) \
    / avg_perf(mu_d, dims_d, acts_d["static_cf"])
assert cum_d["static_cf"] == 0.0                           # context-free single best == oracle: no routing problem
assert gain_d <= 0.0                                       # dimension stats add nothing; a noisy dim prior even hurts
assert gain_d < gain_static                                # they help under complementarity, not under a dominator
assert cum_d["fb_ctx"] > cum_d["static_cf"]                # online EXPLORATION is pure overhead when routing is moot

# 5) ADVERSARIAL: a mis-specified (noisy) verifier degrades the feedback router toward static.
fb_clean = feedback_router(mu, dims, R, contextual=True, noise=0.0)
fb_noisy = feedback_router(mu, dims, R, contextual=True, noise=0.6)
cum_clean = float(np.cumsum(per_task_regret(mu, dims, fb_clean))[-1])
cum_noisy = float(np.cumsum(per_task_regret(mu, dims, fb_noisy))[-1])
assert cum_noisy > 2.5 * cum_clean                         # noisy feedback multiplies regret (~2.8x here)
assert cum_noisy > 0.4 * cum["static_db"]                  # ... collapsing a large fraction back toward static
assert cum_clean < cum_noisy < cum["static_cf"]            # it lands between clean feedback and no feedback

print("cumulative regret (lower is better):")
for k in ("oracle", "fb_ctx", "fb_cf", "static_db", "static_cf"):
    print(f"  {k:10s} CumReg={cum[k]:7.2f}  AvgPerf={avg_perf(mu, dims, acts[k]):.4f}")
print(f"first->last decile mean per-task regret:")
print(f"  static_cf {sc_f:.4f}->{sc_l:.4f} (x{sc_l/sc_f:.2f})   "
      f"static_db {db_f:.4f}->{db_l:.4f} (x{db_l/db_f:.2f})   "
      f"fb_ctx {fb_f:.4f}->{fb_l:.4f} (x{fb_l/fb_f:.2f})")
print(f"relative AvgPerf gain from task-dimension stats: static {gain_static*100:.1f}%  online {gain_online*100:.1f}%")
print(f"boundary (dominator): static_cf CumReg={cum_d['static_cf']:.2f}  dim-stat gain={gain_d*100:.2f}%")
print(f"adversarial verifier: clean CumReg={cum_clean:.2f}  noisy CumReg={cum_noisy:.2f}  "
      f"(static_db={cum['static_db']:.2f})")
print("all assertions passed")

Run output:

cumulative regret (lower is better):
  oracle     CumReg=   0.00  AvgPerf=0.8500
  fb_ctx     CumReg= 214.06  AvgPerf=0.8143
  fb_cf      CumReg=1455.01  AvgPerf=0.6075
  static_db  CumReg=1191.29  AvgPerf=0.6515
  static_cf  CumReg=1367.19  AvgPerf=0.6221
first->last decile mean per-task regret:
  static_cf 0.2246->0.2161 (x0.96)   static_db 0.1969->0.1865 (x0.95)   fb_ctx 0.1335->0.0159 (x0.12)
relative AvgPerf gain from task-dimension stats: static 4.7%  online 34.0%
boundary (dominator): static_cf CumReg=0.00  dim-stat gain=-2.94%
adversarial verifier: clean CumReg=214.06  noisy CumReg=606.79  (static_db=1191.29)

Read the assertions, not the toy dollars. Five things hold. The static routers' per-task regret barely moves from the first decile to the last (x0.96, x0.95): they keep paying the same gap, so cumulative regret is near-linear. The feedback router's per-task regret collapses (x0.12): it learns the per-dimension winner, so its cumulative regret is sublinear and lands at roughly a sixth of the best fixed heuristic. Task-dimension statistics are a real lift, small offline (+4.7%) and large once the router also learns online (+34%), which is the direction of the paper's 15.3% finding and not a claim to reproduce its magnitude. Under a dominator the trivial context-free router already equals the oracle (regret 0), dimension statistics buy nothing (a noisy prior even loses 2.9%), and the online explorer is pure overhead. And a verifier that is right only 40% of the time nearly triples the feedback router's regret, dragging it a large fraction of the way back toward the no-feedback baseline: the honest failure mode.

How to use it¶

Point it at a heterogeneous pool. ACRouter is the routing policy in front of several model endpoints, so it slots into the gateway placement described in LLM request routing. The pool should be genuinely complementary; two similarly-ranked-but-redundant models give the router nothing to exploit.
Give it task dimensions. The single cheapest win is per-dimension performance statistics (which model has historically been best on this language, task type, or difficulty band). The paper's diagnostic is that this alone recovers most of the gap; the online loop then refines it.
Verify with execution, not vibes. Correctness signals must be grounded (unit tests, sandboxed run, compile-and-run), because the Verifier score is the only new information in the loop. Where a dimension cannot be executed, an LLM-as-judge is a fallback, not the default.
Put cost in the reward. Route on quality-per-dollar by weighting the Verifier score positively and the token cost negatively, so the router does not silently converge on the most expensive model.

How to develop and integrate it¶

ACRouter ships as a research recipe, not a library, so integration is building the three modules against your pool. A faithful, unexecuted skeleton of the loop:

# acrouter_loop.py -- REFERENCE TEMPLATE (not executed). Faithful shape of the C-A-F loop.
# The executed numpy block above is the validated core; this is the production wiring around it.
from dataclasses import dataclass

@dataclass
class Feedback:
    score: float            # Verifier performance in [0,1]
    cost: float             # monetary cost from token usage and list prices

class ACRouter:
    def __init__(self, orchestrator, verifier, memory, models,
                 w_perf=1.0, w_cost=0.1):
        self.orch, self.ver, self.mem, self.models = orchestrator, verifier, memory, models
        self.w_perf, self.w_cost = w_perf, w_cost      # reward = w_perf*score - w_cost*cost

    def step(self, task):
        # Context: metadata + dimension prior + kNN recall of similar past tasks
        ctx = {"task": task,
               "prior": self.mem.dimension_prior(task.dimension),
               "recall": self.mem.knn(task.embedding, k=10)}
        a = self.orch.decide(ctx, self.models)          # Action: pick a model index
        out = self.models[a].run(task)                  # Execute
        score = self.ver.score(task, out)               # Feedback: execution-grounded score
        fb = Feedback(score=score, cost=out.cost)
        reward = self.w_perf * fb.score - self.w_cost * fb.cost
        self.mem.write(task, a, fb, reward)             # Memorize -> next Context
        return a, fb, reward

The Orchestrator stays small. Keep it cheap enough that the routing turn is a fraction of the generation it gates; a sub-1B fine-tune or even a well-fed heuristic is the paper's regime. See agent harness architecture for why the surrounding program, not a bigger router, carries reliability.
The Verifier is the dependency to get right. Reuse an existing execution sandbox and test harness; the scoring contract here is the same one that gates promotions in agent evaluation. Budget engineering effort here, because the adversarial result shows a weak Verifier erases the router's advantage.
Memory is an online vector store. Key on task embedding; store model, score, cost, and trace; query by cosine kNN; bound the size and evict FIFO so recall stays fast and reflects the current model pool.
Cold-start from offline priors. Before Memory fills, the loop leans on the per-dimension prior fitted on a probing set. This is why the "static" routers in the code are not worthless: they are the router's day-zero behaviour, which the online loop then improves on.

How to run it in production¶

Warm-start, then adapt. Ship with the offline per-dimension priors in place so the router is useful from the first task, and let the online loop take over as Memory accumulates. Do not launch cold.
Monitor regret, not accuracy. The operating metric is cumulative regret against a periodically-recomputed per-task oracle, plus per-dimension win rates and quality-per-dollar. A rising regret slope on a dimension means the pool changed or the Verifier drifted.
Guard the Verifier as the trust root. Alert on Verifier disagreement with held-out ground truth, sandbox failures, and judge-vs-execution divergence. Treat a Verifier regression as a routing incident, because it corrupts every statistic downstream.
Bound and isolate Memory. Cap entries, evict FIFO, and salt keys per tenant if the router serves multiple teams, for the same confidentiality reasons as own or rent a coding model gives for a shared prefix cache.
Keep the escape hatch. ACRouter is the policy that decides when the owned pool suffices and when to escalate to a frontier model; it does not replace the hatch, it earns its cost. That is exactly the role own or rent a coding model assigns to it.

How to maintain it¶

Re-probe when the pool changes. Adding or retiring a model, or a silent vendor update, invalidates the per-dimension priors and the Memory's stored outcomes. Re-fit the priors on a fresh probing set and consider flushing stale Memory for the affected models.
Re-fit the Orchestrator on drift. The routing policy is trained on a probing set; when the traffic distribution or the model ranking shifts materially, re-fit it and re-check regret on a fresh oracle.
Treat the constants as version-specific. The reward weights, the kNN neighbour count, the Memory bound, and the Orchestrator size come from one 2026 paper on one benchmark. Re-tune them for your pool and traffic; do not assume they transfer.
Re-baseline the benchmark. Coding tasks leak into training corpora over time, so regenerate evaluation tasks from recent work rather than freezing CodeRouterBench forever, the same discipline own or rent a coding model applies to its internal eval.

Results¶

All figures are as reported by the paper (Zhou et al., 2026); verify against the source before quoting, and see the honest-discrepancy note in the opening callout.

The 15.3% headline. Augmenting a vanilla LLM router with task-dimension performance statistics gives a 15.3% relative gain in average performance and surpasses a heuristic router built on the same dimension-level priors. The paper reads this as evidence that routing is bottlenecked by an information deficit, not by the router's reasoning.
CodeRouterBench. Roughly 10K task instances with verified scores from eight frontier LLMs (the paper's pool spans the Claude, GPT, Qwen, GLM, Kimi, and MiniMax families), split into about 7,080 probing tasks, 2,919 in-distribution test tasks, and 176 out-of-distribution agentic-programming tasks (the 7,080 plus 2,919 make the 9,999 in-distribution tasks the official repository lists). Most dimensions are scored by execution (pass@1 in a sandbox); the rest use proxy metrics with an LLM-as-judge. Tasks are repurposed from more than fifteen existing coding benchmarks (including HumanEval, MBPP, and SWE-bench).
Cumulative regret. ACRouter reports the lowest cumulative regret on the in-distribution stream among the compared routers (static heuristics, trained classifiers, contextual bandits, and single-model policies), with the per-task oracle at zero by construction. The paper's central claim is this regret ordering, which the executed toy above reproduces qualitatively.
Out-of-distribution generalisation. On the 176 agentic-programming tasks, static learners degrade sharply (the paper reports four of its five trained baselines falling below a random router's 31.25%: LogReg at 19.64%, RouteLLM-BERT at 21.43%, TF-IDF+MLP at 13.39%, and RouteLLM-MF at 8.93%), while ACRouter's online adaptation holds up and beats an always-use-the-strongest policy on both performance and quality-per-dollar.

Failure modes¶

Mis-specified Verifier. A noisy or gameable correctness signal poisons Memory and the priors, and the router degrades toward a static classifier. The adversarial case in the code shows a 40%-correct verifier nearly tripling regret. The Verifier is the trust root; budget for it accordingly.
Memory poisoning and staleness. Stored outcomes from a since-updated model, or from a period of Verifier error, mislead the kNN recall. Bound Memory, evict FIFO, and flush entries for models that changed.
Cold-start before priors accumulate. With empty Memory and no offline prior, the loop routes near-randomly until it has explored. Warm-start from a probing set; do not launch cold.
No routing headroom. If one model dominates every task type, routing cannot help and the online explorer is pure overhead, losing to a trivial fixed router (the boundary case). Confirm complementarity before deploying a router.
Reward weights misconfigured. Too small a cost penalty and the router drifts to the most expensive model; too large and it starves quality. The weights are in the loop, not a post-hoc filter, so a bad setting shapes every decision.
Distribution shift beyond what was tested. The out-of-distribution evidence is 176 agentic tasks from one benchmark; genuinely novel traffic may still break the priors. Monitor regret and re-probe.
Orchestrator cost creep. A router that grows large enough to reason well per request eats the latency and cost budget it was meant to save. Keep it small; the paper's own finding is that information, not a bigger router, is the lever.
Over-trusting single-paper numbers. The exact per-router figures differ between the paper's tables and its repository, and this is one recent paper on one benchmark. Reproduce the regret ordering on your own pool rather than quoting the point estimates.

References¶

Agent-as-a-Router: Agentic Model Routing for Coding Tasks (Zhou et al., 2026), arXiv abstract: https://arxiv.org/abs/2606.22902 . Formalises routing as a Context-Action-Feedback loop (Orchestrator, Verifier, Memory); reports a 15.3% relative gain from task-dimension performance statistics over a vanilla LLM router; introduces CodeRouterBench (~10K instances, 8 frontier LLMs) and reports ACRouter at the lowest cumulative regret in-distribution with out-of-distribution generalisation.
Official implementation and CodeRouterBench data: https://github.com/LanceZPF/agent-as-a-router . The repository's own recomputed out-of-distribution baseline table diverges from the paper's Table 13 by 5 to over 30 points on most rows (its own maintainer notes flag one baseline's published number as off by about 30 points); cumulative-regret figures stay closer. Treat both sources as approximate, and the out-of-distribution figures as materially unstable.