Skip to content
Markdown

AOI: trainable multi-agent cloud diagnosis (Autonomous Operations Intelligence)

Scope: AOI (Autonomous Operations Intelligence, arXiv 2603.03378), a trainable multi-agent SRE framework built from a read-write separated Observer/Probe/Executor/Compressor runtime, Observer-level GRPO training, and a Failure Trajectory Closed-Loop Evolver, evaluated on AIOpsLab's 86-task benchmark. Covers the permission architecture, the GRPO advantage and reward formulation, the Evolver's repair-versus-augmentation split, and the paper's own results tables and ablations. This is a distinct project from the other same-acronym paper in this KB, AOI: AI-Oriented Operations (arXiv 2512.13956, a different research group); the two papers share only a three-letter name, not authors, architecture, or results, and should never be conflated. For the broader field context see agentic AIOps and its production-deployed worked example OpsAgent; the training algorithm itself is GRPO; the read-write separation pattern generalizes the isolation principles in agent sandboxing and isolation.

All benchmark numbers on this page (Tables 2-4, Appendices B-E) are the paper's own AIOpsLab runs; no AIOpsLab cluster or Qwen3-14B training run was reproduced here. The two Python blocks are executed and asserted: one models the Table 1 agent/store permission matrix, Algorithm 1's main loop, and the Executor's evidence-and-whitelist gate; the other models the Eq. 1 GRPO advantage, the Eq. 3 six-dimension step reward with its hard-penalty rule, and the best@k/avg@k metrics applied to a population constructed to honor Table 9's stability-bucket counts (the paper publishes only bucket totals, not per-round pass/fail identity per task, so the reconstructed best@k curve is a metric-definition check, not a reproduction of Figure 7). Code status: the PDF's own "Code and Data Availability" section says the review repository was anonymized (anonymous.4open.science/r/aoi-C8C7, returns HTTP 401 as of 2026-07-16); the de-anonymized project page named in the arXiv front matter, https://github.com/OpenEdgeHQ/aoi, is live and public (GitHub API: "private": false, "visibility": "public", language Python, pushed 2026-02-11) and its README describes the same Observer/Probe/Executor/Compressor architecture, so that repository is the one cited below.

What it is

AOI targets three obstacles the authors give for why LLM-agent SRE has not reached enterprise production: proprietary operational data cannot leave the estate, so only small locally-deployed models (under 100B parameters) are viable; permission-governed environments make unsafe action execution a hard blocker, not a tuning problem; and closed, static systems cannot improve from their own failures.1 It answers with three integrated pieces: a trainable diagnostic system that applies Group Relative Policy Optimization (GRPO) to distill expert-level diagnostic behavior into a locally-deployed open-weight model; a read-write separated execution architecture that keeps diagnosis and remediation on strictly different privilege rails; and a Failure Trajectory Closed-Loop Evolver that mines the system's own unsuccessful trajectories and turns them into corrective training signal and inference-time guidance.1

The runtime (Section 3) is four specialized agents coordinating over three memory stores, not a single prompted LLM. Observer plans, tracks hypotheses, and decides the next action (Probe, Execute, or Submit), but never touches the environment directly. Probe issues read-only commands (kubectl get, describe, logs) with up to Kmax=5 exploration rounds per iteration. Executor issues state-changing commands behind a whitelist filter, with two-stage error recovery and a "look before you leap" option to invoke a single Probe round before acting. Compressor deduplicates and semantically compresses raw tool output into the Observer's sole information source, and is stateless: every iteration is processed independently so errors cannot silently accumulate across the trajectory.2 Table 1's access-control matrix over three stores, Mraw (raw environment output), Mtask (task queue and hypothesis state), and Mcomp (compressed context), makes the separation structural rather than a prompt convention: Observer can only read Mcomp; Probe and Executor can write Mraw but not read it back; only the Compressor moves information from Mraw into Mcomp.2

Long-horizon coherence comes from dual-timescale memory: long-term memory H is a running list of per-iteration summaries S_i, and short-term memory is the full compressed context C_{n-1} from the immediately preceding iteration; at iteration n the Observer conditions its decision on (H_{n-2}, C_{n-1}).2 Each iteration is a fixed four-stage pipeline: Decision (Observer emits an action and instruction), Interaction (the routed agent executes in the environment and writes Mraw), Compression (the Compressor produces C_n within a fixed token budget), and Caching (the platform stores C_n and appends the Observer's summary to H).2

GRPO trains two different things at two different granularities. Observer GRPO (Section 3.3) optimizes single-step decisions: for a context x, a group of G candidate actions {y_i} are sampled, an LLM judge scores each on six dimensions, and the group-normalized advantage A_i = (R(x,y_i) - mu_G) / (sigma_G + eps) drives the policy gradient.3 Evolver GRPO (Section 4) optimizes whole-trajectory generation: given a seed trajectory tau, it samples G candidate corrected command sequences and scores them on validity, completeness, correctness, and effectiveness, using the same group-relative advantage form.4 Training data for Observer GRPO comes only from Evolver-repaired or Purifier-cleaned successful trajectories; the Purifier strips retries and dead-end exploration from a successful run down to the minimal command sequence that reached the correct diagnosis.1

Why use it

  • Read-write separation alone is worth more than the base model. With an identical GPT-4o-mini base and no task-specific training, AOI reaches 58.1% overall on the full 86-task benchmark against the reference ReAct-style AOL-agent's 14.7%, a roughly 4x improvement the paper attributes to Observer being free to explore diagnostically without risking a state mutation.6 On the Qwen3-14B (best@5) rows, the same architectural change takes Detection from STRATUS's 75.0% to 100%, RCA from 7.7% to 30.8% (a 4x / +300% relative gain -- note the paper's own "+150% over STRATUS" RCA claim in Section 5.2 arithmetically matches the GPT-4o-mini row's RCA change instead, 15.4% -> 38.5%, not this Qwen3-14B row), and Mitigation from 15.4% to 46.2% (3x), because Executor-level safety gates prevent the cascading failures STRATUS suffers when it mutates state before diagnosis is complete.6
  • A 14B open-weight model with this architecture beats a frontier model with a simpler one. Qwen3-14B + AOI reaches 66.3% best@5 overall (solving 57/86 tasks) against Claude Sonnet 4.5 + AOL-agent's 57.0% (49/86), a 9.3-point gap. (The abstract's headline "24.4 percentage points" figure is a different comparison -- it is the gap between AOI's untrained runtime, 66.3% best@5, and the prior state-of-the-art STRATUS's Qwen3-14B best@5 Overall score, 41.9%, not a comparison against Claude Sonnet 4.5.)6 Sonnet still wins outright on Mitigation (76.9% vs AOI's 46.2%) and RCA is where it is weakest (15.4%, tied with GPT-4o-mini+STRATUS), which the paper reads as frontier models being strong at remediation pattern-matching but no better than a well-scaffolded 14B model at multi-step causal reasoning.6
  • Observer GRPO training generalizes to fault types it never saw. Trained on only 23 tasks across 11 fault types, the 14B Observer lifts avg@1 from 33.7% (untrained AOI) to 42.9% on 63 held-out tasks spanning 15 unseen fault types, surpassing Claude Sonnet 4.5's 41.3% on the identical held-out set without any multi-run sampling.7 The gain concentrates in Detection (54.5% -> 90.9% vs Sonnet, +36.4pp) and RCA (8.3% -> 16.7%, +8.4pp), the two task types the paper argues reward systematic diagnostic search over pattern recognition; Mitigation stays flat at 14.3% because Observer optimization cannot directly improve commands the Executor generates.7
  • The Evolver converts failure into a measurable reliability gain, not just a headline accuracy bump. On the 37 tasks Claude Sonnet 4.5 originally failed, adding Evolver-generated corrected plans as prompts raises end-to-end avg@5 by 4.8 points (24.9% -> 29.7%) and shrinks the best@5-avg@5 gap from 29.2pp to 18.9pp, a 35% reduction the paper reads as reproducibility rather than luck: Detection's own gap alone halves from 50pp (100% best@5, 50.0% avg@5) to 26pp (90% best@5, 64.0% avg@5).8 Repair quality itself improves under GRPO training: an LLM-judge score of the Evolver's repaired plans rises from a mean of 7.18 to 8.27 (out of 10) while its standard deviation drops from 0.97 to 0.49, which the paper reads as the trained Evolver learning structural correction patterns rather than memorizing command strings.8

When to use it (and when not)

  • Use the pattern where an SRE agent needs write access to a real environment and proprietary telemetry cannot leave the estate; the whole design exists to make a sub-100B locally-deployed model safe and competent enough to replace a closed frontier API for this constraint.1
  • Use Observer GRPO when you have even a small set of successful trajectories (the paper trained on 23 tasks) and need the gains to transfer to fault types outside that set; the held-out evaluation is deliberately a strict fault-type split with zero overlap between train and test types.5
  • Do not expect uniform gains from GRPO training. It is not free lunch: the same training that lifts Detection by 25.5 points (untrained AOI 65.5% -> trained 90.9%, avg@1 per the abstract's and Section 5.3's own framing of these Table 3 numbers -- though Figure 10's caption in the same paper labels this identical delta "best@5," a source inconsistency we flag rather than silently resolve) drops Localization by 4.5 points (22.7% -> 18.2%), because the trained Observer learns to explore roughly 9 more steps per task on average, which helps find an anomaly but hurts pinpointing the faulty component among several similar candidates; both of the paper's two GRPO-degraded tasks are Localization instances that flip from 4/5 to 0/5 successes.9 Net effect across the 63 held-out tasks is +9 (11 improved, 2 degraded), positive but not monotonic, so re-run the paper's per-task-type breakdown on your own fault mix before trusting an aggregate delta.9
  • Do not treat this as a validated production system yet. Unlike this KB's OpsAgent page, which documents 53 days of live Lenovo deployment, AOI's own Limitations section states production deployment is future work ("we plan to deploy AOI in production SRE environments to validate the use and productization potential"); every number on this page is an AIOpsLab benchmark run.10
  • Expect a hard capability floor. 29 of 86 tasks (33.7%) never succeed across five sampling rounds, GPT-4o-mini, or any GRPO variant tested; the paper characterizes these as systematic capability gaps (MongoDB auth recovery needing Helm-specific knowledge, multi-service localization needing causal reasoning the architecture does not yet provide) rather than noise, and recommends routing them to human escalation rather than more retries.9

Architecture

flowchart TB
  SRE["SRE troubleshooting workflow<br/>seed trajectory"] --> JUDGE{"Judge: classify outcome"}
  JUDGE -->|"success, 49 seeds"| PURIFIER["Purifier: strip retries and<br/>dead-ends to minimal sequence"]
  JUDGE -->|"failed, 37 seeds"| EVOLVER["Trajectory Evolver:<br/>GRPO-sampled repair, G=4"]
  PURIFIER --> TRAINDATA["Observer GRPO training set<br/>23 tasks, 11 fault types"]
  TRAINDATA --> GRPOTRAIN["Observer GRPO:<br/>group-normalized advantage,<br/>6-dim step reward"]
  EVOLVER --> CORRECTED["Corrected diagnostic plan<br/>structured prompt"]
  GRPOTRAIN --> OBSERVER
  CORRECTED -.->|"guidance at inference"| OBSERVER

  subgraph RUNTIME["AOI Multi-Agent Runtime"]
    OBSERVER["Observer: decide Probe, Execute<br/>or Submit; reads Mcomp only"] -->|"Probe"| PROBE["Probe: read-only kubectl<br/>get/describe/logs, up to 5 rounds"]
    OBSERVER -->|"Execute"| EXECUTOR["Executor: whitelisted<br/>state-changing commands"]
    PROBE --> MRAW[("Mraw: raw environment output")]
    EXECUTOR --> MRAW
    MRAW --> COMPRESSOR["Compressor: stateless dedup<br/>plus LLM semantic compression"]
    COMPRESSOR --> MCOMP[("Mcomp: compressed context")]
    MCOMP --> OBSERVER
  end

  OBSERVER -->|"Submit"| RESULT["Root-cause / mitigation submission"]
  RESULT --> JUDGE

The left half is Figure 1's Closed-Loop Evolution Pipeline: a Judge classifies each completed SRE workflow, successful ones get purified into Observer GRPO training data, failed ones get repaired by the Evolver into structured guidance the Observer receives as a prompt at inference time. The right half is Figure 2's runtime: Algorithm 1's main loop, in which Observer never sees Mraw directly and every diagnostic or remediation action is routed through the Compressor before Observer sees the result.2 Budget exhaustion is explicit in the algorithm, not a hang: after N=15 iterations (Table 5) without a Submit, the loop returns a timeout submission.5

How to use it

The reference configuration trains and serves a single open-weight model: Qwen3-14B as the base, LoRA fine-tuning at rank 64 / alpha 128 / learning rate 1e-5, GRPO group size G=4, batch size 16, 3 epochs, on 2xA100 GPUs with vLLM for inference; the Evolver's reward model is Claude Opus 4.5 used only as an offline judge, never in the serving path.5 Table 5's runtime hyperparameters are the starting point for a new deployment: 15 max iterations, up to 5 Probe rounds per iteration, a 4096-token context budget per iteration, 10-summary long-term memory capacity, and a 47-pattern Executor whitelist.5 The public code (github.com/OpenEdgeHQ/aoi) wraps AIOpsLab as a git submodule and drives it end to end:

# Reference template from the public README (github.com/OpenEdgeHQ/aoi). Not executed
# here: it needs a Kind/Kubernetes cluster, Helm, and an LLM API key.
git clone --recurse-submodules https://github.com/OpenEdgeHQ/aoi.git
cd aoi
pip install -r requirements.txt
cd AIOpsLab && pip install -e . && cd ..

cp .env.example .env   # set API_SOURCE, API_KEY, API_BASE, MODEL
kind create cluster --config AIOpsLab/kind/kind-config-x86.yaml
cd AIOpsLab/aiopslab && cp config.yml.example config.yml  # set k8s_host: kind

When adapting the benchmark's 26 fault types to a real estate, mirror the paper's nested, leakage-free data split rather than a random holdout: build the Evolver's training set from every successfully-resolved historical incident, take a strict fault-type subset of that (zero type overlap with your held-out evaluation set) for Observer GRPO, and keep every test case unseen by both components, exactly as the paper enforces Dobs_train subset of Devolver_train subset of Dall so the combined system's evaluation carries no leakage between the two trained pieces.5

How to develop and extend it

  • Reward dimensions are where behavior gets tuned, not the base model. Context Instruction and Context Namespace carry 60% of the Observer's step reward by design, reflecting the authors' bet that diagnostic reasoning quality and target accuracy dominate information gain per step; Format is rule-based (a JSON parse failure is a hard R=0.09 penalty regardless of the other five scores) so malformed output can never win a GRPO group by accident.3 If you extend the reward, keep a rule-based floor for structural validity separate from the LLM-judged dimensions, the same separation the executed reward function below enforces.
  • Seed provenance is deliberately swappable. The paper uses Claude Sonnet 4.5 trajectories on AIOpsLab as a stand-in for expert SRE records purely because that model provided enough high-quality successes on the benchmark; the Evolver's design does not care whether seeds come from a frontier model, human runbooks, or the system's own historical successes.4 Point the Purifier and Evolver at your own historical incident tickets once you have enough resolved ones, and success-vs-failure classification (the Judge in Figure 1) becomes the only part that needs a human-in-the-loop sign-off.
  • The Evolver's own stated limitation is a good next-extension target. Section 7 flags that the Evolver currently only emits corrected command sequences as static prompts; the authors call out producing synthetic environment feedback via a simulator, or running the Evolver as a live runtime agent for dynamic plan refinement, as unexplored future architecture, not something the current release does.10
  • Localization is the component that most needs task-aware tuning before you trust GRPO broadly. Because the same reward that improves Detection degrades Localization (Appendix D.4's root-cause analysis: over-exploration surfaces multiple anomaly candidates and the trained model picks the wrong one), a task-type-conditioned reward or a separate exploration budget per task type is the change the paper's own analysis points toward but does not implement.9

How to run it in production

Treat AOI the way this KB treats any pre-production agent framework: as a candidate to gate against your own incident history before it gets write access, not as a deployed reference. Three production-shaped facts from the paper matter for that gate. First, the read-write separation is not a compliance tax: the paper's own framing is that constraining the action space improved diagnostic success rather than reducing it, because evidence accumulation before mutation avoided the cascading state corruption STRATUS exhibited when it mutated before diagnosis completed.10 Second, GRPO training on 23 tasks generalized to 15 unseen fault types without collapsing, but it did so unevenly across task types, so any production rollout should track success by task type (Detection/Localization/RCA/Mitigation), not a single blended accuracy number, exactly because the blended 42.9% overall hides a Localization regression.7 Third, budget planning should use the paper's own diminishing-returns curve: best@1 is 31.4%, best@2 is 51.2% (a single retry captures most of the easy wins, +19.8pp), and gains flatten sharply after that (best@3 58.1%, best@5 66.3%); the paper recommends k=2 sampling rounds for cost-sensitive deployments and k=3 (which captures 88% of the achievable improvement) for higher-stakes ones, rather than assuming more samples keeps paying off.9

Executed: read-write separation and the Executor gate

This models Table 1's permission matrix, Algorithm 1's main loop, and the least-privilege gate the Introduction describes ("high-risk write commands are technically isolated and can only be triggered after sufficient evidence is gathered and verified"), plus the Executor's whitelist filter. It is a model of the mechanism, not a run of the real Observer/Probe/Executor/Compressor agents.

# aoi_readwrite_gate.py - validated: AOI's read-write separated execution
# architecture (Table 1 permission matrix, Section 3.1 key invariants,
# Algorithm 1 main loop) and the Executor's whitelist gate (Table 5:
# "Executor whitelist: 47 command patterns"). Models the mechanism; does not
# run AOI or Qwen3-14B.
from __future__ import annotations

from dataclasses import dataclass, field
from typing import Literal

Store = Literal["Mraw", "Mtask", "Mcomp"]
Mode = Literal["R", "W"]

# Table 1: agent x store access-control matrix ("-" = no access).
PERMISSIONS: dict[str, dict[Store, str]] = {
    "Observer":   {"Mraw": "-",  "Mtask": "RW", "Mcomp": "R"},
    "Probe":      {"Mraw": "W",  "Mtask": "R",  "Mcomp": "-"},
    "Executor":   {"Mraw": "W",  "Mtask": "R",  "Mcomp": "-"},
    "Compressor": {"Mraw": "R",  "Mtask": "-",  "Mcomp": "W"},
}


class PermissionError_(Exception):
    pass


def access(agent: str, store: Store, mode: Mode) -> None:
    """Enforce Table 1: raise unless the agent's grant covers this mode."""
    grant = PERMISSIONS[agent][store]
    if mode not in grant:
        raise PermissionError_(f"{agent} has no {mode} access to {store} (grant={grant!r})")


# A toy whitelist standing in for the paper's 47 Executor command patterns
# (Table 5); the exact 47 patterns are not published, only the count.
EXECUTOR_WHITELIST = {"kubectl rollout restart", "kubectl scale", "kubectl delete pod"}


def executor_run(cmd: str, evidence_gathered: bool) -> str:
    """Section 3.1 / Section 1: Executor 'operates under strict whitelists'
    and high-risk write commands 'can only be triggered after sufficient
    evidence is gathered and verified'."""
    if not evidence_gathered:
        raise PermissionError_("Executor blocked: no verified evidence yet (least-privilege gate)")
    if cmd not in EXECUTOR_WHITELIST:
        raise PermissionError_(f"Executor blocked: {cmd!r} not in whitelist")
    return f"executed: {cmd}"


@dataclass
class Stores:
    raw: dict = field(default_factory=dict)
    task: dict = field(default_factory=dict)
    comp: dict = field(default_factory=dict)


def compress(raw_n: dict) -> dict:
    """Compressor: 'stateless, processing each iteration independently to
    avoid error accumulation' (Sec 3.1, invariant 3). Pure function of the
    CURRENT iteration's raw output only, no closure over prior calls."""
    return {"summary": f"{len(raw_n)} raw records", "n_records": len(raw_n)}


def run_loop(max_iterations: int, decisions: list[tuple[str, dict]]) -> tuple[list[dict], str]:
    """Algorithm 1: Observer decides an action in {Probe, Execute, Submit}
    each iteration; only Mcomp reaches the Observer (invariant 1); budget
    exhaustion (Table 5: max_iterations=15) submits a timeout, never hangs."""
    stores = Stores()
    history: list[dict] = []
    evidence_gathered = False
    for n, (action, payload) in enumerate(decisions[:max_iterations], start=1):
        access("Observer", "Mcomp", "R")           # Observer only ever reads Mcomp
        if action == "Submit":
            return history, "submitted"
        if action == "Probe":
            access("Probe", "Mraw", "W")
            stores.raw[n] = payload
            evidence_gathered = True
        elif action == "Execute":
            access("Executor", "Mraw", "W")
            result = executor_run(payload["cmd"], evidence_gathered)
            stores.raw[n] = {"result": result}
        c_n = compress(stores.raw[n])
        access("Compressor", "Mcomp", "W")
        stores.comp[n] = c_n
        history.append(c_n)
    return history, "budget_exhausted"


# 1. Legal path: Probe gathers evidence, then a whitelisted Execute succeeds,
#    then Submit. Table 5 caps this at max_iterations=15.
decisions = [("Probe", {"q": "kubectl get pods"}),
             ("Execute", {"cmd": "kubectl rollout restart"}),
             ("Submit", {})]
hist, status = run_loop(max_iterations=15, decisions=decisions)
assert status == "submitted" and len(hist) == 2

# 2. Adversarial: Observer must never read Mraw directly, even if a caller
#    tries to route around Mcomp (invariant 1, the basis for the paper's
#    "Observer never directly interacts with the environment" claim).
try:
    access("Observer", "Mraw", "R")
    assert False, "Observer must not be able to read Mraw"
except PermissionError_:
    pass

# 3. Adversarial: Probe writes raw but cannot read it back, all information
#    must flow back through the Compressor (invariant 2).
access("Probe", "Mraw", "W")
try:
    access("Probe", "Mraw", "R")
    assert False, "Probe must not be able to read Mraw back"
except PermissionError_:
    pass

# 4. Adversarial: least-privilege gate. Execute before any Probe evidence is
#    denied even for a whitelisted command.
try:
    executor_run("kubectl scale", evidence_gathered=False)
    assert False, "Executor must not fire before evidence is gathered"
except PermissionError_:
    pass

# 5. Adversarial: a non-whitelisted command is rejected even with evidence in
#    hand (the 47-pattern whitelist, Table 5).
try:
    executor_run("rm -rf /", evidence_gathered=True)
    assert False, "non-whitelisted command must be rejected"
except PermissionError_:
    pass

# 6. Budget exhaustion: an Observer that never submits is cut off at
#    max_iterations and does not hang (Algorithm 1, line 19).
never_submits = [("Probe", {"q": f"probe-{i}"}) for i in range(20)]
hist2, status2 = run_loop(max_iterations=15, decisions=never_submits)
assert status2 == "budget_exhausted" and len(hist2) == 15

# 7. Compressor statelessness: compressing same-size raw payloads, with an
#    unrelated intervening call, must be independent of call history
#    (invariant 3, no error accumulation across iterations).
c_a = compress({"x": 1, "y": 2})
compress({"unrelated": "call", "with": "different", "size": 4})
c_b = compress({"p": 9, "q": 8})
assert c_a["n_records"] == c_b["n_records"] == 2

print("legal path status:", status, "| iterations:", len(hist))
print("budget-exhausted after", len(hist2), "of", len(never_submits), "attempted iterations")
print("all read-write separation and whitelist-gate assertions passed")

Executed output:

legal path status: submitted | iterations: 2
budget-exhausted after 15 of 20 attempted iterations
all read-write separation and whitelist-gate assertions passed

Executed: GRPO advantage, the six-dimension reward, and best@k/avg@k

This models Eq. 1's group-normalized advantage, Eq. 3's weighted six-dimension step reward with its rule-based hard penalty, and Section 5.1.2's best@k / avg@k metrics. The best@k reconstruction honors Table 9's task-stability bucket counts (14/16/27/29 of 86 tasks); the paper reports only those bucket totals, not which round each task succeeded in, so the per-round assignment below is our construction and the resulting curve is a check of the metric definitions, not a reproduction of Figure 7's exact points (best@5 lands at 66.3% by construction, since that value is fixed by the bucket counts alone: (86-29)/86; the intermediate points depend on the random seed).

# aoi_grpo_reward.py - validated: AOI's Observer GRPO group-normalized
# advantage (Eq. 1), the six-dimension weighted step reward with its
# rule-based hard penalty (Sec 3.3.2), and the best@k / avg@k multi-run
# metrics (Sec 5.1.2) applied to a population honoring the paper's Table 9
# task-stability buckets (14/16/27/29 of 86 tasks). Models the mechanisms;
# does not reproduce Qwen3-14B numbers.
from __future__ import annotations
import numpy as np

# Sec 3.3.2 / Appendix A.2: six reward dimensions and default weights.
WEIGHTS = {"format": 0.10, "summary": 0.15, "action": 0.10,
           "context_instruction": 0.30, "context_namespace": 0.30,
           "confidence": 0.05}
assert abs(sum(WEIGHTS.values()) - 1.0) < 1e-9
assert WEIGHTS["context_instruction"] + WEIGHTS["context_namespace"] == 0.60  # paper's "60%" claim


def step_reward(scores: dict[str, float], valid_json: bool) -> float:
    """Eq. 3: R(x,y) = sum_d w_d * s_d/10. Format is rule-based: a JSON
    parse failure triggers a hard penalty R=0.09 regardless of the other
    five scores (Sec 3.3.2)."""
    if not valid_json:
        return 0.09
    return sum(WEIGHTS[d] * scores[d] / 10.0 for d in WEIGHTS)


def group_advantage(rewards: np.ndarray, eps: float = 1e-6) -> np.ndarray:
    """Eq. 1: A_i = (R(x,yi) - mu_G) / (sigma_G + eps)."""
    mu = rewards.mean()
    sigma = rewards.std()
    return (rewards - mu) / (sigma + eps)


# 1. A candidate with strong context scores beats one that nails Format and
#    Action but gives vague reasoning: Context Instruction + Context
#    Namespace alone are 60% of the weight.
strong = {"format": 10, "summary": 8, "action": 10,
          "context_instruction": 9, "context_namespace": 9, "confidence": 7}
shallow = {"format": 10, "summary": 5, "action": 10,
           "context_instruction": 3, "context_namespace": 3, "confidence": 8}
r_strong, r_shallow = step_reward(strong, True), step_reward(shallow, True)
assert r_strong > r_shallow

# 2. Adversarial: perfect content scores but malformed JSON is capped at the
#    hard penalty, below every well-formed candidate regardless of content.
perfect_but_broken = step_reward(
    {"format": 10, "summary": 10, "action": 10,
     "context_instruction": 10, "context_namespace": 10, "confidence": 10},
    valid_json=False)
assert perfect_but_broken == 0.09
assert perfect_but_broken < r_shallow

# 3. GRPO group of G=4 (Table 5): advantages are exactly zero-mean, and the
#    highest-reward candidate gets the largest positive advantage.
G = 4
group_rewards = np.array([r_shallow, r_strong, 0.09, step_reward(strong, True) - 0.05])
adv = group_advantage(group_rewards)
assert abs(adv.mean()) < 1e-9
assert int(np.argmax(adv)) == int(np.argmax(group_rewards))

# 4. Adversarial: a degenerate group where every candidate scores identically
#    (sigma_G = 0) must not produce NaN/inf; the eps term in Eq. 1 exists
#    exactly for this case.
degenerate = np.array([0.5, 0.5, 0.5, 0.5])
adv_degenerate = group_advantage(degenerate)
assert np.all(adv_degenerate == 0.0) and np.all(np.isfinite(adv_degenerate))

# 5. best@k / avg@k (Sec 5.1.2) over a population honoring Table 9's task-
#    stability buckets: 14 tasks 5/5, 16 tasks 3-4/5 (split 8/8), 27 tasks
#    1-2/5 (split 14/13), 29 tasks 0/5, 86 tasks total. Per-round pass/fail
#    assignment within a task's success count is our reconstruction (the
#    paper reports only bucket totals); the metric functions are exact.
rng = np.random.default_rng(0)
n_tasks, n_rounds = 86, 5
counts = np.array([5] * 14 + [4] * 8 + [3] * 8 + [2] * 14 + [1] * 13 + [0] * 29)
assert len(counts) == n_tasks

runs = np.zeros((n_tasks, n_rounds), dtype=bool)
for t, k in enumerate(counts):
    idx = rng.choice(n_rounds, size=k, replace=False)
    runs[t, idx] = True


def best_at_k(runs: np.ndarray, k: int) -> float:
    return float(runs[:, :k].any(axis=1).mean() * 100)


def avg_at_k(runs: np.ndarray, k: int) -> float:
    return float(runs[:, :k].mean() * 100)


best_curve = [best_at_k(runs, k) for k in range(1, 6)]
avg5 = avg_at_k(runs, 5)
# Structural properties any valid best@k/avg@k curve must obey, independent
# of our synthetic per-round assignment:
assert best_curve == sorted(best_curve)                    # best@k is monotone non-decreasing
assert best_curve[-1] <= 100.0 and best_curve[0] >= 0.0
assert avg5 <= best_curve[-1]                               # avg@5 never exceeds best@5
# A 0/5 task can never contribute to best@k for any k; a 5/5 task always does.
assert runs[counts == 0].any(axis=1).sum() == 0
assert runs[counts == 5].all(axis=1).sum() == 14

print(f"reward: strong={r_strong:.3f} shallow={r_shallow:.3f} broken={perfect_but_broken:.3f}")
print(f"group advantages (G={G}): {np.round(adv, 3).tolist()}, mean={adv.mean():.6f}")
print(f"degenerate-group advantages: {adv_degenerate.tolist()}")
print(f"reconstructed best@k 1..5: {[round(b,1) for b in best_curve]}, avg@5={avg5:.1f}")
print("all GRPO reward and best@k/avg@k assertions passed")

Executed output:

reward: strong=0.895 shallow=0.495 broken=0.090
group advantages (G=4): [-0.267, 0.972, -1.522, 0.817], mean=0.000000
degenerate-group advantages: [0.0, 0.0, 0.0, 0.0]
reconstructed best@k 1..5: [39.5, 54.7, 59.3, 62.8, 66.3], avg@5=38.8
all GRPO reward and best@k/avg@k assertions passed

best@5 (66.3) matches Figure 7's reported point exactly, as it must by construction; best@1 through best@4 in this run (39.5/54.7/59.3/62.8) land close to but not identical with the paper's own curve (31.4/51.2/58.1/62.8), because those intermediate points depend on which of the 5 rounds each partially-successful task landed in, information the paper does not publish per task.

Failure modes

Failure mode Cause Fix
Localization accuracy drops after GRPO training The reward optimizes end-task completion; the trained Observer explores ~9 more steps per task on average, which surfaces multiple anomaly candidates and picks the wrong one for precision-critical tasks.9 Track success by task type, not a blended average; consider a task-type-conditioned reward or a separate exploration budget for Localization.
Mitigation accuracy stays flat despite Observer training Remediation commands are generated and executed by the Executor, not the Observer; Observer GRPO cannot directly improve command quality.7 Train or extend the Executor's command-generation policy separately if Mitigation is the bottleneck; do not expect Observer-only training to move it.
A GRPO group produces NaN or divide-by-zero advantages All G candidates score identically in one training step (sigma_G = 0). Eq. 1's + eps term is required, not optional; the executed block's degenerate-group case demonstrates it holds advantages at exactly zero rather than blowing up.
Well-formed but shallow diagnostic reasoning scores competitively A reward implementation that under-weights Context Instruction / Context Namespace against Format/Action lets superficially correct outputs win GRPO groups. Keep those two dimensions at their paper-default 60% combined weight, and keep Format's hard penalty rule-based and separate from LLM-judged content scores.
Benchmark scores read as production-ready Every number in this paper is an AIOpsLab run; the paper's own Limitations section states production deployment is still future work.10 Do not cite Table 2-4 numbers as production evidence; if you need a validated production deployment in this KB, see OpsAgent's Lenovo results instead.
Confused with the other "AOI" paper in this KB Two unrelated groups independently chose the acronym AOI for different systems (this page's "Autonomous Operations Intelligence" vs arXiv 2512.13956's "AI-Oriented Operations"). Always cite the arXiv ID alongside the acronym; link to AOI: AI-Oriented Operations when disambiguation matters.
Citing the anonymized review repo as unavailable The PDF's own Code Availability section points at an anonymous 4open.science link that returns HTTP 401. Use the de-anonymized project page instead, github.com/OpenEdgeHQ/aoi, verified public via the GitHub API as of 2026-07-16.
~30% of tasks treated as a training target 29/86 tasks fail across every sampling round, model, and GRPO variant the paper tested; the paper characterizes these as capability gaps, not stochastic noise.9 Route these fault classes to human escalation rather than budgeting more sampling rounds or retraining against them.

References

  • Yang, Chen, Zheng, Li, Li, Tu, Xiao, Pang, Zhang, Li, Long, Ai, Yang, Shi, "AOI: Turning Failed Trajectories into Training Signals for Autonomous Cloud Diagnosis," arXiv:2603.03378: https://arxiv.org/abs/2603.03378
  • Code and pre-trained models (de-anonymized project page, verified public): https://github.com/OpenEdgeHQ/aoi
  • AIOpsLab benchmark (Chen et al., MLSys 2025): https://arxiv.org/abs/2501.06706 and https://github.com/microsoft/AIOpsLab
  • STRATUS, the multi-agent baseline AOI compares against: https://arxiv.org/abs/2506.02009
  • Shao et al., "DeepSeekMath" (GRPO): https://arxiv.org/abs/2402.03300
  • Hu et al., "LoRA: Low-Rank Adaptation of Large Language Models": https://arxiv.org/abs/2106.09685
  • Kwon et al., "Efficient Memory Management for Large Language Model Serving with PagedAttention" (vLLM): https://arxiv.org/abs/2309.06180
  • Yang et al., "Qwen3 Technical Report": https://arxiv.org/abs/2505.09388
  • Anthropic, Claude Sonnet 4.5 system card: https://www.anthropic.com/claude-sonnet-4-5-system-card
  • Saltzer and Schroeder, "The Protection of Information in Computer Systems" (least-privilege principle the architecture cites): https://doi.org/10.1109/PROC.1975.9939

Related: AIOpsLab · AOI: AI-Oriented Operations · Agentic AIOps · Agentic incident management: OpsAgent · GRPO (group relative policy optimization) · Agent Sandboxing and Isolation

  1. AOI (arXiv:2603.03378v3, submitted 2026-03-16, revised 2026-03-17), Gradient / Soochow University / UC Santa Cruz / Georgia Institute of Technology / University College London / Cookiy.ai / WeJoy / ByteDance. Abstract and Section 1 (Introduction): three deployment challenges (proprietary data access, unsafe action execution, closed systems unable to learn from failure); three components (GRPO-trained diagnostic system, read-write separated execution, Failure Trajectory Closed-Loop Evolver); footnote 1 computes the ~215GB deployment cost of a 100B-parameter model under FP16 (~200GB weights + ~10GB KV cache for a 14K-token context + overhead) as the stated reason locally-deployed models must stay under 100B. Purifier strips redundant commands from successful trajectories (Sec 3.3.1). "Code and Data Availability" section: anonymized review repository at anonymous.4open.science/r/aoi-C8C7; "The repository will be made public upon paper acceptance." ↩↩↩↩

  2. Section 3 (AOI Runtime Architecture). Table 1: agent roles, permissions, and the Mraw/Mtask/Mcomp access matrix (Observer: Mraw "-", Mtask "R/W", Mcomp "R"; Probe: Mraw "W", Mtask "R", Mcomp "-"; Executor: Mraw "W", Mtask "R", Mcomp "-"; Compressor: Mraw "R", Mtask "-", Mcomp "W"). Section 3.1: three key invariants (Observer cannot read raw outputs; Probe/Executor write raw but cannot read it, all information flows through Compressor; Compressor is stateless per iteration). Section 3.2.1: four-stage pipeline (Decision, Interaction, Compression, Caching). Section 3.2.2: dual-timescale memory, long-term H_{n-2} = {S1,...,S_{n-2}}, short-term C_{n-1} = Compress(RawOutputs_{n-1}). Algorithm 1 (Appendix A.1): main loop with explicit budget-exhaustion return (E.Submit(timeout)) after N iterations; lazy compression (line 17, after the decision, not before) so the Observer sees fresh evidence when deciding but only compressed summaries persist. ↩↩↩↩↩

  3. Section 3.3 (Observer Step-Level Policy Optimization) and Appendix A.2 (Algorithm 2). Eq. 1: group-normalized advantage A_i = (R(x,y_i) - mu_G)/(sigma_G + eps), mu_G = (1/G) sum_j R(x,y_j). Eq. 2: policy gradient over sampled group. Eq. 3 and Section 3.3.2: six reward dimensions and default weights, Format (w=0.10, rule-based, JSON validity; parse failure triggers hard penalty R=0.09), Summary (w=0.15), Action (w=0.10), Context Instruction (w=0.30), Context Namespace (w=0.30), Confidence (w=0.05); Context Instruction + Context Namespace stated as "60% of the total weight." ↩↩

  4. Section 4 (Trajectory Evolver). Eq. 4: tau* = pi_evolve(tau, problem); repair for failed seeds, augmentation for success seeds. Section 4.2: seeds defined and categorized (success/failed); Section 4.2: "we use Claude Sonnet 4.5 trajectories on AIOpsLab as a proxy for such expert records... The Evolver's design is agnostic to seed provenance." Eq. 5: same group-normalized advantage form applied to Evolver corrections, scored on Validity/Completeness/Correctness/Effectiveness. Section 4.4 and Appendix A.3/Figure 5: three-stage integration (Failure Collection, Correction Generation, Guidance Injection); corrected plans delivered to the Observer as a "[Corrected Diagnostic Plan]" structured prompt, explicitly guidance not a rigid constraint. ↩↩

  5. Section 5.1 (Experimental Setup). Section 5.1.1 and Appendix B.3/Figure 6: nested data split D_obs_train (23 tasks, 11 fault types) subset of D_evolver_train (49 successful Sonnet trajectories) subset of D_all (86 tasks); D_obs_test (63 held-out tasks, 15 unseen fault types plus 15 training-fault-type tasks Sonnet failed); D_evolver_test (37 failed Sonnet trajectories, a strict subset of D_obs_test, so all 37 Evolver test cases fall within the Observer's 63 held-out tasks). Section 5.1.2: best@k (any of k runs succeeds) and avg@k (mean success rate across k runs) defined. Section 5.1.3: Qwen3-14B base, LoRA rank 64/alpha 128/lr 1e-5, GRPO group size G=4, 2xA100 GPUs, vLLM inference. Table 5 (Appendix B.1): max iterations 15, max Probe rounds/iteration 5, context budget 4096 tokens/iteration, long-term memory capacity 10 summaries, Executor whitelist 47 command patterns; GRPO batch size 16, 3 epochs, reward model Claude Opus 4.5, 49 training / 37 test samples for the Evolver. Appendix B.2: AIOpsLab is 88 scenarios reduced to 86 due to 2 deprecated scenarios. ↩↩↩↩↩

  6. Table 2 (AOI runtime comparison, full 86-task benchmark, %) and Section 5.2 prose. GPT-4o-mini+AOL-agent: Detection 25.0, Localization 9.5, RCA 7.7, Mitigation 7.7, Overall 14.7. GPT-4o-mini+STRATUS: 78.1/25.0/15.4/23.1/43.0. GPT-4o-mini+AOI: 90.6/32.1/38.5/53.8/58.1 ("Architecture yields 4x improvement over vanilla agents," 58.1/14.7 = 3.95x). Claude Sonnet 4.5+AOL-agent (single-run): 68.8/53.6/15.4/76.9/57.0. Qwen3-14B (best@5/avg@5)+STRATUS: 75.0/41.3, 32.1/11.4, 7.7/4.6, 15.4/15.4, 41.9/22.1. Qwen3-14B (best@5/avg@5)+AOI: 100/66.9, 53.6/27.9, 30.8/7.7, 46.2/23.1, 66.3/38.6. Prose: Detection "100% best@5 vs STRATUS's 75%" (Qwen3-14B best@5 row); RCA "largest relative gains (+150% over STRATUS)" -- this figure arithmetically matches the GPT-4o-mini row's RCA change (15.4->38.5, +150.6%), not the Qwen3-14B best@5 RCA row cited alongside it in the same paragraph (7.7->30.8, +300%), even though the surrounding sentences (Detection, Mitigation) are discussing Qwen3-14B rows -- an inconsistency in the paper's own prose, not introduced by this page; Mitigation "improves 3x over STRATUS" (46.2 vs 15.4, Qwen3-14B best@5); "Qwen3-14B + AOI (66.3% best@5) outperforms Claude Sonnet 4.5 + AOL-agent (57.0%), solving 57 vs 49 tasks" (a 9.3-point gap; distinct from the abstract's "24.4 percentage points" figure, which is 66.3% best@5 minus STRATUS's Qwen3-14B best@5 Overall score of 41.9%, not minus Sonnet's 57.0%). ↩↩↩↩

  7. Table 3 (Observer GRPO on held-out fault types, 63 tasks, %) and Section 5.3 prose. Sonnet 4.5 (AOL-agent): Det 54.5, Loc 40.9, RCA 8.3, Mit 57.1, Overall 41.3. AOI (Untrained): 65.5/22.7/6.7/14.3/33.7. AOI (Observer-GRPO): 90.9/18.2/16.7/14.3/42.9. Abstract and Section 1: "lifting avg@1 from 33.7% to 42.9%, surpassing Claude Sonnet 4.5 (41.3%) without multi-run sampling" -- this is the paper's own repeated label for Table 3's numbers. Section 5.3.1: Untrained-to-trained delta (Figure 10) is Detection +36 points (65.5->90.9, prose rounds to "+36.4pp" when compared against Sonnet: 54.5->90.9), Localization -4.5 (22.7->18.2, this is the untrained-vs-trained AOI comparison, not vs Sonnet); "the trained Observer prioritizes high-confidence fault indicators over exhaustive exploration"; Appendix E, ~9 more exploration steps for GRPO-trained vs base. Note: Figure 10's own caption labels these same Detection/Localization deltas "(best@5)," contradicting the abstract's and Section 5.3's "avg@1" label for the identical Table 3 numbers -- a source inconsistency (caption vs. prose) we flag rather than silently resolve; this page follows the abstract/prose "avg@1" label since it is stated more prominently and repeatedly. Sonnet-vs-trained comparison: "improvement concentrates in Detection (+36.4 points) and RCA (+8.4 points)" (54.5->90.9 and 8.3->16.7). Mitigation unchanged 14.3->14.3 because "remediation commands are generated and executed by the Executor, so Observer optimization cannot directly improve the quality of the final repair actions." ↩↩↩↩

  8. Table 4 (component ablation on D_evolver_test, 37 tasks, best/avg %; first three rows 5 runs, last row 4 runs) and Sections 5.3.2/5.4. Base: Det 100/50.0, Loc 54/26.2, RCA 27/7.3, Mit 0/0, Overall 54/24.9. Evolver-prompts: 90/64.0, 54/24.6, 18/12.7, 0/0, 49/29.7. Observer-GRPO: 90/64.0, 38/21.5, 36/29.1, 0/0, 49/33.5. Observer-GRPO+Evolver-prompts: 100/72.5, 31/19.2, 36/25.0, 0/0, 49/33.8 ("+8.9 point improvement over Base," 24.9->33.8). Section 5.4.1: Evolver-as-Prompt end-to-end validation (Table 4, avg@5 +4.8%, i.e. Base 24.9 -> Evolver-prompts 29.7) and LLM-judge repair-quality scoring (Claude Opus 4.5) across Validity/Completeness/Correctness/Effectiveness (Figures 3-4). Section 5.4.4: variance analysis, Base best@5-avg@5 gap "29.2pp (54.1%-24.9%)", Evolver-prompts gap "18.9pp (48.6%-29.7%)"; Detection gap 100/50.0 -> 90/64.0 (50pp to 26pp). Section 6 (Discussion) and Section 8 (Conclusion): "GRPO-trained Evolver ... mean LLM judge score 7.18->8.27, std 0.97->0.49"; variance reduction stated as 35% ((29.2-18.9)/29.2 = 35.3%). ↩↩

  9. Appendix B.2 (fault types in the 37 failed cases: service failures 31%, misconfigurations 26%, authentication errors 18%, pod failures 15%, network issues 10%; failed-case counts by task type: Detection 10, Localization 13, Mitigation 3, RCA 11). Appendix B.3, Tables 6-7: 11 training fault types / 23 tasks (Table 6); 15 test-only fault types / 48 tasks plus 15 training-fault-type tasks Sonnet failed = 63 held-out (Table 7). Appendix C.1/Figure 7: best@k curve, best@1=31.4%, best@2=51.2% (+19.8pp), best@3=58.1%, best@4=62.8%, best@5=66.3%; k=2 recommended for cost-sensitive deployments, k=3 "captures 88% of the maximum achievable improvement" for high-stakes ones. Appendix C.2, Table 8: per-round success by task type (R1-R5), Overall avg 38.6%; Mitigation flat at 23.1% every round ("failures are capability-limited rather than stochastic"). Appendix C.3, Table 9: task stability distribution over 86 tasks, 5/5 consistently solved 14 (16.3%), 3-4/5 mostly solved 16 (18.6%), 1-2/5 occasionally solved 27 (31.4%), 0/5 never solved 29 (33.7%). Appendix C.4: the 29 never-solved tasks cluster as Localization 13 (astronomy_shop dependency chains), RCA 9 (temporal fault-propagation reasoning), Mitigation 7 (domain-specific remediation, e.g. MongoDB auth recovery, Helm chart upgrades). Appendix D, Figure 11: task-level changes after GRPO training, net +9 tasks (11 improved, 2 degraded, rest unchanged); Appendix D.4: both degraded tasks are Localization (pod_failure_hotel_res-localization-1 and product_catalog_service_failure-localization-1, each 4/5 -> 0/5), root-caused to over-exploration surfacing multiple similar-symptom candidates. Appendix E, Table 10: Observer GRPO training-set composition by task type (Detection 10/43.5%, Localization 6/26.1%, Analysis 1/4.3%, Mitigation 6/26.1%, of 23 total; test set 22/22/12/7 of 63). Table 11: average exploration steps, Observer-GRPO 10.9 vs Base 1.9 (+9.0 average across task types). ↩↩↩↩↩↩↩

  10. Section 6 (Discussion): "Safety mechanisms improve capability... Read-write separation forces evidence accumulation before mutation, preventing the cascading failures we observed in STRATUS where premature remediation attempts corrupted system state"; "Capability boundaries are task-specific," 29/86 tasks fail consistently across five Qwen rounds, GPT-4o-mini, and GRPO variants, attributed to systematic gaps (e.g. MongoDB auth recovery needing Helm-specific knowledge) rather than random failure. Section 7 (Limitations): the Evolver "currently generates corrected command sequences as structured prompts"; extending it to "produce synthetic system feedback via environment simulators or to serve as a runtime agent for dynamic plan refinement" is explicitly future work, "beyond the scope of this work." "On the applied side, we plan to deploy AOI in production SRE environments to validate the use and productization potential of the framework within real-world incident response workflows" (i.e., not yet validated in production as of this paper). ↩↩↩↩