Markdown

When to compact¶

Scope: the decision of when an agent should compress its own history, treated as a policy rather than a threshold, and the separate question of what a compressor should be told to keep. This page covers why a token-count trigger is uncorrelated with the thing you care about, the rubric-gated probe and the KV-cache mechanics that make it affordable, how to optimise the compression instruction without touching weights, the measured headroom between a fixed schedule and perfect timing, and the cost and latency this actually buys. Training the trigger instead of scaffolding it is compaction-aware RL; making the compression reversible is agentic context management; the surrounding concepts are in context and memory.

Three NumPy-free stdlib blocks below are executed and asserted in this page (Python 3.11). They rebuild a payback model from SelfCompact's own cost formula and price table (arXiv 2606.23525), recompute its answer-transition and oracle tables, and audit ACON's published results (arXiv 2510.00615) including one sentence that pairs numbers from two different rows. The benchmark numbers are the papers'; the payback model, the standard-error reasoning and the audit conclusions are ours. No agent was run.

What it is¶

Every deployed agent harness compacts. The question is only who decides when.

Trigger	Fires on	Knows whether a sub-task just finished	Training needed
Fixed threshold (the default in shipped products)	context length crossing a percentage of the window	no	none
Keep-last-N / delete-all	same threshold, cruder retention	no	none
Tool exposed to the model	whenever the model emits the call	in principle	none, but base models rarely call it
Rubric-gated probe (SelfCompact)	a periodic yes/no verdict against structural conditions	yes, that is what the rubric asks	none
Learned trigger (AutoCompact, ACM)	the policy's own tool call, trained	yes	SFT and RL, plus an annotation pipeline

A fixed threshold fires on how much context exists, which is uncorrelated with whether the agent is in the middle of a derivation, halfway through verifying a hypothesis, or has just cleanly resolved a sub-question. SelfCompact's contribution is to make that a two-part scaffold: a summarisation tool the model can invoke, plus a short rubric that is appended at periodic probe points and asks the model to return COMPRESS or CONTINUE against conditions requiring verbatim evidence quoted from the trajectory. Neither half works alone. The tool alone fires unevenly across models (some reflexively, some never); the rubric alone cannot act.

ACON attacks the orthogonal half. It leaves the trigger as a length threshold and instead optimises the compression guideline, the natural-language instruction the compressor receives, by running the agent with and without compression, collecting the tasks where it succeeds uncompressed and fails compressed, and asking an optimiser model to write feedback explaining what the compression destroyed. It then optionally distils that optimised compressor into a smaller model. Both stages are prompt-space work with no gradient on the agent, which is what makes it applicable to an API-only agent.

Why it matters¶

A badly timed compaction is not neutral, it is destructive. Tracking answer transitions across 12 fixed-interval summarisation calls, SelfCompact finds 1,486 wrong-to-correct and 1,009 correct-to-wrong. 40.4% of all transitions destroy a correct answer. The net is positive, but the churn is enormous relative to the net.
The timing decision is worth several times the mechanism. Block 2 derives this from the paper's own tables: summarising at all buys +2.5 points over no compaction, and an oracle that follows the same schedule but skips the call whenever the current answer is already correct adds +11.5 more. Timing is worth 4.6x the mechanism.
A good trigger is cheaper than no compaction, not more expensive. Every token generated after a compaction attends to a short summary. SelfCompact measures per-question cost dropping 67%, 63% and 33% on BrowseComp-Plus across three models, despite adding an extra LLM call at every probe.
The compression instruction is a tunable with real range. ACON's optimised guideline is worth +7.7 points on AppWorld over a naive compression instruction, without touching a weight.
It generalises past the context limit. AutoCompact reports gains at a 256k window where the threshold never fires at all, which means the benefit is not only about fitting; stale exploration in context degrades reasoning on its own.

When to use it (and when not)¶

Use a rubric-gated trigger when your agent runs on an open-weight model in a tool loop, your provider or engine gives you prefix KV caching, and episodes are long enough that several compactions fire.

Use guideline optimisation when your agent is behind an API you cannot fine-tune, or when you want the compressor to be a small cheap model and need it to punch above its size.

Do not use either when:

The remaining horizon is short. Block 1 shows the payback is measured in remaining LLM calls, not in a ratio. Compacting on the second-to-last call is pure overhead.
You are latency-sensitive and not token-sensitive. ACON's own Table 4 measures history compression at 13.9% lower API cost and 19.7% higher median wall-clock per task; observation compression at 17.8% lower cost and 39.2% higher latency. Compression trades latency for tokens, in that direction, every time.
You have not tried the trivial baselines. On MiMo-V2-Flash and BrowseComp-Plus, keeping the last 3 turns and discarding the rest scores 63.5 at $0.15 per question against SelfCompact's 62.9 at $0.16. A one-line retention rule beat the rubric on both axes in that cell.
Your engine re-prefills on every call. The whole cost argument depends on appending to a cached prefix rather than re-encoding it. Without prefix caching the probe costs a full re-prefill and the economics invert. See prompt caching.

Architecture¶

flowchart TB
  subgraph LOOP["Generation loop, one inference engine"]
    Y["assistant turn y_t"]
    CHK{"probe point?<br/>t mod N == 0"}
    PR["append rubric prompt P_R<br/>KV prefix of y_1..t reused"]
    RV{"rubric verdict"}
    PS["append summariser prompt P_S<br/>KV prefix still reused"]
    SUM["generate summary y~"]
    RESET["hard reset: context = x + y~<br/>one-time prefill of the new prefix"]
    Y --> CHK
    CHK -->|"no"| Y
    CHK -->|"yes"| PR
    PR --> RV
    RV -->|"CONTINUE: pop the probe,<br/>trajectory left unchanged"| Y
    RV -->|"COMPRESS"| PS
    PS --> SUM
    SUM --> RESET
    RESET --> Y
  end
  GUIDE["ACON: optimise what P_S asks for<br/>from success-vs-failure contrast"] -.->|"offline, no weight update"| PS
  DIST["ACON: distil the compressor<br/>into a small model"] -.-> SUM

How to use it¶

Block 1: when a compaction pays for itself¶

SelfCompact states that "compaction wins iff L/l > 10" and credits its Appendix C with working it out. Appendix C gives the per-question cost formula, the calculation procedure and the OpenRouter price table, and no such derivation. Rebuilding the break-even from that formula gives something more useful than a ratio: a payback horizon in remaining LLM calls.

print("### BLOCK B1 ###")
# ---- SelfCompact's own single-rate cost model (arXiv 2606.23525, App. C) -----
# COST(q) = (p_cache * N_prompt + p_out * N_out) / 1e6, every prompt token billed
# at the cache rate. Prices in USD per 1M tokens, from the paper's Table 8.
PRICES = {                       # model: (p_in, p_cache, p_out)
    "GLM-4.7-Flash": (0.07, 0.01, 0.40),
    "MiMo-V2-Flash": (0.10, 0.01, 0.30),
    "MiniMax-M2.5": (0.30, 0.03, 1.20),
}


def payback_calls(model, L, ell, verdict_tokens=60):
    """Remaining LLM calls after which one compaction has repaid itself.

    Cost of NOT compacting over R further calls : R * p_cache * L
    Cost of compacting                          : probe (p_cache*L + p_out*v)
                                                + summariser (p_cache*L + p_out*ell)
                                                + R * p_cache * ell
    """
    _, p_cache, p_out = PRICES[model]
    fixed = 2 * p_cache * L + p_out * (verdict_tokens + ell)
    per_call_saving = p_cache * (L - ell)
    assert per_call_saving > 0, "a summary longer than the trajectory never repays"
    return fixed / per_call_saving


# The paper asserts "compaction wins iff L/ell > 10" and credits Appendix C with
# working it out. Appendix C gives the cost formula and the price table and no
# such derivation. Rebuilding it from the formula gives no horizon-free
# threshold: the ratio sets the payback horizon, not a win/lose line.
for model in PRICES:
    r10 = payback_calls(model, 20_000, 2_000)      # L/ell = 10, the claimed line
    r20 = payback_calls(model, 40_000, 2_000)      # L/ell = 20, the paper's low end
    r80 = payback_calls(model, 80_000, 1_000)      # L/ell = 80, the paper's high end
    print(f"  {model:15s} payback calls: L/l=10 {r10:5.1f} | L/l=20 {r20:4.1f} | L/l=80 {r80:4.1f}")
    assert 5.5 < r10 < 7.0 and 3.5 < r20 < 4.5 and r80 < 3.0

# Even at a ratio far below the claimed threshold, compaction still repays; it
# just needs a longer remaining horizon. There is no ratio at which it never does.
r2 = payback_calls("GLM-4.7-Flash", 4_000, 2_000)
assert 45 < r2 < 46
print(f"  at L/l=2 it still repays, after {r2:.0f} further calls")

# ---- does the measured cost table agree? (Table 4, BrowseComp-Plus) ---------
bcp_cost = {"GLM-4.7-Flash": (0.12, 0.04), "MiniMax-M2.5": (0.19, 0.07),
            "MiMo-V2-Flash": (0.24, 0.16)}       # (no compaction, SelfCompact)
drops = {m: round(100 * (a - b) / a) for m, (a, b) in bcp_cost.items()}
assert drops == {"GLM-4.7-Flash": 67, "MiniMax-M2.5": 63, "MiMo-V2-Flash": 33}
# One cell where a trivial baseline wins on BOTH axes: Keep-last-N on MiMo.
assert 63.5 > 62.9 and 0.15 < 0.16          # accuracy and USD/question
print("  BrowseComp-Plus cost reduction:", drops)
print("  Keep-last-3-turns beats SelfCompact on MiMo BC-Plus: 63.5 vs 62.9 at $0.15 vs $0.16")

Output:

### BLOCK B1 ###
  GLM-4.7-Flash   payback calls: L/l=10   6.8 | L/l=20  4.3 | L/l=80  2.6
  MiMo-V2-Flash   payback calls: L/l=10   5.7 | L/l=20  3.7 | L/l=80  2.4
  MiniMax-M2.5    payback calls: L/l=10   6.8 | L/l=20  4.3 | L/l=80  2.6
  at L/l=2 it still repays, after 45 further calls
  BrowseComp-Plus cost reduction: {'GLM-4.7-Flash': 67, 'MiniMax-M2.5': 63, 'MiMo-V2-Flash': 33}
  Keep-last-3-turns beats SelfCompact on MiMo BC-Plus: 63.5 vs 62.9 at $0.15 vs $0.16

The compression ratio does not decide whether compaction wins; it decides how fast it pays off. At the paper's reported 20x to 80x shrinkage, one compaction repays after 2.4 to 4.3 further calls. At the claimed 10x break-even it takes 5.7 to 6.8. Even at 2x it still repays, after about 45 calls. The rule to implement is therefore not "compact when the ratio exceeds 10" but "do not compact when fewer than about five calls remain", which is a different and more actionable guard, and one no fixed-percentage trigger enforces.

Two mechanics carry the whole cost argument, and both are worth stating explicitly because they are easy to lose in an implementation:

Append, do not substitute. Both the rubric probe and the summariser instruction are appended as a fresh user-role message to the live context (x, y_1..t). The KV cache of the trajectory is preserved across the call, so the probe pays prefill only on its own instruction. Rebuilding the prompt from scratch to insert the instruction, which is the natural thing to write, throws the cache away and pays an O(L^2) re-encode on every probe.
The reset is where the saving lives. After a COMPRESS verdict the context becomes (x, y~) and every later token attends to the summary rather than the trajectory. The probe and the summary generation are rounding errors against that.

Block 2: what bad timing costs, and what perfect timing would buy¶

print("### BLOCK B2 ###")
# ---- what a badly timed trigger actually costs ------------------------------
# Table 2: answer transitions over 12 fixed-interval summarisation calls,
# Qwen3-4B-Instruct-2507 on IMO-AnswerBench.
wrong_to_correct, correct_to_wrong = 1486, 1009
total = wrong_to_correct + correct_to_wrong
assert total == 2495
assert round(100 * correct_to_wrong / total, 1) == 40.4
# Table 3: what those transitions net out to, and what perfect timing would buy.
no_compaction, fixed_interval, selfcompact, oracle = 38.9, 41.4, 45.5, 52.9
assert round(oracle - fixed_interval, 1) == 11.5 and round(oracle - no_compaction, 1) == 14.0
mechanism = fixed_interval - no_compaction      # what summarising at all buys
timing = oracle - fixed_interval                # what perfect timing adds on top
assert round(timing / mechanism, 1) == 4.6
print(f"  regressive transitions: {100 * correct_to_wrong / total:.1f}% of {total}")
print(f"  summarising at all buys {mechanism:+.1f} pts; perfect timing adds "
      f"{timing:+.1f} more ({timing / mechanism:.1f}x)")
# The realised rubric captures a third of that headroom.
assert round((selfcompact - fixed_interval) / timing, 3) == 0.357
print(f"  the rubric captures {(selfcompact - fixed_interval) / timing:.1%} of the headroom")

# ---- is the math comparison budget-matched? (Table 1) -----------------------
# rows: model -> (no-compaction avg, fixed-interval avg, selfcompact avg,
#                 kilo-token budgets for the same three)
math_table = {
    "Qwen3-4B-Instruct": (38.7, 41.5, 45.1, 16, 44, 48),
    "Qwen3-30B-A3B": (50.6, 54.9, 56.4, 16, 26, 29),
    "Qwen3.5-9B": (32.5, 40.1, 47.3, 16, 90, 93),
    "Qwen3.5-4B": (21.9, 30.7, 33.8, 16, 64, 67),
}
for name, (n0, nf, ns, b0, bf, bs) in math_table.items():
    print(f"  {name:20s} vs no-compaction {ns - n0:+5.1f} at {bs / b0:.1f}x tokens | "
          f"vs fixed-interval {ns - nf:+4.1f} at {bs / bf:.2f}x tokens")
    assert bs > bf                       # SelfCompact always gets the larger budget
# The paper matches fixed-interval to "within +/- 3k tokens"; one of four is 4k.
gaps = [bs - bf for (_, _, _, _, bf, bs) in math_table.values()]
assert gaps == [4, 3, 3, 3] and max(gaps) > 3
# The 18.1-point headline is a 5.8x token-budget comparison, not a compaction win.
imo_feb = {"none": 34.2, "fixed": 44.9, "self": 52.3}
assert round(imo_feb["self"] - imo_feb["none"], 1) == 18.1
assert round(imo_feb["self"] - imo_feb["fixed"], 1) == 7.4
assert round(93 / 16, 1) == 5.8
print(f"  HMMT Feb / Qwen3.5-9B: +18.1 vs a 16k baseline at 5.8x tokens, "
      f"+7.4 vs fixed-interval at 1.03x")

Output:

### BLOCK B2 ###
  regressive transitions: 40.4% of 2495
  summarising at all buys +2.5 pts; perfect timing adds +11.5 more (4.6x)
  the rubric captures 35.7% of the headroom
  Qwen3-4B-Instruct    vs no-compaction  +6.4 at 3.0x tokens | vs fixed-interval +3.6 at 1.09x tokens
  Qwen3-30B-A3B        vs no-compaction  +5.8 at 1.8x tokens | vs fixed-interval +1.5 at 1.12x tokens
  Qwen3.5-9B           vs no-compaction +14.8 at 5.8x tokens | vs fixed-interval +7.2 at 1.03x tokens
  Qwen3.5-4B           vs no-compaction +11.9 at 4.2x tokens | vs fixed-interval +3.1 at 1.05x tokens
  HMMT Feb / Qwen3.5-9B: +18.1 vs a 16k baseline at 5.8x tokens, +7.4 vs fixed-interval at 1.03x

The oracle number is the single most useful figure in this literature. It is not a method; it is an upper bound on what any trigger policy can be worth, obtained by cheating (skip the summarisation whenever the current answer is already correct) while keeping the schedule otherwise identical. It says the ceiling on trigger quality is 4.6x the value of the summarisation mechanism itself. SelfCompact's rubric realises 35.7% of that headroom without training, which leaves most of it on the table and is the honest case for the learned triggers in compaction-aware RL.

Read the math results against fixed-interval, not against no-compaction. The paper's abstract quotes improvement "over a no-summarization baseline by up to 18.1 points on math", and that is arithmetically correct. But the no-compaction row is capped at a 16k budget in every block while SelfCompact spends 93k on the cell that produces 18.1. Against fixed-interval summarisation at a comparable budget, the same cell is +7.4. The agentic-search numbers do not have this problem: there SelfCompact is both more accurate and cheaper than the baseline, which is the stronger result and the one to cite. The paper's own matching tolerance also slips once: it claims fixed-interval is matched to within 3k tokens, and the Qwen3-4B-Instruct block is 44k against 48k.

Block 3: what optimising the guideline is worth¶

print("### BLOCK B3 ###")
# ---- ACON (arXiv 2510.00615): what guideline optimisation is worth ----------
# Tables 1 and 2, history compression, gpt-4.1 agent and compressor.
appworld = {"none": (56.0, 9.93), "prompting": (43.5, 6.93),
            "acon_ut": (51.2, 7.17), "acon_utco": (56.5, 7.33)}
office = {"none": (76.84, 7.27), "prompting": (71.58, 4.40),
          "acon_ut": (74.74, 4.93), "acon_utco": (72.63, 4.54)}


def peak_cut(table, key):
    return 100 * (table["none"][1] - table[key][1]) / table["none"][1]


assert round(peak_cut(appworld, "acon_utco"), 1) == 26.2      # abstract's lower bound
assert round(100 * (10.35 - 4.71) / 10.35, 1) == 54.5         # abstract's upper bound
# The abstract compares task success to compression BASELINES, and that holds.
# Against the no-compression upper bound the same rows are much flatter.
assert round(appworld["acon_utco"][0] - appworld["prompting"][0], 1) == 13.0
assert round(100 * (appworld["acon_utco"][0] / appworld["prompting"][0] - 1)) == 30
assert round(appworld["acon_utco"][0] - appworld["none"][0], 1) == 0.5
assert round(office["acon_ut"][0] - office["none"][0], 2) == -2.10
print(f"  AppWorld: +13.0 pts (+30% relative) over naive prompting, "
      f"+0.5 over no compression, peak -{peak_cut(appworld, 'acon_utco'):.1f}%")
print(f"  OfficeBench: {office['acon_ut'][0] - office['none'][0]:+.2f} vs no compression, "
      f"peak -{peak_cut(office, 'acon_ut'):.1f}%")

# The 8-objective-QA sentence pairs a peak number with a dependency number from
# a different row. Only ACON UT *history* beats no-compression on both EM and F1.
qa = {                          # row: (EM, F1, peak, dependency)
    "none": (0.366, 0.488, 10.35, 3.32),
    "prompting_hist": (0.376, 0.478, 4.73, 1.66),
    "acon_ut_hist": (0.373, 0.494, 4.71, 1.57),
    "acon_ut_obs": (0.364, 0.475, 4.97, 1.28),
}
beats = [k for k, v in qa.items()
         if k != "none" and v[0] > qa["none"][0] and v[1] > qa["none"][1]]
assert beats == ["acon_ut_hist"]
cut = lambda k, i: 100 * (qa["none"][i] - qa[k][i]) / qa["none"][i]
assert round(cut("acon_ut_hist", 2), 1) == 54.5      # the claimed peak number
assert round(cut("acon_ut_hist", 3), 1) == 52.7      # NOT the claimed 61.5
assert round(cut("acon_ut_obs", 3), 1) == 61.4       # this is where 61.5 lives
print(f"  8-obj QA: ACON UT history cuts peak {cut('acon_ut_hist', 2):.1f}% and "
      f"dependency {cut('acon_ut_hist', 3):.1f}%; the quoted 61.5% is the "
      f"observation row ({cut('acon_ut_obs', 3):.1f}%), which loses on EM")
# The naive prompting baseline still wins EM outright.
assert qa["prompting_hist"][0] > qa["acon_ut_hist"][0] > qa["none"][0]

# ---- where the gain in the optimiser actually comes from (Table 3) ---------
opt = {"o3 + contrastive": 51.2, "o3, no contrastive": 50.6,
       "gpt-4.1 + contrastive": 47.6, "gpt-5 + contrastive": 50.6}
total_gain = opt["o3 + contrastive"] - appworld["prompting"][0]
contrastive = opt["o3 + contrastive"] - opt["o3, no contrastive"]
optimiser_model = opt["o3 + contrastive"] - opt["gpt-4.1 + contrastive"]
assert round(total_gain, 1) == 7.7
assert (round(contrastive, 1), round(optimiser_model, 1)) == (0.6, 3.6)
assert round(100 * contrastive / total_gain, 1) == 7.8
print(f"  guideline optimisation is worth {total_gain:+.1f} over naive prompting; "
      f"the contrastive mechanism is {contrastive:+.1f} of it ({100 * contrastive / total_gain:.1f}%), "
      f"the optimiser model {optimiser_model:+.1f}")

# ---- the tokens-versus-latency trade (Table 4) ------------------------------
practical = {"No compression": (0.331, 73.24), "ACON history": (0.285, 87.68),
             "ACON observation": (0.272, 101.92)}
base_cost, base_lat = practical["No compression"]
for name, (c, lat) in practical.items():
    if name == "No compression":
        continue
    print(f"  {name:18s} cost {100 * (c / base_cost - 1):+5.1f}% "
          f"latency {100 * (lat / base_lat - 1):+5.1f}%")
assert round(100 * (0.285 / 0.331 - 1), 1) == -13.9
assert round(100 * (87.68 / 73.24 - 1), 1) == 19.7
assert round(100 * (101.92 / 73.24 - 1), 1) == 39.2

Output:

### BLOCK B3 ###
  AppWorld: +13.0 pts (+30% relative) over naive prompting, +0.5 over no compression, peak -26.2%
  OfficeBench: -2.10 vs no compression, peak -32.2%
  8-obj QA: ACON UT history cuts peak 54.5% and dependency 52.7%; the quoted 61.5% is the observation row (61.4%), which loses on EM
  guideline optimisation is worth +7.7 over naive prompting; the contrastive mechanism is +0.6 of it (7.8%), the optimiser model +3.6
  ACON history       cost -13.9% latency +19.7%
  ACON observation   cost -17.8% latency +39.2%

ACON's framing is careful and the numbers behind it are worth restating in absolute terms. The abstract claims a 26 to 54% peak-token reduction "while improving task success over existing compression baselines", and both halves check out: 26.2% on AppWorld, 54.5% on 8-objective QA, and the accuracy comparisons in Figure 1 are against the naive prompting baseline. Against the no-compression upper bound rather than against the baselines, the same rows read +0.5 points on AppWorld, -2.1 on OfficeBench and +0.007 EM on 8-objective QA. That is the right way to read it: ACON is a method for getting compression close to free, not for making it profitable.

One sentence pairs numbers from two different rows. The paper states that on 8-objective QA ACON "surpasses the no compression baseline in EM/F1 while reducing peak tokens and dependency by 54.5% and 61.5%, respectively". Only one row beats no-compression on both EM and F1, ACON UT under history compression, and its reductions are 54.5% and 52.7%. The 61.5% belongs to ACON UT under observation compression, whose EM (0.364) is below the no-compression baseline (0.366). Also worth noting in that table: the naive prompting baseline scores 0.376 EM, above both ACON rows and above no compression, at essentially the same peak tokens.

The named mechanism is not where the gain comes from. ACON's contribution is contrastive feedback: compare trajectories that succeed uncompressed with the same tasks failing compressed, and use the difference to rewrite the guideline. Its own Table 3 ablates that. Removing the contrast and optimising on failures alone costs 0.6 points; swapping the optimiser model from o3 to gpt-4.1 costs 3.6. Guideline optimisation as a whole is worth 7.7 points over a naive instruction, of which the contrastive mechanism accounts for 7.8%. The practical reading is that you should do prompt optimisation on your compressor with the strongest optimiser you can afford, and the specific contrastive recipe is a refinement.

Compression buys tokens and costs wall clock. ACON's Table 4 is the number most papers in this area omit: with a gpt-4.1 agent and a Qwen3-14B compressor on AppWorld, history compression cuts estimated API cost 13.9% and raises median per-task latency 19.7%; observation compression cuts cost 17.8% and raises latency 39.2%. The extra compressor call is on the critical path. If your SLO is time-to-answer rather than dollars, this is the wrong lever. See agent loop economics.

How to develop with it¶

Write the rubric as evidence-quoting conditions, not as a vibe. SelfCompact's probe enumerates a small set of conditions, each requiring the model to quote verbatim evidence from the trajectory. Fire when a sub-task has resolved or the trajectory is converging; suppress when the model is mid-derivation or stuck. The conditions are task-specific: competition math and agentic search get different rubrics.

Ablate the rubric before you trust the tool. SelfCompact's Table 5 removes the rubric and keeps the tool: BrowseComp-Plus accuracy goes 45.6 (no compaction), 50.0 (fixed interval), 51.9 (tool without rubric), 54.1 (tool with rubric). Without the rubric the scaffold barely beats a fixed schedule and loses to it on two of the three benchmarks in that table. Knowing when to compact is what the tool alone does not supply.

Instrument where your triggers actually fire. SelfCompact plots the context length at each firing against a 30%-of-window threshold, and its rubric-driven firings skew well to the left of it on all three models. If your learned or rubric-driven trigger fires at the same place a threshold would, it is not doing anything.

For guideline optimisation, run the two stages separately and keep both. ACON's utility step (UT) maximises task reward and its compression step (CO) then shortens the guideline. UT improves accuracy and reduces tokens on all three benchmarks; CO reduces tokens further and can cost accuracy. The paper's own recommendation is UTCO for verbose noisy environments and UT where precise fact retention matters, and its numbers support that: on AppWorld UTCO beats UT by 5.3 points, on 8-objective QA UT beats UTCO by 0.038 EM.

Distil the compressor, keep the agent large. ACON distils the optimised compressor into Qwen3-14B, Qwen3-8B and Phi-4 with LoRA and claims the students retain over 95% of the teacher. That claim rests on a bar chart with no accompanying table, and the paper's own limitations section concedes "a performance gap remains between our distilled models and their teacher models" and attributes it to using 100 examples per domain. Treat 95% as the target to verify on your own workload, not as a delivered result. The separate observation that gpt-4.1-mini works as a compressor with no distillation at all is the cheaper thing to try first.

How to maintain it¶

Version the rubric and the guideline together with the model id. Both are prompts tuned against one model's behaviour, and neither transfers silently.
Re-run the guideline optimisation when the environment changes. The contrastive set is built from tasks that succeed uncompressed and fail compressed in your environment. New tools or new observation formats invalidate it.
Keep the uncompressed control arm. Every claim on this page is a difference against a no-compression run. Without one in CI you cannot tell a compression regression from a model regression.
Alert on the firing-position distribution, not the firing count. A drift of firings toward the threshold means the rubric has stopped discriminating and you are paying for a probe that reproduces a percentage rule.
Track probe verdict balance. A rubric that returns COMPRESS on nearly every probe has degenerated into fixed-interval compaction with extra latency.

How to run it in production¶

Guard the tail of the episode. Add a hard rule that suppresses compaction when fewer than roughly five LLM calls are expected to remain, per Block 1. No published trigger does this and it is the cheapest win available.
Verify prefix caching is actually hitting. Appending the probe preserves the cache only if nothing upstream rewrites the prompt. A harness that re-serialises messages, re-orders tools, or injects a timestamp invalidates the prefix and silently converts a near-free probe into a full re-prefill. Measure cache-hit tokens per call, not just total tokens.
Budget latency, not just tokens. Every probe is a generation on the critical path and every compaction is a summary generation plus a fresh prefill of the compacted prefix. ACON's measured 19.7% to 39.2% latency increase is the number to plan against.
Keep a trivial baseline in the comparison forever. Keep-last-3-turns beat the rubric on both accuracy and cost in one of nine published cells. It costs nothing to keep it as a control arm and it will occasionally win.
Do not let the summariser and the agent drift apart. If you run a separate compressor model, pin its version alongside the agent's. The compressor swap experiment in compaction-aware RL shows a 6.5-point spread on SWE-bench Verified from changing only that component.

Failure modes¶

Compacting mid-derivation. The failure the whole rubric exists to prevent, and the one that produces 40.4% regressive transitions under a fixed schedule.
Compacting near the end of an episode. Pure overhead, and invisible to any percentage-of-window trigger.
Rebuilding the prompt to insert the probe. Destroys the KV prefix and turns a near-free judgement into an O(L^2) re-encode.
Shipping the tool without the rubric. Base models call it reflexively or not at all; SelfCompact's ablation shows this landing roughly where a fixed schedule does.
A rubric that always says yes. Fixed-interval compaction with an extra LLM call per interval.
Reading a compaction gain that is really a budget gain. The 18.1-point math headline compares a 93k-token run against a 16k one.
Optimising the guideline on the wrong contrast. Failures alone are worth 0.6 points; the paired success-and-failure contrast is the signal, and a weak optimiser model costs more than dropping the contrast entirely.
Assuming compression lowers latency. It lowers tokens and raises wall clock, measurably and in that direction.
Trusting a distillation claim sourced from a chart. ACON's over-95% retention has no table behind it and its own limitations section walks it back.

References¶

Self-Compacting Language Model Agents (SelfCompact, rubric-gated triggering): https://arxiv.org/abs/2606.23525
SelfCompact code: https://github.com/tianjianl/selfcompact
ACON: Optimizing Context Compression for Long-horizon LLM Agents: https://arxiv.org/abs/2510.00615
AutoCompact: Learning When to Compact Context in Long-Horizon Coding Agents (project page, no paper or code as of 2026-08-06): https://autocompact.github.io
LLMLingua (the extractive compression baseline in ACON's tables): https://arxiv.org/abs/2310.05736
MemGPT / Letta (paged external memory, the dialogue-oriented ancestor): https://arxiv.org/abs/2310.08560
AppWorld (ACON's primary benchmark): https://arxiv.org/abs/2407.18901
BrowseComp-Plus (SelfCompact's agentic-search benchmark): https://arxiv.org/abs/2508.06600
TextGrad (the natural-language optimisation ACON builds its guideline step on): https://arxiv.org/abs/2406.07496