The filesystem as agent memory¶
Scope: the medium deployed agents actually use for long-term memory, a directory tree of markdown files the agent reads, writes, and reorganizes through generic file tools, and what measurement says about it. This page covers the role decomposition that makes the setting analysable, the memory shapes worth comparing, what organizing the store does and does not pay for, why the tool set is as strong a lever as the model, and the store-health properties that decide whether any of it survives growth. The concept-level treatment of agent memory is in context and memory; the tool-level compression alternative is in agentic context management; the procedural half of the store is in agent skills.
Two primary sources. Zhou, Yu, Wei et al., "Filesystem-based Memory for LLM Agents: Organization, Evolution, and Sustainability" (arXiv 2607.26637, submitted 2026-07-29), the empirical study. Ning, Tieu, Fu et al., "Code as Agent Harness: Toward Executable, Verifiable, and Stateful Agent Systems" (arXiv 2605.18747v1, 2026-05-18), a survey supplying the framing.
What is and is not reproduced. No LLM was run and none of the study's benchmark numbers were reproduced; every figure attributed to the study is quoted. The Python block is a working memory filesystem built and queried on real files by this page: it implements the integration contract and the search agent, and derives its cost figures from bytes actually read off disk. Its numbers are properties of that model, not measurements of the paper's systems, and they are labelled as such throughout. It exists to make the mechanisms testable, and in two places it reaches a sharper conclusion than the paper states.
Before quoting any single ordering from the study, note its own stated noise: re-judging a fixed cell moved about two questions (1.3 points), implying that cell held roughly 154 questions, while column sizes elsewhere run from 32 to 158. One question is therefore worth 3.1 points on the smallest tier and 0.6 on the largest, so a four-point gap is decisive on one tier and a coin flip on another. The authors say the same: gaps of a few questions on small tiers are "suggestive rather than settled".
What it is¶
The survey supplies the frame. It argues that in agentic systems code has stopped being only an output and become "an operational substrate for agent reasoning, acting, environment modeling, and execution-based verification", and organises that claim into three layers: the harness interface (code for reasoning, for acting, and for modelling the environment), harness mechanisms (planning, memory and context engineering, tool use, a plan-execute-verify control loop, and harness optimization), and scaling the harness to multiple agents over shared code artifacts. Its stated goal for the field is "executable, verifiable, and stateful AI agent systems".
Filesystem memory is what the stateful pillar looks like in deployment, and it arrived by convergence rather than design. Coding agents already live on files, so files became the natural way to extend their context: a vendor memory tool exposing a directory behind six generic file operations, coding agents keeping written notes in an indexed memory folder, and agent context files and skills shipping as repository markdown. The medium earns its place by being inspectable, portable, operable with tools the agent already masters, and natively hierarchical, since folders form a taxonomy whose names are its labels.
The study formalises the setting as three roles around one memory filesystem:
| Role | Contract |
|---|---|
| Management agent | Integrates an incoming chunk into the store and organizes what is there, producing the next store state |
| Search agent | Answers a query against the store, with cited sources |
| Execution agent | Supplies task trajectories, which are distilled into skills |
One store serves both declarative content (facts, events, preferences, rules) and procedural content (reusable skills), which mirrors how deployed harnesses actually work rather than splitting them into separate subsystems.
Against that, three memory shapes are compared: an agent-organized hierarchy (the default under study), a verbatim dump (write everything down, organize nothing), and chunk retrieval (conventional embedding search over pieces).
The reason this needed a study is that the default rests on two untested assumptions: that an agent can keep a growing store organized as memories accumulate, conflict, and go stale, and that this organization pays.
Why the question matters¶
- The context window is not memory. It is ephemeral and, as the study notes, "degrades long before it is full", which is what pushed persistent external memory into being a first-order component of agent design rather than an optimization.
- The filesystem is already the default, so its properties are load-bearing whether or not anyone measured them. Research largely passed over the medium while building bespoke alternatives (paged context, extracted fact stores, temporal knowledge graphs, self-linking note networks, summary banks, embedding-organized trees), each behind its own interface.
- Organization has a real cost. Curation is model calls, and the study finds it "never gets cheaper per episode". If it does not pay, that spend is waste.
- The answer turns out to be conditional, and the conditions are things you control: the size of the material, which agent consumes the memory, and the tools in hand.
When to organize the store (and when not)¶
Organize when the material is large and dense. This is the one unambiguous win, and it is a cost win. Below the size where scanning gets expensive, a verbatim dump searches at near parity.
Do not organize when:
- You are hoping it improves answers. No agent in the study converts organization into better correctness. On one tier the curated store is the worst option by 40 points.
- Your management agent is weak. Taxonomy adherence erodes as stores grow for all but the strongest management agent measured, and a curation pass can silently condense content unless an explicit preservation rule is added.
- Your execution agent is strong and you are storing skills. The winner flips: a verbatim episode log serves a strong execution agent best, distilled guidance serves a weak one.
- You have not measured your own tiers. Column sizes in the study run from 32 to 158 questions, and re-judging moves about two questions. Most single orderings are suggestive rather than settled.
Architecture¶
flowchart TB
STREAM["Incoming stream<br/>(conversation turns, documents)"] --> MGMT
TRAJ["Execution agent<br/>task trajectories"] --> DISTIL["distil into skills"]
DISTIL --> MGMT["Management agent<br/>integrate + organize"]
MGMT --> STORE[("Memory filesystem<br/>directory tree of markdown<br/>declarative + procedural in ONE store")]
STORE --> MGMT
Q["Query"] --> SEARCH["Search agent<br/>answer with cited sources"]
STORE --> SEARCH
SEARCH --> ANS["Answer + citations"]
TOOLS["Tool set<br/>(sandboxed shell / memory-tool functions / search tooling)"] -.->|"shapes the store as strongly<br/>as the model does"| MGMT
TOOLS -.-> SEARCH
SHAPES["Alternatives:<br/>verbatim dump, chunk retrieval"] -.->|"compared against"| STOREThe dashed tool-set edge is the finding most people would not predict: it is a lever over memory organization, not a neutral wrapper.
How to use it: build the store, then measure it¶
The block below is a working memory filesystem, not an arithmetic check over the paper's tables. It implements the management agent's integration contract (route, dedupe, supersede), a search agent that navigates the taxonomy, and the two degenerate curation behaviours the paper documents. Files are written to and read from a real temporary directory, and every cost is derived from bytes actually read during a query.
"""A working memory filesystem with the paper's three roles, on real files.
Implements the management-agent integration contract (route, dedupe, supersede),
a search agent that navigates a taxonomy, and the degenerate curation behaviours
the paper documents. Every cost below is DERIVED from bytes actually read off
disk during a query, not copied from the paper's tables.
"""
import re
import shutil
import tempfile
from pathlib import Path
TAXONOMY = {"diet": "food", "music": "media", "film": "media", "commute": "travel"}
# ============================================================ the memory store
class MemoryFS:
"""A directory tree of markdown files, the medium deployed agents use."""
def __init__(self, root: Path):
self.root = root
self.root.mkdir(parents=True, exist_ok=True)
self.read_bytes = self.read_calls = 0
self.list_bytes = self.file_bytes = 0
def write(self, rel: str, front: dict, body: list[str]) -> None:
p = self.root / rel
p.parent.mkdir(parents=True, exist_ok=True)
fm = "\n".join(f"{k}: {v}" for k, v in front.items())
p.write_text(f"---\n{fm}\n---\n" + "\n".join(body) + "\n")
def read(self, rel: str) -> tuple[dict, list[str]]:
raw = (self.root / rel).read_text()
self.read_bytes += len(raw.encode()) # charged: the agent saw it
self.file_bytes += len(raw.encode())
self.read_calls += 1
_, fm, body = raw.split("---\n", 2)
front = dict(l.split(": ", 1) for l in fm.strip().split("\n") if ": " in l)
return front, [l for l in body.strip().split("\n") if l]
def list_dir(self, rel: str = "") -> list[str]:
p = self.root / rel if rel else self.root
names = sorted(c.name for c in p.iterdir())
self.read_bytes += len("\n".join(names).encode()) # listings cost too
self.list_bytes += len("\n".join(names).encode())
self.read_calls += 1
return names
def exists(self, rel: str) -> bool:
return (self.root / rel).exists()
def files(self) -> list[str]:
return sorted(str(p.relative_to(self.root)) for p in self.root.rglob("*.md"))
def total_bytes(self) -> int:
return sum((self.root / f).stat().st_size for f in self.files())
def reset_meter(self) -> None:
self.read_bytes = self.read_calls = 0
self.list_bytes = self.file_bytes = 0
# ============================================== management agent: three shapes
# The paper finds management agents build wildly different trees from the same
# stream: one sharded into 122 files, one consolidated into 2. The shape is the
# variable that decides whether organisation buys anything.
def route_sharded(subject: str, attribute: str) -> str:
"""Partition along BOTH dimensions a query names."""
return f"{TAXONOMY[attribute]}/{subject}.md"
def route_consolidated(subject: str, attribute: str) -> str:
"""One file per topic. Organised-looking, but it grows with the corpus."""
return f"{TAXONOMY[attribute]}/all.md"
class ManagementAgent:
def __init__(self, fs, router, dated_updates=True, drift=0.0):
self.fs, self.router = fs, router
self.dated_updates, self.drift = dated_updates, drift
self.seq = 0
def integrate(self, subject: str, attribute: str, value: str, turn: int) -> None:
rel = self.router(subject, attribute)
if self.drift and ((self.seq * 9973) % 100) / 100.0 < self.drift:
rel = "misc/" + rel.split("/", 1)[1] # taxonomy violation
self.seq += 1
front, body = (self.fs.read(rel) if self.fs.exists(rel) else ({}, []))
line = f"- {subject} {attribute}: {value} (turn {turn})"
if line in body:
return # exact duplicate
prior = [l for l in body if l.startswith(f"- {subject} {attribute}:")]
if prior and self.dated_updates:
body = [l + " [superseded]" if l in prior and "[superseded]" not in l
else l for l in body]
body.append(line)
self.fs.write(rel, {"topic": rel.split("/")[0]}, body)
def live_value(body, subject, attribute):
"""What the store currently asserts. Superseded lines are not live."""
hits = [l for l in body if l.startswith(f"- {subject} {attribute}:")
and "[superseded]" not in l]
if len(hits) != 1:
return None # zero or ambiguous: cannot answer
return re.search(r": ([^(]+) \(turn", hits[0]).group(1).strip()
def latest_value(lines, subject, attribute):
"""A verbatim dump keeps chronology, so a scanning agent resolves by recency.
An organised store cannot: reorganisation destroys chronological order, which
is exactly why it must carry explicit supersede state."""
hits = [l for l in lines if l.startswith(f"- {subject} {attribute}:")]
if not hits:
return None
newest = max(hits, key=lambda l: int(re.search(r"\(turn (\d+)\)", l).group(1)))
return re.search(r": ([^(]+) \(turn", newest).group(1).strip()
# ============================================================ the search agent
def search_navigate(fs, subject, attribute, router):
"""List the root, descend into the labelled folder, read the matching file."""
fs.reset_meter()
want = router(subject, attribute)
top = want.split("/")[0]
if top in fs.list_dir() and want.split("/")[1] in fs.list_dir(top):
_, body = fs.read(want)
v = live_value(body, subject, attribute)
if v:
return v, fs.read_bytes, fs.read_calls
for r in fs.list_dir(): # contract broken: sweep
for name in fs.list_dir(r):
_, body = fs.read(f"{r}/{name}")
v = live_value(body, subject, attribute)
if v:
return v, fs.read_bytes, fs.read_calls
return None, fs.read_bytes, fs.read_calls
def search_scan(dump, subject, attribute):
lines = [l for l in dump.split("\n") if l]
return latest_value(lines, subject, attribute), len(dump.encode()), 1
def build_dump(records):
return "\n".join(f"- {s} {a}: {v} (turn {t})" for s, a, v, t in records) + "\n"
def corpus(n_subjects, updates_per=3):
"""Each subject states a preference, then changes it. Only the last is true."""
recs, truth, turn = [], {}, 0
for i in range(n_subjects):
s = f"user{i:03d}"
for attribute in TAXONOMY:
for u in range(updates_per):
turn += 1
recs.append((s, attribute, f"{attribute}_v{u}", turn))
truth[(s, attribute)] = f"{attribute}_v{u}"
return recs, truth
def build(recs, path, router, **kw):
shutil.rmtree(path, ignore_errors=True)
fs = MemoryFS(path)
mgmt = ManagementAgent(fs, router, **kw)
for s, a, v, t in recs:
mgmt.integrate(s, a, v, t)
fs.reset_meter()
return fs
work = Path(tempfile.mkdtemp())
try:
def probes_of(n):
return [(f"user{i:03d}", a) for i in range(n) for a in TAXONOMY]
def measure(n, router, tag):
recs, truth = corpus(n)
fs = build(recs, work / f"{tag}{n}", router)
dump = build_dump(recs)
pr = probes_of(n)
nb = nc = sb = lb = fb = 0
for s, a in pr:
v, b, c = search_navigate(fs, s, a, router)
assert v == truth[(s, a)], (tag, n, s, a, v)
nb += b; nc += c; lb += fs.list_bytes; fb += fs.file_bytes
v, b, _ = search_scan(dump, s, a)
assert v == truth[(s, a)]
sb += b
q = len(pr)
return {"dump_kb": len(dump.encode()) / 1024, "nav": nb / q, "scan": sb / q,
"calls": nc / q, "saving": 1 - nb / sb, "files": len(fs.files()),
"listing": lb / q, "file": fb / q}
# ---- 1. correctness parity holds before any cost claim is meaningful ----
a8 = measure(8, route_sharded, "sh")
c8 = measure(8, route_consolidated, "co")
assert a8["files"] > c8["files"] # genuinely different shapes
# ---- 2. the payoff is a property of the SHAPE, not of "being organised" --
sizes = [4, 16, 64]
sharded = [measure(n, route_sharded, "sh") for n in sizes]
consol = [measure(n, route_consolidated, "co") for n in sizes]
# Sharded: bytes read per query stay nearly flat while the corpus grows 16x.
assert sharded[-1]["dump_kb"] / sharded[0]["dump_kb"] > 15
assert sharded[-1]["nav"] / sharded[0]["nav"] < 4.0 # sublinear, not flat
# ...so its saving climbs toward 1 as material grows.
assert [round(s["saving"], 3) for s in sharded] == sorted(
round(s["saving"], 3) for s in sharded)
assert sharded[0]["saving"] < 0.85 < sharded[-1]["saving"]
assert sharded[-1]["saving"] > 0.96
# ...but the residual growth is the DIRECTORY LISTING, not the file. Sharding
# converts file-read cost into fanout cost, and past some width the listing
# is the bill. This is the design constraint sharding actually imposes.
assert sharded[0]["file"] > sharded[0]["listing"] # small: file dominates
assert sharded[-1]["listing"] > sharded[-1]["file"] # large: listing dominates
assert sharded[-1]["listing"] / sharded[0]["listing"] > 5 # grows with fanout
assert sharded[-1]["file"] / sharded[0]["file"] < 1.2 # the file does not
# Consolidated: reads grow WITH the corpus, so the saving is a constant
# factor and does not improve. This is "organised" that buys nothing extra.
assert consol[-1]["nav"] / consol[0]["nav"] > 10
assert abs(consol[-1]["saving"] - consol[0]["saving"]) < 0.05
assert sharded[-1]["saving"] - consol[-1]["saving"] > 0.35
# Both are hierarchies of markdown. Only one is sublinear in the corpus.
assert consol[-1]["files"] == 3 and sharded[-1]["files"] == 64 * 3
# ---- 3. the effort trade: more calls, far fewer bytes -------------------
assert sharded[-1]["calls"] >= 3.0 # list, list, read
assert sharded[-1]["nav"] < sharded[-1]["scan"] / 15
# ---- 4. write defect: an undated update leaves the stale fact live ------
recs2, truth2 = corpus(4)
good = build(recs2, work / "dated", route_sharded, dated_updates=True)
bad = build(recs2, work / "undated", route_sharded, dated_updates=False)
pr2, total = probes_of(4), 4 * len(TAXONOMY)
good_ok = sum(search_navigate(good, s, a, route_sharded)[0] == truth2[(s, a)]
for s, a in pr2)
bad_ok = sum(search_navigate(bad, s, a, route_sharded)[0] == truth2[(s, a)]
for s, a in pr2)
assert good_ok == total and bad_ok == 0
# Nothing was lost: all three values are on disk. It is a WRITE defect that
# makes the store unreadable, not a storage failure.
_, body = bad.read("food/user000.md")
assert sum(1 for l in body if l.startswith("- user000 diet:")) == 3
assert latest_value(body, "user000", "diet") == truth2[("user000", "diet")]
# ---- 5. silent condensation during a reorganisation pass ---------------
def reorganize(fs, preserve):
out = MemoryFS(Path(str(fs.root) + f"_r{int(preserve)}"))
shutil.rmtree(out.root, ignore_errors=True)
out = MemoryFS(Path(str(fs.root) + f"_r{int(preserve)}"))
for rel in fs.files():
front, body = fs.read(rel)
out.write(rel, front, body if preserve else body[-1:])
out.reset_meter()
return out
kept, lost = reorganize(good, True), reorganize(good, False)
kept_ok = sum(search_navigate(kept, s, a, route_sharded)[0] == truth2[(s, a)]
for s, a in pr2)
lost_ok = sum(search_navigate(lost, s, a, route_sharded)[0] == truth2[(s, a)]
for s, a in pr2)
assert kept_ok == total
assert lost_ok < total and lost_ok / total == 0.75
# The condensed store is 71% SMALLER, which looks like successful tidying...
assert lost.total_bytes() < kept.total_bytes()
assert lost.total_bytes() / kept.total_bytes() < 0.30
# ...while reads get MORE expensive, because every lookup that now misses
# falls back to sweeping the tree. Size moves one way, cost the other, and
# neither points at the defect. Only answer quality does.
lb = sum(search_navigate(lost, s, a, route_sharded)[1] for s, a in pr2) / len(pr2)
kb = sum(search_navigate(kept, s, a, route_sharded)[1] for s, a in pr2) / len(pr2)
assert lb > kb
# ---- 6. taxonomy erosion costs economics, not answers -------------------
recs3, truth3 = corpus(16)
clean = build(recs3, work / "clean", route_sharded, drift=0.0)
eroded = build(recs3, work / "eroded", route_sharded, drift=0.5)
pr3 = probes_of(16)
def adherence(fs):
fl = fs.files()
return sum(1 for f in fl if not f.startswith("misc/")) / len(fl)
assert adherence(clean) == 1.0 and adherence(eroded) == 0.75
def verdicts(fs):
right = wrong = miss = 0
for s, a in pr3:
v = search_navigate(fs, s, a, route_sharded)[0]
if v == truth3[(s, a)]:
right += 1
elif v is None:
miss += 1
else:
wrong += 1
return right, wrong, miss
assert verdicts(clean) == (len(pr3), 0, 0)
right, wrong, miss = verdicts(eroded)
# Erosion does not make the store admit ignorance. It makes it CONFIDENTLY
# WRONG: the supersede marker is scoped to one file, so splitting a subject's
# history across folders leaves an unsuperseded stale line standing as live.
assert wrong > right and miss == 0
assert (right, wrong, miss) == (31, 33, 0)
# And retrieval gets CHEAPER as this happens, because the split files are
# smaller. Cost monitoring reports an improvement while answers rot.
cb = sum(search_navigate(clean, s, a, route_sharded)[1] for s, a in pr3) / len(pr3)
eb = sum(search_navigate(eroded, s, a, route_sharded)[1] for s, a in pr3) / len(pr3)
assert eb < cb
# So adherence is the leading indicator; neither cost nor miss rate is.
print("all filesystem-memory mechanism assertions passed")
print(" shape corpus files bytes/query saving")
for n, s, c in zip(sizes, sharded, consol):
print(f" sharded {s['dump_kb']:6.1f}KB {s['files']:6d} "
f"{s['nav']:11.0f} {s['saving']:8.1%}")
print(f" consolidated {c['dump_kb']:6.1f}KB {c['files']:6d} "
f"{c['nav']:11.0f} {c['saving']:8.1%}")
print(f" undated updates answerable: {bad_ok}/{total} (dated: {good_ok}/{total})")
print(f" silent condensation answerable: {lost_ok}/{total}, "
f"store shrank to {lost.total_bytes()/kept.total_bytes():.0%}, "
f"bytes/query rose {lb/kb-1:.0%}")
print(f" taxonomy adherence {adherence(eroded):.0%}: "
f"right/WRONG/miss {right}/{wrong}/{miss}, "
f"bytes/query {cb:.0f} -> {eb:.0f} (cheaper while wrong)")
finally:
shutil.rmtree(work, ignore_errors=True)
Executed output:
all filesystem-memory mechanism assertions passed
shape corpus files bytes/query saving
sharded 1.7KB 12 280 83.7%
consolidated 1.7KB 3 838 51.2%
sharded 6.8KB 48 414 94.1%
consolidated 6.8KB 3 3268 53.3%
sharded 27.6KB 192 944 96.7%
consolidated 27.6KB 3 13060 53.9%
undated updates answerable: 0/16 (dated: 16/16)
silent condensation answerable: 12/16, store shrank to 29%, bytes/query rose 16%
taxonomy adherence 75%: right/WRONG/miss 31/33/0, bytes/query 414 -> 310 (cheaper while wrong)
"Organised" is not one thing, and only one version is sublinear¶
The paper reports that organisation cuts retrieval cost and that the payoff tracks material size, and separately that management agents build wildly different trees from the same stream: one sharded into 122 files, another consolidated into two files with 210 heading sections. The model shows those two observations are the same observation.
Both routers below produce a tidy topic hierarchy of markdown. They behave completely differently as the corpus grows:
| corpus 1.7 KB | 6.8 KB | 27.6 KB | |
|---|---|---|---|
| sharded bytes/query | 280 | 414 | 944 |
| consolidated bytes/query | 838 | 3,268 | 13,060 |
| sharded saving vs dump | 83.7% | 94.1% | 96.7% |
| consolidated saving vs dump | 51.2% | 53.3% | 53.9% |
The corpus grows 16x. Sharded reads grow 3.4x; consolidated reads grow 15.6x, tracking the corpus almost exactly. So the sharded store's advantage compounds with scale while the consolidated store's stays pinned at a constant factor.
The rule that falls out is sharper than "organise your memory": the hierarchy has to partition along the dimension the query names. Queries here name a subject and an attribute, so a store keyed by both is sublinear and a store keyed only by topic is not. A management agent that produces a beautiful two-file taxonomy has organised nothing that retrieval can use. This is a testable property of a store, and it is worth testing, because both shapes look organised in a directory listing.
Sharding converts file cost into fanout cost¶
The residual 3.4x growth in the sharded store is not the file. The block meters directory listings separately from file reads, and they cross over: at the small corpus the file dominates, at the large one the listing does, growing more than 5x while the file barely moves. Navigating a wide directory means reading its index, and past some width that index is the bill.
That gives a second design rule the paper does not state: sharding is not free, it trades read volume for fanout. If a folder is going to hold thousands of entries, the store needs a real index file or a second level of nesting, or the listing quietly becomes the cost you were trying to avoid.
The write contract is what makes the store readable¶
The paper reports a management agent that "inconsistently records changed preferences as dated updates, leaving stale facts standing as live traits". The block reproduces that failure and shows how total it is.
Turn off supersede marking and keep everything else identical, and answerability goes from 16/16 to 0/16. Nothing was lost: all three versions of every preference are still on disk, and the block asserts it by reading the file back and recovering the correct value by recency. The store is complete and unreadable at the same time, because a reader looking at an organised store has no way to tell which of three unmarked lines is current.
This is the sharpest argument for the supersede contract, and it explains an asymmetry that is easy to miss. A verbatim dump preserves chronology, so a scanning agent resolves a changed preference by recency and needs no supersede state at all. An organised store destroys chronology the moment it groups by subject, so it must carry explicit supersede state or it cannot answer. Organising the store creates the obligation. Skip it and organisation is strictly worse than the dump it replaced.
Two defects, and neither is visible in the metrics you would watch¶
The block runs the paper's other documented failure, a reorganisation pass that condenses content without a preservation rule, and then measures taxonomy erosion. Both are silent in exactly the wrong way.
| Defect | Answers | Store size | Bytes/query |
|---|---|---|---|
| Silent condensation | 12/16 | 29% of original | +16% |
| Taxonomy erosion (75% adherence) | 31 right, 33 wrong, 0 missing | unchanged | -25% |
Condensation makes the store 71% smaller, which reads as successful tidying, while reads get more expensive because every lookup that now misses falls back to sweeping the tree.
Erosion is worse. Splitting a subject's history between misc/ and its correct folder does not make the store admit ignorance: it produces 33 confidently wrong answers and zero misses, and retrieval gets 25% cheaper because the split files are smaller. A dashboard watching cost and error rate sees an improvement and no errors.
The mechanism behind that is worth stating on its own, because it is a property of the medium rather than of any model: the supersede marker is scoped to a single file. Any routing inconsistency splits a subject's history across two files, and the stale line in one of them was never marked, so it stands as live. Two fixes follow directly. Make routing deterministic and immutable per (subject, attribute) so a history can never split, or lift supersede state into an index that is not per-file. The paper observes this failure as a model-capability problem; the model here says it is a scoping problem that a stronger model only partly hides.
The practical consequence for monitoring: taxonomy adherence is the leading indicator, and neither retrieval cost nor miss rate is. Both defects move cost in a direction that looks fine, and erosion never produces a miss to alert on.
How to develop with it¶
Four rules below are derived from the model above rather than quoted, and each is testable against a store you already have.
- Test that your hierarchy partitions along the query dimension. Take a real query, and check whether the store lets you reach the answer without reading content that belongs to other subjects. A topic-only taxonomy fails this and still looks organised. Sharded and consolidated stores in the model differ by 43 points of saving at the same corpus size.
- Cap directory fanout, or add an index. Sharding moves cost from file reads to directory listings, and at the largest corpus modelled the listing is already the dominant term. A folder heading for thousands of entries needs a second level or a real index file.
- Make routing deterministic and immutable per key. The supersede marker is file-scoped, so any inconsistency that splits one subject's history across two files silently breaks it. If routing can drift, lift supersede state into an index instead.
-
Never let a reorganisation pass drop lines. Condensation is the documented degenerate behaviour, and in the model it costs a quarter of answers while making the store look tidier and cheaper.
-
Give the management agent an explicit preservation rule. The documented degenerate behaviour is silent condensation during reorganization. The instruction has to say that reorganizing may move and rename but may not drop content.
- Make updates dated rather than destructive. The stale-facts failure came from changed preferences being recorded inconsistently, leaving superseded traits standing as current. Superseding an entry is a different operation from editing it.
- Write the taxonomy contract down and check adherence as a metric. Adherence erodes as stores grow for all but the strongest management agent, so it needs monitoring rather than trust.
- Keep declarative and procedural content in one store. That is the deployed shape the study formalises, and it is what lets a task trajectory become a retrievable skill without a second subsystem.
- Pick the skill representation from the consumer, not from taste. A verbatim episode log serves a strong execution agent best; distilled guidance serves a weak one. The winner flips.
How to run it in production¶
- Budget the reader, not the writer. A 14x cost span sits on the search-agent tier and at most 2x on store shape. That is where both your quality and your bill live.
- Meter curation separately from retrieval. Curation "never gets cheaper per episode", so it is a recurring cost proportional to intake, not an amortising investment. Retrieval cost is per query. These scale on different axes and should not sit in one line item.
- Watch the verbatim episode log. The one liability the study finds growing with the store is its serve-everything retrieval.
- Treat the store as inspectable, because it is. The medium's real advantage over bespoke memory representations is that a human can read the directory, diff it, and revert it. Keep it in version control and that advantage becomes an audit trail.
- Isolate the store per tenant. It is a filesystem the agent writes to, so the isolation rules in sandboxing and isolation apply directly.
How to maintain it¶
Store health held within every horizon the study measured, which is the encouraging half. Conversational stores created their few files early and afterwards only edited them, never deleting; on skills, early memories survived and a stronger management agent maintained files in place rather than replacing them. Stores only became more useful as they grew, and accumulated experience substituted for execution-agent capability.
Organization is the weaker half. Taxonomy adherence erodes as most stores grow, and only the strongest management agent tracked held it. So the maintenance question is not whether content survives, it is whether the structure you are paying for still exists after a few thousand integrations. Sample the store periodically and check it against its own contract.
Failure modes¶
- A taxonomy that does not partition along the query dimension. Looks organised, reads linearly in the corpus, and caps the payoff at a constant factor.
- Unbounded directory fanout. Sharding converts read cost into listing cost; past some width the index is the bill.
- File-scoped supersede state plus drifting routes. Produces confidently wrong answers with zero misses, and retrieval gets cheaper while it happens.
- Monitoring cost or miss rate for store health. Both defects modelled here move those metrics in the reassuring direction. Adherence to the taxonomy contract is the leading indicator.
- Organizing a small store. Near-parity search cost, real curation spend, no correctness gain.
- Expecting organization to improve answers. No agent measured converts organization into better correctness, and one configuration lost 40.6 points.
- Silent condensation during reorganization. The documented degenerate curation behaviour; add an explicit preservation rule.
- Destructive preference updates. Superseded facts left standing as current, which a stronger model only half fixes.
- Upgrading the management agent to fix answer quality. Capability there buys organizational style, not correctness, unless writing is already failing.
- Reading a single ordering off a small tier. One question is worth 3.12 points at n=32. Most reported gaps on small tiers are not resolvable.
- Treating the tool set as neutral. Replacing it reshapes the store as strongly as swapping the model.
- Assuming these results generalise. This is one study, on specific benchmarks, with small question counts and a stated re-judging noise of about two questions. It converts the filesystem default from an assumption into a design space; it does not settle it.
References¶
- Zhou, Yu, Wei et al., "Filesystem-based Memory for LLM Agents: Organization, Evolution, and Sustainability": https://arxiv.org/abs/2607.26637
- Ning, Tieu, Fu et al., "Code as Agent Harness: Toward Executable, Verifiable, and Stateful Agent Systems": https://arxiv.org/abs/2605.18747
- Code as Agent Harness reading list: https://github.com/YennNing/Awesome-Code-as-Agent-Harness-Papers
- MemGPT / Letta (the OS-style paged-context alternative): https://arxiv.org/abs/2310.08560
- Cognitive architectures for language agents (the declarative and procedural memory framing): https://arxiv.org/abs/2309.02427
- Voyager (skills distilled from experience into a reusable library): https://arxiv.org/abs/2305.16291
- Lost in the Middle (why the context window is not memory): https://arxiv.org/abs/2307.03172
Related: Context and memory · Agentic context management · Agent skills · Skill optimization · Agent harness architecture · Self-improving harnesses · Always-on agents and persistent state · Agent tools and function calling · Hierarchical agent decomposition · Harness effect on token economics · Agent loop economics · Agent evaluation · Evaluation integrity and anti-gaming · RAG versus CAG · Agent sandboxing and isolation · Agent observability