The action-execution boundary: from proposed intent to irreversible effect¶
Scope: what happens after an agent's action has been approved and before an irreversible effect lands in an external system of record. Covers the typed intent as the only thing a model may emit, an execution service that holds the credential so the agent never does, a durable ledger with content-addressed idempotency keys, reconciliation against the external system as the authority on what actually happened, and the operational controls (action-class kill switch, anomaly circuit breaker, shadow mode, staged authority) that let authority grow from evidence. The decision layer sits upstream and is covered elsewhere: whether an action may run at all is the agent policy engine, how much human oversight it needs is risk-tiered approval, whether the user really asked for it is intent verification, and what credential the agent carries is agent identity and access. The journal-before-dispatch rule and the uncertain-not-failed verdict are already established for session-local mutations in detached agent sessions and supervision; what this page adds is content-addressed keys, reconciliation against an external authority, and orphan detection. Note the word "intent" carries two meanings across these pages: here it is the model's proposed action object, while on intent verification it is the user's out-of-band signed attestation that they asked for it. They are different objects with different hashes and both can be required for one action.
The Python block was executed; its output is pasted verbatim. It models the ledger contract, not any specific vendor API.
What it is¶
A boundary with a one-line rule: the model proposes, deterministic systems authorise and execute.
Concretely, four components with four different trust levels:
| Component | Trust | Owns |
|---|---|---|
| Agent | untrusted output | producing a typed intent, nothing else |
| Policy and risk service | trusted, deterministic | may this intent proceed, at what oversight tier |
| Execution service | trusted, credential-holding | constructing, signing and submitting the real call |
| Ledger and reconciler | trusted, durable | what was proposed, what was sent, what actually happened |
The agent never holds the credential, never formats the outbound request, and never learns whether an action succeeded except by being told. Generated text does not reach the execution layer; a validated object does.
This is the zero-trust policy decision point and policy enforcement point split applied to an agent: the policy service decides, the execution service enforces, and the two are separate processes with separate credentials. What that buys is bounded: a compromised model cannot bypass the deterministic boundary or exceed its configured envelope. It can still propose a schema-valid action inside that envelope, and the executor will perform it, which is why the envelope is the security control and the separation only guarantees the envelope is applied. What zero-trust architecture does not give you is the third component here, the durable record of what was actually attempted, which is the part that turns a decision into an auditable action.
Why use it¶
Because approval and execution answer different questions, and most agent stacks only build the first.
A policy engine answers may this run. It does not answer did it run exactly once, what does the external system think happened, or what is to be done about the request that was in flight when the process died. Those are execution questions, and they do not go away because the policy said yes.
The failure that motivates the boundary is mundane. An agent proposes an action. The call is dispatched. The process dies before the acknowledgement arrives. On restart the agent sees an action with no result and does the natural thing: it tries again. If the first call landed, the effect happened twice. If the recovery logic instead marks it failed, and the first call landed, the ledger and reality now disagree and nobody will notice until a reconciliation nobody wrote.
In a distributed system you cannot promise exactly-once delivery across the network boundary. An exactly-once effect is reachable, but only when the receiver honours the key: idempotent submission, durable state, and reconciliation against the authority get you there together, and none of them is sufficient alone. Case 6 in the model below makes that dependency visible by running the identical ledger against a venue that honours the key and one that does not.
When to use it (and when not)¶
Use it when the action is irreversible and external: moving money, submitting an order to a venue, provisioning or destroying infrastructure, sending a message to a customer, changing permissions, publishing. The tell is that an external system, not your workspace, becomes the record of what happened.
Do not build it for reversible in-workspace effects. An agent editing files in a scratch copy, running tests, or drafting output needs a sandbox and a diff, not a ledger (sandboxing and isolation). Applying this machinery there buys latency and nothing else.
The boundary case is worth naming: an action that is reversible in principle but expensive to reverse, such as a large deployment, belongs on this side of the line. "Reversible" should mean reversible within the time an incident allows.
Architecture¶
flowchart TB
M["Agent<br/>(untrusted output)"] -->|"typed intent object"| V["Schema validation<br/>reject, do not coerce"]
V --> P["Policy and risk service<br/>deterministic, deny by default"]
P -->|"tier: approve or multi-approve"| H["Human or second decision path"]
P -->|"denied"| D["Terminal: rejected, journalled"]
H --> L
P -->|"tier: validate (automated check only)"| L["Ledger<br/>content-addressed key, journal before dispatch"]
L --> X["Execution service<br/>holds the credential, signs the call"]
X --> EXT["External system of record"]
EXT -.->|"acknowledgement"| L
R["Reconciler (standing job)"] -->|"reads"| EXT
R -->|"repairs uncertain rows, raises orphans"| L
KS["Kill switch per action class"] -.->|"halts"| X
CB["Circuit breaker on anomaly rate"] -.->|"trips"| X
L --> AUD["Immutable audit: model version, context, tool outputs, decision, order id"]Note what the reconciler reads. It does not read the agent's memory or the transcript. It reads the external system, because that is the only party whose opinion is authoritative.
How to use it: the typed intent¶
The intent is data, and it is untrusted data. Give it a schema and reject anything that does not match, rather than coercing a near-miss into something valid.
{
"intent_version": "1",
"action": "submit_order",
"instrument": "XYZ-PERP",
"side": "buy",
"order_type": "limit",
"quantity": "4",
"limit_price": "100.00",
"strategy_id": "st-114",
"rationale": "free text, for the audit record only, never parsed by the executor",
"context_digest": "sha256:...",
"proposed_by": {"model": "…", "harness_version": "…", "session": "…"}
}
Three properties that are easy to get wrong:
- The executor reads fields, not prose.
rationaleexists for the audit trail. If any downstream component parses it, the boundary has a hole in it: a prompt injection that reaches the model's output now reaches the execution path (prompt-injection defense). - The intent names the model and the context digest. Without these, an audit cannot answer "what did the system know when it proposed this", and a post-incident review of a bad action has nowhere to start.
- The intent does not carry a credential, an endpoint, or a signature. Those belong to the execution service. An intent that names its own endpoint is an SSRF primitive.
- Quantities and prices are decimal strings, not JSON numbers. A JSON number is parsed as a binary float in most stacks, and a binary float cannot represent a decimal price exactly, so the same economic order computed two ways produces two different fingerprints. Validate them with a schema that requires a finite positive decimal, sets
additionalProperties: false, and enumeratessideandorder_type. The executed model refuses floats, zero, negatives and non-finite values at the boundary rather than discovering them in the venue's rejection.
Modern tool-calling protocols make this cheaper than it used to be. In the Model Context Protocol a tool declares an inputSchema describing the arguments the model supplies, and the server is expected to validate against it, so the intent arrives as a checked object rather than something parsed out of a completion. Wire that schema and validate server-side; do not hand-roll the parse (tools and function calling). The companion outputSchema describes the structured result the server returns, which is a different direction and does not constrain what the model proposes.
How to develop with it: the ledger contract¶
Three properties carry the boundary. The model below asserts all three, then attacks them.
# execution_ledger.py -- validated: the boundary between a model's proposed action and an
# irreversible external effect. The first draft of this model got the identity rule wrong,
# and the corrected version is the point of the page:
# (1) operation identity is a token MINTED AT APPROVAL, not a hash of the payload. Hashing
# the payload silently merges two deliberate identical actions into one, which is
# data loss, not idempotency. The payload fingerprint is kept SEPARATELY and compared,
# which is what Stripe and AWS do: same token plus different payload is an error;
# (2) effect-bearing fields and audit fields are separated, so a rationale or a context
# digest cannot change identity, and decimal spelling cannot either;
# (3) a crash between journal and dispatch leaves the row uncertain, and it stays uncertain
# until a visibility horizon passes, because the venue's read view can lag its writes;
# (4) the divergence sweep enumerates the venue's records by VENUE id and groups them, so a
# duplicate effect under one token is visible. Collapsing the venue into a dict keyed by
# token hides exactly the failure the sweep exists to find;
# (5) none of this yields an exactly-once effect on its own. The receiver has to honour the
# token. The last case runs identical code against a venue that does and one that does not.
# Standard library only.
import copy
import hashlib
import json
import unicodedata
from decimal import Decimal, InvalidOperation
TERMINAL = {"rejected", "failed", "acknowledged"}
UNCERTAIN = {"submitted"}
# Fields that determine WHAT HAPPENS. Everything else on the intent is audit metadata and
# must not participate in the fingerprint, or a reworded rationale looks like a new order.
EFFECT_FIELDS = ("action", "instrument", "side", "order_type", "quantity", "limit_price")
DECIMAL_FIELDS = ("quantity", "limit_price")
def canon_effect(intent):
"""Canonical effect payload. Text is NFC-normalised; decimal fields are parsed and
re-rendered so that 100.0, 100.00, 0100.00 and 1E2 are one value; floats are refused,
because a binary float cannot represent a decimal price and would make the fingerprint
depend on how the number was computed."""
out = {}
for field in EFFECT_FIELDS:
if field not in intent:
raise ValueError(f"intent missing effect field: {field}")
value = intent[field]
if isinstance(value, float):
raise TypeError(f"float in {field}: use a decimal string")
if field in DECIMAL_FIELDS:
try:
number = Decimal(value)
except (InvalidOperation, TypeError, ValueError):
raise ValueError(f"{field} is not a decimal: {value!r}")
if not number.is_finite() or number <= 0:
raise ValueError(f"{field} must be a finite positive decimal: {value!r}")
out[field] = format(number.normalize(), "f")
elif isinstance(value, str):
out[field] = unicodedata.normalize("NFC", value)
else:
raise TypeError(f"{field} must be a string, got {type(value).__name__}")
return out
def fingerprint(intent):
body = json.dumps(canon_effect(intent), sort_keys=True, separators=(",", ":"))
return hashlib.sha256(body.encode()).hexdigest()[:32]
class Ledger:
"""Durable record of proposed effects, keyed by the approval-minted operation id.
Single-writer: a real store needs a unique index on that id and a compare-and-set
transition, or two replicas can both pass the in-process check in submit()."""
def __init__(self):
self.rows = {}
self.dispatches = []
def propose(self, operation_id, intent):
"""An operation id is minted once, when the action is approved. Re-proposing the
SAME id is a retry. A different id is a different action, even byte-identical."""
fp = fingerprint(intent)
row = self.rows.get(operation_id)
if row is None:
self.rows[operation_id] = {"intent": copy.deepcopy(intent), "fingerprint": fp,
"state": "proposed", "venue_id": None, "sent_at": None}
return operation_id
if row["fingerprint"] != fp:
# Stripe errors when incoming parameters differ from the original; AWS answers
# IdempotentParameterMismatch. Neither dedups it away.
raise ValueError(f"operation {operation_id} reused with a different payload")
return operation_id
def validate(self, operation_id, policy):
row = self.rows[operation_id]
if row["state"] != "proposed":
raise ValueError(f"cannot validate from state {row['state']}")
ok, reason = policy(row["intent"])
row["state"] = "validated" if ok else "rejected"
row["reason"] = reason
return row["state"]
def submit(self, operation_id, venue, now=0.0, crash_after_journal=False):
"""Journal `submitted` BEFORE the outbound call. Guards raise rather than assert, so
`python -O` cannot strip the state machine."""
row = self.rows[operation_id]
if row["state"] in TERMINAL:
return row["state"]
if row["state"] == "submitted":
return row["state"]
if row["state"] != "validated":
raise ValueError(f"cannot submit from state {row['state']}")
row["state"], row["sent_at"] = "submitted", now
self.dispatches.append(operation_id)
if crash_after_journal:
raise SystemExit("process died between journal and dispatch")
row["venue_id"] = venue.submit(operation_id, row["intent"])
row["state"] = "acknowledged"
return row["state"]
def reconcile(self, venue, now=0.0, visibility_horizon=60.0):
"""Repair against the venue's own records. Two rules the naive version gets wrong:
an uncertain row is NOT written off before the venue's read view has had time to
catch up, and the sweep enumerates venue records individually so that two effects
under one operation id are visible as a duplicate."""
by_op = {}
for venue_id, op in venue.records():
by_op.setdefault(op, []).append(venue_id)
repaired, divergent = [], []
for op, row in self.rows.items():
if row["state"] not in UNCERTAIN:
continue
if op in by_op:
row["state"], row["venue_id"] = "acknowledged", by_op[op][0]
repaired.append((op, "confirmed"))
elif now - row["sent_at"] >= visibility_horizon:
row["state"] = "failed"
repaired.append((op, "absent-past-horizon"))
else:
repaired.append((op, "still-uncertain-inside-horizon"))
for op, venue_ids in sorted(by_op.items()):
row = self.rows.get(op)
if len(venue_ids) > 1:
divergent.append((op, tuple(venue_ids), "duplicate-effect"))
elif row is None:
divergent.append((op, tuple(venue_ids), "orphan-no-ledger-row"))
elif row["state"] != "acknowledged" or row["venue_id"] != venue_ids[0]:
divergent.append((op, tuple(venue_ids), f"ledger-says-{row['state']}"))
return repaired, divergent
class Venue:
"""Stand-in for the external system of record. `honour_token=False` models a venue that
ignores the idempotency token, which is the case the ledger alone cannot survive."""
def __init__(self, honour_token=True):
self._log = []
self.calls = 0
self.honour_token = honour_token
def submit(self, operation_id, intent):
self.calls += 1
if self.honour_token:
for venue_id, op in self._log:
if op == operation_id:
return venue_id
venue_id = f"V-{len(self._log) + 1:04d}"
self._log.append((venue_id, operation_id))
return venue_id
def records(self):
"""Every record, by venue id. NOT a dict keyed by operation id: that would collapse
a duplicate effect into one entry and hide it from reconciliation."""
return list(self._log)
def limit_policy(max_notional):
def policy(intent):
effect = canon_effect(intent) # re-validates types and ranges
notional = Decimal(effect["quantity"]) * Decimal(effect["limit_price"])
if notional > max_notional:
return False, f"notional {notional} over limit {max_notional}"
return True, "within limits"
return policy
INTENT = {"action": "submit_order", "instrument": "XYZ-PERP", "side": "buy",
"order_type": "limit", "quantity": "4", "limit_price": "100.00",
"rationale": "audit only, never parsed by the executor",
"context_digest": "sha256:abc123"}
POLICY = limit_policy(Decimal("1000"))
# --- (1) A retry of the same approved operation submits once.
led, venue = Ledger(), Venue()
led.propose("op-1", INTENT)
led.validate("op-1", POLICY)
led.submit("op-1", venue)
led.propose("op-1", dict(INTENT)) # the model re-ran and re-derived the same intent
assert led.submit("op-1", venue) == "acknowledged"
assert (len(led.dispatches), venue.calls) == (1, 1)
print(f"1 retry of op-1 -> dispatches={len(led.dispatches)} venue_calls={venue.calls} "
f"effects={len(venue.records())}")
# --- (2) THE CORRECTION. Two deliberate, separately approved, byte-identical orders are two
# actions. Payload-hash identity merges them and loses one; token identity keeps both.
led2, venue2 = Ledger(), Venue()
for op in ("op-a", "op-b"):
led2.propose(op, dict(INTENT))
led2.validate(op, POLICY)
led2.submit(op, venue2)
assert len(venue2.records()) == 2, venue2.records()
assert fingerprint(INTENT) == fingerprint(dict(INTENT)) # identical payloads
print(f"2 two separately approved identical orders -> effects={len(venue2.records())}; "
f"their payload fingerprints are equal ({fingerprint(INTENT)[:8]}), so payload-hash "
"identity would have delivered 1")
# --- (3) Same token, changed payload, is an error and never a silent dedup.
try:
led2.propose("op-a", dict(INTENT, quantity="400"))
raise AssertionError("a changed payload under a reused token must not be accepted")
except ValueError as exc:
print(f"3 op-a reused with quantity 400 -> rejected: {exc}")
# --- (4) Audit fields do not touch identity; decimal spelling does not either; floats and
# non-positive quantities are refused at the boundary rather than in the venue's rejection.
assert fingerprint(dict(INTENT, rationale="reworded", context_digest="sha256:zzz")) == \
fingerprint(INTENT)
spellings = {p: fingerprint(dict(INTENT, limit_price=p))
for p in ("100.00", "100.0", "0100.00", "1E2")}
assert len(set(spellings.values())) == 1, spellings
for bad, why in ((dict(INTENT, limit_price=100.0), "float"),
(dict(INTENT, quantity="-4"), "negative"),
(dict(INTENT, quantity="0"), "zero"),
(dict(INTENT, quantity="NaN"), "non-finite")):
try:
fingerprint(bad)
raise AssertionError(f"{why} accepted")
except (TypeError, ValueError) as exc:
last = str(exc)
print(f"4 rationale and context_digest do not change identity; "
f"{sorted(spellings)} all hash to {list(spellings.values())[0][:8]}; "
f"float, negative, zero and NaN all refused (last: {last})")
# --- (5) A crash between journal and dispatch is uncertain, and nothing reached the venue.
led3, venue3 = Ledger(), Venue()
led3.propose("op-c", INTENT); led3.validate("op-c", POLICY)
try:
led3.submit("op-c", venue3, now=0.0, crash_after_journal=True)
except SystemExit:
pass
assert led3.rows["op-c"]["state"] == "submitted" and venue3.calls == 0
assert led3.submit("op-c", venue3) == "submitted" # restart must not re-dispatch
assert len(led3.dispatches) == 1
print(f"5 crash after journal -> state={led3.rows['op-c']['state']}, "
f"journalled={len(led3.dispatches)}, venue calls={venue3.calls}")
# --- (6) THE SECOND CORRECTION. An uncertain row is not written off on one negative read.
# The venue's read view can lag its writes, so absence only becomes failure past a stated
# visibility horizon. Inside the horizon the honest answer is "still uncertain".
rep, _ = led3.reconcile(venue3, now=10.0, visibility_horizon=60.0)
assert rep == [("op-c", "still-uncertain-inside-horizon")] and led3.rows["op-c"]["state"] == "submitted"
print(f"6 reconcile at t=10s inside a 60s horizon -> {rep[0][1]}, state still "
f"{led3.rows['op-c']['state']}")
# --- (7) The order surfaces late, before the horizon expires, and is confirmed rather than
# having been wrongly written off.
venue3._log.append(("V-0001", "op-c"))
rep, div = led3.reconcile(venue3, now=30.0)
assert rep == [("op-c", "confirmed")] and div == [] and led3.rows["op-c"]["state"] == "acknowledged"
print(f"7 the same row at t=30s, order now visible -> {rep[0][1]}, venue_id="
f"{led3.rows['op-c']['venue_id']}")
# --- (8) Past the horizon with the venue still empty, it is written off, and it stays under
# the sweep: if it ever appears, the next reconcile reports the divergence.
led4, venue4 = Ledger(), Venue()
led4.propose("op-d", INTENT); led4.validate("op-d", POLICY)
try:
led4.submit("op-d", venue4, now=0.0, crash_after_journal=True)
except SystemExit:
pass
rep, _ = led4.reconcile(venue4, now=120.0)
assert rep == [("op-d", "absent-past-horizon")] and led4.rows["op-d"]["state"] == "failed"
venue4._log.append(("V-0001", "op-d"))
_, div = led4.reconcile(venue4, now=180.0)
assert div == [("op-d", ("V-0001",), "ledger-says-failed")], div
print(f"8 written off at t=120s, appears at t=180s -> {div[0][2]}")
# --- (9) The classic orphan: a venue record with no ledger row at all.
venue4._log.append(("V-0002", "op-unknown"))
_, div = led4.reconcile(venue4, now=200.0)
assert ("op-unknown", ("V-0002",), "orphan-no-ledger-row") in div
print(f"9 venue record with no ledger row -> "
f"{[d for d in div if d[0] == 'op-unknown'][0][2]}")
# --- (10) THE THIRD CORRECTION, and the reason the sweep enumerates venue records rather
# than keying a dict by token: a venue that ignores the token produces two effects for one
# operation, and the duplicate has to be VISIBLE. Two ledger replicas, no shared state.
for honour in (True, False):
v = Venue(honour_token=honour)
for _ in range(2):
rep_ledger = Ledger()
rep_ledger.propose("op-e", dict(INTENT))
rep_ledger.validate("op-e", POLICY)
rep_ledger.submit("op-e", v)
_, div = rep_ledger.reconcile(v, now=0.0)
kinds = [d[2] for d in div]
assert v.calls == 2 and len(v.records()) == (1 if honour else 2)
assert ("duplicate-effect" in kinds) == (not honour), (honour, div)
print(f"10 two replicas, venue honours token={honour} -> calls={v.calls}, "
f"effects={len(v.records())}, reconcile flags {kinds or 'nothing'}")
# --- (11) A terminal row is inert. Replaying a rejected action never dispatches.
led5, venue5 = Ledger(), Venue()
led5.propose("op-f", dict(INTENT, quantity="400"))
assert led5.validate("op-f", POLICY) == "rejected"
assert led5.submit("op-f", venue5) == "rejected" and led5.dispatches == []
print(f"11 replay of a rejected action -> state={led5.rows['op-f']['state']}, "
f"reason={led5.rows['op-f']['reason']}, dispatched={len(led5.dispatches)}")
print("all assertions passed")
Executed output:
1 retry of op-1 -> dispatches=1 venue_calls=1 effects=1
2 two separately approved identical orders -> effects=2; their payload fingerprints are equal (fa43b085), so payload-hash identity would have delivered 1
3 op-a reused with quantity 400 -> rejected: operation op-a reused with a different payload
4 rationale and context_digest do not change identity; ['0100.00', '100.0', '100.00', '1E2'] all hash to fa43b085; float, negative, zero and NaN all refused (last: quantity must be a finite positive decimal: 'NaN')
5 crash after journal -> state=submitted, journalled=1, venue calls=0
6 reconcile at t=10s inside a 60s horizon -> still-uncertain-inside-horizon, state still submitted
7 the same row at t=30s, order now visible -> confirmed, venue_id=V-0001
8 written off at t=120s, appears at t=180s -> ledger-says-failed
9 venue record with no ledger row -> orphan-no-ledger-row
10 two replicas, venue honours token=True -> calls=2, effects=1, reconcile flags nothing
10 two replicas, venue honours token=False -> calls=2, effects=2, reconcile flags ['duplicate-effect']
11 replay of a rejected action -> state=rejected, reason=notional 40000 over limit 1000, dispatched=0
all assertions passed
Six things this pins down, and three of them are corrections to the first version of this page. They are stated rather than quietly absorbed, because each was a design defect that looked right.
Operation identity is a token minted at approval, not a hash of the payload. Case 1 is the easy half: re-proposing the same operation id after the model re-ran is a retry, and it submits once. Case 2 is the half the first draft got wrong. Two deliberate, separately approved, byte-identical orders are two actions, and their payload fingerprints are equal, so content-addressed identity would have merged them and silently delivered one. That is data loss wearing an idempotency costume. Stripe and AWS both key on a caller-supplied token and compare the payload separately; neither treats payload equality as operation identity.
The fingerprint is kept, and used for mismatch detection only. Case 3: the same token with a changed payload is an error, not a dedup. This is settled practice elsewhere, and the exact behaviour is worth copying: AWS answers IdempotentParameterMismatch, Stripe compares incoming parameters against the original and errors when they differ, and the IETF Idempotency-Key draft specifies a 422 for that case and a 409 while the original is still in flight. That draft is an expired Internet-Draft rather than an RFC, so treat it as a well-argued convergence of practice and not as a standard you can require conformance to.
Effect and audit are separated, and decimals are normalised. Case 4: a reworded rationale or a different context digest must not change identity, and 100.00, 100.0, 0100.00 and 1E2 are one price. Floats, zero, negatives and non-finite values are refused at the boundary. This matters because the fingerprint is what decides whether a retry is the same order.
A crash in the submit window is uncertain, not failed. Case 5 is the failure that motivates the design, and the counts are the point: one dispatch was journalled and the venue received zero calls, which is exactly why a restart cannot know which happened in the real case.
Absence is not failure until a stated visibility horizon has passed. Cases 6 to 8 are the second correction. The first version wrote a row off as failed on a single negative read, which contradicted its own evidence that a venue's read view can lag its writes. Now an uncertain row inside the horizon reports as uncertain, an order that surfaces at thirty seconds is confirmed rather than having been wrongly buried, and a row written off past the horizon stays under the sweep so that a late appearance is reported as a divergence. Case 9 is the classic orphan, a venue record the ledger has never heard of, which is what a credential used outside the boundary looks like.
The sweep enumerates venue records, and the receiver is where the guarantee actually lives. Case 10 is the third correction and the most consequential. Keying the venue's inventory by operation id collapses two effects into one entry, which hides precisely the failure reconciliation exists to find. Enumerating by venue identifier and grouping makes the duplicate visible. The same case then shows where an exactly-once effect comes from: two ledger replicas with no shared state each dispatch, and the venue produces one effect or two depending entirely on whether it honours the token. The ledger bounds what you send. Only the receiver can bound what happens, and the reconciler is what tells you which of those you are dealing with.
Two limits the model does not hide. The uniqueness check in submit is single-writer: a real deployment needs a unique index on the operation id and a compare-and-set transition, or two processes can both pass it, which is exactly the case 10 setup. And the state-machine guards raise rather than assert, because python -O strips assertions and a boundary whose enforcement disappears under an interpreter flag is not a boundary.
How to run it in production¶
The credential lives in the execution service. A credential broker that mints a short-lived, scoped token for the agent is an improvement over a static secret, and it is still weaker than an agent that never holds a credential at all (agent identity and access covers the broker pattern). Network policy should make the external endpoint unreachable from the agent's namespace, so a compromised agent cannot route around the boundary even holding a token.
Kill switch per action class, outside the agent's control. Revoking the agent's identity is a blunt instrument and a necessary one, but the switch you reach for during an incident is narrower: halt submit_order while leaving cancel_order working. That has to be a property of the execution service, reachable when the agent is unresponsive, unreachable by the agent itself.
Circuit breakers on anomaly rate, not just on spend. Token-budget breakers are the familiar kind (the harness effect covers the spend-side version, including halting a model that re-issues a byte-identical failing call). The one this boundary needs trips on the shape of the actions: rejection rate above a threshold, notional per unit time, repeated near-identical intents, or intents against instruments or accounts the agent has not touched before. Trip open and page; do not throttle silently.
Shadow mode before live mode. Run the whole pipeline with the execution service in propose-only: intents are generated, validated, ledgered, and compared against what a human or the incumbent system actually did, but nothing is submitted. This is a different thing from shadowing a harness change (governing self-modifying agents); here the executor is what you are testing. The output is a disagreement rate you can argue about before it costs anything.
Expand authority from evidence, on stated axes. Authority is not one dial. Separate the axes and move one at a time: value ceiling per action, aggregate exposure, instrument or account allowlist, and actions per interval. Each expansion should name the evidence that justified it and the condition that reverses it.
The oversight tier is deliberately not on that list. Risk-tiered approval enforces a hard floor under which an irreversible or regulated action can never be classified auto, however clean its track record, and this page's scope is exactly those actions. Evidence buys a larger envelope inside the tier; it does not buy the tier itself.
Keep the audit immutable and complete. Model and harness version, the context digest, the tool outputs the intent was derived from, the policy decision and its reason, the ledger transitions with timestamps, and the external identifier. Anything less and a post-incident review turns into archaeology.
How to maintain it¶
- Run reconciliation as a standing job, not as recovery. If it only runs after an incident, its first run during an incident is also its first real test.
- Alarm on orphans at any rate above zero. An orphan means an effect happened outside the boundary. That is a security finding, not a data-quality one.
- Age out uncertain rows into an alert. A row that has been
submittedfor longer than the external system's own settlement window is a page. - Mint the operation id once, at approval, and never derive it from the payload. Deriving it merges deliberate repeats; case 2 measures that.
- Set the visibility horizon from the venue's documented settlement behaviour, not from a round number. It is the interval before absence may be treated as failure, and getting it too short buries live actions.
- Alarm on
duplicate-effectat any rate above zero, separately from orphans. It means the receiver did not honour the token, which invalidates the whole guarantee rather than one row. - Enforce key uniqueness in the store, not in the process. A unique index plus a conditional state transition; the in-process check is a fast path, not the guarantee.
- Check the venue's token retention window. Providers prune idempotency keys after a bounded period (Stripe documents removing them after at least 24 hours), and a retry that arrives after pruning is treated as a new request. Both the visibility horizon and the uncertain-row ageing alarm must fire well inside that window.
- Version the intent schema and reject unknown versions. A schema that silently accepts extra fields will eventually accept one that matters.
- Rehearse the kill switch on a schedule. A switch nobody has pulled is a switch nobody knows the blast radius of (agent identity and access makes the same point about revocation).
- Re-run the disagreement measurement after every model change. Shadow mode is not a one-time gate; a model upgrade is a new proposer.
Failure modes¶
- The agent holds the credential. Every other control becomes advisory, because the agent can bypass them.
- Free text parsed downstream. A rationale field consumed by the executor turns prompt injection into an execution primitive.
- Operation identity derived from the payload. Two deliberate identical actions merge into one and the second is silently lost. Mint the token at approval; keep the payload fingerprint separately and error on mismatch.
- A reused token with a changed payload accepted. The opposite failure, and equally silent. It must be an error.
- Binary floats or unnormalised decimals in the fingerprint. The same price computed or spelled two ways looks like two different orders, so a retry becomes a second effect.
- Audit fields inside the fingerprint. A reworded rationale makes a retry look like a new action.
- The intent stored by reference. A later mutation changes what the audit record says was proposed under a key that no longer matches its own content.
- Retry-on-no-acknowledgement. The single most common way to produce a duplicate effect.
- Reconciliation against internal state. Comparing the ledger with the agent's memory confirms nothing; only the external system is authoritative.
- Divergence checked only as a set difference, or against a venue view keyed by token. An order that appears after its row was written off has a ledger row, so it is not an orphan by that test. A duplicate effect under one token collapses to a single entry and disappears entirely. Enumerate the venue's records individually and group them.
- Absence on one read treated as failure. A lagging read view buries a live action. Require a visibility horizon, and keep written-off rows under the sweep.
- Exactly-once claimed without checking the receiver. Exactly-once delivery is not achievable across a network boundary, and an exactly-once effect depends on the venue honouring the key. Confirm in the venue's own documentation that it does, and what its key retention window is; otherwise the design is at-least-once with extra steps.
- Authority expanded by success rate alone. A clean run of small actions is not evidence about large ones; expand on the axis you measured.
- Kill switch inside the agent's blast radius. If the agent can reach it, it is not a kill switch.
References¶
- IETF HTTPAPI, the Idempotency-Key HTTP header field, revision 07, expired and archived rather than an RFC (422 on key reuse with a changed payload, 409 while the original is in flight): https://datatracker.ietf.org/doc/draft-ietf-httpapi-idempotency-key-header/
- Stripe API, idempotent requests (parameter comparison on reuse; keys pruned after at least 24 hours): https://docs.stripe.com/api/idempotent_requests
- AWS EC2, ensuring idempotency in API requests (
IdempotentParameterMismatch, regional and zonal scope): https://docs.aws.amazon.com/ec2/latest/devguide/ec2-api-idempotency.html - NIST SP 800-207, Zero Trust Architecture (the policy decision point and policy enforcement point split this boundary instantiates): https://csrc.nist.gov/pubs/sp/800/207/final
- Model Context Protocol, tools and structured output (
outputSchema,structuredContent): https://modelcontextprotocol.io/specification/2026-07-28/server/tools
Related: Agent policy engine · Risk-tiered approval · Intent verification · Agent identity and access · Tools and function calling · Sandboxing and isolation · Agent security threat model · Agent observability · Agentic systems index · Glossary