Copy for agents Markdown

Changelog

Scope: what is new in this knowledge base, newest first. Each entry lists the pages added or substantially updated on a date, so you can see where the KB grew without hunting through the nav. Dates track the git history of the docs/ tree; each page also shows its own last-updated date in the footer.

How this page is maintained: when you ship new pages or a substantial update, add an entry at the top under a new date heading. Link every new page, group related pages, and record only real, shipped changes. Keep the older entries as they are.

2026-08-09

• New section and category: Agentic Cybersecurity & SysOps, a new top-level nav group with a routing hub, agentic cybersecurity and SysOps index, covering agents that find, prove, and report vulnerabilities, the runtime layer that contains them, and agents that resolve production incidents. Every page names the exact repository, commit, and license it was examined at, and carries a core mechanism executed and asserted rather than paraphrased. The agentic systems index gains an inbound pointer to it.
• New page: AI SRE agents with OpenSRE, on Tracer-Cloud/opensre (Apache-2.0, public alpha) at commit dceb3c3d: a framework for incident-response agents that correlate logs, metrics, traces, and deploys across 60-plus integrations, plus a synthetic-incident environment to train against. The executed block reproduces the reversible identifier masking (platform/masking): per-kind stable placeholders, a one-pass unmask that never lets a shorter placeholder clobber a longer one, counter resume across a state round trip, and the contextual-detector false positive where the word was after a bare ns keyword is masked as a namespace, which is why masking is defence-in-depth and not a privacy boundary. The real upstream context.py/detectors.py were also driven directly and round-tripped a realistic k8s alert byte-for-byte.
• New page: Autonomous web pentesting with Shannon, on KeygraphHQ/shannon (AGPL-3.0) at commit 760a1402: a multi-agent, white-box pentester built on the Pi agent core that reads source, plans attacks, and reports only exploited findings. The executed block rebuilds the exact AGENTS prerequisite map from session-manager.ts and schedules it into five dependency waves, confirming pre-recon is the sole root, report the sole sink, the five vulnerability agents fan out in parallel, and each exploit agent waits only on its own class's vulnerability agent.
• New page: Earned-verdict pentesting with ptai, on 0xSteph/pentest-ai (MIT, v1.1.0) at commit 96c3bf7f: an MCP-driven scanner whose findings stay candidates until a machine oracle re-proves them N of N, with a false-control twin that rejects any signal it cannot attribute to the payload. The executed block reproduces the N-of-N replay with intensity scaling (safe=2, normal=3, aggressive=5) and the control-twin abstain; the real engine/verifier.py was also driven and confirmed a non-pass yields candidate, never refuted.
• New page: Claude Code pentest subagents, on 0xSteph/pentest-ai-agents (MIT, VERSION 3.4.0) at commit c8dcc809: prompt-only Claude Code subagents for authorized offensive work with a mandatory scope-guard block on every execution-capable agent. The executed block models the declare-then-validate scope gate, the absolute hard-refusal list, and the tier2-requires-guard invariant; run against the real agents/ directory, all 13 Bash-capable agents of 52 routable ones carry the guard, and the routable count exceeds the README's marketed "50".
• New page: Agentic pentest orchestration: PTT and CrewAI, comparing two control models: GHOSTCREW (0xSojalSec/PentestAgent, MIT) drives a dynamic Pentesting Task Tree, TARS (osgil-defense/TARS, MIT, archived, pinned to crewai==0.28.6) drives a fixed sequential crew. The executed block reproduces PTT bottom-up status propagation with blocked-branch isolation and the CrewAI sequential hand-off, contrasting tree autonomy against pipeline determinism.
• New page: LLM red-teaming with promptfoo, on promptfoo/promptfoo (MIT) at commit 49c0f6d7, which ships 53 red-team plugin modules and 31 strategy modules mapped to OWASP, MITRE ATLAS, and NIST AI RMF. The executed block reproduces the plugin/strategy/grader split: base64, rot13, and leetspeak strategies defeat a keyword filter while preserving intent, and the grader, not the wrapper, decides whether the target actually complied.
• New page: Offensive security tooling in Rust, on skerkour/black-hat-rust (MIT) at commit 24c86c53. The ch_02 tricoder scanner was built and run on hardware: it compiled in about 19 seconds and ./tricoder localhost panicked at src/main.rs:37 on an .unwrap() of a reqwest JSON-decode error, the exact failure ch_03 fixes by rewriting main as async fn -> Result with ? propagation. The executed block reproduces the crt.sh subdomain set-cleaning (dedup, drop wildcards, drop the bare target) and the ch_02 unwrap crash.
• New page: eBPF runtime security with Cilium, on cilium/cilium (Apache-2.0) at commit 8c0423e9 (default branch 1.21.0-dev; pin a stable 1.20.x), framed as the containment layer under agentic workloads. The executed block reproduces the identity-based ingress decision (matchLabels plus matchExpressions, deny-by-default once a policy selects an endpoint) and makes the key point concrete: an L3 label match admits a compromised peer too, so egress and L7 rules, not L3 identity alone, contain a matched-but-compromised workload.
• New page: Tau: a minimal, readable coding agent, on huggingface/tau (Apache-2.0, v0.3.8) at commit 0217633f, a Python port of Pi's minimalist coding agent with a clean brain/app/TUI split. The executed block reproduces the agent-loop state machine (a turn is a model response plus its tool executions, halting on a text turn or max_turns, filtering empty error turns out of the next request); the real run_agent_loop was also driven with a scripted provider and confirmed every case.
• New page: Self-improving RLM agent (Prime Agent), on PrimeIntellect-ai/prime-agent (MIT) at commit a18809e0: a coding and research agent built on the Recursive Language Model (context as variables, subagents as function calls in a persistent REPL) and a Continual Harness that refines durable state through evidence-backed updates. The executed block shows why a programmatic-subagent agent needs both a recursion-depth cap and a shared token budget: depth alone does not stop a wide fan-out, and budget alone does not stop deep recursion. The project states plainly that its isolation is a lifecycle boundary, not a security sandbox.
• New page: NVIDIA DGX Spark playbooks, on NVIDIA/dgx-spark-playbooks (Apache-2.0) at commit 1fb66f05, 66 vendor-validated playbooks for the Blackwell DGX Spark desktop device covering framework setup, local inference and fine-tuning, agent stacks, and multi-Spark clustering over 200GbE. The executed numpy block sizes the ring-all-reduce bound (busbw = algbw times 2(N-1)/N) for two and three Sparks, showing why the 200GbE link, not Blackwell compute, caps a multi-node step and why the NCCL playbook tells you to tune buffer size.
• New page: Decentralized inference with Petals, on bigscience-workshop/petals (MIT, dormant since September 2024, pinned to transformers==4.43.1) at commit 22afba62: BitTorrent-style serving where each server holds a span of transformer blocks and the client chains them over a DHT. The executed block reproduces the Dijkstra min-latency routing over a (server, block) graph, showing the cache-allocation penalty on the entry edge, not raw RTT, drives the route, that servers are used to their span end before a switch, and that a coverage gap fails loudly with MissingBlocks.
• New page: Kernel-backed agent runtimes, on the harness design where an agent session owns a persistent interpreter process and that interpreter is its only model-facing tool: capabilities arrive as importable callables instead of tool schemas, working state survives context compaction because it does not live in the transcript, and privileged operations return through a typed host bridge that keeps credentials, transcript writes, and child lifecycle outside the interpreter. Two executed blocks carry the argument. The first runs two real kernel processes over real socketpairs with a single-threaded request loop and shows that a host reply routed onto the request channel never wakes the cell awaiting it, that the same reply is then read into the next request slot and desynchronises the stream, and that the host had replied in both runs so the failure is the channel and not a missing response, while the durable namespace answers correctly across turn boundaries in both. The second shows that the frame-correlation rule which correctly drops stale output silently discards a channel-open frame whose parent cell has already gone idle, and that both orderings accept identical output and reject the identical tampered frame, so a test that only checks stdout passes on the broken dispatcher. States plainly that the interpreter process is a lifecycle boundary and not a security boundary.
• New page: Detached agent sessions and supervision, on the process and protocol design that lets an agent session outlive the client that started it: a supervisor owning routing, attachment, and health but no execution, one worker per session tree, resident versus client-owned worker lifecycles, transcript leases keyed on the canonical path, and supervisor-generation fencing so a superseded supervisor cannot keep commanding an adopted worker. The executed block checks three recovery rules against their naive versions and finds that a bare sequence cursor does not error after a worker restart, it simply never delivers the post-restart events because their numbers sit below a cursor from a stream that no longer exists, while a generation-aware cursor detects the retired generation and falls back to a snapshot; that replaying every missed boundary of a ten-minute heartbeat after a three-hour outage queues 18 prompts where coalescing queues one and still advances the schedule to the real boundary; and that a mutation journaled before dispatch but left unresolved by a crash must be reported uncertain rather than replayed, with the mutation running exactly once across two submits.
• New page: Recursive agent delegation, on the contract where spawning a sub-agent returns an admission handle immediately and never the child's answer, results arrive as explicit messages or files, unknown options and unavailable models fail rather than being ignored or substituted, and a parent-scoped registry survives compaction, interpreter restart, and parent restore. The executed block establishes the reconciliation identity between per-node own usage and the root aggregate, shows that summing the totals each node reports inflates a 740k-token session to 1.5M because the deepest child's tokens are counted once at every ancestor, and shows the opposite error in the same tree: a parent holding 20% of a 200k context window reports 370% of it if the billed aggregate drives the compaction trigger. Also bounds the fan-out a depth cap permits, from 4 concurrent sessions at depth 1 to 40 at depth 3 under three-way branching. One predicted figure in the draft was wrong and was corrected against the executed result.
• Updated: Token-in, token-out gains a section on what to do when the exact-prefix invariant cannot hold at all, which is the case for an environment that compacts its own context, a harness that rewrites history, and a sub-agent handoff that returns only a summary. A fourth executed block segments a trajectory into maximal runs where extension provably holds, yielding two training samples from a five-step trajectory that breaks once rather than five or one corrupted one, confirms that merging is cheaper than per-step scoring, and adds two adversarial cases: a prompt that contains but does not start with the prior text must not merge, and an off-by-one truncation of the previous completion must segment. Records the prove-or-refuse rule for a renderer that bridges turns.
• Updated: Async and disaggregated RL systems gains a section treating the rollout fleet as a serving deployment: one global router with admin traffic addressed to engines directly, trajectory-scoped consistent hashing so every turn of a multi-turn rollout returns to the engine holding its KV, balancing on in-flight requests rather than scrape-lagged metrics because a group of rollouts is a burst of same-prefix requests, prefill-to-decode ratios that follow rollout shape (roughly 3:1 agentic, 1:2 non-agentic), and KV pooling across nodes when block keys omit the instance id. Adds the serving-side cost of exporting expert-routing decisions for trainer-side replay, the reported KV transfer packaging defect for disaggregated prefill and decode, and the note that a weight-transport fallback to the filesystem changes a run's staleness profile and belongs in its logged configuration.
• New page: Activation obfuscation on untrusted accelerators, covering GELO (arXiv 2603.05035v2, 2026-03-06): a confidential GPU holds hidden states and mixes each batch by a fresh secret invertible matrix on the token axis, so a non-confidential accelerator can run the attention projections on obfuscated data and return an exactly recoverable result. The page explains why left-mixing commutes with a weight multiply, why security reduces to a single-batch blind source separation problem, and what the untrusted device still learns. Two executed blocks carry the analysis. The first confirms the correctness identity, then shows that an orthogonal transform leaks the hidden-state covariance to machine precision and preserves the token-similarity spectrum exactly, verifies that shield rows shift the observed Gram matrix by precisely the shield covariance, and shows that the same mitigation masks far less than it appears to, since a free trace rescaling collapses its Gram error from 11.33 to 1.38 and the top-10 principal subspace still aligns at 5.1 times chance, and quantifies its low-precision cost: a tenfold error increase at the paper's own cond(A) < 100 limit and near-total corruption two orders of magnitude beyond it. The second reproduces the paper's Appendix A cost model exactly and then audits its headline claim, finding the offloadable share is 33.2% of per-token linear algebra rather than 76%, since the feed-forward network the scheme does not touch is twice the size of everything it does. That 33.2% is the supremum over all context lengths, so 76% is unreachable for what the scheme offloads under any denominator; the two readings that do reach 76% both include the FFN or the attention core. Grouped-query attention makes the offloaded tensors 44% smaller on both GQA models tested, leaving the share at 17.0% on Llama-3-70B and 18.1% on Qwen3-32B, which is the higher of the two because Qwen3 runs a 8192-wide attention block over a 5120-wide residual stream. Also records that the trusted-side security cost is 6.3 times the GEMM being offloaded, a ratio no interconnect changes, that Hopper's confidential mode bounce-buffers every one of the 64 to 160 crossings per forward pass, and that the paper leaves unresolved whether the secret transform is refreshed per batch or per projection, which is worth two orders of magnitude in cost or the security argument. Includes the 2019 ObfNet learned-obfuscator predecessor (arXiv 1912.09859) as a worked example of an obfuscation scheme whose entire privacy evaluation is ten volunteers failing to recognise the output, with no inversion attack attempted and the adversary holding the obfuscator weights it generated.
• New page: Covariant obfuscation for private LLM inference, covering AloePri (arXiv 2603.01499v2, ByteDance and Nanjing University, 2026-03-30): the client transforms weights and data together offline so that an unmodified vLLM serves an obfuscated model, with no TEE, no inference-time protocol, and a private token alphabet. The executed block reimplements the paper's Algorithm 1 rectangular key matrices, confirms the feed-forward network commutes exactly, and then derives from first principles the mechanism behind the paper's own Table 4 ablation: because the attention score is an inner product between query and key projections of the same input, key matrices, rotations and diagonal scalings all cancel and the score stays bit-identical to plaintext, which is why score recovery sits unchanged at 87.14% until rotary block permutation is added. A control confirms nothing else in the construction contributes, so protection of attention scores rests entirely on rotary embeddings. It also separates the two components that carry residual error, attention deliberately through the block permutation and RMSNorm unavoidably because it divides by a norm, and shows that residual falling from 3.6% to 1.4% as the model widens once the key matrices are sampled as Algorithm 1 specifies. A second executed block bounds the paper's Theorem 4 using only the definition of its own terms and finds the guarantee improves the composed privacy budget by under one part in a billion, and is vacuous when the optional token-perturbation mechanism is omitted, which is the configuration that produced the accuracy numbers. Prices the consequence the paper never states: the secret is client-generated, so every tenant needs its own obfuscated checkpoint, which removes cross-tenant weight sharing and means ten tenants of DeepSeek-V3.1-Terminus cost about 13.9 TB of weights at the bfloat16 the paper evaluates, and 80 hours of offline obfuscation. Notes that the paper's "less than 5% of tokens recovered" is correctly scoped to DeepSeek-V3.1-Terminus but does not generalise, since the same table reports 25.05% for Qwen3-14B; that the efficiency claim rests on 17-token prompts at concurrency 4 with no throughput measurement and posts a physically impossible speedup on the 14B model at both concurrency levels; and that the paper's attribution of its bfloat16 failure to numerical overflow cannot be the mechanism, since bfloat16 shares float32's exponent range.
• New page: Membership inference against fine-tuned LLMs, covering SOFT (arXiv 2506.10424, USENIX Security 2025, Purdue and Cisco Research): attacks that score near chance against a pre-trained checkpoint reach a ten-attack average of 0.819 AUC after three epochs of full fine-tuning on the same data, with the strongest single attack at 0.943 to 0.955, and exposure rises with model size, epochs, and LoRA rank. The executed block implements both metrics from scratch, reproduces every published aggregate in the paper's Tables 1 through 4 exactly, and then constructs the case that motivates reporting true positive rate at low false positive rate: a defence that flattens the bulk of the loss distribution while leaving a small subgroup memorised scores at chance on AUC while still fingering members at nine times the false positive rate. Records two findings from reading the released code at commit 7ca2b7b. The selection rule differs from the published algorithm in both threshold and schedule: the paper computes a cutoff from mean validation loss each epoch, while the code takes a fixed bottom-half quantile on the first epoch only and then replaces every sample on all later epochs, so the default three-epoch run is one epoch of selective obfuscation followed by two of wholesale replacement. And the shipped paraphrase path sends the private fine-tuning data to the OpenAI API, hard-coded to gpt-4o-mini-2024-07-18 rather than the GPT-4 and Claude-3.5 the paper names, which is disqualifying for the restricted corpora the defence exists to protect; a local neighbour-based alternative ships but is not what the module's entry point calls. Also notes that adaptive attackers who know the selection strategy raise true positive rate at 1% false positive rate from 0.033 to 0.149.

2026-08-07

• New page: Object-oriented agents (NOOA), covering NVIDIA-labs OO Agents (arXiv 2607.20709, submitted 2026-07-22) and the nooa package, where an agent is a Python class: methods are the actions, typed fields are the state, docstrings are the prompts, type annotations are runtime-enforced contracts, and a method whose body is ... becomes an LLM loop while a method with a real body stays deterministic. The page carries the paper's six model-facing capabilities (typed I/O, pass by reference, code as action, loop engineering, object state, model-callable harness APIs) as a reusable rubric for grading any harness, with the caveat that its fourteen-framework comparison is scored by an interested party against commits pinned 2026-07-07 to 2026-07-09. The executed block runs against nooa 0.0.8 from PyPI and probes the four mechanisms that need no model, producing three results that qualify the paper. The bounded-preview mechanism is real and its 1M-row case renders a million dicts in 98 characters, but the paper's compact one-line example for a 100-element list is max_length=10 while the shipped parameter default is prefill_format.max_length=25, at which the same argument occupies 31 lines. The in-process cell guard rejects every API section 3.4 names (eval, exec, compile, input, __import__, globals, wildcard import, dunder access, sys.exit, while True) and allows every escape the repo's own README names (open(), pathlib writes, importlib, reflection), which is the README being honest rather than the guard being broken; two further gaps are that rebinding e = eval is not caught because alias tracking is fed only by import statements, and that the blocking-call check resolves names against the live session so a same-cell import time then time.sleep(5) passes a check that fires on the next cell. And module-level constants never reach the agent, contrary to section 3.4's "imports, methods, and constants... are injected as locals": injection keeps modules, classes, and callables only, so a config constant is invisible while every module-level callable including underscore-prefixed ones is injected and rendered into the prompt, with @hidden the only filter. Four more findings come from reading the paper against its own tables and the repo at commit 10c6846. The reasoning-mode pass rates cannot be reconciled with Table 1: GPT-5.5 is 440/440 (100.0%) there but 99.5/98.6 off/on in the text, Nemotron 3 Nano is 91.6% against 52.5 rising to 84.8, and the same sentence names a "Super-v3" that is not one of the ten models. Reading tests/capability/config.yaml, two of the six stress families (error_recovery, task_decomposition) weight an LLM methodology judge at 0.5, and that judge is nemotron3-nano-30b, itself one of the ten models under test and the weakest at 91.6%. On Terminal-Bench 2.0 the dominance claim does not hold: PI beats NOOA in two of five columns (75.3 against 73.0 at GPT-5.5 xhigh, 65.2 against 64.0 at Opus 4.6 off), NOOA gains nothing from xhigh over high, and OpenCode gets worse as effort rises (60.7 to 52.8). The CyberGym L1 ranking mixes network protocols, since the three closed systems near NOOA's 86.8% are all marked network status unknown and the 83.5% open comparator ran with an open network; the clean like-for-like the paper does not foreground is Codex network-blocked at 64.9%, so NOOA leads by 21.9 points. The ARC-AGI-3 result is split into its solid half (a controlled ablation holding the agent fixed and swapping only the store: 50.2% RHAE with the typed memory subsystem against 38.4% with plain markdown files, both GPT-5.5) and its soft half (the 85.1% headline is a different model, the "6.4x harness effect" is footnoted as indicative by the authors because evaluation budgets differ, and every configuration is a single 25-game fleet with no repeat). Operationally the page is blunt that the containment boundary is the operating system: the paper's own hostile-workload fleet layers a Landlock and seccomp per-cell sandbox over the in-process guard rather than trusting the guard alone, and choosing a sandboxed code mode trades away the pass-by-reference that is most of the reason to pick this interface. Also recorded: the shipped capability suite has grown to 40 families and 116 instances against the paper's 36 and 88, so HEAD yields 5,800 records rather than 4,400; MCP configuration no longer expands ${VAR} from the host environment; tracing disables silently when the viewer is absent; and renaming a method is a behaviour change because the name is part of the prompt.
• Updated: agent harness architecture gains the six-capability rubric as a grading axis for any harness, and the filesystem as agent memory gains the NOOA ARC-AGI-3 ablation as an outside data point on what a typed store buys over a directory of markdown.

• New page: Cross-model KV cache transfer, on reusing a KV cache produced by one model inside a different model so a routed or cascaded request skips re-prefill, built from NVIDIA's arXiv 2608.03893 with C2C (arXiv 2510.03215) as the trained-fuser alternative. The load-bearing finding is that the abstract's "retains 73-98% of standalone accuracy on four pairs" is a mean over five benchmarks, four of which are log-likelihood classification. Reading the per-benchmark column, two of those four pairs retain 18.2% and 36.6% of GSM8K, the only chain-of-thought generation benchmark in the set: Llama 3.1 8B to 70B scores 14.78 where the 70B target scores 81.12. Two structural facts make this easy to miss and are recorded: the source-layer count k is selected by maximising the mean of the log-likelihood benchmarks, so GSM8K is a holdout by construction and nothing in tuning protects it, and the paper's own out-of-sample check uses PIQA, BoolQ and ARC-Easy, all classification and all easier than the selection set, reporting a reassuring 96 to 100% while testing none of the capability that collapsed. An executed NumPy block carries the mechanism the paper is right about, that error placement rather than error magnitude decides downstream quality: holding Frobenius norm and R-squared identical (both 0.9551) while aiming the residual into the query subspace versus its complement moves attention-output cosine from 0.9251 to 0.9885, which is why the paper's nonlinear MLP recovers up to +36.8 points on failure pairs by redistributing error rather than fitting better, and why it is slightly worse where ridge already works. The same block verifies the closed-form ridge solve against an augmented least-squares reference to 9.16e-16 and shows the RoPE strip-and-re-rotate round trip is exact, so a content-space fit calibrated at positions 0 to 511 re-applies at position 30000 to 1.33e-15. The storage formula reproduces all three published parameter counts exactly (1.07B, 1.61B, 3.36B), which reveals something the paper does not state: dividing its reported GB by its reported parameters gives 4.0 bytes per weight, so the 4 to 12 GB figures are fp32 and bf16 storage is an unmeasured halving. Fleet cost is quadratic, up to P(P-1) ordered pairs or roughly 130 GB for five models, against host memory rather than VRAM. Also recorded: the ablation table's dramatic RoPE row is an induced fit-versus-evaluation mismatch, while the fair "minus all RoPE" row is a wash on every benchmark and the appendix says so, making RoPE stripping a portability argument rather than a measured accuracy win; retention is strongly asymmetric, with Llama 8B to 70B retaining 94% small-to-large and 37% large-to-small at the same calibration R-squared of 0.84; calibration R-squared correlates negatively with retention across pairs (r = -0.20) while attention-output cosine reaches +0.57; raw retention credits 69.2% to a mapper scoring below chance, which floor normalization places at -7.7%; and the published latency spread of 2.7x to 25.1x excludes shipping the mapped cache to the target process, which the paper states it does not measure.

• New page: Cross-layer KV sharing, covering the layer axis of KV compression through CLA (arXiv 2405.12981, NeurIPS 2024) for the pretraining case and xKV (arXiv 2503.18893, ICML 2026) for the post-training case. The KB already treated the layer as a routing axis; it had nothing on layers sharing one cached tensor. Three executed NumPy blocks carry it. The first reproduces the dissociation xKV is built on: on synthetic caches constructed to share a token basis, mean token-wise cosine similarity is -0.010 against +0.002 for an independent-basis control, a gap of 0.012 that cannot separate the two groups, while CKA reads +0.842 against +0.037, a gap of 0.805. Cosine is why the earlier interpolation-based approach was capped near 1.2x compression. The second block turns that into a byte decision and includes the adversarial control: at matched 95% reconstruction energy, one shared basis plus per-layer coefficients costs 2.39x less than per-layer SVD on the aligned group and 1.5x more on the independent group, giving the operational rule that CKA should screen candidate layer groups before any factorization is enabled, rather than grouping every W consecutive layers uniformly. The third block isolates the property most likely to be misread in production: CLA reduces bytes stored by its sharing factor (2.00x, 2.91x, 4.00x at CLA2/3/4) while bytes read per decode step stay at exactly 1.00x in every configuration, because a layer reusing an earlier layer's cache must still read it. The paper states this directly, and it means CLA is a capacity lever whose throughput benefit is indirect, via larger batches and longer contexts; a latency benchmark at fixed batch size will correctly measure nothing. Also recorded: CLA3 on 32 layers gives 2.909x rather than 3x because the factor does not divide depth; layers sharing a cache must sit in the same pipeline stage or communicate KV across the boundary; CLA models prefer higher learning rates than their baselines (2.25e-3 against 1.5e-3 at 1B) so the comparison only holds when both are tuned; CLA2 beat larger sharing factors at matched footprint; and at 3B the authors report an outcome they did not expect, with MQA-CLA2 beating the same-head-dimension MQA model outright rather than trading perplexity for half the cache. xKV's factorization is defined over pre-RoPE keys, which is flagged as an integration trap.

• New page: Multi-agent KV reuse, on why prefix caching stops working the moment agents in a conversational system see the same text behind different prefixes, built from KVCOMM (arXiv 2510.12872, NeurIPS 2025) with AutoGen (arXiv 2308.08155) supplying the workload shape rather than any performance claim. An executed block separates the two halves of the alignment problem: a segment moved from offset 100 to offset 356 differs by a raw max of 7.100 in its keys, large enough that ignoring it corrupts the cache, and de-rotating then re-rotating removes that positional component exactly (8.88e-16) because RoPE is orthogonal, leaving only the content deviation from a different preceding context, which is what the anchor pool estimates. The arithmetic block is the one that changes how the headline should be read: because both prefill cost terms are linear in new-token count at fixed context, the saving is exactly the token ratio with no extra credit from attention being quadratic, so a reuse rate of rho is bounded by 1/(1-rho). A 70% reuse rate therefore cannot exceed 3.33x, while the paper's advertised peak of 7.8x corresponds to 87.6% reuse (8.06x in the model), the top of its reported 70 to 87.6% range. Both published numbers are real but they are different operating points, and the abstract pairs the reuse floor with the speedup peak in one sentence; the honest planning numbers are the paper's own means of 2.24x at the shortest prefix setting, 6.72x at the longest, and about 6.7x in the three-agent configuration. A further block shows why large teams benefit more, with the reusable share of prefill rising from 50.0% at two agents to 80.0% at eight in a fully-connected topology, which is why the headline is drawn from the best-placed agent in the largest run. Anchor pool behaviour is modelled as the cache-tuning problem it is, with hit rate climbing from 54.7% to 92.4% as the pool grows from 5 to 40 under frequency-biased eviction, so the paper's V=20 default is a starting point rather than a law. Also recorded: the v2 revision states that the original submission's TTFT calculation omitted the first token's decoding latency, so v1 speedups used a more favourable definition; performance correlates with request order because the admission criterion depends on which samples arrive first; and the comparison baseline CacheBlend fell from 82.0% to 57.1% on GSM8K in the same harness, a reminder that approximate KV reuse can fail catastrophically and silently on reasoning while aggregate metrics look fine.

• Updated: KV cache token eviction and compaction gains two sections for the compaction-family papers in the same batch. A third wall joins the page's existing two: the compressed cache is only good for one query. Query-aware evictors score importance from the queries present at prefill, so in the multi-query case that most long-context deployments actually run, where one expensive context serves many questions, the compression never amortizes and must be repaid per query. KVzip (arXiv 2505.23416) is covered as the query-agnostic answer, scoring importance by how much each KV pair contributes to reconstructing the original context, with 3-4x size reduction and about 2x lower FlashAttention decoding latency to 170K tokens, and with its context-dependent and context-independent modes distinguished because only one carries per-context overhead. A separate section covers Fast KV Compaction via Attention Matching (arXiv 2602.16284), which constructs synthetic keys and values reproducing the original attention outputs rather than selecting a subset of real ones, reporting up to 50x compaction in seconds and roughly two orders of magnitude faster than the Cartridges baseline it exceeds. A comparison table separates selection from construction on output rows, cost profile, high-ratio behaviour, paged-allocator fit and interpretability, noting that construction sidesteps the page's block-reclamation wall by emitting a small dense cache instead of scattered survivors and pays instead with a per-context compaction pass. A citation trap is recorded in the footnote: pdftotext on the KVzip PDF renders the en-dash in "3-4x" as the digit 9, producing a nonsensical "394x" in both places the figure appears, while other arXiv PDFs in the same batch preserve en-dashes normally; the arXiv HTML abstract is authoritative.

• New page: Gradient inversion under federated averaging, covering Geiping, Bauermeister, Dröge and Moeller (arXiv 2003.14053v2, NeurIPS 2020) as the protocol-level follow-on to the gradient-leakage page: given that one gradient inverts, does aggregating over local steps, mini-batches and epochs protect anyone. The paper's answer is no, and the page separates what it measured from what it asserted. Multiple local steps buy almost nothing (19.77 dB at one step against 19.39 dB at one hundred, same learning rate), and the single configuration that defeated the attack used a learning rate the authors themselves note "would lead to a divergent training update, and as such does not provide useful model updates". Table 2 is reproduced in full: single-image reconstruction sits at 25.05 dB while every multi-image setting lands between 14.66 and 16.92, and the two 8-image batch-size-8 rows at 1 and 5 epochs (16.49 and 16.58) show epochs do not help the defender. An executed NumPy block implements the paper's Proposition 3.1 as running code rather than restating it: it recovers a network input through three stacked fully-connected layers with ReLU between them at MSE 1.1e-33, no optimizer, against hand-derived gradients checked to 2.7e-10 by central differences. Three findings come out of that. The analytic attack is magnitude-invariant for free, because the recovery is a ratio: scaling the entire gradient by 1e-6, roughly what training does to gradient norms, leaves reconstruction at machine precision while the Euclidean distance saturates at the norm of the original, which is exactly the property the paper's cosine objective was engineered to give the optimization attack. The proposition's technical condition is not usable as a defense: it never failed across 2000 random networks, and a hand-built network with an entire ReLU layer forced off does block the walk-back but drives dL/dW1 to exactly zero, so blocking the attack this way and training the network are mutually exclusive. And the page supplies a mechanism for the effect the paper calls its most surprising, that "the distortions arising from batching are not uniform": the mixture weight for sample k at the attacker's chosen row is exactly the model's residual on that class, p_k[i] - 1[c_k == i], asserted to 1e-12, so exposure is set by per-sample prediction error. In the near-uniform-prediction regime one sample of eight holds 50.9% of the row against 7.01% for the least exposed, a 7.3x spread. Batch size is therefore an average protection with no per-user floor, and the samples that leak most are the ones the model handles worst, which the data owner can neither see nor control. Also recorded: the architecture study points opposite to intuition, with width raising average PSNR from 19.02 to 22.94 (wider is easier, not harder) and depth barely mattering (22.04, 21.59, 20.98 across ResNet-18/34/50) while ImageNet reconstruction through a ResNet-152 still succeeds; zero-padded convolutions are named as a privacy risk against circular padding, though on one inset figure rather than a sweep; the cosine attack is a trade and not a strict win, scoring 18.00 against the Euclidean attack's 46.25 on the untrained shallow network DLG was demonstrated on; the per-architecture PSNRs carry standard deviations of 2.84 to 6.83 dB, and the displayed width-comparison images differ by 8 dB where their averages differ by 3.9; and the threat model is honest-but-curious, with the authors noting in the same breath that "the attack is near-trivial under weaker constraints on the attacker".

• New page: Gradient leakage in distributed training, covering DLG (arXiv 1906.08935v2, NeurIPS 2019) with its released implementation at commit d21007f, and iDLG (arXiv 2001.02610v1). The KB had no page on gradient inversion at all, and the security section treated data-in-transit as a transport problem rather than a reconstruction one. An executed NumPy block carries the argument. It first derives the closed form the whole attack family rests on: for z = Wx + b the weight gradient is (dL/dz) x^T and the bias gradient is dL/dz, so any row divided by its bias entry returns the layer input exactly, recovered here at MSE 5.1e-34 with no optimizer, no priors and no iterations, against hand-derived gradients checked to 3.6e-10 by central differences. It reproduces iDLG's label rule at 5000/5000 trials, against the 89.9 / 83.3 / 79.1% that DLG's own optimizer scored on MNIST / CIFAR-100 / LFW by iDLG's Table 1. It measures what batching does: the recovered row is exact for the dL/dz-weighted mean at every batch size (about 1e-33) and never matches an individual sample, so batching mixes rather than hides. The load-bearing result is the defense sweep, which runs DLG's own Table 3 list against the same gradient and finds that every defense the paper marks as successful still leaks. Int-8, marked as defending at a cost of 22.6 accuracy points, returns the sample at MSE 4.9e-6; Gaussian noise at sigma 1e-2, costing 31 points, at 2.3e-4; even sigma 1e-1, which drives CIFAR-100 accuracy to 1% or below, leaves MSE at 1.3e-2 and cosine 0.983, inside the paper's own recovered band. Magnitude pruning has to reach 99% rather than the paper's stated tolerance of around 20%, and a sixth block explains why: weight-gradient row i scales with |dL/dz_i| and the true-class row carries the largest such scalar, so global pruning removes it last, leaving the attacker's row at 100% of its entries at a 20% global ratio and 87.5% at 70% where the tensor overall retains 29.8%. The conclusion drawn is that defendability in Table 3 is a property of one 2019 L-BFGS attack, not of gradient sharing. The page also records the constraints the original results were produced under, which are routinely dropped when DLG is cited: ReLU replaced with Sigmoid and strides removed because the attack needs a twice-differentiable model, every experiment on randomly initialized weights, batch size up to 8 and resolution up to 64x64, and iDLG's own restriction of its 100% claim to per-sample gradients. The random-init constraint was later measured, and Geiping et al. (arXiv 2003.14053) Table 1 is quoted for it: the DLG formulation scores 46.25 PSNR on the untrained shallow network it was demonstrated on and 6.90 on a trained ResNet20-4, where a cosine-similarity objective holds at 13.95, so ReLU, a trained model and large batches are all reasoning against the 2019 attack only. Four disagreements between the paper and its released code are recorded: main.py instantiates a four-layer LeNet where the paper claims ResNet-56 (which is why iDLG says it follows DLG's settings and uses LeNet), the repository's ResNet cannot run at all because BasicBlock.forward calls F.Sigmoid and torch.nn.functional defines only lowercase sigmoid, the script runs 300 iterations where the paper states 1200, and initialization is uniform_(-0.5, 0.5) on every weight and bias rather than PyTorch defaults. A topology table separates the case this is not a threat for (single-tenant DDP or FSDP on a fabric you own, where the attacker would already hold the dataset) from the cases where it is: a rented parameter server, untrusted peer ranks in a decentralized reduction, cross-organization federated training, WAN training, and gradients left in object stores, checkpoints or debug logs. Defenses are ordered by whether they survive a stronger attack, putting secure aggregation and trusted local aggregation ahead of noise, and flagging that DLG's own extrapolation from fixed-ratio pruning to Deep Gradient Compression was never run end to end and that DGC's error feedback defers pruned components rather than discarding them.

• Updated: Prima.cpp heterogeneous home-cluster inference is re-anchored from the v1 preprint to arXiv 2504.08791v3, published at ICLR 2026. The revision changed more than a version string: the title, the author list (five to eleven), the method's name (piped-ring to pipelined-ring), and the headline against exo and dllama (v1's "5-8x latency and 12-24x TTFT" became v3's "at least 18x lower TPOT and 42x lower TTFT"), and eleven appendices are new. A second executed block audits v3's own tables and finds the abstract's two headline numbers come from two different clusters: "On four consumer home devices, a 70B model reaches 674 ms/token TPOT ... and a 32B model with speculative decoding achieves 26 tokens/s" pairs a home-cluster result with a figure from a separate four-node Linux testbed carrying two RTX 4090s. On the home cluster the same 32B model posts 89 ms/token (11.2 tok/s) in the paper's own Table 9, against the 55 ms baseline of the testbed that produced the 26 tok/s; applying the best speculative gain measured anywhere in the paper (45%, on Llama 3.3-70B) to the home cluster gives about 20.4 tok/s, not 26. Section 4.1's "674 ms/token (442 ms/token with speculative decoding)" has the same problem, since 442 is Table 10's Llama 3.3-70B whose own baseline is 803 ms. The speculative-decoding claim itself checks out and is honestly scoped: 25-45% across 14-70B holds for all six such rows, while the 7B and 8B rows it excludes gain 10% and 0%. The block also derives the rule behind Halda's most counterintuitive result, where dropping a GTX 1080Ti node and moving its two layers to another node's CPU improved TPOT from 156 to 146 ms/token despite a 30x FLOPS disadvantage: every device costs a round trip per token on the ring, so a device earns its slot only when the compute it lifts off the critical path exceeds one hop, a threshold that rises with network latency. New operational content from v3: a context-length safe region beyond which TPOT rises sharply or OOMs (about 4K tokens on a low-memory cluster, 16K on a medium-memory one, for 70B Q4K), network sensitivity showing bandwidth is irrelevant above roughly 50 Mbps while 100 to 200 ms RTT costs 68%, concurrency that peaks at 8-16 requests then declines, background-app degradation of 5-18% for everyday apps against 66% for a realtime 3D game, and energy accounting putting distributed inference 91-99% below single-device per device and 28% above a datacenter server in total. Recorded as a maintenance finding: v3 removed the limitations section v1 carried, and all five occurrences of "limitation" in v3 describe a competing system rather than prima.cpp, so the page keeps the v1 cautions and says where they come from.

• Updated: Privacy-aware split inference over a WAN gains the measured negative result the page was missing. Before settling on lookahead, the paper tried vanilla Jacobi decoding over the WAN and got about 1.5 tok/s against roughly 8.3 tok/s sequential, a 5.5x regression, because blocks "rarely commit more than 1-2 tokens per iteration for language modeling" while every iteration still pays a full round trip and ships a larger tensor. That makes the n-gram machinery load-bearing rather than an optimization. The page also now carries the fixed-overhead ceiling from the paper's partnership projection: at 5 ms RTT per-step time is about 48 ms and throughput about 21-26 tok/s, so once RTT stops dominating, the 43-46 ms of local compute, cloud compute and serialization is the only remaining target. Both pages now cross-link, since they reach the same conclusion from opposite directions: latency, not bandwidth, is what limits a ring or a split across a slow network.

• New page: Verified task synthesis for agent RL, on manufacturing executable agent tasks when there are not enough real ones, built from RST (arXiv 2608.05466) and checked against the artifacts the authors actually released rather than the paper's own summary. The recursive loop runs solution first: extend solve.sh, realign the verifier, instruction and Dockerfile to it, then accept only when the reference solution passes the private verifier in a fresh sandbox and every checked requirement is stated publicly or discoverable in the workspace. An executed NumPy block turns the paper's two published curves into the number the paper does not state: the text-only gates absorb 28.8% of attempts at R1 and 32.1% at R15 before any sandbox is allocated, an accepted task costs about 2.5 sandbox validations once bounded repair is counted, and it reserves 1.68 sandbox-hours at the pool's median timeouts. Inverting that against the published $0.05 per task gives a break-even container price of $0.030 per vCPU-hour, so the headline cost is a model-token figure with no container bill in it; the paper publishes no cost breakdown anywhere. The same block records that "attempt" is never defined and that R1's own numbers require about eight attempts per seed. Independent measurement of the released 37,484-task pool found four things the paper does not say: 75.0% of bundles are still labelled difficulty = "easy" and 58 are labelled hard on a pool whose late rounds a frontier model passes 2.5% of the time; 37,459 of 37,484 tasks request exactly 1 vCPU, 2 GB and 6 GB, unchanged while the median solution grew more than fivefold, so neither difficulty nor resources were realigned by the rewrite; 56 shipped instructions exceed the 180-word limit the generation-quality gate is described as rejecting; and the release carries no synthesis-round label, so the R1 to R15 curves that carry the paper's entire difficulty argument cannot be reproduced from it. A dedicated section records four places where the paper disagrees with itself or with its release: the conclusion reports the RL headline as 46.07 and +11.82% where the abstract and Table 4 both say 49.44 and +20.00%; R11 through R14 appear in no figure or table, so "no observed ceiling" rests on a series with four consecutive unreported points; the PPO actor starts from base weights but its value head is warm-loaded from a prior terminal-agent critic, so the relative gains are not a clean base comparison; and the released trajectory corpus was produced by four models, two of which are named nowhere in the paper.

• New page: Sizing the agentic rollout sandbox fleet, the container-capacity model the KB was missing for RL tasks that are workspaces rather than prompts, derived here from the 327,189 released RST trajectories and the 37,484 task.toml resource requests. A sweep line over the start and finish timestamps gives a peak of exactly 3,000 concurrent sandboxes against a mean of 467.6, a 15.6% duty cycle that says this fleet should be bought at peak on interruptible capacity; 191,099 sandbox-hours over 17.0 days produced 62,511 passing rollouts, or 3.06 sandbox-hours each. Because a task asks for one vCPU and 2 GB, an ordinary 96-core node is CPU-bound at 96 slots and the whole 3,000-sandbox peak fits on 32 machines, which is why this cost never appears in a GPU capacity plan. The traffic shape is the other half: one busy sandbox generates 48.1 prefill tokens and 1.72 decode tokens per second, a 28:1 ratio, and cached_tokens is zero on all 238,415 rows that report tokens. Fitting median input tokens against turn count gives a quadratic with R2 = 0.9956 against 0.8560 for a linear fit, with a stable 824-token per-turn context increment: the signature of re-sending the whole transcript every turn. Prefix caching is therefore worth (turns + 1) / 2 on prefill, 6x at the measured median of 11 turns and 11x at the p90 of 21, for no change to the training run. The executed block asserts the model against the measured medians and records where it fails, undershooting two-turn trajectories by 22.9% where fixed prompt overhead dominates. Operational findings: 29.4% of rollouts carried an exception and those passed 0.97% of the time against 26.65% for clean ones; 25.0% of rows carry no reward at all and 27.1% no token counts, so completion is not success and a quarter of the telemetry is missing.

• Updated: Agentic and tool-use RL now routes its "tool execution is a fleet, not a call" bullet to the measured sizing page instead of leaving the capacity question open, and RLVR, synthetic data generation and the rollout fleet sizing runbook cross-link the two new pages.

2026-08-06

• New page: When to compact, on the compaction trigger as a policy rather than a percentage-of-window rule, built from SelfCompact (arXiv 2606.23525) and ACON (arXiv 2510.00615). Three executed stdlib blocks carry the load. The first rebuilds a payback model from SelfCompact's own cost formula and OpenRouter price table, because the paper credits its Appendix C with deriving "compaction wins iff L/l > 10" and Appendix C contains the formula, the procedure and the prices but no such derivation. Reconstructed, the ratio does not decide whether compaction wins, it decides how fast it repays: 2.4 to 4.3 remaining LLM calls at the reported 20x to 80x shrinkage, 5.7 to 6.8 at the claimed 10x line, and still repaying at 2x after about 45 calls. The actionable rule that falls out is a tail guard (do not compact with fewer than roughly five calls left), which no fixed-percentage trigger enforces. The second block recomputes the answer-transition table: across 12 fixed-interval summarisation calls, 1,009 of 2,495 transitions (40.4%) turn a correct answer wrong, summarising at all is worth +2.5 points, and an oracle that keeps the same schedule but skips the call when the answer is already correct adds +11.5 more, so trigger quality has a ceiling worth 4.6x the mechanism and the published rubric realises 35.7% of it. The same block shows the 18.1-point math headline is a 93k-token run against a 16k baseline (5.8x); against fixed-interval at a matched budget the same cell is +7.4, and the paper's own "within 3k tokens" matching tolerance slips to 4k in one of four blocks. The third block audits ACON: its 26 to 54% peak-token claim reproduces (26.2% and 54.5%) and its accuracy comparisons are correctly stated against compression baselines, but against the no-compression upper bound the same rows are +0.5 on AppWorld, -2.1 on OfficeBench and +0.007 EM on 8-objective QA; one sentence pairs a 54.5% peak reduction from the history-compression row with a 61.5% dependency reduction that belongs to the observation row, whose EM is below the no-compression baseline; and the named contrastive-feedback mechanism accounts for 0.6 of the 7.7 points that guideline optimisation buys (7.8%) while the choice of optimiser model is worth 3.6. It also surfaces the number most of this literature omits, ACON's Table 4: history compression costs 13.9% less and takes 19.7% longer per task, observation compression 17.8% less and 39.2% longer. Compression trades latency for tokens, in that direction, every time. Also recorded: on one of nine published cells, keeping the last 3 turns beats the rubric on both accuracy and cost.
• New page: Compaction-aware RL, on training the policy that compresses its own context, drawn from ReSum and ReSum-GRPO (arXiv 2509.13313), SUPO (arXiv 2510.06727), CompactionRL (arXiv 2607.05378) and the AutoCompact project page. Three executed NumPy blocks. The first derives the two biases a compaction boundary introduces: normalising the advantage over the expanded segment list rather than over the rollouts shifts the baseline by 0.3 and turns clean plus-or-minus-1 advantages into minus 2 against plus 0.5 as soon as segment count correlates with reward, and segment-level loss averaging hands the compacting rollouts 80% of the gradient mass on 65.8% of the tokens. Both papers that measured these agree with the direction: SUPO's ablated normalisation scores 42.1 on CodeGym against a plain GRPO baseline of 44.5, and CompactionRL loses 6.8 points without token-level normalisation. The second block reconstructs CompactionRL's cross-trajectory GAE against a reference GAE over the concatenated rollout and separates what the correction does from what it claims: it restores the terminal reward's discount exactly (error 3.3e-16 at every token, against a 1.44x over-credit of the first token without it) but leaves the value-bootstrapping error at each segment boundary, 33.5% relative on the pre-final segments and zero on the last. Sweeping the paper's length-adaptive lambda over three readings of an undefined "response length" shows the first segment of a four-segment rollout retaining 5.3%, 39.2% or 62.6% of its weight, so the earliest summary in an episode may be learned twenty times more slowly than the last depending on a parameter the paper does not pin down. The third block audits the four sets of published results: SUPO's CodeGym +3.2 is a like-for-like effective-length comparison but 0.51 standard errors on 128 tasks, while its BrowseComp-Plus +14.0 is 2.01 standard errors and gives the trained model 3x the effective context; ReSum's training-free 4.5-point average reproduces to the decimal from Table 1 but its abstract's "further 8.2%" matches none of the three natural baselines in Table 2 (4.40, 6.74, 8.90), and two cells show ReSum-GRPO trailing methods it is presented as beating; CompactionRL's +7.0 and +3.1 shrink to +2.3 and +0.9 against its own RL baseline trained at twice the context budget, the Terminal-Bench figure sitting at 0.70 standard errors on 89 tasks, and, most usefully, its Table 1 shows that swapping in a stronger off-the-shelf summariser buys +5.0 points where the entire RL campaign buys +5.5. AutoCompact is recorded as a project page with no paper, no code, no weights and no numeric table, cited for its mechanism only.
• Updated: Agentic context management for the ACM code and checkpoint release that landed after the page was written. The repository (MIT, commit f06f90e, pushed 2026-08-03) and a nine-item Hugging Face collection of three post-trained checkpoints and six rollout and logprob datasets are now public. A new executed block imports the released tool registry at that pinned commit and confirms the two-tool design this page describes, while surfacing two things the paper does not state: only browsecomp and browsecomp-plus are registered, so the DeepSearchQA and SWE-Bench Verified columns of the paper's tables have no tool set or data loader in the repo, and the no-query ablation rewrites manage_context's description as well as removing query_memory, so the two arms differ in what the agent is told and not only in what it can call. The README's own disclosure that the on-policy distillation is run off-policy against a cached teacher logprob store because of GPU supply is recorded, along with the reproduction cost (a 397B teacher in bf16 on 8xB200 or fp8 on 8xH100, pyserini BM25 needing a JDK, and the BrowseComp-Plus corpus cloned separately).

2026-08-05

• New page: Rollout reuse under policy lag, covering arXiv 2608.01418v1 (PNPO) as the prefix-normalized point on the importance-weighting ladder, and the epoch count as a capacity lever rather than only an algorithm choice. The paper publishes no code, so nothing is claimed against a released implementation, and the training runs were not reproduced. Three executed NumPy blocks rebuild the load-bearing claims from the paper's own numbers. Back-solving the two published step times (283 s at one epoch, 510 s at four) recovers 207.3 s of generation and 75.7 s of optimizer work per epoch, which reproduces both step times and both reported step counts exactly, and shows generation is 73.3% of a one-epoch step and 40.7% of a four-epoch one. At the matched 2,400-update budget that is 21.25 hours against 47.17, a 2.22x saving on exactly one quarter as many generated responses, against a hard ceiling of 3.74x that no amount of further reuse can pass. The second block establishes that PNPO is an interpolation rather than a new family: its weight is exactly the token-local ratio at the first position and exactly GSPO's sequence statistic at the last, and the sqrt(L/t) gate factor cancels the 1/sqrt(t) shrinkage of the prefix mean, holding acceptance between 92.0% and 97.3% across all 15,360 positions where a flat tolerance would accept 1.25% at the first. The same block exposes a confound the paper does not name: at the terminal position both objectives act on the identical statistic, but PNPO runs at (7e-4, 9.5e-4) against GSPO's (3e-4, 4e-4), about 2.3x wider, so part of the four-epoch gap could be clip width. The third block pins two implementation traps by execution: with a mean per-token log-ratio of 0.05 the exact cumulative ratio leaves float32 range at token 1,843 and float64 range at token 14,286 of a 15,360-token response while the prefix mean stays at 1.05, and renormalizing after gating turns a response contributing 0.3594 into one contributing 0.9985, silently converting rejection into reweighting. Three honesty findings are recorded rather than smoothed over. The headline 3.00-point gap is mostly GSPO degrading under reuse (-2.18 from its own one-epoch peak) rather than PNPO improving (+0.19), so the result is that reuse is roughly free for PNPO and expensive for GSPO and GRPO (-1.20). The reported 22.4-hour first passage for four-epoch GSPO corresponds to step 158 of a 150-step run at the paper's own 510 seconds per step, and its four-epoch reward panel ends visibly below the 0.25 threshold, so the "6.4 hours earlier" claim sets an observed crossing against an extrapolated one; the one-epoch values and four-epoch PNPO's both fall inside their runs. And on 30-problem benchmarks the four-epoch AIME separations are 1.19 and 0.91 problems wide from a single seed, with Table 1 selecting each benchmark peak independently over 15 evaluations so its average is not achieved by any single checkpoint. The verl integration is given against the current @register_policy_loss registry and actor.yaml keys, and TRL is noted as exposing num_iterations and an importance_sampling_level that has no prefix level.

• New page: Harness-R1: learning to edit runtime harnesses, covering arXiv 2608.02276v1 and the released implementation at commit b9d9853. It separates the trained 9B engineer from the frozen target, maps the four lifecycle hooks, same-batch delta reward, cold-start SFT, and eight-candidate online GRPO, and records the reported 44.3% to 53.6% average gain without presenting the benchmark as independently reproduced. The source-only release suite passes all 17 tests. An executed NumPy audit reproduces the displayed 9.3-point gain, rejects task-identity drift, shows aggregate cancellation and held-out regression, and exposes that zero-reward invalid outputs receive positive relative advantage when the valid candidate group is harmful. Production guidance adds a disjoint regression gate, external code isolation, signed patch manifests, canary rollout, and per-hook rollback because the paper's reward is transductive and does not price runtime cost. A second pass against the paper and the pinned commit tightened five claims and added one defect found by execution. The transfer footnote now names the three regressing cells the paper does enumerate in Appendix E, WebShop on Llama-3.1-70B, ALFWorld on Qwen2.5-72B, and WebShop on Gemma-4-31B-it, none of which is the primary target, which is what makes the three-of-sixty count exact rather than an inference. The page also records that the released training script still passes the KL flags at a coefficient of zero, that group standard-deviation normalization is on unless disabled, that WebShop identity enforces manifest agreement rather than the literal seed 233, and that the shipped demo is a stored evaluation moving 1/10 to 5/10 rather than the Appendix H case that moves 2/10 to 5/10. The endpoint tool-call probe the release marks as important is now included, with the correction that its README form omits the required --tool-choice and --output arguments and exits 2 during argument parsing before contacting any endpoint.

2026-08-04

• New page: Policy dissemination for WAN rollout fleets, covering arXiv 2602.02192 (ECHO-2, v5) as a pattern rather than a product, since the framework publishes no repository and its authors' public org ships Parallax, Lattica, and Symphony but no RL implementation. The page fills a gap the existing sizing arithmetic left open: rollout fleet sizing assumes weight sync is negligible against a step, which fails once the fleet is a WAN away, and that runbook now names the assumption and routes to the corrected rule. Three executed blocks rebuild the load-bearing claims. The capacity model reproduces both reported fleet sizes (8 workers at a safety factor of 1.0, 9 at the default 1.1) from a single back-solved per-worker rate, fires the feasibility gate that no fleet size clears, and shows the rule degenerating to plain rate matching as broadcast time goes to zero, which is the only reason the publication period appears in it at all. The broadcast model is built independently from link rates and payload size and lands on the source from both directions: a 1312 s floor against the measured 1437 s, and 3936 s for the star-limited case at 9 workers on a 300 Mbps uplink against the roughly 4000 s its figure shows. That agreement is also the evidence for an assumption the paper never states, that snapshots are full bf16 copies. The floor is set by the per-worker downlink, not the topology, so peer forwarding fixes scaling and only delta weight sync moves the floor: at the 1% to 5% densities measured there the same 22-minute transfer becomes 13 to 66 seconds. The cost model separates two margins the paper's abstract and conclusion combine. The 33% to 36% saving against a colocated baseline is real but bundles halving the learner from 8 GPUs to 4 with the rollout move, never ablated apart, and each step runs 8.2% longer. The 4.6% saving against an already-disaggregated system is a price ratio: the published $1,914 and $1,826 totals divide by their fleet rates to the identical run length of 36.6667 hours, so the comparison holds wall clock fixed and cannot measure the scheduler it is presented as isolating; 83 commodity workers is break-even, 84 is a loss, and a step 5% slower erases the margin entirely. Also recorded: the staleness envelope (within roughly 5% of synchronous for a budget of 6 or less, divergence at 11 under unmodified GRPO), cost-aware activation by dollars per rollout per second, and the fault asymmetry where losing one relay costs 16.45% of throughput against the 12.5% one worker represents, because a fan-out-1 chain stalls everything behind it.

• New page, then two correction passes: autoresearch-rl separates the public 0.4.0 wheel from a later private source snapshot, maps inner model RL, outer experiment PPO, and source-only continuous learning as distinct loops, and provides an executed three-file local campaign with a separate frozen evaluator. The audit rejects the project's strongest trust claim: prepare.py is only a diff-scope boundary, many shipped recipes report their objective from mutable training code, and local processes can read the same held-out files. Everything attributed to 0.4.0 was produced by running it, including the sdist suite (605 passed, 41 failed, 6 skipped, every failure tracing to example fixtures the wheel and sdist do not ship) and five complete campaigns. The command target runs train_cmd twice per iteration when eval_cmd is absent, which an executed run confirms at the file level (40, 120, 40, 120, 40 progress lines for a steps: [20, 60] grid), while Basilica caches the training outcome instead. A loss = ... line anywhere in the output suppresses the generic fallback and silently drops a custom objective; JSON is never parsed; Basilica's parser additionally rejects spaces around the sign and requires one of eight known keys. The campaign forecaster stopped an executed six-point grid at five under max_iterations of 6, 7, and 12, on both objective directions, losing a point that scored about 4,300 times better; re-running the same grid under controller.parallel completed all six, because that engine never calls the forecaster. The page's runnable block isolates the cause on a noiseless power law: the offset is grid-searched over only 0.5, 0.8, and 0.9 times min(y), so the fit overshoots at the tail and the rule stops improving curves while leaving flattened ones alone, and the same forecaster is direction-corrected inside the trial-level guard but not at campaign level. Resume does not recover the lost points, because policy state is written as an empty dict and never read back. LLM policies fall back to random after 240 s of retry backoff with no trace of it in any artifact. The guide also covers policy and target extension seams, the limits of the deterministic finite-difference learned policy, source-only shadow and human-gated campaigns, production artifacts, and the external isolation and authentication controls the framework does not provide.

2026-08-03

• New page: Parametric long-term memory modules, covering arXiv 2607.27919 and the released LUMIA-Group/MemoryDecoder-at-Scale code. Two executed blocks rebuild the parts that decide whether the technique works. The first models the sharded Faiss pipeline and reproduces the paper's approximately 250x dense-to-sparse storage ratio exactly (250.5x at 64.95 retained pairs per row against a 50,304-token vocabulary), then shows the saving is a vocabulary effect that halves with the vocabulary and collapses as K grows. It also demonstrates the two silent defects the pipeline cannot report: at one probed centroid the sharded search recovers 29% of the true neighbours with no error raised, and merging each shard's best candidate without a global re-rank returns a different distribution. The second block reproduces all six published averages and the paper's own "improves 47 of 51" claim, and finds that all three regressions are TruthfulQA, once at every scale, so calibration is a systematic cost rather than noise. The 39% parameter saving belongs to the mixed-scale pairing (410M backbone plus 6.9B memory, 7.31B against 12B); the matched-scale pairing at the top of the same table is 13.8B and therefore larger than the 12B model it beats. Because the interpolation weight is tuned per benchmark on a validation split, and optima move in opposite directions for knowledge-heavy and reasoning-heavy traffic, a single production value gives back a large share of the benefit. The page also records that the paper's own memorization probe measures a leak as well as a feature: on identical data and budget, strict 16-token suffix reproduction rises from 42.4% to 49.7% against a continued-pretraining baseline, and a domain-anchor probe from 22.6% to 56.5%.

• New page: Agentic AI detection and response, covering the ADR paper (MLSys 2026, arXiv 2605.17380) and the uber/ADR release at commit 73873e1. The executed block re-derives the published confusion matrix, the per-tactic breakdown (which sums exactly to 28 of 42), and the blended cost and latency from the two cascade paths, then tests what the headline numbers actually establish. Tier 1 forwards about 83% of the attacks Tier 2 could catch, so triage, not the reasoning agent, sets the recall ceiling: removing it lifts recall from 0.667 to 0.805 at 29% more cost. Zero false positives on 260 benign tasks bounds the false-positive rate near 1.15% with 95% confidence, not at zero, which at the reported 10,000 sessions per day is roughly 115 escalations daily; at a 0.1% production prevalence the same rate drops alert precision to about 5%, and a perfect recall barely moves it. The paper's own deployment section agrees, labelling the analyst review queue 34% true positive, 17% true positive but non-malicious, and 49% false positive. Three source-level findings are recorded rather than smoothed over: the triage ablation's 0.805 recall is not attainable as any integer count out of 42 attacks while the cost and latency deltas beside it reproduce exactly; the abstract's "2 to 4x better F1" understates its own widest gap at 4.49x; and the repository ships 303 tasks against the paper's 302. Prevention and the offline Explorer are described but not released, and two of the four baselines are paper-only numbers after their code was removed for licensing.

• New page: The RL orchestrator control loop, on the component that sits between rollout generation and the trainer in an asynchronous post-training run, checked against AReaL at commit d31701c. The executed block drives the real admission formula, min(max_concurrent - running, (S + version + 1) * B - (accepted + running)), through full runs and breaks it three ways. With uniform completion times the configured bound is exact and tight. With variable completion times, observed age overshoots it (7 versions at a configured 4) because rollouts finish out of admission order, and the mean still tracks the configured value, so this hides in aggregate metrics. Removing the staleness term makes off-policy age come out at exactly concurrency / batch_size - 1, which turns a fleet-sizing decision into a data-distribution decision. And resuming a checkpoint at version 40 with a zeroed accepted counter opens 344 admission slots instead of 24 and produces data 42 versions stale where 2 was configured; a concurrency cap hides the size of the burst without restoring the bound. The page also covers per-token version stamping under partial rollouts (head, tail, and a run-length encoding, which is why the knob bounds the oldest contributing version rather than an average), the admission-side against ship-side choice for where to enforce the bound, and the drain and empty-batch guards that fail silently when they are missing.

2026-08-02

• New page: Privacy-aware split inference over a WAN, covering arXiv 2602.16760 and its released implementation. Refitting the published 7B projection recovers the sequential path's measured 43.0 ms fixed overhead but an impossible 32.8 ms for lookahead, while the paper's ablation gives 52.1 to 72.6 ms for lookahead against 42.5 ms sequential. Re-projecting with measured lookahead overhead lowers the 20 ms result from 18.6 to 16.8 tok/s and reverses the reported RTT trend. The 221-sample inversion marginals reconstruct exactly, but paired token outcomes are unavailable, so they do not support the independent-samples significance test previously reported here. The executed transport model counts both prefill directions, yielding 537 MB for a 32k 7B prompt and 8.59 GB for a 128k projected-405B prompt; the released 20 MiB WebSocket cap and dense per-step attention masks make long-context operation impossible without protocol changes. The privacy claim is scoped to a semi-honest cloud that lacks the locally adapted layer weights.

• New page: How RAG paradigms scale, covering arXiv 2607.26497 and its 28-tier nested enterprise corpus. BM25 first overtakes the file agent at the measured 8.5M-token tier; at the previously omitted shared 66,932-document tier, the reported scores are 59.5 for BM25, 56.7 for the file agent, 38.1 for DenseRAG, and 30.9 for LinearRAG. Endpoint arithmetic approximates the paper's graph-build projections but cannot reproduce fitted coefficients that were not published, and the projections extend up to about 200 times beyond measured scale. BM25 is Pareto-undominated at the four evaluated generative-model-token horizons, with embedding, latency, and hardware cost explicitly outside that result. The strongest control changes both the tool interface and its instructions, taking the agent from 36.9 to 69.4 while calls and tokens fall sharply; the main text and appendix disagree on whether the prompt was adapted. Session, protocol, and judge deltas are reported as sensitivity evidence rather than converted into an unsupported universal resolution floor.

• New page: Cookbook: running DeepSeek-V4-Flash on your own hardware, audited against pinned model, GGUF, llama.cpp, vLLM, and SGLang revisions. Summing all 48 safetensors headers reproduces the 166,878,536,440-byte index exactly: 147.17 GB is target routed-expert weight, 10.27 GB is three attached DSpark draft blocks, and 9.44 GB is everything else. The official 154.99 GB GGUF omits all 4,705 attached draft tensors and has a 7.82 GB non-routed weight floor. llama.cpp's --cpu-moe executes expert operations on the CPU rather than streaming selected weights to the GPU, so the 3.45 GB selected-weight figure supports a host-memory ceiling model, not PCIe token-rate claims. The page distinguishes target-model MTP depth one from the three physical draft blocks, vLLM's seven-token example from the checkpoint's five-token DSpark default, and generic Transformers prompt failure from current engines' native DeepSeek-V4 renderers. Both Python blocks, including the revision-pinned encoder probe, pass; the GPU launch templates remain explicitly unexecuted.

• New page: Auditing autonomous research agents, covering arXiv 2605.26340 and its four forensic checks across 75 papers. The reported 0/337 versus 42/201 reference rates are strong aggregate evidence, but references are clustered within 30 papers, so the page labels the row-level Fisher value as naive rather than treating it as a valid decisive test. The abstract's uniqueness is corrected to the conjunction of zero hallucinated references, perfect score verification, and highest method-code alignment; Sakana AI-Scientist v2 also records zero hallucinated references alone. Table 3 contradicts the claim that every system matches the human on every task, while ScientistOne itself does beat the human on all five. Its 9.4% and 4.0% gaps to the best TXN and LLM-SQL entries are descriptive and do not establish a causal cost of verifiability. Soundness remains every system's lowest review dimension, and the automated-review author overlap and shared model family are separated from the more mechanical integrity checks.

• New page: Reconstructing agent memory instead of replaying it, covering arXiv 2607.28272 and the MemHarness release at commit 0cafec5. The paper assigns trajectory advantage to reconstruction and action tokens, but both released benchmark scripts disable dedicated adaptor training and do not directly optimize reconstruction completions; this blocks a code-level reproduction of the stated joint-token mechanism. Memory-off inference retains 75.0% and 78.9% of the point gap over plain GRPO in two single-run benchmarks, which motivates a serving ablation rather than proving that retrieval is optional. The released utility counter credits every retrieved memory in a successful episode, including guidance later rejected by reconstruction, so it measures association rather than acceptance-aware causal utility. The duplicated OOD row is recorded as a reporting concern rather than proof of reuse, and the numerical identity between RL + Raw Memory and EvolveR does not identify them as the same run. Raw replay trails GRPO by 6.3 points on ALFWorld but leads by 6.5 on WebShop, preventing a generic negative-transfer claim.

• Updates from the hidden-technical-debt series on agent runtime, harness, and evaluation infrastructure. Agent sandboxing and isolation gains the six-component runtime definition (compute substrate, snapshot-capable filesystem, tool surface, network boundary, state model, lifecycle controller), a cold-start comparison across containers, gVisor, Kata, Firecracker, and V8 isolates, and the training-versus-production runtime divergence table with its three resolution paths: co-locate, define a versioned runtime contract, or inject 5 to 10% tool errors during training so the policy learns recovery. Agent harness architecture gains the training-versus-production harness table and the symmetric failure pair (over-shackling in training produces a policy helpless outside the allowlist; under-fencing in production ships the training action space and patches it after the first injection), the three optimisation surfaces ordered by cost to change, and the removability test: a component you can delete in an hour is an option, one that takes a week is debt. It also records that harness benchmark results are statements about a harness-model pair and do not survive a model swap, cross-referenced to the three-harness spread measured in the RAG scaling study. Evaluating agents gains the control-plane and data-plane split, the five evaluation surfaces (output, trace, memory, environment, internals) with the failure modes each catches, the minimum trace-record field list, and the checkpoint-branching experiment designs that make perturbation and ablation cheap once state mutation is checkpointed.

2026-08-01

• New page: Frontis-MA1 and OpenMLE, covering arXiv 2607.28568 and the released FrontisAI/OpenRSI stack at commit c16ebd6. The page treats the three layers as separable infrastructure (task gym, operator training, evolutionary search) and reconstructs the cluster topology the paper's headline hides: the "12 hours on one RTX 4090 at 12 GB" budget is the sandbox allowance, the shipped profile also carries an 18-hour model_plus_sandbox_time_budget covering inference, and reproduction needs 1x8 GPUs for the 30B model and 2x8 for the 35B. Appendix B.5 is reimplemented in numpy and cross-checked against the shipped adaptive_reward_advantage_utils.py over 500 random reward groups, agreeing to 1e-9; the executed block shows why adaptive bounds matter (sixteen genuinely different log-loss programs span 0.44% of the reward range under leaderboard bounds and 80% under policy-derived ones) and puts the paper's 4.0x upper-tail figure inside a measured 1.67x to 6.34x band. Seven findings come out of the source rather than the abstract. The frontier comparison "exceeds GPT-5.5 + Codex" is 2 medal outcomes out of 66 against a single-run reference on a row whose own standard deviation is 2.8x the margin, while the post-training comparison it sits next to is a solid 3.8 standard errors. The claim that the domain harness "consistently" beats general coding scaffolds is contradicted by the paper's own GLM-5.2 row, where Claude Code wins Valid Rate and Human Rank. The "held-out" transfer benchmark shares 11 of its 17 authors with this paper, including both project leaders and the corresponding author, while Figure 2 describes the evaluation as third-party. The sandbox job service the paper describes and Table 11 ticks is not in the repository: only three httpx clients for /api/v1/jobs are, and the one file named sandbox_exec.py is 29 lines of in-process importlib. All four shipped launchers enable an undocumented flag that returns reward exactly 1.0 whenever a rollout group has a single finite score, executed and confirmed at a score of 0.001, which inverts the paper's own upper-tail principle on precisely the sparse-success tasks RL is meant to fix. The 84.3% search-efficiency headline is per-token; the absolute new-best count rose 7.4%. And the whole release is CC BY-NC 4.0.

2026-07-31

• New page: The filesystem as agent memory, covering arXiv 2607.26637 framed by the code-as-agent-harness survey (arXiv 2605.18747). Rather than checking the study's tables, the page builds a working memory filesystem on real files: the management agent's integration contract (route, dedupe, supersede), a search agent that navigates the taxonomy, and the two degenerate curation behaviours the study documents, with every cost derived from bytes actually read off disk. Four results come out of the model, two of them sharper than the study states. First, "organised" is not one thing: a store sharded along both query dimensions reads 3.4x more as the corpus grows 16x and reaches a 96.7% saving against a verbatim dump, while a topic-only consolidated store reads 15.6x more and stays pinned at 54%, so the hierarchy has to partition along the dimension the query names. Second, sharding converts file-read cost into directory-listing cost, and at the largest corpus modelled the listing already dominates, which makes fanout a design constraint. Third, removing supersede marking takes answerability from 16/16 to 0/16 with nothing lost from disk, because organising a store destroys the chronology a verbatim dump uses to resolve changed facts by recency. Fourth, taxonomy erosion splits a subject's history across folders and breaks the file-scoped supersede invariant, producing 33 confidently wrong answers and zero misses while retrieval gets 25% cheaper; silent condensation likewise shrinks the store to 29% while raising per-query cost 16%. Both defects move cost and miss rate in the reassuring direction, so taxonomy adherence is the only leading indicator.

2026-07-30

• Correction pass across eleven pages, driven by an external source review. Each item narrows an over-general claim or adds evidence that was missing, and every cited source was re-verified against its primary text before the edit. Kimi K3 serving: the TP=32 conclusion is now scoped to the SGLang recipe on 32x H100-80GB, since Moonshot officially supports vLLM, SGLang and TokenSpeed, and the divisibility result (architectural) is separated from the memory result (hardware and engine specific). Reasoning-effort control: the "cheapest quality knob" absolute is gone, replaced by the manipulation-check finding from arXiv 2605.16938 that on GPT-OSS the effort parameter behaves as an upper budget rather than a real-time dial, with a high-to-low token fold change near 1.0 on arithmetic tasks. Agentic kernel generation: adds arXiv 2607.17979, where Agent-Assisted kernels beat Full-Agent artifacts across five FlashInfer-contest operators on B200, with two autonomous artifacts falling below the vendor baseline at 0.27x and 0.83x, corroborating this page's guarded-fallback finding from the opposite direction. Agentic context management: separates byte-exact archival storage from retrieval, with executed assertions showing storage recall of 1.0 alongside zero retrieval recall for queries that share no surface term with the archived message. GEPA: adds the project's documented seed-prompt guidance and the structural consequence that a flat-scoring seed cannot pass the strict minibatch improvement gate. Hybrid serving: distinguishes exact state checkpoints from HYPIC's near-exact segment composition (arXiv 2607.01299), including its measured 1.71-point quality gap from full recompute. Training data I/O: replaces the GIL and sharding absolutes with the documented CPython position (threads remain appropriate for I/O-bound work, free-threaded builds are experimental and off by default) and names the three costs of sharding. Agentic vulnerability scanning: append-only storage now enables rather than provides safe resumability, with the additional conditions stated. Experience distillation: the 3.8% SFT figure is bounded to the paper's exploration-heavy trajectories. SDPO: the scoring-cost claim is qualified by the paper's own limitation that it grows for smaller models with shorter generations, and adds the forgetting comparison where the off-policy SFT baseline costs 2x the generations, underperforms, and forgets worse. Recursive language models: adds the depth 0-3 ablation from arXiv 2512.24601, where depth 0 beats every sub-calling variant on CodeQA, the advantage over base GPT-5 is consistent only beyond 2^14 tokens, and RLM cost is bimodal (median cheaper, mean higher).

• New page: GEPA: reflective prompt evolution, covering arXiv 2507.19457 (ICLR 2026 Oral) and the reference implementation gepa-ai/gepa at commit 0310bb7b4952. The Pareto candidate selector is ported faithfully from gepa_utils.py and validated. Its domination test is not classical Pareto dominance: candidates are considered from lowest aggregate score upward and removed only while every validation front they represent retains another active member. A final survivor is the last representative of at least one front, not necessarily an original sole leader; exact aggregate ties inherit front traversal order. The executed block also prices the two-stage acceptance gate (4,000 rollouts against 21,000 at a 15% acceptance rate, with the crossover at b/n_pareto) and audits Table 1 by reproducing all four published aggregates from the per-task cells. Three findings from that audit: the "35x fewer rollouts" headline is rollouts-to-best-candidate (678 against GRPO's 24,000), where the total budget ratio on the same task is 6.7x; MIPROv2 regresses below the unoptimized baseline on three of six tasks, dropping 7.33 points on AIME-2025; and GEPA+Merge is worse than plain GEPA on three tasks, falling to 28.23 on IFBench, below both the 36.90 baseline and GRPO's 35.88, which contradicts the table caption's claim that both variants beat GRPO everywhere except AIME. The page carries the real gepa.optimize, dspy.GEPA, and optimize_anything APIs and the GEPAAdapter contract, and separates peer-reviewed results from project-blog and vendor claims.

• New page: Reasoning-effort control, covering how low/high/max effort modes are actually trained, from L1/LCPO (arXiv 2503.04697), the Kimi k1.5 group length penalty and long2short (arXiv 2501.12599), the Kimi K2.5 Toggle alternation (arXiv 2602.02276), Kimi K3's per-problem budget override, and DeepSeek-V4's effort specialists (arXiv 2606.19348). All four published reward shapes are implemented and asserted. The executed block surfaces three traps: under LCPO-Exact a correct answer more than 1/alpha tokens off target scores below a wrong answer that hits the target (3,333 tokens at alpha=3e-4); LCPO-Max returns exactly zero for both wrong answers and correct answers over budget, so a GRPO group of either gives zero advantage; and the K3 override is a training-reward cliff, not a serving cap, where one token past tau*b0 flips a correct long-horizon trajectory to -1. The cost model separates per-request feasibility from complete correct answers per million generated tokens and explicitly excludes prefill and fixed serving costs.

• New page: Multi-teacher on-policy distillation, from MOPD (arXiv 2606.30406), MiMo-V2-Flash (arXiv 2601.02780), and Kimi K3's consolidation of nine domain-by-effort experts. All eight published normalised scores in Table 2 are reproduced from the paper's own raw accuracy columns, which pins the aggregation order: averaging within a domain first gives 0.9373, averaging across the five benchmarks gives 0.9226. The executed block proves the policy-gradient form is exact rather than approximate (finite-difference gradient matches to 1e-7 because the score-function expectation vanishes) and shows why the top-k loss needs its bias-correction term: the naive truncated reverse KL can be driven below the teacher's own value by moving student mass off the top-k set. Table 4 supplies one controlled teacher replacement: the normalised score falls from 0.9373 to 0.6003 with policy gradient and to -1.1898 with top-k. That supports a conservative same-origin default and a KL preflight, not a universal ancestry rule.

• New page: Cookbook: serving Kimi K3 across four nodes, from the Kimi K3 technical report plus a published 32x H100 deployment walkthrough. The executed block derives that the common divisors of 96 attention heads and 7,168 hidden dimension are exactly {1,2,4,8,16,32} and that only TP=32 survives the checkpoint-size filter, since TP=16 needs a 90.86 GiB checkpoint share per GPU against 79.19 GiB usable. It keeps the 45.43 GiB TP=32 checkpoint share separate from the walkthrough's 59.63 GB runtime allocator observation, reproduces the report's fine-grained prefix-cache example, and shows why MXFP4 experts need a W4A16 kernel on Hopper. The multi-node launch is a complete but explicitly unexecuted third-party reference template whose container image must be resolved to an immutable digest.

• New page: DeepSeek-V4: compressed sparse attention, from arXiv 2606.19348. It separates CSA (compress every 4 tokens, then select top-k) from HCA (compress every 128, no selection) and shows the layer mix, not either compression rate, is the design knob: at 1M context a CSA layer holds 250,000 entries against HCA's 7,812. The executed block identifies that mHC's stated manifold is the set of doubly stochastic matrices and confirms the paper-order float32 Sinkhorn-Knopp loop reaches a 7.15e-7 residual after 20 iterations, approximately rather than exactly. It also makes the efficiency claim's regime explicit: the core attention stays at top-k while the lightning indexer still scores every preceding compressed block, so total per-token work is not constant with context.

• New page: Latent reasoning with outcome-reward RL, from SLPO (arXiv 2607.19691v2). It covers the Gaussian surrogate over MC-dropout latent transitions and the hazard-rate stopping head that converts a fixed thinking budget into a variable horizon. All 24 comparisons behind the "improves Pass@8 and Pass@16 in all 12 settings" claim are checked, and the executed block records the caveat that belongs beside it: mean deterministic-accuracy gain is 1.20 points against 2.53 for Pass@8, with one published cell at exactly 0.00. It also shows the variance estimator's 1/(K*d) normalisation is biased low by exactly (K-1)/K and separates the paper's sub-stochastic first-stop training mass (0.90424 plus 0.09576 no-stop mass) from capped generation, which absorbs the remainder at the final step.

• New page: Hierarchical agent decomposition, from Matryoshka Agent (arXiv 2607.25090). It covers the Orchestrator/Sub-Agent/Tool split, the Solution Refinement Tree, and branch-level downstream-return labelling. The executed block constructs the slow-starter case where immediate-score and downstream-return labelling genuinely disagree, and makes the tree cost explicit: depth-4 binary expansion executes 30 non-root attempts to produce 16 terminal leaves and 15 common-prefix preference pairs. It confirms ranking-NCE is exactly DPO at two candidates and reads the HumanRank labels correctly: SFT adds 0.0448, RL adds 0.0299, the controlled all-o4 architecture gain is 0.0633, and the trained 4B Orchestrator remains 0.0105 below the all-o4 Matryoshka configuration. The table does not isolate a numerical weak-Sub-Agent effect.

• New page: Agentic context management, from ACM (arXiv 2607.23809). It contrasts agent-invoked lossless compression (manage_context offloads raw messages to disk under a summary id; query_memory retrieves them) against summary agents that discard history on a threshold trigger. The audit of Table 2 records three things the abstract does not: the headline 27/16/8% are relative improvements, worth 15.7, 5.8 and 4.1 absolute points; the stated "around 20%" peak-token reduction is 14.3, 10.9 and 15.3% for the post-trained model, a mean of 13.5%, with only the untrained Base variant on SWE-Bench reaching 22.0%; and five of nine baseline cells raise peak tokens above plain ReAct, ReSum on DeepSearchQA by 72%.

• New page: Agentic CUDA kernel generation, from Kernel Forge (arXiv 2607.24762). It covers MCTS with progressive widening, the minimisation UCT, the compile-and-launch repair loop, and guarded fallback. The executed Amdahl analysis identifies the paper's percentages as shares of the captured operator region, not full-model latency: the 1.52x headline operator contributes 1.00041x inside that region, ResNet-50 reaches about 1.033x there, and Stable Diffusion 3.5 brackets between 1.005x and 1.045x with the guard but is about 14.2x slower without it. Full-model examples are explicitly parameterized by the missing region fraction. The UCT assertions use only realizable child/parent visit counts, and the inclusive widening boundary is tested directly.

• Updated: Self-improving harnesses now covers RHI (arXiv 2607.15524) and AREX (arXiv 2607.21461). The RHI section specifies the prompt-level harness representation, the accumulated pairwise-preference history as a momentum-semantic signal, and the improvement-rate stopping rule, with an executed block showing the rule converges in a few iterations and that accumulating comparisons sharpens the direction only when each comparison beats chance: a biased judge gets confidently wrong faster. The headline result connects to reasoning-effort control, since a better harness at low reasoning effort exceeded the same model at maximum effort while cutting cost by up to 60%, with output tokens nearly constant. The AREX section covers the discovery-verification asymmetry and the constraint-wise audit loop.

• Updated: Delta weight sync gains a section on optimizing the full refit path, from NeMo-RL discussion 3315. It covers topology-aware NCCL broadcast, coalesced collectives, and CUDA-stream overlap of gather with broadcast, with reported refit times falling from 15s to 1.5s (Qwen3 30B-A3B), 23s to 5s (235B-A22B), and 70s to 14s (DSV3). The executed model explains why the smallest model got the largest speedup: in the modelled MoE case 96% of naive broadcast time is collective-launch overhead, so coalescing is worth about 24x there and about 2% on a dense checkpoint, with the crossover where one tensor's bytes take as long as one launch. It also bounds phase overlap at 2x and computes what a 10x refit is worth per RL step at 3, 15 and 40% refit shares.

• Updated: Evaluating AI agents on cybersecurity tasks gains a section on reading vendor scores, prompted by Microsoft's MAI-Cyber-1-Flash and MDASH announcement. It records the architecture claim (a compact specialist absorbing up to 90% of tasks with escalation to a larger model, at a claimed 50% cost saving) and shows the 90/10 split and the saving are internally consistent only if the specialist is about 2.25x cheaper per task. It then separates the announced 95.95% on CyberGym from the roughly 22% level-1 reproduction result already on the page: these are different tasks at unstated difficulty, more than 4x apart, and the announcement states no level, subset, or scoring definition and ships no paper or weights.

• New page: Recursive language models, from the alphaXiv blog post "Reinforcing Recursive Language Models" and the released SkyRL implementation. It models the blog's inherited-root-advantage objective separately from the code: the current multi-paper environment gives child depths reward zero, the generator retains independently assigned child rewards, and no parent-advantage overwrite was found despite a contradictory config docstring. The executed block validates 1/k normalization on regular and irregular trees, shows that the recurrence is one plus mean child-subtree weight rather than universally linear in maximum depth, and models finite-concurrency service waves. The blog remains a single reported experiment with no seeds or ablations; the release mismatch is surfaced rather than silently resolved.

• New page: Training data I/O, covering SPDL (arXiv 2504.20067) and the AIStore/WebDataset work (arXiv 2001.01858). It separates the layout decision from the concurrency decision and shows both are sharply structured: thread-pool scaling is Amdahl's law with the GIL-held fraction as the serial part, so a pipeline at f=0.50 gains 21% from an eightfold thread increase against a hard 2x ceiling, and sharding converts a latency-bound job (97.8% of time in per-object latency at one file per sample) into a bandwidth-bound one that is within 0.44% of the floor at 10,000 samples per shard while cutting shufflable units from a million to a hundred. The page states plainly that under its modelled IPC cost processes keep the throughput lead until the GIL-release fraction exceeds 0.989, so the case for threads rests on CPU and resident memory rather than throughput alone.

• Updated: On-policy distillation now covers SDAR (arXiv 2605.15155) alongside GKD, OPD², and SEED. It specifies SDAR's detached gated auxiliary loss and its three gate constructions, and adds a runnable block computing all three on the same signals. Two results are recorded: soft-OR gating amplifies a token the privileged teacher rejected whenever the student was uncertain there, which is exactly what gap gating exists to suppress, and the beta parameter degenerates at both ends, to a uniform 0.5 gate as beta approaches zero and to a hard sign filter as it grows. The page also records that SDAR's first author is an author of SEED and that the gap gate matches SEED's functional form, so the two are one line of work rather than independent replications.

• New page: RL with rich feedback (SDPO), from arXiv 2601.20802 and official source commit 7c457fc1. It covers treating the policy conditioned on environment feedback as its own teacher and re-scoring the original rollout rather than sampling a new one. The executed validation treats top-K as a K-plus-tail coarsening applied after choosing reverse KL or JS; an adversary that redistributes fixed probability within the tail leaves all coarsened divergences at zero while full reverse KL, forward KL, and JS remain nonzero. It also demonstrates that identical weights under different contexts produce nonzero JS, so EMA controls parameter motion rather than creating the teacher signal, and replaces invented throughput with exposed-overhead algebra whose inputs must be measured.

• New page: Zero RL at trillion scale, from Ring-Zero (arXiv 2607.12395). It separates the systems problems from the algorithm: the Megatron/SGLang logit gap that destabilizes the importance ratio, targeted FP32 on the attention softmax and LM head, all-to-all context parallelism for MLA layers and a single AllGather for linear-attention layers in place of ring attention, and the four-stage pipeline with a self-distillation reset. The executed model shows BF16 softmax barely moves the mean ratio error while widening p99 about 6x and the maximum 7.3x, and models centered one-sigma trajectory drift as std(log ratio) * sqrt(L). A four-case sign test corrects the paper's blanket PPO description: low-ratio positive-advantage tokens remain active, while the negative-advantage half clips; the stop-gradient objective retains a separate every-token coverage property.

• New page: Internalizing agent experience, covering Experience Distillation (arXiv 2607.21051) and skill-library approaches (SkillRL, arXiv 2602.08234). It explains why supervised fine-tuning on raw trajectories recovers only 3.8% of the in-context gain where one-step branch distillation recovers at least 64.8%, and derives the branch-packing economics: the naive one-example-per-branch-point form is quadratic in trajectory length, processing 434,600 tokens to supervise 7,200 on a 40-turn trajectory against 28,750 when packed, a 15.1x reduction that grows to 59.9x at 160 turns. Including the initial observation, the teacher-generated context share is 23.1%, 25.0%, and 25.3% at 5, 40, and 160 turns and approaches a 25.4% asymptote rather than remaining exactly invariant.

• New page: Agentic vulnerability scanning at repository scale, built on vercel-labs/deepsec at commit 76c03d67b1a47882c9e4bb738b57253155307945. It covers the free-regex-then-paid-agent funnel, the append-only FileRecord model that gives resumability and multi-worker scale, revalidation for false-positive suppression, the fact that source snippets go to the configured provider, and the sandbox isolation posture. The standalone Bun 1.3.14 audit loads all 200 matcher modules and executes all 1,830 declared examples; 1,829 match on a candidate path. The single failure is a real regex defect, a leading \b before a quote character class in framework-server-action. A second executed JS block reproduces the boundary failure and verifies the corrected pattern. The audit also shows framework-untrusted-fetch gating more narrowly inside match() than its filePatterns advertise.

• New page: Anatomy of an autonomous agent intrusion (July 2026), a case study of the Hugging Face and OpenAI incident accounts. It distinguishes the roughly 4.5-day reconstructed campaign from roughly 2.5 days inside Hugging Face, traces the two dataset-loader injection vectors, and follows the metadata-to-cluster escalation through a shared system:masters connector credential. The executed analysis finds the phase-table label count 1,092 below the daily action count but does not guess whether omitted labels, overlap, or both explain the net gap. It reconstructs the staging encoding, measures 1.7 token-TTL intervals after first mint, and correctly treats the 271 lateral-movement phase labels as insufficient to infer detector precision. OpenAI's account was retrieved and its still-evolving status is explicit.

• New page: Serving hybrid recurrent plus full-attention models, from vLLM's Kimi K3 day-0 release post. It covers the hybrid cache manager holding paged KV and constant-size recurrent state under one scheduler, why in-place state updates break ordinary prefix reuse, the interval-based and Marconi-style retention policies, multi-tier partial-hit reconciliation, and the tail-zeroing correctness requirement when the state crosses a disaggregation boundary under heterogeneous tensor parallelism. All seven cited vLLM pull requests were checked and are merged, with dates recorded. The executed cache-economics model derives the state-versus-KV crossover, the square-root optimum for the checkpoint interval, and an adversarial case where cache-on-second-hit collects nothing because every prefix is seen exactly twice.

• New page: Attention-FFN disaggregation (AFD) for MoE serving, covering arXiv 2605.28302. It specifies the all-pairs bipartite pairing, attention-side routing that removes every FFN-to-FFN collective, the FusedMoE.forward_pre_routed expert-kernel bypass, the four-stage microbatch pipeline, and frequency-ordered placement of per-layer traffic versus the once-per-request KV transfer. Its executed model restricts the paper's Equation 1 to its three- or four-microbatch steady-state domain and handles M=1 as serial latency. It adds a KV-capacity result the paper does not state: AFD's memory effect flips sign with the split, so FFN-heavy layouts can leave less total KV capacity than an aggregated one. The page separates the abstract's infeasibility claim from the body, which credits aggregated chunked prefill for most throughput panels and attributes part of the non-AFD infeasibility to a narrow parallelism search. Engine support was checked directly: vLLM PR #29772 is open and unmerged, PR #33961 is an open draft, and the SGLang RFC is unmerged.

2026-07-20

• New reference page: Inference system map, separating API contracts, prefill and decode, schedulers, KV state, kernels, parallelism, quantization, speculation, disaggregation, routing, admission, autoscaling, observability, rollout, security, and cost. Four Mermaid diagrams map the conceptual dependencies, request path, distributed topology, and replica lifecycle. A real llama.cpp server was built from tag b10069, loaded with a pinned SmolLM2 GGUF model, and tested through health, model discovery, chat generation, usage accounting, and malformed JSON; the page preserves its exact output. Current SGLang v0.5.15.post1 and vLLM v0.25.1 GPU launch paths use independently verified OCI digests and an immutable Qwen3 revision, with the no-GPU execution boundary stated.

• Updated the RL weight-synchronization set after an independent review of PULSE, vLLM PR #40096, verl PR #6974, slime's main-branch disk protocol, the Composer 2 report, Fireworks' deployment account, TRL PR #5417, and the Hugging Face delta-sync article. Delta weight sync now separates measured BF16 compute-visible sparsity from byte-delta and runtime-tensor protocols, records release and topology boundaries, and validates index-plus-value integrity and exact overwrite reconstruction. Focused updates cover verl, slime, TRL, async RL systems, GRPO variants, GRPO, DiLoCo, the RL library index, and the RL cluster bring-up recipe. The review corrects released versus main-only versus unmerged status, slime's patched /pull_weights requirement and current hooks, verl's executable config inconsistencies, PULSE anchor and BF16-view semantics, Composer 2's four-service multi-region architecture, and the extrapolated status of TRL's 405B figures.

• New reference page: Post-training system map, separating SFT, DPO, RL, RLVR, GRPO, LoRA, QLoRA, trainer parallelism, rollout strategy, actor/learner placement, synchronization, precision planes, and experiment control. Four Mermaid diagrams map the conceptual taxonomy, end-to-end lifecycle, distributed actor/learner system with nested prefill/decode inference, and the outer autoresearch-rl loop. The page covers FSDP and other trainer axes, colocated and disaggregated synchronous or asynchronous RL, rollout controls, quantized actors and KV caches, delta-sync boundaries, and serving conversion. Its standard-library architecture planner and rollout rate model were executed with boundary and failure cases, and the pasted output was compared byte-for-byte.

• New page: Harness Handbook: behavior-first code localization, covering arXiv 2607.13285 and the project repository at commit dc222e64d2a2bddb5b293ba7b93d4a54aeb32e38. The page specifies the L1/L2/L3 handbook, state registers, Behavior-Guided Progressive Disclosure, locator revalidation, resynchronization, deployment gates, and a runnable adversarial routing model. The Python-only static phase was executed against the generator's own 37-file source; the advertised automatic multi-language path instead reproduced a Tree-sitter Node.kind crash. The paper's planning-only improvements are separated from patch correctness, and its 3.3-point Figure 5 result is preserved despite a conflicting 16.3-point lower bound in the prose. The public repository omits the paper's benchmark artifacts and target revisions.
• Updated: On-policy distillation now covers On-Policy Delta Distillation (arXiv 2607.15161) and SEED (arXiv 2607.14777). It derives and executes the OPD² centered teacher/base advantage and SEED's sigmoid-gated sampled-token loss, adds architecture, selection, cluster-capacity, monitoring, and adversarial failure guidance, and distinguishes reported gains from their evidence limits. The OPD² repository contains no released implementation. SEED's core loss matches its source at commit 2cf2fadca3c5aba28da68e8e1405182ba8d90e6c, but public launchers disagree with the paper table on update counts and Search KL coefficient.

2026-07-18

• Updated the cookbook-depth remediation after an independent adversarial pass. NVML now accumulates nvml_field_read_errors_total monotonically once per scrape, keeps Prometheus rendering side-effect-free, propagates watcher-detected GPU loss through the reconnect loop, and emits a byte-stable executed transcript. STRATUS replaces the crewai run Job entry point, which swallowed child failures and returned zero, with direct uv run run_crew execution that preserves a failing exit code. AOI: Autonomous Operations Intelligence separates administrator-owned OpenEBS/Prometheus bootstrap from the long-running agent identity and binds the controller ServiceAccount to namespaced permissions in the real Hotel Reservation target namespace. Warp-level CUDA primitives now returns nonzero when no device is available, so compile-only CI cannot be reported as a successful device validation. ReContext now classifies its random-weight HTTP scorer as an integration smoke test and records that its river-question ranking fails a production semantic-relevance check. Real GPU, credentialed LLM, checkpoint/rollback, and multi-host execution boundaries remain explicitly open where they were not run.

2026-07-17

• Updated, cookbook-depth audit remediation (see notes/cookbook-depth-audit-2026-07-16.md for the source review): the following pages were brought from surrogate-validated or unexecuted reference material to genuine, executed evidence, or honestly reclassified where hardware access made real execution impossible. Cloud-OpsBench: real repo clone at a pinned commit, real Pydantic tool schemas, a scripted adapter driven through the actual harness, three adversarial cases. Agentic paper replication: the released paper-replication skill installed and run end to end through a real two-target workspace to the actual completion gate, including a rejected forbidden-pattern-matching provenance record and a genuine structural-target failure/correction cycle. STRATUS: real repo clone; found and reproduced a real KubectlLinter construction bug (AttributeError on a nonexistent ResourceType.REPLICASET) and documented the real dry-run-based confinement mechanism. AOI: Autonomous Operations Intelligence: real repo clone; confirmed no whitelist artifact exists despite the paper's Table 5 claim, and a real hyperparameter drift between the paper and the shipped ObserverGRPOConfig. PithTrain: pinned commit, real config/data schemas, a real CPU-executable data-pipeline and validate-correctness skill run. NVML: a real nvidia-ml-py exporter exercising init/shutdown, Xid event registration, and batched field reads, with real, reproduced NVMLError_DriverNotLoaded output on a driverless host. PyTorch Symmetric Memory: direct package introspection of torch==2.13.0+cpu found symm_mem.get does not exist; corrected to the real hdl.get_remote_tensor/wait_signal API. Prefill-as-a-Service: reclassified as an architecture/economics study; a real lmcache==0.5.1 install found and corrected a nonexistent nixl_store adapter type. Warp-level CUDA primitives: real nvcc compilation across three architectures plus cuobjdump -sass disassembly confirming real SHFL.DOWN/VOTE.ANY/MATCH.ANY instruction emission. Kubernetes GPU-node resource isolation: a real kind cluster with the documented kubelet policy; found strict-cpu-reservation crash-loops kubelet v1.31.0; verified Node Allocatable, CPU/Memory Manager state, cgroup enforcement, a real scheduler rejection, and a real drain cycle. Datacenter interconnect and RDMA fabric metro extension: reclassified as design/reference pages, with the missing ## Architecture heading added to both. coding-harness-benchmarking: a real task pack, scorer, reset script, and randomized runner executed end to end. Recipe: Mesh LLM production deployment: a real single-node mesh-llm install and serve, a real two-process invite-token mesh with owner attestation verified, executed trust/revocation and owner-control round trips, SIGKILL peer-death and mid-request node-death transcripts, and three real defects found: a missing-libnccl install trap, a models certify gap against raw-GGUF-served models, and an invalid [owner_control] example (inherited from upstream docs/USAGE.md) that the shipped config validate rejects. ReContext: the released repository's real attention-extraction and replay code executed end to end at its pinned dependency versions. cuda-checkpoint, colibri, DFloat11, CUPTI, and transformers continuous batching gained real, executed builds, installs, or serving smoke tests at their real hardware/dependency boundaries. Six additional pages (always-on-agents-persistent-state, AOI: AI-Oriented Operations, centralized KV cache placement, The Harness Effect, knowledge distillation methods, KV cache fundamentals) received an explicit reference/concept classification sentence so their scope is not mistaken for an installable technology.

• Updated: Managed inference endpoints on SageMaker closes its cookbook-depth audit gaps. The deploy template's <TAG-FROM-CATALOG> placeholder is replaced with a real image URI resolved by executing sagemaker==3.16.0's offline registry lookup (sagemaker.core.image_uris.retrieve against the bundled huggingface-vllm.json) on 2026-07-17 and cross-checked against the DLC catalog the same day; the v2 SDK line (2.257.5, current) does not ship the GPU huggingface-vllm registry and raises FileNotFoundError, also executed. The previously promised but code-less pathways now exist as version-pinned reference templates, every call shape, waiter schedule, enum, and integer bound verified against the installed boto3==1.43.50 service models and each labeled unexecuted against AWS: the endpoint_in_service waiter with FailureReason surfacing (30 s polls, 120 attempts, verified locally), real-time invoke_endpoint with the container's actual contract (vLLM v0.21.0's schema-dispatched /invocations, optional model field, max_completion_tokens over the deprecated max_tokens, all from the tagged source), application-autoscaling registration plus target tracking on endpoint/<name>/variant/AllTraffic with the two scale-in protection knobs (DisableScaleIn, SuspendedState), the async pathway (AsyncInferenceConfig, invoke_endpoint_async with the 128,000-byte inline-Body cap and TTL/timeout ranges from the API reference), blue/green canary and linear updates with auto-rollback alarms and AWS's documented both-fleets billing warning, and teardown in dependency order (scaling policy, scalable target, endpoint via the endpoint_deleted waiter, config, model) with orphan verification via list_* and describe_scalable_targets plus the list of what deletion never removes (S3 artifacts and async payloads, log groups, alarms).

• Updated: CUPTI closes an audit gap in its development section, which previously validated only the interval-union arithmetic in Python and never exercised CUPTI itself. It now carries a complete, minimal Activity API client in C (asynchronous buffer callbacks, cuptiActivityEnable for kernel/memcpy/runtime/driver kinds, 8-byte-aligned buffers per ACTIVITY_RECORD_ALIGNMENT, record iteration terminating on CUPTI_ERROR_MAX_LIMIT_REACHED, dropped-record accounting via cuptiActivityGetNumDroppedRecords, and the flush/disable/finalize shutdown lifecycle), compiled for real with gcc -Wall -Wextra -Werror against Ubuntu 24.04's libcupti-dev 12.0.146~12.0.1-4build4 headers, with the exact compile command shown. The run on the driverless build host is reported verbatim as the real no-GPU boundary: cuptiGetVersion succeeds host-side, then cuptiActivityRegisterCallbacks fails with CUPTI_ERROR_NOT_INITIALIZED (15), which cupti_result.h defines as "CUPTI is unable to initialize its connection to the CUDA driver"; no GPU execution is claimed. Also records two header-verified traps: the kind-to-struct mapping is version-specific (CUpti_ActivityKernel2 through Kernel9 coexist in one header, with Kernel9 current for CUDA 12.0), and cupti_version.h's comment table says "v19 : CUDA Toolkit 12.0" while the shipped define is CUPTI_API_VERSION 18. Two new failure-mode rows cover the CUPTI_ERROR_NOT_INITIALIZED-at-registration signature and silent trace holes from unchecked drop counters.

2026-07-16

New page: Cloud-OpsBench: a reproducible State Snapshot benchmark for agentic RCA, on arXiv 2603.00468 (CUHK/Sun Yat-sen University): 452 Kubernetes fault cases across 40 fault types replace live-cluster evaluation with a frozen "State Snapshot" digital twin (a Generator/Executor/Verifier pipeline pre-computes roughly 487 tool-call responses per case against a Google Online Boutique testbed), scored on both outcome accuracy and process-centric metrics (trajectory alignment, tool relevance/coverage, invalid-action and zero-tool-diagnosis rates) across seven evaluated LLMs. Carries an executed, assert-validated Python model of the paper's Section 4.2.2 scoring formulas (Exact/In-Order/Any-Order Match, Tool Relevance, Tool Coverage), including two adversarial cases (a causal-order-violating "lazy leap" trajectory and a schema-mismatched action). Cross-linked from AIOpsLab, agentic AIOps, and OpsAgent.

New page: DevOps-Gym: benchmarking AI agents across the DevOps cycle, on arXiv 2601.20882 (ICLR 2026, UC Santa Barbara/NUS/Berkeley/UCLA/Google): a Terminal-Bench-harnessed benchmark spanning build-and-configuration, monitoring, issue resolving, and test generation over 30+ real Java/Go repositories, plus 14 chained end-to-end pipeline tasks where full four-stage success is 0% for every evaluated agent/model pair. The best evaluated setup (Claude Code + Claude-4-Sonnet) scores 51.85% on build, 20.56% on monitoring, 23.87% on issue resolving, and 13.87% on test generation. Carries an executed, assert-validated Python model of the paper's Appendix B decontamination metrics (Levenshtein ratio, prefix match, longest common substring, sentence BLEU) and its repository-exclusion rule applied to the paper's own Table 3 numbers, exposing that Table 3 reports two different contamination aggregates (Max, Avg) without disambiguating which one gates exclusion. The page flags, with exact section/table pointers, several internal numeric inconsistencies in the paper itself: four task-count accountings (Section 3.1 prose, Appendix Table 5, the introduction's "704," and the public repo's README) that do not arithmetically reconcile with each other, and a narrative sentence in Section 4.2 that misquotes Table 1's own build-and-configuration score. Cross-linked from AIOpsLab, benchmarking coding harnesses, evaluating agents, and evaluation integrity and anti-gaming.

New page: STRATUS: transactional non-regression for autonomous SRE, on arXiv 2506.02009 (NeurIPS 2025, UIUC/IBM/Tsinghua): a four-agent LLM system (detect, diagnose, mitigate, undo) built around Transactional Non-Regression (TNR), a formalized safety property that bounds a mitigation attempt as a K=20-action transaction committing only if severity does not increase, else a stack-based Undo agent restores the exact prior state. On AIOpsLab and ITBench it solves 69.2% and 50.0% of mitigation problems with GPT-4o, at least 1.5x the next-best agent, at 3.6x the time and 4.3x the cost of the strongest baseline (which the paper says lacks STRATUS's safe-retry capability, not that it is single-shot). Carries an executed Python block reproducing Table 1's four worked TNR transaction examples with two adversarial cases (a K-bound violation, a mid-transaction crash). Cross-linked from AIOpsLab, agentic incident management, agentic AIOps, and risk-tiered approval gates.

New page: SREGym: a live benchmark for AI SRE agents with high-fidelity failure scenarios, on arXiv 2605.07161 (UIUC/University of Toronto): a live, Kubernetes-native SRE-agent benchmark of 90 curated problems built from 50 fault primitives across 139 services, with ambient noise and metastable/concurrent/correlated failure modes, holistically scored in one continuous session rather than decomposed sub-scores. Diagnosis success spans 38.9-72.6% and mitigation 57.3-78.5% across four frontier agent-model pairs (including STRATUS, which shares four authors with SREGym, disclosed on the page), collapsing to 10.3-48.7% end-to-end on SREGym-unique failure classes. Carries an executed Python block modeling the Section 2.5 checklist diagnosis-oracle formula with adversarial edge cases. Cross-linked from AIOpsLab, agentic AIOps, evaluating agents, and agentic incident management.

New page: AOI: trainable multi-agent cloud diagnosis (Autonomous Operations Intelligence), on arXiv 2603.03378: a read-write separated Observer/Probe/Executor/Compressor runtime with Observer-level GRPO training and a Failure Trajectory Closed-Loop Evolver, evaluated on AIOpsLab's 86-task benchmark. The untrained runtime alone reaches 66.3% best@5 (vs 41.9% for STRATUS), a GRPO-trained 14B Observer reaches 42.9% avg@1 on 63 held-out tasks (beating Claude Sonnet 4.5's 41.3%), and the Evolver lifts avg@5 by 4.8 points while cutting run-to-run variance by 35%. Carries two executed, assert-validated Python blocks modeling the permission matrix/Algorithm 1 main loop and the GRPO advantage/reward formulation. This is a distinct project from the other same-acronym paper in this batch, AOI: AI-Oriented Operations (arXiv 2512.13956); both pages carry an explicit disambiguation note so the two are never conflated. Cross-linked from AIOpsLab, agentic incident management, agentic AIOps, and GRPO.

New page: AOI: context-aware multi-agent operations (AI-Oriented Operations), on arXiv 2512.13956: an Observer/read-only-Probe/guarded-Executor role split tied together by a potential-based dynamic scheduler and an LLM-compressed three-layer memory, formalized as a Dec-POMDP with Lyapunov stability and information-preservation theorems. On a self-built 1,000-scenario AIOpsLab-environment simulation plus Loghub-derived logs it reports 94.2% task success, a 34.4% MTTR reduction over its strongest self-implemented baseline, and 72.4% context compression at 92.8% information preservation; no code repository is cited anywhere in the paper. Carries an executed Python block modeling the sliding-window arithmetic, the Lyapunov contraction recursion, and the Appendix A.4 risk gate, exposing that the paper's own deployed default (overlap ratio rho=0.5) sits exactly on the boundary its own Theorem 2 excludes. Explicitly disambiguated from the unrelated same-acronym AOI: Autonomous Operations Intelligence (arXiv 2603.03378). Cross-linked from AIOpsLab, agentic incident management, agentic AIOps, and agent context and memory.

New page: AIOpsDoom: subverting LLM-driven AIOps via telemetry manipulation, on arXiv 2508.06394 (RSA Conference 2025 / USENIX Security 2026): the first security analysis of LLM-driven AIOps. AIOpsDoom, a zero-privilege, fully automated attack, plants adversarial reward-hacking payloads in telemetry and hits a 90% average attack success rate over 180 trials against ReAct/Flash agents on GPT-4o/GPT-4.1 (0% for generic prompt injection), evading three prompt-injection defenses. AIOpsShield, the paper's structured-telemetry sanitizer (design-documented, not released), blocks all of them with statistically insignificant utility loss. Carries an independently written, executed Python model of AIOpsShield's two-stage taint-identification/field-abstraction design with a suffix-injection adversarial case. Cross-linked from AIOpsLab, agent security threat model, prompt-injection defense, and agentic AIOps.

New page: UnderSpecBench: action-boundary violations in underspecified DevOps instructions, on arXiv 2607.02294: a 69-task-family, 2,208-prompt benchmark isolating instruction underspecification from task difficulty, scoring Claude Code, Codex, and OpenCode with deterministic, side-effect-based oracles. 55.8-67.8% of acted runs cross a target or scope boundary; target ambiguity degrades action quality far more than blast-radius wording; OverScope concentrates on shared control-plane surfaces (59.8-77.2%) versus bounded-object ones (14.4-37.6%). Every task family is grounded in a real incident or CVE (the 2017 GitLab database deletion, CVE-2019-5736, a 2026 Cursor/Claude Opus 4.6 production-deletion incident). Carries an executed Python block reproducing the paper's Cohen's kappa human-validation result (kappa=0.860) from its published confusion matrix. The paper states a release is planned but no working repository was found as of 2026-07-16. Cross-linked from risk-tiered approval gates, agent security threat model, intent verification, and the policy engine.

New page: TRACE: turn-level reward assignment via credit estimation, on arXiv 2607.13988 (Microsoft Research/University of Wisconsin-Madison): a critic-free, dense credit-assignment method for long-horizon agentic RL that turns a frozen reference model's gold-answer log-probability into a log-ratio state value, then uses its turn-to-turn Temporal-Difference change as the turn's reward, mixed with the standard GRPO outcome advantage. On the closed-web BrowseComp-Plus benchmark it raises Qwen3-4B from 7.2 to 35.6 and Qwen3-30B-A3B from 8.4 to 42.6, pure RL with no cold-start SFT. Carries two executed, assert-validated numpy blocks: one reproduces the paper's own Appendix A.2 worked-example numbers to 1e-3; the other models the K-step TD backup with terminal-outcome fill and the GRPO zero-std advantage, with adversarial edge cases. No code or checkpoint release is linked from the paper as of 2026-07-16. Cross-linked from GRPO, reward design for RL post-training, agentic and tool-use RL, and RLVR.

Fix: DSpark speculative decoding (DeepSeek) now cites the paper's arXiv listing (arXiv:2607.05147, posted 2026-07-06) instead of only the DeepSpec repository PDF, which was the paper's sole citable location when this page was first written.

New page: Benchmarking coding harnesses on a local model, for comparing Qwen Code, Codex CLI, Claude Code, or another coding harness while holding the model, endpoint, hardware, repository, permissions, and paired task set fixed. It uses an exact two-sided McNemar/binomial test over discordant task outcomes, treats an inconclusive result as neither a win nor proof of equivalence, and reports token use, latency, and cost separately from quality. The stdlib validator covers decisive, inconclusive, all-tie, and input-failure cases. Shell and configuration examples are explicitly fast-moving reference templates checked against the current official Qwen, Ollama, OpenAI, and Anthropic documentation. Cross-linked from running local coding agents, agent harness architecture, evaluating agents, and own or rent a coding model.

New page: Speculative decoding economics: the walltime-cost model and when it stops paying, using Leviathan et al.'s Theorem 3.8 only as a low-batch analytical screen. It distinguishes expected output tokens per verification round from accepted draft tokens and requires direct latency, goodput, power, memory, and accepted-output-throughput measurements at the intended concurrency. The cost model prices a measured throughput multiplier under the same offered load and SLO, including any additional hardware allocation and its break-even bound. Current vLLM dynamic-depth behavior is documented with its tested-method, runner, and data-parallel limitations; the Vizuara EAGLE3 regression remains an unreplicated practitioner warning, not a general performance model. Cross-linked from speculative decoding, evaluating speculative decoding, and DSpark speculative decoding.

New page: Runbook: NCCL socket fallback despite InfiniBand/RoCE present, the progressing-job counterpart to the NCCL-hang runbook. It separates three evidence paths that require different remedies: NCCL selected NET/Socket; a forced verbs test failed during RDMA initialization or registration; or an expected inter-node NET/IB channel lacks /GDRDMA. The procedure checks process and file-based NCCL configuration, exact HCA selection, container verbs-device and network access, InfiniBand logical and physical states, RoCE QP/GID errors, memory-lock failures, relevant PCIe ACS bridges, NCCL_NET_GDR_LEVEL, and DMA-BUF or nvidia_peermem. Verification uses the same nccl-tests baseline and, when the direct GPU path changes, NVIDIA's CUDA DMA-BUF ib_write_bw test. Cross-linked from the NCCL-hang runbook, RDMA and RoCE performance tuning, the operational runbooks index, and the troubleshooting dispatcher.

New page: Always-on agents: persistent state, governance, and the AOEP protocol, distilled from Ding, Nannapaneni, Liu, and Zhang's survey "Always-On Agents" (arXiv:2606.30306). Gives a six-axis diagnostic framework (authority, scope, mutability, provenance, recoverability, actionability) for durable state and a ten-stage lifecycle. The paper's 435-work coding finds retrieval and writing dominant while rollback is the rarest stage at 27/435. AOEP-v0 scores governance obligations separately from negative-invariant leakage so a system that stores nothing cannot win by amnesia. A stdlib boundary model checks authority monotonicity, scope non-expansion, deletion propagation, provenance preservation, and rollback traceability against governed and adversarial trajectories. Cross-linked from context and memory, governing self-modifying agents, and identity and access.

New page: The Harness Effect: orchestration-layer token economics, covering Sayed Ali et al.'s controlled swap of two orchestration layers over the same 22 tasks and six models (Writer, Inc.; arXiv:2607.06906). It reports the paper's aggregate -41% cost, -44% latency, and -38% token results while stating that the paper supplies no confidence intervals, significance tests, or repeated-run variance. The page separates the six proposed mechanism families from measured outcomes, flags the vendor conflict of interest and single frozen baseline, and shows that the rounded $0.21 to $0.12 figures imply -42.9%, so the abstract's -41% cannot be reconstructed without unrounded inputs. Cross-linked from agentic loop economics, loop engineering, and harness architecture.

New page: ReContext (recursive evidence replay), on Zhao et al.'s training-free long-context inference method (arXiv:2607.02509). Each round obtains fresh question-cue attention from a fixed set of 16 released layer-head pairs, accumulates selected sentence spans, and constructs the final prompt as [context; evidence; question]. The page records the paper body's R=2, K=8, and context-only setup alongside the released repository's dataset-specific R, K, and candidate-source differences. It reports the paper's 0.24 to 0.30 mean accuracy result and its 62-minute versus 44-minute CLIPPER benchmark runtime without converting that batch result into a per-request decode-latency claim. A NumPy model checks the recurrence, fresh-round readout, deduplication, and an uninformative-head negative control. Cross-linked from context and memory and KV cache fundamentals.

New page: Agentic paper replication (evidence-gated reproduction), on Hans and Bilionis's Paper-replication workflow (arXiv:2607.02134): a coding-agent skill that records paper claims as targets, requires run, provenance, comparison, and report evidence before marking a target MATCHED, and gates workspace completion on that external state rather than the agent's final message. The paper reports 12 runs across four scientific-ML papers with all 158 targets matched; the page also preserves the authors' independent finding that 2 of 39 scalar anchors fell outside a fixed external threshold despite passing their workspace-specific rules. A stdlib completion-gate validator rejects numerically correct results with missing provenance or failed workspace validators. Cross-linked from evaluating agents, evaluation integrity and anti-gaming, and Workspace-Bench.

2026-07-15

New page: Prefill-as-a-Service: Cross-Datacenter Prefill with LMCache, a full read of arXiv 2604.15039 v2 (Moonshot AI/Tsinghua, the Mooncake lineage): selective offload of long uncached prefills to remote compute-dense clusters with once-per-request KV transfer over ~100 Gb/s Ethernet, viable only for hybrid-attention/MLA models. Carries an executed reproduction of the paper's profiling-fed analytical case study from its published inputs (Table 5 profiling plus the truncated log-normal workload): the 49.6% offload fraction, ~13 Gb/s egress, 3.24 req/s Lambda_max, the implied BS_max=20 decode constant, and an independent bisection of the routing threshold (17.8K vs the paper's 19.4K, deviation explained). The page separates the roughly 4x and 13x measured KV-size ratios for comparable model pairs from the paper's separate estimated 36x Ring-memory reduction, and flags that the equal-cost gain is about 15%, not 54%. It pins the LMCache MP deployment surface to commit 76744ce2514f3444e9a6fb06d67b23739fa93b51, including the MultiConnector NIXL and LMCacheMP prerequisites, hybrid-model alignment constraints, and the not-bit-exact caveat.

New page: Centralized KV Cache Placement: When a Shared Store Pays, answering "could decentralized decode use a centralized KV cache?" with a decision model for once-per-request prefix fetch, full-KV streaming, and per-layer query transfer. Carries an executed, assert-validated first-order break-even model (net_bw > kv_bytes_per_token x prefill_tok_s / compression, about 13.1 Gb/s for Llama-3.1-70B FP16 and about 3.7 Gb/s with CacheGen-class compression) plus an RTT-aware conservative bound for sequential WAN transfers. The page rejects full-KV streaming across a WAN, while retaining DistAttention as a valid low-latency-fabric design: Infinite-LLM reports roughly 0.075 to 0.36 ms query-transfer time in Figure 4 and overlaps communication with computation. PrfaaS (arXiv 2604.15039 v2) is treated as a conditional hybrid-attention/MLA case backed by a profiling-fed analytical study, not a deployed-system measurement: comparable model pairs reduce transferred KV by roughly 4x or 13x, while the separate 36x figure is an estimated Ring-memory reduction. The corresponding conditional wording is carried into Cross-WAN Model-Parallel Inference and Disaggregated Inference.

New page: Molt (NVIDIA NeMo Labs), covering NVIDIA-NeMo/labs-molt at commit d35ac06695078fabde5c48e7915b335fe849b573: NVIDIA's agentic-first, single-actor RL post-training framework built on Ray, vLLM, and AutoModel/FSDP2. Claims were checked against the implementation in molt/agents/base.py, molt/trainer/algorithm/advantage.py, and molt/models/loss.py. Two executed NumPy validations reproduce the rloo/grpo/dr_grpo advantage estimators, including PyTorch's sample-standard-deviation default for GRPO, and the token/seq/geo importance-sampling correction gate. The dependency-bound agent examples are labelled unexecuted reference templates and preserve Molt's message and tool schema instead of rebuilding a single user prompt. Added to RL libraries's landscape table, focused-pages list, and selection guidance.

Full remediation pass on notes/ai-systems-performance-engineering-book-audit-2026-07-15.md, the adversarial audit of this KB against Chris Fregly's AI Systems Performance Engineering. Every highest-priority Chapter 3/4 defect and every proposed new-content item from that audit was fixed or written, each re-verified against current primary sources (NVIDIA/Kubernetes/PyTorch/NCCL documentation, pinned upstream commits) rather than trusted from the book or from memory.

Defect fixes across 13 existing pages:

• NUMA Affinity and CPU Pinning: fixed the -1 (unknown) NUMA node handling to fail closed instead of silently binding to node 0; corrected the claim that NUMA policy does not survive exec (it does, per set_mempolicy(2)); corrected the memlock/pinned-memory fallback claim (generic mlock/CUDA pinned APIs fail outright, they do not silently substitute swappable memory).
• Host OS and Kernel Tuning for GPU Nodes: separated generic mlock limits from CUDA host-allocation errors and required callers to check CUDA return codes; fixed the io_uring/O_DIRECT conflation and the POSIX_FADV_DONTNEED-on-dirty-pages misconception; reframed THP, governors, and dirty-page budgets as workload-specific settings; added an executed CPU-list and dirty-ratio profile validator.
• Topology-Aware GPU Scheduling in Kubernetes: added the missing Memory Manager Static/reservedMemory configuration (Topology Manager policy alone does not NUMA-align memory); removed the nonexistent NCCL_PORT_RANGE variable in favor of net.ipv4.ip_local_port_range; clarified that Kubernetes only guarantees MIG slices land on one node, never one physical GPU; corrected the stale blanket "MIG disables P2P" claim to the current R570 same-GPU-only P2P support matrix.
• MIG (multi-instance GPU) and GPU Containerization Performance: fixed the MPS-shares-one-address-space error (Volta+ MPS has per-client isolated address spaces) and the MIG-P2P-blanket-unsupported claim (same-GPU P2P is supported since driver R570); stopped presenting MPS as a hard-isolation alternative to MIG.
• PyTorch CUDA Caching Allocator Tuning: softened the unqualified "most OOMs are fragmentation" claim to a testable hypothesis with no published incidence-rate backing.
• NCCL Collectives and Algorithm Selection: fixed ring all-reduce per-link volume; stopped describing tree steps as full-buffer transfers; corrected PAT to its AllGather/ReduceScatter scope; replaced deprecated channel variables with CTA controls; added an executed nccl-tests busbw correction validator.
• RDMA and RoCE Performance Tuning: stopped routine GID-index pinning; encoded NCCL_IB_TC as DSCP << 2 | ECN with ECN bits owned by the fabric policy; corrected the RoCE netdev example; put DMA-BUF ahead of legacy nvidia_peermem; added an executed one-byte traffic-class validator.
• DDP (Distributed Data Parallel): fixed the self-contradiction between "one fully overlapped all-reduce per step" and the page's own later, correct bucketing explanation; DDP issues one all-reduce per gradient bucket, and the final bucket is normally exposed after backward, not hidden.
• Runbook: NCCL Hang / Collective Stall: fixed node loops and the Gloo group passed to monitored_barrier(); corrected TORCH_NCCL_ASYNC_ERROR_HANDLING; added NCCL RAS; changed recovery to process exit plus launcher restart because PyTorch documents same-process process-group reinitialization as unsupported and untested.
• SHARP: In-Network Reduction: replaced the nonexistent NCCL_SHARP_DISABLE with the documented NCCL_COLLNET_ENABLE=0/NCCL_IBEXT_DISABLE controls.
• GPU Execution Model: SM, Warp, SIMT: corrected the "64 warps/2,048 threads held across recent generations" claim; Blackwell compute capability 12.x drops to 48 warps/1,536 threads.
• KV Cache Transfer with NIXL: pinned the core lifecycle to NIXL v1.3.1, distinguished standalone genNotif from transfer-attached notifications, preserved default pull versus optional push direction, and clarified that recompute preserves correctness while trading decode capacity for availability; added an executed request-ownership validator.
• Across 23 pages that prescribed NCCL_NET_GDR_LEVEL=SYS and/or NCCL_NVLS_ENABLE=1 as fleet-wide defaults (recipes, RL library pages, cluster orchestration pages, training pages, runbooks): reframed both as auto-selected/gated behavior in current NCCL, with the explicit override reserved for a measured, profiled exception rather than boilerplate.

New pages and sections:

• Kubernetes GPU-Node Resource Isolation: Node Allocatable, kubeReserved/systemReserved/reservedSystemCPUs, Guaranteed QoS composed with the CPU/Memory Manager, node-pressure eviction, taints, cgroup v2 I/O limits (and their absence at the pod-spec level), and NoSwap/LimitedSwap.
• Warp-Level CUDA Primitives: documents shuffle, vote, match, __activemask, and __syncwarp; computes ballot masks before divergence, guards inactive shuffle sources, and uses an executed model that rejects missing participants instead of assigning deterministic corruption to undefined behavior.
• PyTorch Symmetric Memory: distinguishes one-sided get from collective SymmMem operations, uses LOCAL_RANK for device selection, captures the returned one-shot all-reduce tensor, and retains the alpha and same-order rendezvous constraints.
• cuda-checkpoint: Process-Level GPU State Checkpoint: documents the suspend/restore workflow and current driver milestones: 550 base support, 570 process-tree and Driver API additions, 580 migration, 595 ARM, and 610 cuIpcGetMemHandle; retains the remaining UVM and shareable-handle IPC limitations.
• Dynamic Parallelism and Device-Initiated Launch: distinguishes CDP, device graph launch, and host-launched PDL; corrects the PDL signatures and dependency synchronization wording; adds an executed pending-launch budget validator.
• CUDA Toolkit and Runtime: corrects same-major cubin direction, models generic, architecture-specific, and family-specific images separately from the physical GPU target, and treats driver compatibility as a necessary precheck rather than an iff model of torch.cuda.is_available().

Two production-shaped correctness/capacity lessons added to existing pages, both with executed, self-checking Python validations rather than asserted claims.

• Speculative decoding: new section "Roll back every position-indexed live state," covering why cropping only the KV cache after a rejected draft is insufficient when a decoder maintains other state indexed by committed position. Every such state must return to the same accepted boundary before the next verification round. Includes an executed simulation that reproduces correct lockstep rollback and the stale-state failure without asserting a model-specific mechanism that the cited sources do not document. Cross-linked from a new failure-mode table row and from Cross-WAN model-parallel inference's speculative-decoding-over-WAN section.
• Cross-WAN model-parallel inference: new subsection "Size materialized dequantization buffers only when the engine creates them," under "Does MoE make this easier?", correcting the inference that FP32 accumulation implies expert-sized FP32 allocations. The runnable sizing model applies only after an allocator trace confirms that an engine materializes complete dequantized experts and keeps multiple buffers live; fused tile-dequantization kernels have a different working set. The failure-mode entry carries the same condition instead of turning the hypothetical model into a claim about vLLM or TensorRT-LLM. The existing [^wan-specdec] citation was also strengthened with the source's published throughput progression and draft-compile speedup figures for its worked example.

New page:

• Recipe: Mesh LLM Production Deployment: the operational bring-up Mesh LLM & Skippy Stage Splits does not cover: pinned installation, service supervision, private-mesh identity and shipped trust/revocation commands, release attestation's build-provenance-not-runtime-attestation limit, load-balancer health checks, deployment-time model certification, reliability guardrails, upgrade and versioned rollback, continuous monitoring, and node decommissioning. The page distinguishes a detached tmux session, which survives a normal SSH disconnect but not a process crash or reboot, from production supervision. It also states that the trust store remains local and operators must distribute the policy. None of the procedure has been executed against a live multi-node mesh in this KB, and no independent production report was found.

Build-content remediation pass: three pages whose "how to develop" sections had degraded into decision-framework prose without anything buildable were given real, source-verified code.

• P2P Transport for Decentralized Inference: added the missing What/Why/When/How structure and a buildable iroh/QUIC example with ALPN-separated control and data planes. The example binds a claimed endpoint ID to connection.remote_id(), admits only identities present in an explicit static allowlist, and removes admission state when the authenticated control connection closes. It was compiled with cargo check against iroh v1.0.2 at commit c3ccf502c3881444811fbb3a3a0eeaf850594dba. The production section identifies the remaining gaps: deadlines, per-identity quotas, distributed allowlist and revocation state, stable identity storage, concurrent membership transitions, and relay-path monitoring.
• Prefill-as-a-Service: replaced the "how to develop" bullet list with an executed, adversarially tested length-threshold router core covering the exact threshold, partial prefix reuse, and fail-toward-local behavior on a dead link. The router accepts an explicit uncached-token count from a tenant-aware cache index or native longest-prefix API. It does not derive token coverage from LMCache MP's found_keys: at commit 76744ce2514f3444e9a6fb06d67b23739fa93b51, that result aggregates rank-expanded ObjectKey entries and cannot identify the longest fully reusable prefix, especially under tensor parallelism. The page also clarifies that LMCache's prefetch endpoint performs a state-changing L2-to-L1 promotion rather than a side-effect-free lookup.
• Centralized KV Cache Placement: added an executed pre-flight fetch-vs-recompute gate built on the page's first-order break-even model and a canonical cache key. The key covers token IDs, model and tokenizer revisions, dtype, attention layout, parallel world size, KV rank and group, chunk size, and tenant salt; its tests include delimiter-collision and field-difference cases. The page separates the policy layers: the pre-flight gate chooses fetch versus recompute before transfer, while vLLM's kv_load_failure_policy decides whether a load failure aborts or recomputes, with availability and tail-latency tradeoffs.

2026-07-14

Added an MGX section to NVIDIA DGX, HGX & MGX systems (renamed from "DGX & HGX systems"), closing a gap flagged when comparing DGX/HGX/"others": MGX is NVIDIA's open, modular server reference architecture, structurally different from HGX's fixed 8-GPU SXM baseboard, an OEM picks chassis, CPU (x86, Grace, Vera, other Arm), GPU (SXM or PCIe), and networking independently. Three primary sources fetched directly: the current MGX product page, a recent NVIDIA developer blog citing $2-4M R&D savings per platform and 200+ ecosystem partners, and the original May 2023 COMPUTEX announcement citing the smaller original three-quarters-cost/six-month-time figures and naming the launch OEM partners (ASRock Rack, ASUS, GIGABYTE, Pegatron, QCT, Supermicro). The page records both sets of figures as dated snapshots rather than picking one, and adds the operational trap the modularity creates: unlike "HGX," the label "MGX" alone does not imply NVSwitch, Fabric Manager, or any specific networking is present on a given vendor SKU.

Three pages closing the datacenter-to-datacenter gap in the interconnect coverage, plus a map page tying the full stack together. Two independent research passes fetched primary sources directly (Meta's engineering blog series, NVIDIA's InfiniBand and long-haul product pages, Juniper's coherent-optics and PFC documentation, AWS/Azure interconnect docs) rather than relying on recalled figures, and the load-bearing numbers (Meta's 10x Backbone optics tiers, the NVIDIA Chicago-Ashburn training benchmark, PFC's cable-length-driven headroom mechanism) were independently re-fetched a second time before being cited.

• Networking, the DC-to-DC layer: Datacenter interconnect (DCI): dark fiber, DWDM, and private backbones (the physical layer beneath everything else: dark fiber's unamplified reach, 400ZR's 120 km and OpenZR+'s up-to-2,000 km coherent-optics tiers, Meta's distance-tiered FR/LR/ZR optics strategy and its Prometheus/BAG gigawatt-scale multi-building architecture, Google's B4, and what AWS Direct Connect/Azure ExpressRoute Global Reach actually let a tenant connect. The validated model derives the speed-of-light latency floor and cross-checks it against NVIDIA's own published Chicago-Ashburn measurement: a 9.79 ms theoretical RTT floor against a 21 ms measured figure, meaning more than half the measured latency at 1,000 km is routing and switching overhead, not distance) and RDMA/InfiniBand fabric extension across metro distances (why RoCEv2 does not simply stretch to metro distance: PFC's lossless headroom is provisioned from cable length, and the validated model shows a 40 km hop needs roughly 198x the buffer headroom a switch ASIC's 100 m default assumes. Distinguishes two genuinely different NVIDIA products that get conflated, the IB router, which segments a fabric into isolated subnets, from MetroX, which extends one fabric's reach over DWDM, and records Meta's own ATSW aggregation layer as the production RoCE answer, which accepts oversubscription rather than re-provisioning PFC for distance).
• Reference map: GPU communication protocols: a map from GPU-to-GPU to DC-to-DC (answers "how many protocols connect GPUs" directly: the number depends on which of four layers is being counted, intra-node NVLink/PCIe, intra-DC InfiniBand/RoCE, the three non-interchangeable DC-to-DC answers, physical DCI, fabric extension, and overlay, and the algorithm layer, DiLoCo-class infrequent sync, that decides whether the DC-to-DC link needs to be fast at all. Routes every reading path by use case: single-DC training, multi-DC/multi-building training, RL post-training, inference serving, and bare-metal cluster bring-up).

Three pages on the server-side tooling that inspects and observes a GPU node from underneath the tools most operators actually type. Each is grounded in a primary source that was read directly rather than recalled: NVIDIA's nvml.h (NVML_API_VERSION 13) and nv-reg.h (driver 610.43.03) were downloaded and grepped, not quoted from memory.

• Node administration: NVML: the library under nvidia-smi, DCGM and every GPU exporter (the C library every GPU tool on your node is built on, and therefore the source of your dashboards' semantics). Leads with the metric that misleads: utilization.gpu is documented in nvml.h as "Percent of time over the past sample period during which one or more kernels was executing on the GPU", so a kernel occupying one SM out of 132 reports 100%, and any capacity plan built on it is measuring the wrong thing. Records that the metric is unavailable entirely on MIG-enabled GPUs, and that a freshly loaded driver reports a spurious utilization spike caused by ECC memory scrubbing. The executed model decodes the clock-event bitmask into an alert decision and proves three traps: GpuIdle (0x1) makes an idle GPU report a nonzero mask, so if (mask) alert() pages at 3am on a machine nobody is using; a bare HwSlowdown is ambiguous by NVIDIA's own note ("May be also reported during PState or clock change"), so it needs a sustained run before it pages while the specific thermal and power-brake bits page on the first sample; and a future NVML bit must be surfaced by masking with ~ALL rather than silently dropped. Also records the partial rename verified in the header: six reasons became nvmlClocksEventReason* with same-value deprecated aliases, but HwSlowdown (0x8), HwThermalSlowdown (0x40) and HwPowerBrakeSlowdown (0x80) kept the nvmlClocksThrottleReason* spelling, so a blanket rename does not compile; that nvmlDeviceGetTemperature is now DEPRECATED(13.0) in favour of nvmlDeviceGetTemperatureV; that without persistence mode volatile ECC counts reset every time a compute app runs, which makes the counter mean something other than what the dashboard says; that nvmlDeviceGetFieldValues returns success if any field populated, so the per-field nvmlReturn must be checked or the exporter publishes undefined values; and that nvidia-ml-py is NVIDIA's binding and the distribution that actually ships the pynvml module, while the separately-named pynvml package is a deprecated redirector from 12.0.0 onward (a stale pin below 12 is the real hazard, since those versions shipped a divergent module of their own).

• CUDA tooling: CUDA-GDB: device debugging and GPU core dumps (device-aware debugging, and the post-mortem path that is the only part of it fit for a production cluster). The defining constraint is quoted from the manual: "Any GPU hitting a breakpoint will pause all the GPUs running CUDA on that system", so an interactive breakpoint on a shared 8-GPU node is a node-wide outage until the human at the prompt resumes, containable only by pinning CUDA_VISIBLE_DEVICES. Explains why the usual levers have stopped working: CUDA_LAUNCH_BLOCKING=1 cannot localize a fault inside a CUDA graph (the whole graph is one launch), and cudaErrorIllegalAddress "leaves the process in an inconsistent state and any further CUDA work will return the same error", so an in-process error handler cannot read the state it needs. Carries the production core-dump recipe with the three things that actually matter on a cluster: size (a full dump includes global memory, which vLLM measured at "hundreds of GiB" for a model that fills an 80 GB GPU), path (the default is the current working directory, which on a pod is the ephemeral layer that dies with the fault), and standing cost (arming dumps installs an error barrier constraining warp exit on every kernel, per CU_COREDUMP_NO_ERRBAR_AT_EXIT, which is why it is armed on the retry rather than on the fleet). The executed model is the debugger's focus arithmetic: it proves lane == threadIdx.x % 32 is false for any block whose x extent is not a multiple of 32 (a thread the operator reads as lane 23 is lane 15 to cuda-gdb), and that an unguarded kernel's out-of-bounds threads do not scatter but land in exactly one warp, the last of the last block, in the lanes above the remainder. Surfaces one inconsistency between NVIDIA's own sources rather than picking a side: the driver API documents the default dump filename as core.cuda.HOSTNAME.PID while the CUDA-GDB guide documents core_TIME_HOSTNAME_PID.nvcudmp. All three new pages were rewritten after an independent adversarial fact-check against the primary sources found defects in the first drafts. The most serious: the CUDA-GDB page had claimed the debugger's warp coordinate equals tid // 32, when the manual defines warpid as the "Warp index inside the current SM", a scheduler-assigned slot that no arithmetic can recover from (block, thread); the vLLM walkthrough the page already cited disproves the original claim outright, reporting sm 124 for the first thread of the first block. The executed model now computes only what the programming model fixes (the linear thread ID and the lane) and proves the block-relative warp index is not injective across co-resident blocks. Also corrected: Kepler was removed in CUDA 12.0 and Maxwell/Pascal/Volta in 13.0 (not "deprecated in 12.8"); set cuda break_on_launch defaults to none and takes four values; set cuda api_failures has five modes, and it checks API return codes, so it does not pinpoint an async kernel fault; the skip_abort flag is reported by vLLM as carrying "a significant bug, which may cause illegal memory access errors on the GPU to be ignored", which the first draft recommended without the warning; the CUPTI page had attributed DCGM's own "resource is in use" error to the profiling user, who actually sees a failed driver-resource reservation or CUPTI_ERROR_HARDWARE_BUSY; and its claim that NVIDIA publishes no overhead number was falsified by Nsight Systems' own "up to 70%" figure for Unified Memory page-fault collection. On the NVML page, one assertion in the executed block was a tautology and has been replaced with a real test of the alias identity.

• Also updated: Compute Sanitizer was brought onto the WHAT/WHY/WHEN/HOW contract it predated. It gains an Architecture section and, more usefully, a Failure modes table recording the trap that silently defeats the whole point of running it in CI: --error-exitcode defaults to 0, so a sanitizer job that detects real errors still exits zero and the pipeline goes green. Also records that the sanitizer cannot share a process with the debugger or a profiler, that -G can hide the very race being hunted, and that racecheck never sees global-memory races between blocks.

• Profiling methodology: CUPTI: the profiling interface under Nsight, PyTorch and DCGM (the layer you administer rather than call, since every nsys run, torch.profiler block and DCGM profiling metric passes through its permission model). Draws the line the docs draw: tracing is cheap, unprivileged and composable, profiling is expensive, privileged and exclusive, and nearly every operational surprise follows from that. Documents the permission surface end to end (ERR_NVGPUCTRPERM, the NVreg_RestrictProfilingToAdminUsers regkey with its required initramfs rebuild, CAP_PERFMON from R565, --cap-add=SYS_ADMIN in containers, and the R610+ nvidia-capabilities scheme that NVIDIA says will replace the regkey) and settles the default from source rather than from prose: NVIDIA's nv-reg.h comment says "0: Do not restrict GPU counters (default)" while the code five hundred lines further down compiles in NV_DEFINE_REG_ENTRY(..., 1), so the operative default is restricted and the comment is stale. Records the exclusivity that makes dcgm-exporter and an engineer's profiling run fight over one reservation, with DCGM's own dcgmi profile --pause workflow and the consequence that utilization panels go blank (DCGM publishes BLANK values while paused). The executed model reconstructs GPU busy time from an activity buffer and shows the standard implementation is wrong: summing kernel durations across overlapping streams reports 135% busy, an impossibility that reads merely as a high number, while the interval union is bounded by wall clock and, inverted, exposes the launch gap (2.5% of the window on one stream). It is proven invariant across 200 random buffer orderings because "CUPTI doesn't guarantee any ordering of the activities in the activity buffer", and it shows an unguarded correlationId join raising KeyError when API records are partly missing. Also records that the Event and Metric APIs were dropped outright in CUDA 13.0 (CUPTI_ERROR_LEGACY_PROFILER_NOT_SUPPORTED), that multiple tracing subscribers became possible only in CUDA 13.3 with driver r610+ while CUPTI's own troubleshooting section still documents the old single-subscriber rule (an internal contradiction the page flags rather than resolves), and that no NVIDIA source claims Nsight Compute is CUPTI-based, so this page does not either.

A batch of ten external sources was triaged for this KB. Six landed as edits or new pages, recorded below; four were evaluated and skipped with reasons recorded rather than silently dropped: NVIDIA's "GPU Performance Background" guide is already exceeded by this KB's own Blackwell-era roofline, occupancy, and memory hierarchy pages; arXiv 2206.01288 turned out to be the DT-FM paper that is already the primary source behind geo-distributed training placement; arXiv 2501.05323 ("Distributed Learning and Inference Systems: A Networking Perspective") is a pure vision paper with zero experiments, tables, or code, out of scope for an operations-focused KB; and Fireworks' "Frontier RL Is Cheaper Than You Think" blog post is already cited verbatim, numbers and all, in delta-weight-sync.

• Training frameworks: PithTrain: agent-native MoE training and agent-task efficiency (distilled from arXiv 2605.31463v1, an ~11K-line, three-layer MoE training framework from CMU, Xlue, and NVIDIA that matches or exceeds Megatron-LM throughput on 4 of 5 tested configurations, built on a DualPipeV pipeline schedule extending DeepSeek-V3's open-source DualPipe scaffold with the actual compute-communication overlap. Its real contribution is agent-task efficiency (ATE) and ATE-Bench, the inverse of SWE-bench: a fixed Claude Opus 4.7 agent against 20 real training-framework tasks, varying the framework instead of the agent. The executed model reconstructs the benchmark's own median-of-three arithmetic from Appendix C's raw per-attempt data, cross-checks all 24 medians byte-exact against the paper's own Table 7, and shows the abstract's "62%/64%" headline compresses two different tasks against two different baseline frameworks into one sentence, plus a real case, drawn from the paper's own numbers, where PithTrain used more Agent Turns than a baseline).
• Serving, single-box: Colibri: GLM-5.2 local inference on 25 GB of RAM (JustVugg/colibri, a single ~2,400-line C engine that runs Z.ai's 744B-parameter GLM-5.2 MoE by keeping a 9.9 GB int4 dense skeleton resident and streaming its routed experts from a roughly 370 GB disk container through a per-layer LRU cache). Validates the sigmoid MoE router's top-k selection in numpy against a brute-force reference, including an adversarial case reproducing the rounding-scale sensitivity behind the project's own documented greedy-output non-determinism. Maturity is treated honestly: the repository is 13 days old, ships no tagged releases, and had same-day crash and memory-safety fixes at the pinned commit; two internal inconsistencies in the README (a 75x256 expert-count arithmetic mismatch, and a parameter count that drifts across sources) are flagged rather than silently resolved. Cross-linked to DwarfStar (ds4), Prima.cpp, and the datacenter-scale vLLM GLM-5.2 cookbook.
• Also updated: Profiling GPUs: Nsight Systems and Nsight Compute was re-verified against the live Nsight Compute Profiling Guide (v2026.2.1) and Nsight Systems User Guide (v2026.3) rather than the book alone. Corrects a real gap: ncu's default is the basic section set, not full, which the page previously left ambiguous. Adds --replay-mode (kernel/application/range/app-range), SourceCounters with its -lineinfo requirement, PM and Warp Sampling, and a new production section on MIG, MPS, Green Context, and CUDA Graph profiling caveats, including the verbatim-quoted trap that enabling graph profiling silently drops instruction-level source metrics for every kernel in that run, not only the graphed ones. Corrects the CI guidance to --csv --print-units base --print-summary per-kernel and the ncu_report Python module (confirmed not pip-installable, ships in extras/python/) instead of grepping raw metric text, and confirms that "Global Memory Load Efficiency" and "average sectors per request" are not literal Nsight Compute UI labels in the current docs, the modern equivalent is the Sectors/Req column. Adds a torchrun multi-process nsys recipe and the --pytorch=autograd-shapes-nvtx auto-NVTX flag, both from the official User Guide; Michael Carilli's nsys cheat-sheet gist is cited separately as independent community corroboration that the torchrun-wrapping approach works in practice. Also corrects the multi-node %q{} output-templating example, which named an undocumented and node-local %q{LOCAL_RANK} variable in place of the User Guide's actual third documented variable, %q{PMI_RANK} (MPICH), and fixes the PM Sampling interval for GA10x+ from "sub-microsecond" to the documented floor of 1000 ns.
• Also updated: DFloat11: lossless BFloat16 compression gains a note on the wider lossless entropy-coding landscape, ZipNN (arXiv 2411.05239) and ZipServ (arXiv 2603.17435), the latter claiming a fused kernel that beats cuBLAS and vLLM while staying lossless, architecturally distinct from DF11's decompress-then-matmul split. It also records brianbell-x/weight-compression's GLM-5.2 (753B) experiment: the byte-split codec is real and reproducible (24.967% smaller, bit-exact across all 59,509 tensors via verify.py), but the 30.168% "K15" ratio is unverified accounting only and the "0.733x BF16" GEMV number is an unfused, dense-only prototype not validated at scale, neither of which should be read as lossless compression beating BF16 throughput.
• Also updated, cross-checked against T-Systems' July 2026 write-up of an autoresearch agent that cold-tuned a GLM-5.2 (744B) serving deployment on a B200 cluster (Parisio et al., AI Foundation Service), verified against the raw fetched post rather than a summary: AI-assisted performance optimization gains a concrete instance of the "no reliable oracle" failure mode one layer up from kernels, where a synthetic fixed-rate benchmark reported doubled throughput while production trace replay found an 18x worse p50 time-to-first-token; autonomous experimentation loops adds serving-deployment-topology search as a domain example beyond training and HPO, and flags that the source's "26 runs, +110% throughput" claim ships with no grid- or random-search baseline; and the GLM-5.2 vLLM cookbook adds the source's externally reported FP8-requires-TP=8-minimum-on-B200 OOM boundary, attributed as third-party and not independently confirmed by this KB.

A close read of Chapters 3 and 4 of Chris Fregly's AI Systems Performance Engineering against this KB's existing coverage found those chapters already deeply mined in prior sessions (the OS/NUMA/hugepage/container/Kubernetes-topology pages and the NCCL/RDMA/SHARP/NIXL pages all carry precise Ch.3/Ch.4 footnotes). Three narrow, genuine gaps remained and are closed below; no new pages were needed.

• Also updated: NCCL Collectives & Algorithms gains a communicator-lifecycle section this KB was missing entirely: the measured cost of creating and destroying a communicator every iteration (48 ms of overhead added to a 0.5 ms all-reduce on 2 ranks; 2-3 minutes instead of 2-3 seconds to stand up 32 per-rank communicators), the fix of initializing once and reusing (new_group(), or batched multi-communicator setup via ncclGroupStart/ncclCommInitRank/ncclGroupEnd), persistent zero-copy user-buffer registration (ncclCommRegister/ncclCommDeregister, cross-linked to its SHARP payoff), and the previously undocumented NCCL profiler plugin API (NCCL_PROFILER_PLUGIN and its five callbacks: init, startEvent, stopEvent, recordEventState, finalize).
• Also updated: Runbook: NCCL Hang / Collective Stall adds two root causes it did not previously cover: a mismatched PyTorch-bundled versus system-installed NCCL build (which can hang or silently degrade a job with no crash to grep for), and ephemeral-TCP-port exhaustion during NCCL's out-of-band bootstrap handshake when net.ipv4.ip_local_port_range is too narrow.
• Also updated: Reliability, RAS & Failure Modes adds Meta's published interruption root-cause table for the Llama 3 405B 54-day, 16,384-H100 pretraining run (arXiv:2407.21783, cross-checked against contemporary press coverage of the same report), giving the page's "failure is steady-state" claim a concrete, cited breakdown instead of an abstract MTBF formula alone: GPU-attributable faults account for over 55% of interruptions, with faulty GPUs alone the largest single category at 30.1%.

2026-07-13

• Inference optimization: DFloat11: lossless BFloat16 compression (entropy-coding the BF16 exponent field so a checkpoint lands at 67.6% to 69.5% of its original size with bit-for-bit identical outputs, and the GPU kernel that decompresses every weight on every forward pass. Distilled from arXiv 2504.11651v3 (NeurIPS 2025) plus the upstream repository and its issue tracker. Covers the measurement the method rests on (the exponent carries about 2.6 bits of its 8, while sign and mantissa are near-incompressible, so only about 40 of 256 exponent values ever occur), the hierarchical lookup tables that get a 24-to-32-bit maximum code length under the SRAM limit by splitting the Huffman tree into 8-bit subtrees, and the two-phase kernel that resolves the variable-length write-position dependency by decoding every exponent twice. The executed numpy model builds the format end to end, proves the roundtrip is bit-exact over 200,000 weights, shows a hierarchical decoder returning identical output to a flat LUT in 3.00 KB against 128 KB, and reproduces the paper's ~2.6-bit entropy finding from a synthetic normal tensor. Its adversarial cases set the boundary: uniform exponents do not expand the tensor, they degenerate to a flat 8-bit code and return exactly 16.000 bits per weight, buying nothing while still costing a decompression kernel; an INT8-rounded tensor whose 16-bit word carries only 7.994 bits still costs DFloat11 10.375, independently reproducing the maintainer's own stated reason for not supporting quantized checkpoints; and one flipped bit in the encoded stream silently rewrites 156,083 of 200,000 weights, raising nothing, which is why artifacts must be checksummed. The page leads with the honest trade the abstract's "negligible overhead" framing obscures: when the BF16 model already fits, DFloat11 is strictly slower, by 1.4x to 2.5x on a single GPU and up to 4.4x on 4xA100 for Llama 3.3 70B, per the paper's own Appendix G, with the upstream README conceding "approximately 2x slower" at batch 1. It is a memory-capacity tool, not a speed tool, and it never beats BF16-on-GPU anywhere in the paper. Records that the headline 405B-on-one-node claim is arithmetic (551.22 GB against 640 GB) and is never benchmarked: no H100 or 8x80GB node appears in any experiment, which uses A5000, A100-40GB and Quadro RTX 8000. Also records that v1's table reports worse ratios than v3's for the same models and that v3 silently drops the Gemma rows, which compress worst; that upstream is close to dormant (last commit 2025-11-24, PyPI pinned at 0.5.0, Hugging Face org silent since 2025-09-30, vLLM support closed as not_planned by a stale bot); that CUDA 12.0 and 12.1 fail outright on the shipped PTX; that LoRA and fine-tuning are unsupported because the weight attribute is deleted; and that the community fork now carrying the ecosystem currently ships broken losslessness on FLUX.2-klein, which is the one property the format exists to provide).
• Distributed training: Geo-distributed training placement adds a WHAT/WHY/WHEN/HOW guide and scripts/geo_placement.py. The NumPy-only tool validates fabric matrices, preserves bidirectional transfer time when symmetrising directional measurements, models explicit message counts, solves bottleneck matching and the Hamiltonian stage path exactly, searches the balanced partition heuristically, and emits a tested device-to-rank map. The page includes a multi-size NCCL measurement template, workload-byte sizing, rank-0 rendezvous derivation, a one-GPU-per-host launch contract, production network guidance, and executed brute-force equivalence and failure tests. Inbound routing was added from pipeline parallelism and overlay and mesh networking. The source audit is pinned to DT-FM commit cda948f02111ad194ce108d4c65b85258afd0c19 and records the paper/repository matrix differences, the scheduler's FP32 proxy versus the shipped FP16 buffer, the sharded-parameter-server model versus shipped selections, and the distinction between emulated world-wide cases and the later two-region FluidStack run.

Six pages distilled from a batch of Hugging Face engineering posts, each broadened well past the post with primary-source verification against upstream code. Every page carries an executed, adversarially-validated numpy or stdlib model with its real output pasted in, plus a Mermaid diagram. Every page was then attacked by an independent adversarial reviewer that re-derived the numbers, mutation-tested the asserts, and re-fetched the citations; the defects it found are fixed and the honest limits are recorded below rather than laundered.

• Inference internals: KV cache fundamentals (the tensor and API layer beneath the KB's eight existing KV pages: why keys and values are cached and the query is not, since causal masking means a past query row is never re-read while every K and V row is re-read by every future query; the [batch, num_kv_heads, seq_len, head_dim] layout and its dim=-2 append; and the layer-based Cache surface in transformers v5. Its highest-value content is three stale-API traps that most tutorials still get wrong, all checked against cache_utils.py on the main branch (__version__ 5.14.0.dev0, 2026-07-13): SlidingWindowCache is now only a back-compat alias for StaticCache, OffloadedCache does not exist at all (offloading is an offloading=True constructor flag whose offload_only_non_sliding default differs between DynamicCache and StaticCache), and the legacy tuple past_key_values format is deleted rather than deprecated. The executed model proves the GQA head map is a real silent-corruption bug class: the classic h % n_kv instead of h // rep is invisible on MHA and MQA fixtures and changes the output only on a genuine GQA shape. Honest limits: the API facts are pinned to a specific pre-release snapshot and will drift, so the maintain section says to re-diff cache_utils.py on upgrade; NVIDIA's widely copied KV byte formula is MHA-only and over-counts a GQA model by num_heads / num_kv_heads; and the 4.7x and 5.2x figures from the popular explainers are batch-1 T4 toy runs, not production speedups).
• Inference internals: Continuous batching in transformers (the transformers-native serving path, which the KB had no coverage of: generate_batch, ContinuousBatchingManager, and transformers serve --continuous-batching, built from the upstream source rather than the announcement posts, because part one of that series contains no API, no code, and no benchmark at all. Covers the paged cache's two reserved blocks, the fifo and prefill_first schedulers, the safety_margin admission reserve that vLLM does not expose under any name, the offload-then-soft-reset answer to KV pressure, and two independent switches that silently kill the decode fast path, one of which fires no warning even when set explicitly. The executed model reproduces the blog's own 693-token padding-waste figure and then refutes the blog's claim that the cost grows quadratically: (n-1)(B-1) is linear in each variable, proven by a zero second difference. Honest limits recorded: the async benchmark is n = 1, its hardware is never disclosed and its model never named, and its headline "22% speedup" is a time reduction (300.6 s to 234.5 s) which is a +28.2% throughput gain, not 22%; its "99.4% GPU utilization" is GPU busy-time from a span trace, not SM occupancy or MFU. No transformers-versus-vLLM throughput ratio is stated anywhere on the page, because none is published under a reproducible method. Also fixes a real citation defect in continuous batching internals, whose Orca footnote credited the paper but linked the vLLM blog; the OSDI '22 paper is now cited properly).
• Inference internals: MoE expert backends and grouped-GEMM kernels (the KB's four MoE pages all sat above the kernel; this one covers how a MoE layer actually executes, from the eager loop to batched GEMM to grouped GEMM to block-sparse to fully fused, and the packed 3D weight layout every non-loop kernel demands. Documents the six backends transformers registers behind experts_implementation (the announcement still says three), and the automatic grouped_mm to batched_mm switch at the decode stage that will surprise anyone reading a profiler. Two fact-checks are baked in: the announcement's headline loading benchmark, introduced as measuring "loading speed of large MoE models", actually benchmarks Qwen/Qwen1.5-110B-Chat, whose config declares Qwen2ForCausalLM with no num_experts field, so it is a dense model and those figures exercise the expert-packing path not at all; and the widely quoted "12x faster MoE training" is 7.3x in Unsloth's own gpt-oss BF16 table (712.33 ms against 5226.86 ms at 8192 context, B200, LoRA rank 64), with the larger multipliers measured against v4 rather than v5. The executed model proves the two real grouped-GEMM bugs are silent: an off-by-one in the group offsets corrupts 6 of 96 token rows and raises nothing. Records a distinction that is load-bearing and absent from the KB until now: Hugging Face's expert parallelism is a masked router plus an all-reduce, with no all-to-all dispatch, so enable_expert_parallel=True does not give DeepEP semantics. Also adds an expert-offload-to-CPU section to MoE sparse scaling, grounded entirely in upstream llama.cpp).
• Serving, managed cloud: Managed inference endpoints on SageMaker (closes a total blind spot: managed-cloud inference had zero coverage across 373 pages. Covers the four AWS pathways with the honest matrix, including that serverless inference is GPU-excluded by AWS and therefore cannot serve an LLM at all, and that async is the only pathway with genuine scale-to-zero while its per-hour rate is identical to real-time. Its differentiator is the first quantification of the PaaS-over-IaaS premium in this KB, re-derived to the cent from the AWS Price List bulk API by two independent agents (us-east-1, on-demand, 2026-07-13): SageMaker Hosting costs 1.40x the EC2 rate on single-GPU G shapes, 1.25x on ml.g5.12xlarge and 1.15x on the P family, so the premium shrinks with instance size rather than being a flat rule. The page corrects the source post's "$730/month" for an idle ml.g5.xlarge: that is the EC2 rate, while SageMaker Hosting is $1.408/hr, or $1,028/month, understating idle cost by about 40%. States plainly that no rigorous evaluation of the agent skills exists (two models, one run each, self-reported; the repo's evals leaderboard is a Hub model-index scraper, not a skill-eval harness), and closes the blast-radius gap the source leaves open: the endpoint execution role is tightly scoped, but an agent's own CreateEndpoint rights are bounded by nothing, and one call can commit about $46,200 a month. The executed cost model asserts the honest shape of the async trade, that a crossover duty cycle exists, while showing its position swings from roughly 0.45 to 0.97 across plausible cold-start and burst assumptions, so the shape is the finding and the specific number is not).
• Post-training: Knowledge distillation: method selection (a hub organising distillation by signal, hard text against soft logits, and by data source, teacher-generated against student-generated, filling the KB's gaps on white-box logit KD (Hinton temperature scaling and the T^2 gradient factor) and on distillation infrastructure (teacher servers, generation buffers, top-k logit truncation and its tail bucket). Documents TRL v1.8.0's six distillation trainers where the KB previously listed two, including GOLDTrainer, whose cross-tokenizer support obsoletes the KB's prior "avoid mismatched tokenizers" advice. Foregrounds the source's best insight: frontier teachers are often not bigger models but same-size checkpoints of the same base, each pushed further in one domain with RL, so specialization rather than scale is what makes them good teachers. Publishes a KB-derived finding, not a cited one: TRL's generalized-JSD loss is discontinuous in magnitude at both beta endpoints, which hard-code the full KL while the interior decays, so the loss scales as beta * (1 - beta) and sweeping beta silently rescales the gradient, confounding the divergence with the effective learning rate. It is derived from TRL's source plus PyTorch's documented kl_div semantics and reproduced in numpy, not executed against torch. TRL's "up to 40x" generation-buffer speedup and "~5x" payload shrink are vendor doc claims with no published benchmark and are cited as claimed, never as measured).
• Post-training: Token-In, Token-Out: rollout token correctness (the rollout-loop rule that you never re-encode tokens you have decoded. Re-rendering the messages list between turns silently corrupts the PPO or GRPO importance ratio, because decode-then-encode is not injective; the executed model shows an unchanged policy producing ratios between 0.198 and 0.585 where every one must be exactly 1.0, with no exception raised. The page's strongest content is something the source post never says: the invariant is already shipped production code in TRL v1.8.0, which auto-swaps a patched chat template when the model's own template is not prefix-preserving. The prefix-preservation property test was re-executed against templates pulled live from the Hub, reproducing the source's finding that Qwen3 fails while the other families pass, and it is shipped as the durable artefact instead of the table, because chat templates drift with model revisions. Honest limits: the source contains zero benchmark numbers, no evals, baselines, model sizes, GPU-hours, artefacts, or licence, and its motivating loss-spike failure is asserted with no repro or loss curve, so this is a mechanism claim and not a measured one; and the source's claim that any dummy prefix works is contradicted by TRL's own code, which uses the real tool name because GPT-OSS derives the tool-response header from it. Also upgrades agentic RL's central masking claim from a book footnote to Search-R1's measured ablation, 0.431 masked against 0.343 unmasked, and fixes dead volcengine/verl and inclusionAI/AReaL links repo-wide).

One source in the batch was evaluated and rejected. A vendor post promoting an inference engine was found to be marketing for stock llama.cpp: its base model returns 401, both "verify it live" demos are dead, its "efficiency leaderboard" is hand-coded HTML listing only the vendor's own models, and it ships no code, paper, or harness, while its headline table silently mixes an aggregate batched figure from a different engine with single-stream figures. No page cites it. The one legitimate technique underneath it, keeping routed experts in CPU RAM while attention and the router stay on the GPU, was written up in MoE sparse scaling from upstream llama.cpp sources alone, with the bytes-per-token arithmetic derived rather than quoted.

• Serving, per-technology: vLLM Transformers modeling backend (serving a Hugging Face transformers model in vLLM at the throughput of a hand-written vLLM port, with --model-impl transformers and no vLLM model to write. Covers the torch.fx plus AST rewrite pass added in vLLM PR #47187, the three fusers it applies (GLU to MergedColumnParallelLinear, QKV to QKVParallelLinear, RMSNorm to vLLM's RMSNorm), the compliance rules a custom model must follow to be fusable, and the grep -cE '^Fused:' assertion that is the only way to prove the fusions fired, since a fuser that stops matching leaves the model correct and merely slower. The executed model validates the sharding contract the fusers depend on: fusion on one GPU is bit-exact (0.000e+00 error, so it is a pure performance transform), grouped-query attention makes the merged QKV rows unequal (8192:1024:1024 on Qwen3-32B, so "split it into three" is wrong before tensor parallelism is even considered), and a contiguous split of a merged weight across ranks raises no exception while corrupting 55% of the output elements, which is why every checkpoint tensor is mapped to an explicit shard id. Honest findings recorded against the announcement: the parity headline is real but the improvement is small, because the backend was already at 96.6% to 98.7% of native before the PR, so the fusers close one to three points; every configuration was run once with no variance reported, and the 102% overshoot on the MoE model bounds the harness precision at about the same magnitude as the 1.3% and 1.7% dense-model gains, which therefore should not be read as distinguishable from noise; and the benchmark measures only saturated throughput on a prefill-heavy 1024-in/128-out workload, the regime that most amortizes per-layer Python overhead, so time to first token and inter-token latency, where a Python modeling path is most exposed, are unmeasured in either direction).

2026-07-12

Two pages closing the largest structural gap in the KB's non-colocated-inference coverage: the transport substrate. The existing cross-WAN model-parallel inference page covers the Petals/HexGen/Helix/Parallax scheduling lineage but treats the layer underneath it (finding peers, connecting through NAT, carrying activations) as an assumption. QUIC, iroh, libp2p, gossip, DHT, and NAT hole-punching had zero coverage anywhere in the KB before today. Both new pages carry an executed, adversarially-validated model whose pasted output is byte-identical to the real program output.

• Serving, per-technology: Mesh LLM and Skippy stage splits (an Apache-2.0 P2P mesh that serves one OpenAI-compatible endpoint across NAT-separated, multi-vendor machines: QUIC/iroh transport, gossip membership, Nostr and invite-token discovery, layer-package weight distribution so a peer downloads only its assigned layer range, and a latency-aware stage planner whose cost model is max(hop_latency) * stage_count, verified in its source at skippy-coordinator/src/topology.rs:462. The executed model reproduces the project's published stage-count table exactly, recovers its hidden size independently from two published activation-frame sizes, and derives why the split axis is forced: tensor parallelism needs 2L blocking all-reduces per token where pipeline parallelism needs stages hops, a 20x ratio that is identical at every link latency, matching Megatron-LM's own bsh versus 8bsh(t-1)/t accounting. Honest findings recorded: splitting a model that already fits on one machine makes decode much slower (2-node split at roughly 31% of solo throughput), and it is the network term of the planner's model, not the total, that rises monotonically with stage count; the project's total tok/s metric counts prompt tokens and corresponds to 0.99 generated tok/s in the run that reports it, the RPC serving lane is no longer supported though the name lingers in the events crate and installers, its own EXO_COMPARISON paragraph ends on a "zero inter-device communication during decode" clause that its preceding sentence contradicts, its release attestation is build provenance and explicitly not runtime attestation, and it performs no output verification at all. The q8 activation-wire section is read from the capability JSON rather than the prose summary: 96 records across 95 families, 48 validated, 40 rejected, 8 untested, with deepseek rejected while deepseek2 is validated and gemma3 rejected while gemma3n is validated).
• Distributed communication: P2P transport for decentralized inference (why QUIC rather than TCP for a link multiplexing latency-critical activations against multi-gigabyte weight transfers; NAT traversal, the relay fallback, and gossip admission control. The executed model derives SWARM's "square-cube law" from the transformer's shape, showing boundary arithmetic intensity scales exactly linearly in hidden size, so a 405B model earns 12.7x more compute per boundary byte than an 8B one, then draws the line the literature blurs: that is a bandwidth result, and the decode hop count is stages regardless of model size. A decode token transmits in 0.13 ms and then waits 10 ms, leaving the link about 99% idle, which is why buying bandwidth to fix decode latency is the expensive mistake: the executed lever table prices a 10x link at 1.2% and a 2x activation compression at 0.7%, against 2.00x for halving the stage count and 4.33x for speculative decoding. The floor has exactly three terms (stages, hop RTT, tokens committed per trip), and speculation is the only one you can pull without re-planning placement. It is not free either: pricing the drafter's compute yields a break-even acceptance rate of about 0.12 on a 10 ms WAN hop, rising to 0.34 on a 1 ms same-datacentre hop and 0.35 with a drafter at 40% of the target's cost, so a WAN makes speculation pay at unusually low acceptance rates rather than at any acceptance rate. Corrects a widespread folk belief with measurement: hole punching is statistically indistinguishable between TCP and QUIC in the largest published libp2p study, so UDP buys nothing at the NAT. The often-quoted 70% success rate is conditional on earlier steps succeeding and the authors call it optimistic; the paper reports no end-to-end rate, and compounding its 29% exclusion gives a lower bound rather than an estimate, so the true end-to-end direct-connection rate sits between about 50% and that 70%. A four-stage chain is about 76% likely to contain a relayed hop even on the optimistic figure).

Both pages were rewritten after an adversarial fact-check against the primary sources found 29 defects in their first draft, including an empirical claim attributed to the wrong network fabric and a run the upstream project itself flags as diagnostic. The retracted analysis was replaced with a cross-check against Petals' published bandwidth-versus-latency sweep, whose measured slope independently recovers the hop-count model (2.22 serialized round trips for a three-server chain).

2026-07-11

Eight pages distilled from eight June 2026 arXiv papers, grouped into agent orchestration and routing, agent foundations, an RL training method, and evaluation. Each carries an executed, adversarially-validated numpy or stdlib model with its real output pasted in, plus at least one Mermaid diagram; every headline number is checked against the paper and flagged honestly where the paper is internally inconsistent.

• Agentic systems, orchestration: Sakana Fugu: orchestrator model family (learned orchestrator LLMs that route a single worker (Fugu) or compose up to five-step natural-language workflows (Fugu-Ultra) over a black-box frontier pool, trained by soft-target SFT, sep-CMA-ES, and GRPO; with an executed portfolio-bound model proving a per-query selector beats the best single agent only under complementarity and good routing, and ties it when one worker dominates; the page flags that "beyond any individual agent" is an aggregate claim, with live counterexamples where a pool model beats both Fugu variants, and that the Figure 4 comparison targets are not publicly checkable).
• Agentic systems, routing: Agent-as-a-Router (ACRouter reframes coding-model routing as a Context-Action-Feedback loop over Orchestrator, Verifier, and Memory modules that learn per-task-dimension model performance online instead of classifying each prompt cold; with an executed contextual-bandit toy reproducing static routing's near-linear cumulative regret against the feedback router's sublinear regret, the task-dimension-statistics gain direction, and the noisy-verifier degradation; the reported 15.3% gain is cross-checked and the paper-versus-repository point-estimate gaps are flagged).
• Agent foundations: Agentive systems and the GIC architecture (a position paper distinguishing agentic systems, competence assembled in engineered scaffolding, from agentive systems, capabilities internalized within the model, and proposing the five-component Goal-Identity-Configurator architecture; with an executed locus-based classifier proving the core claim that a maximally capable but fully scaffolded system stays agentic (capability is not agency) and that all five dimensions must be internalized to cross the agentive threshold; no benchmarks, flagged as such).
• Agent foundations: LLM agent communication protocols: a technical taxonomy (nine open-source protocols, MCP, A2A, LAP, agents.json, Agora, ANP, LMOS, ACP, and agntcy, classified along five dimensions of counterparty, payload, interaction state, discovery, and schema flexibility; with an executed interoperability model that reproduces the paper's findings and shows only 13 of 36 protocol pairs interoperate without a bridge; the ANP discovery-classification discrepancy and the abstract-versus-body schema wording are flagged).
• Post-training, RL method: Self-play with human-data regularization (a training method that keeps a self-play RL loop on a minimal safe reward and adds a small on-policy KL term toward a human behavioral-cloning anchor, so 30 minutes of demonstrations, 2500x less than imitation learning, pull an alien-but-competent policy onto the human-compatible equilibrium on a 15-hour single-GPU budget; with a vectorized numpy equilibrium-selection model proving a small lambda breaks the symmetry between equally-good conventions without collapsing competence, bounding the usable window and the over-regularization collapse).
• Post-training, synthetic data: Autodata: an agentic data scientist (a challenger, weak-solver, strong-solver, and judge inner loop that accepts a synthetic example only when it separates a weak from a strong solver, plus a validation-gated meta-optimization loop that rewrites the generating agent itself; with an executed two-loop model proving filtered meta-optimization improves data quality monotonically to a ceiling, an already-optimal generator gains nothing spurious, an unfiltered loop collapses, and a biased filter propagates its bias into the dataset).
• Evaluation: NatureBench (90 research tasks distilled from Nature-family papers, run in NatureGym per-task containers under a web-search-disabled, matched-hardware protocol; the strongest configuration surpasses published SOTA on 17.8% of tasks under the SOTA-normalized g greater than 0.1 criterion, succeeding mostly by methodological translation rather than discovery; with an executed metric model that reproduces 17.8% as 16 of 90, enforces the strict surpass boundary, and guards the undefined baseline-equals-SOTA case; the abstract's "10 versus 12 configurations" gap is flagged).
• Evaluation: LLM-as-a-judge reliability and validity (21 judges across nine providers and three benchmarks, roughly 541,000 judgments: exact-match agreement masks a universal Cohen's-kappa deflation of 33 to 41 points and a consistency-bias paradox where a judge is highly repeatable yet severely position-biased; with an executed stdlib model reproducing the kappa deflation via skewed marginals and building the paradox in one "always pick A" object that is perfectly consistent, chance-level by kappa, and maximally position-biased; a math inconsistency in the paper's own chance-baseline claim is flagged and corrected).

2026-07-10

The platform-scale sibling of the coding-fleet pair: a full solution architecture for selling tokens, not hours, from your own GPUs. The KB had every ingredient (cache isolation, QoS admission, the vLLM recipe, SLOs, capacity planning) but no page composed them into the OpenRouter-shaped product: one API key, a model catalog, per-token billing, multi-tenant isolation you can prove.

• Inference serving, platform engineering, security: Cookbook: an OpenRouter-style inference platform on your own GPUs (the ten invariants that anchor design reviews, from "no cross-tenant KV reuse, ever" to "reject, never silently degrade"; an OpenAI-compatible gateway whose admission pipeline runs price-free quota gates before balance before model resolution, with allowlist request reconstruction so a client-supplied cache_salt is a 400 in any casing, and at any depth inside a passthrough extension object, never a silent drop; HKDF-derived per-key cache salts with control-plane-only rotation, an eleven-row threat model spanning the Early Bird timing oracle, the CacheProbe aggregator trap, and the Whisper Leak packet-timing channel that cache isolation explicitly does not close; a pool-per-model serving plane with a rich validated serving profile, a metrics sidecar because stock vLLM co-serves /metrics with the API on one listener, and NVLink-domain placement constraints; a registry-gated onboarding pipeline with champion/challenger alias flips; effectively-once metering with prompt-inclusive abort accounting and a dedicated arrears ledger; and a two-layer autoscaler whose damped control law is normative).
• Four executed stdlib contracts carry the page. The capacity model reproduces the public anchors (one H100 serves Qwen3-8B at 8K at 44 concurrent sequences fp16 KV and 88 at fp8; Llama-3.1-70B does not fit at TP=2 under the same budget yet serves 51 sequences at TP=4) plus monotonicity across the GPU/dtype/context grid. The isolation contract proves replica-agreement, distinct-principal, and rotation properties of HKDF salt derivation, then rejects seven smuggling shapes including wrong-cased allowlisted fields. The settlement contract proves idempotent apply under triple redelivery, prompt-inclusive abort billing, the arrears floor at a drained balance, the checkpoint-loss bound, and the fanout-aware affordability clamp (with best_of < n rejected). The control-law simulation shows the naive queue-proportional scaler collapsing to minimum replicas mid-burst and paying 84 cold starts against the damped law's 14, with 12 direction changes against 2.

The how-to half of the coding-fleet pair: own or rent a coding model decides, and the new page executes. It is a pure operations cookbook that starts where the decision ends and stops when a developer's editor gets a working diff from a model you run; it deliberately contains no cost model.

• Inference serving, platform engineering: Cookbook: own and run an open-weight coding model (pin Qwen/Qwen3-Coder-Next-FP8 to revision da6e2ed and serve it on vLLM v0.24.0 with the card's qwen3_coder tool parser, which the v0.24.0 source registers as an alias of the docs' qwen3_xml; verify tool calling before anyone depends on it, including the exact wrong-parser signature of HTTP 200 with <tool_call> markup as plain text; front the fleet with LiteLLM 1.91.1 virtual keys, per-developer max_budget/budget_duration, and a 429/5xx-only frontier hatch under a hard monthly cap; wire Claude Code, Codex CLI (wire_api = "responses" is now the only supported value), Qwen-Code, and OpenHands through the gateway's three dialects; sandbox tool execution with scoped mounts and default-deny egress; salt the prefix cache and stamp priorities per tenant; and gate rollout and every upgrade on a paired eval against the live endpoint).
• Two executed stdlib contracts carry the page, with boundary and type-confusion cases exercised directly in the assertions. The tool-call validator rejects the wrong-parser shape by name plus hallucinated tools, truncated calls (on finish_reason, since truncated JSON can parse as a shorter valid prefix), bool-as-int arguments (isinstance(True, int) is True), and partial batches, fail-closed. The gateway state machine proves reserve-then-settle accounting: quota and hard-budget constructor inputs are checked for both type and sign (a bool or a float quota is rejected, not silently coerced), quota is counted against in-flight holds so concurrent requests cannot double-claim, settlement is exactly-once so retries cannot double-spend, client errors never escalate to the hatch, the cap counts reservations so it cannot be overrun (five identical $7.00 escalations against a $20.00 cap admit exactly two), and exhaustion rejects rather than degrades.

2026-07-09

A business-level cookbook that the knowledge base was missing: the org-level decision spine for running an open-weight coding model behind your own developers. Every adjacent page existed (model selection, per-model serving cookbooks, build-vs-rent, capacity planning, LoRA, eval harness, sandboxing) but nothing converted a served fleet into a cost per developer or compared it against a hosted API bill, and nothing sequenced the pieces into one playbook.

• Strategy, inference serving, agent evaluation: Own or rent a coding model, the decision layer rather than the launch shape (five deployment states from hosted API to continued pretraining; the four gates that should stop the project, taken in order: capability, economics, evaluation, operations; replica sizing that catches the total-vs-active-parameter and BF16-vs-FP8 traps, with GLM-5.2 fitting an 8xH200 but not an 8xH100 node and Ornith-1.0 fitting one 8x80GB node; a frontier escape hatch routed in the gateway, grounded in Agent-as-a-Router's CodeRouterBench and its 15.3% relative gain from task-dimension statistics; and the adaptation ladder that puts harness work before weight work, on the evidence that Agentic Harness Engineering moved Terminal-Bench 2 from 69.7% to 77.0% with the same base model and that HarnessX averages +14.5% across five benchmarks). It deliberately carries no vllm serve command; the per-model cookbooks own those, and the end-to-end bring-up is Cookbook: own and run an open-weight coding model.
• Three executed stdlib models carry the page, with 97 assertions across replica sizing, seat economics, and eval sizing. The seat-economics model replaces the usual scan with an exact criterion. Compare the API's bill per seat against the fleet's marginal cost per seat; if the API bills less, no seat count ever makes self-hosting cheaper. At the worked throughput the marginal cost is $20.96/seat against $6.35 (budget), $22.93 (mid), and $68.80 (frontier), so budget is unbeatable and frontier crosses at 51 seats; move both phases to 100,000 prefill and 5,000 decode tokens/s per replica and the marginal cost falls to $3.77, at which point even a budget API becomes beatable, so no "always" claim about self-hosting survives a throughput change. Only the binding phase counts: raising decode alone to 5,000 hands the bottleneck to prefill, lands at $11.98, and rescues nothing. Where a crossover does exist it is not a single number: the fleet steps by a whole replica while the API bills as a line, so the mid-tier decision reverses 21 times between 153 and 1,681 seats. The model also quantifies the 4.35x idle premium of a fleet that bills 730 hours and serves 168, shows that sizing on the 24-hour mean under-provisions by up to 3x, and proves by sweep that prefix caching moving the bottleneck from prefill to decode is not a law (it disappears at 50:1 and 1:1 throughput ratios), because vLLM's automatic prefix caching accelerates prefill and explicitly not decode.
• Three findings the page records against its own earlier draft, each pinned by an assertion that fails if the fix is reverted. Charging every uncached token at the 1.25x cache-write premium, when the prompt caching contract has a third tier at plain 1.0x, overstates the hosted bill and flatters self-hosting: it moves the first crossing from 153 seats to 138 and hides 12 of the 21 reversals. A GPU-only TCO hides that half an engineer ($8,333/month) costs more than two replicas of GPU rent and moves the frontier crossover from 51 to 223 seats. And a published --tensor-parallel-size 2 pins neither GPU nor precision: the same 80B model needs two 80 GB GPUs at FP8 and three at BF16, while the published 8-GPU shapes bound the non-weight budget at 34% for GLM-5.2 and 38% for Ornith-1.0.
• The eval-gate model makes the case for an internal eval concrete: SWE-Bench+ measured 32.67% solution leakage among passing SWE-bench patches and flagged a further 31.08% as suspicious on weak tests, dropping the reported resolution rate from 12.47% to 3.97%, and a contamination model reproduces the consequence that a benchmark can rank a genuinely weaker model above a stronger one. It also records two discrepancies the page found and resolved: the closed-form paired sample size says 385 tasks suffice to call a 5-point gap at 25% discordance, while the exact McNemar test at n=385 returns p=0.082; and "the first n that clears" (412) is a sawtooth artifact, because p(n) is not monotone in n and the tempting +10% remedy lands on n=424, which returns p=0.064 and does not clear. In the checked planning window, 468 clears every n through 1199. A 30-task internal eval cannot resolve anything below an 18-point difference.

2026-07-09

A five-page set on program-aware and heterogeneous inference systems, open RL-as-a-service, and agent evaluation, distilled from four sources: the ThunderAgent paper and repository, Prima.cpp, OpenTinker, and Workspace-Bench. Each carries an executed, adversarially-validated stdlib model. Two listed sources were already covered and are cross-linked rather than duplicated: AIOpsLab (arXiv 2501.06706) by AIOpsLab, and the PULSE weight-update-sparsity work plus the delta-weight-sync blog (arXiv 2602.03839) by delta weight sync.

• Inference serving, agentic systems: program-aware agentic inference (ThunderAgent) (abstracting a multi-turn workflow as an LLM Program so the KV cache, cross-node memory, and tool sandboxes schedule as one unit, with the pause/restore scheduler, the global waiting queue, and the async tool manager; reported 1.48-3.58x serving and 1.79-3.92x rollout speedups on 8xH100; with an executed model proving the shortest-context-first eviction that Lemma 4.1's quadratic recompute cost implies is 53x cheaper than longest-first and equals the brute-force optimum, plus the pause/restore phase-priority ordering, and an honest overshoot case where the greedy heuristic is only near-optimal).
• Inference serving, deployment recipe: Cookbook: ThunderAgent agentic serving in front of vLLM (the step-by-step deployment, the three client-side changes, and an executed KV-cache capacity planner that computes how many concurrent programs a backend holds before it thrashes and what program-aware pausing buys over a request-level scheduler).
• Inference serving, low-resource: Prima.cpp: heterogeneous home-cluster inference (running a 70B model across four everyday devices whose combined memory is 37 GiB via piped-ring parallelism that overlaps mmap disk loading with compute and the Halda scheduler that solves layer-to-device assignment under RAM/VRAM limits; the v1 674 ms/token result and the honest v1-versus-v3 numbering; with an executed LRU model reproducing the paper's prefetch-release conflict, every layer loaded twice when the window exceeds capacity, and the overlap latency that makes a small window win only in the offloaded regime).
• Post-training, RL infrastructure: OpenTinker: open RL-as-a-service (an Apache-2.0 stack that separates a thin client, a GPU-pool scheduler, a verl training core, and a 2x2 environment framework so users run agentic RL without owning the cluster; the paradigm echoes the managed Tinker API by name with no stated affiliation; with an executed model of the scheduler's no-double-allocation GPU admission and the 2x2 environment data contract, and the shipped-credential security caveats to fix before production).
• Agent evaluation: Workspace-Bench (agent file-dependency benchmark) (5 worker profiles, 74 file types, 20,476 files up to 20GB, 388 tasks each with a human-annotated file dependency graph, 7,399 rubrics, Agent-as-a-Judge scoring; best config 67.7%, mean 47.4%, human 80.7%, with the arXiv abstract's stale 60%/43.3% flagged against the body; with an executed model of the rubric pass rate and TCR@p metrics and the Node-F1-versus-Edge-F1 finding that recovering file relations is the universal bottleneck while finding the files is near-solved).

2026-07-09

A four-page set distilled from the JD Technology report "Thousand-GPU Large-Scale Training and Optimization Recipe for AI-Native Cloud Embodied Intelligence Infrastructure" (Guo et al., arXiv 2603.11101, 2026), which opens a domain this KB did not previously cover: training Vision-Language-Action (VLA) robot foundation models at thousand-GPU scale. One anchor page maps the system; three focused pages carry the reusable techniques, each with executed, adversarially-validated numpy.

• Training workloads, embodied AI: embodied VLA training infrastructure (thousand-GPU) (the four-layer LeRobot plus NVIDIA Isaac platform, the DDP scaling law that took GR00T N1.5 from 15 hours to 22 minutes per epoch, and the honest ledger of what each reported speedup actually measures; with an executed model that reconciles the 40-fold and DDP scaling arithmetic and exposes that the stated "3.5x over the LeRobot baseline" is a 4.5x wall-clock ratio by its own quoted numbers).
• Attention kernels: variable-length attention and data packing (varlen FlashAttention that computes only valid tokens plus data packing that concatenates short samples into full sequences, for a reported 1.88x throughput; with an executed model that reproduces the paper's 2.28%-to-89.73% savings curve almost exactly, drives packing padding from over 50% to under 5%, and proves the block-diagonal mask that stops packed samples leaking attention across boundaries; includes the pi0.5 visual-token-pruning case study).
• Inference optimization, quantization: block-wise FP8 quantization for VLA/VLM (128x128 block-wise FP8 post-training quantization of the language backbone with the vision tower kept high-precision, for 36.6% compression and held accuracy; with an executed model building the OCP E4M3 grid that demonstrates the load-bearing, counter-intuitive truth: for floating-point FP8 a symmetric round-trip is nearly scale-invariant so block-wise and per-tensor agree within 0.3%, which is why accuracy is maintained, while the several-fold error win of fine-grained scaling is a fixed-point INT8 phenomenon).
• Post-training and RL: RL-VLA³: triple-level asynchronous RL for VLA models (asynchronous train/inference on disjoint GPUs, a Bmax/Tmax dynamic-batching interaction policy, and streaming micro-batch generation, for a verifiable +59.25% over the co-located baseline at 32 GPUs; with an executed model implementing the dynamic-batching invariants, the sync-versus-async overlap bound in [1, 2], and the size-weighted micro-batch gradient equivalence that makes streaming generation free of accuracy cost; reconciles the abstract's 126.67% and the companion paper's 85.2% as different baselines).

2026-07-07

• Inference optimization, multi-tenancy: tenant cache isolation and noisy-neighbor protection for shared inference (the two problems bundled under "cache multi-tenancy": confidentiality, shared KV/prefix caches leak cross-tenant timing side channels, per the Early Bird paper's 92.3%-accuracy system-prompt recovery and CacheProbe's gateway-isolation audits, and fixed in vLLM via the merged cache_salt PR while SGLang's equivalent request sits closed and unimplemented; and fair-share, where request priority alone does not protect resident KV blocks from a noisy neighbor's eviction pressure, per Resident KV Claims, TokenCake's reserved-partition results, and VTC's token-cost fairness bound; with two executed models, one reproducing vLLM's salted block-hash isolation and the collapse of the timing side channel, one reproducing total cache eviction under plain LRU against full protection under a reserved partition).
• Post-training, MLOps governance: governance registries: prompt and policy versioning, data governance and lineage (treating prompts, agent policies, and training data as governed artifacts instead of untracked strings: MLflow's Prompt Registry pattern of immutable versions plus mutable aliases, applied to prompts and Cedar/OPA policy files the same way the model registry already applies it to checkpoints; dataset license and consent tracking after the Data Provenance Initiative's audit found 70%+ license omission and 50%+ categorization error across 1800+ datasets, with Hugging Face's own documentation confirming the platform does not verify consent for you; and a poisoning-detection gate motivated by the finding that pretraining poisoning attacks need a near-constant absolute document count, not a percentage of the corpus, regardless of model size from 600M to 13B parameters; with two executed models, one proving hash-based drift detection catches an unregistered prompt hotfix, one proving an absolute-count duplicate scan catches the same poisoned cluster at two corpus sizes a percentage-based threshold would not).
• Agentic systems, AI security: risk-tiered human approval gates for agent actions (a four-rung oversight ladder, auto, validate, approve, multi-approve, synthesized from GAIE's graduated-oversight factors and reported 84-97% velocity preservation, "Measuring AI Agent Autonomy"'s impact/oversight axes, and the AURA risk-assessment framework; the Approve rung implemented via MCP's elicitation spec and LangGraph's human-in-the-loop middleware, and the Multi-approve rung via the maker-checker/segregation-of-duties control from COSO and NIST SP 800-53; with two executed models, one proving a hard risk-floor and monotonicity in the tier classifier, one proving the Validate tier always resolves to proceed-or-escalate and the Multi-approve gate rejects both self-approval and a replayed single-approver signature).
• Agentic systems, harness engineering: RAG vs CAG: dynamic knowledge injection architecture (the retrieve-then-rerank RAG pipeline, bi-encoder plus vector index plus cross-encoder, against Cache-Augmented Generation's precompute-once KV cache reuse; NVIDIA's finding that retrieval still helps regardless of context window size, against the CAG paper's own reported numbers showing it is not universally faster once generation time is counted, and the hybrid pattern that caches a stable core while retrieving volatile context; with two executed models, one showing why reranking exists by promoting a truly relevant document a bi-encoder under-ranked, one reproducing the CAG paper's real HotPotQA-Large latencies plus an explicitly illustrative amortization model for when a one-time cache-build cost breaks even).

2026-07-07

A five-page set distilled from Sebastian Raschka's Build a Reasoning Model (From Scratch) (Manning, 2026), covering the parts of the reasoning-model build pipeline this KB did not yet have its own dedicated, executed-and-tested page for: verifier engineering, inference-time scaling, GRPO run diagnostics, hard distillation, and end-to-end GRPO training itself.

• Post-training, RL evaluation: Cookbook: build a math answer verifier for RLVR (brace-depth-aware \boxed{} extraction, LaTeX-to-calculator-form canonicalization, SymPy symbolic equivalence, and multi-part grading; extends RLVR's verifier-engineering section into a full, adversarially-tested implementation, including a real percent-normalization bug the naive strip-only approach introduces and this page's canonicalizer fixes).
• Post-training, inference-time scaling: Inference-time scaling for reasoning: temperature, self-consistency, self-refinement (temperature/top-p sampling mechanics, self-consistency majority voting with a confidence-scored tiebreak implementing what the source book leaves as an exercise, and the confidence-gated critique-revise self-refinement loop; adversarially demonstrates that logprob confidence rewards the model's own certainty, not correctness).
• Post-training, RL operations: Runbook: GRPO training-run health check (reads reward, group-relative advantage mean/std, entropy, and KL together to diagnose zero-variance groups, entropy collapse, KL blowup, reward hacking, or a solved task, with a runnable diagnostic classifier; companion to GRPO variants's conceptual health-metrics guidance).
• Post-training, distillation: Reasoning distillation via SFT on teacher traces (the DeepSeek-R1-Distill recipe: teacher trace generation, correctness filtering against a documented unfiltered-noise trade-off, length-filtering economics, and answer-only loss masking with an executed invariance proof; explicitly scoped against on-policy distillation as its lambda=0 baseline case, not a competing method).
• Post-training, RL training: Cookbook: RL-train a reasoning model with GRPO, end to end (assembles rollout grouping, the verifiable reward, group-relative advantage, sequence-level log-probability, and the ratio/clip/KL loss into one pipeline, with a gradient-checked toy proof that a GRPO update actually shifts probability toward correct answers and that the KL coefficient controls reference drift; adversarially found and documented a real ddof discrepancy, rl-grpo.md's numpy illustration uses population std while the book's own reported advantage figure requires PyTorch's sample-std default, a genuine ~13% difference at group size 4, now cross-referenced from both pages).

2026-07-06

• GPU architecture: register bank conflicts and the instruction control word (the register file's physical bank layout and the per-instruction control word, reuse flags, dependency-barrier indices, stall cycles, that decide real kernel throughput below CUDA C++, PTX, and SASS; sourced from two independent microarchitecture-reverse-engineering papers, Zhang et al.'s Kepler bare-metal tuning study and Jia et al.'s Volta microbenchmarking report, cross-checked against each other and against the CMU 15-418 GPU-architecture lecture for foundational framing; with an executed model validating the bank-conflict rule against both papers' own worked examples, including why Volta's wider register banks tolerate a two-of-three operand collision that Kepler's narrower banks do not).
• Inference serving, non-colocated GPUs: split the combined decentralized-and-distributed-inference page into two, now that each pattern has enough of its own material to stand alone: region-replica routing for non-colocated inference (Pattern A, production-composable) and cross-WAN model-parallel inference (Pattern B, research-stage). Both pages carry the same "does a full replica fit at one site?" decision diagram and link to each other from it, so a reader landing on either pattern is routed to the other when it does not apply. decentralized-distributed-inference.md stays at the old public URL as a thin decision stub carrying the same diagram, so existing external links keep resolving. Each new page also gained an executed model beyond what the combined page had room for: region-replica routing gained a composite routing score that adds an M/M/1 queueing-delay term to the existing fetch-versus-recompute crossover, operationalizing GORGO's "jointly account for KV-cache locality, replica load, and variable network latency" cost model into one decision rule instead of three separate signals; cross-WAN model-parallel inference gained a discrete-time simulation answering how many concurrent request streams it takes to hide a WAN pipeline's per-hop round-trip latency, extending the page's existing WAN-speculative-pipelining and pipeline-parallelism material with a closed-queueing-network model cross-checked against Little's Law.
• Inference serving, non-colocated GPUs: a review of the two split pages caught five issues, fixed here. Cross-WAN model-parallel inference's verification section overclaimed that signed per-stage receipts prove "integrity"; checked against the underlying repository's actual receipt.py and its own design docs, the receipt alone proves attribution and layer-coverage (who is accountable, no layer silently skipped), not that a node's output was computed correctly, a signed but fabricated output passes it fine. Rewrote the section around the repository's real, separate challenge.py mechanism (a trusted-recompute spot check compared by cosine similarity, not hash, because floating-point output is not bit-reproducible across GPUs), presented honestly as a probabilistic, economic deterrent the source itself calls open research at scale, not a cryptographic guarantee. Region-replica routing contradicted itself, stating nothing but requests/responses cross the WAN while its own executed model evaluated fetching the KV cache cross-region; reconciled by scoping the fetch-versus-recompute model to a session-migration decision (reducing a long conversation's per-turn round trip, or a redirect-blocking constraint) distinct from the default per-request redirect, which still moves no KV bytes. Also qualified an overclaim that WAN speculative pipelining is a "free option" (free in the throughput/latency sense the model measures, not in verification compute, network bytes, or payment budget), and corrected "one WAN RTT per pipeline stage" to describe what the coordinator-placement model actually computes, a sum of per-hop latencies within one round trip, not a separate round trip per stage.
• Inference serving, non-colocated GPUs: decentralized and distributed inference across non-colocated GPUs's WAN speculative-pipelining model fixed a conflation a third-party review caught: the acceptance-rate crossover where pipelining depth 4 beats synchronous decode was worded next to a specific round-trip time as if RTT set the threshold. Added two adversarial checks to the executed script itself, proving the crossover is unchanged (0.780 vs 0.770, within Monte Carlo noise) across a 3x change in RTT, while absolute throughput scales with RTT almost exactly (3.00x for a 3x RTT change); the crossover depends only on chunk depth K and pipelining depth D.
• Inference serving, non-colocated GPUs: decentralized and distributed inference across non-colocated GPUs's Parallax citation gained real technical depth in place of description: a quoted excerpt of Parallax's actual water-filling layer-rebalance code, and an executed model that independently reproduces the layer-allocator's own worked example (choosing 2 pipelines over 1 for a 6-node, 70-layer case) by brute-force computing its Z(k) = k^2/s*(k) scoring rule, confirming the published design claim rather than restating it; also notes Parallax's MLX/Apple-Silicon backend as a non-GPU hardware class for Pattern B, and its heartbeat-based dynamic node join/leave as a concrete answer to this page's fault-tolerant-membership requirement.
• Inference optimization, MoE: model-data co-scheduling for expert-parallel all-to-all reduction (clustering experts by measured, context-independent token-expert affinity and jointly scheduling expert placement with request/token routing, instead of treating the two as separate problems; the Attention-DP request-rescheduling and Attention-TP fused-shuffle mechanisms; an executed co-clustering model proving the local-activation-rate gain over affinity-blind placement, and that the gain vanishes without real affinity to exploit).
• Inference optimization, KV cache: per-layer heterogeneous KV cache compression (routing a different eviction ratio and K/V bit-width to each transformer layer under one memory budget, instead of one uniform recipe everywhere; why eviction and quantization sensitivity vary by up to 700x across layers at aggressive ratios; an executed greedy solver proving heterogeneous routing beats the best uniform allocation at matched memory, and that the advantage shrinks when layers are not actually heterogeneous).
• Inference serving, non-colocated GPUs: decentralized and distributed inference across non-colocated GPUs rewrote its speculative-decoding-over-WAN section to remove changelog-style framing and vendor-narrated claims: the WAN acceptance-gated law, coordinator placement, and the integrity/confidentiality line are now presented as durable KB knowledge, cross-checked against and extending this KB's own already-validated speculative-decoding math with a new executed discrete-event simulation, and illustrated with real, cited code excerpts (placement search, signed-receipt verification, fault-tolerant healing) from the underlying open-source implementation rather than paraphrased claims about it.
• Inference serving, AMD GPUs: serving and RL post-training on AMD GPUs (ROCm) (the ROCm stack, engines, and AMD's Quark quantization toolkit for serving known models on Instinct GPUs; the cross-vendor portability question, whether NVIDIA-trained weights can be served and RL post-trained on AMD, answered against verl's cross-hardware RL baselines, an architecture-aware AITER benchmark study, an LLNL cross-vendor numerics study, and production llm-d/OCI MI300X deployments; with an executed model quantifying why AMD recalibrates quantization rather than reusing another vendor's checkpoint).
• Inference serving, non-colocated GPUs: decentralized and distributed inference across non-colocated GPUs gained a section on whether MoE makes this easier (it is a real tension, not a clean win: sparse expert placement and DHT-style lookup help at low concurrency, following the Learning@home lineage that precedes Petals, but the every-layer all-to-all makes production-concurrency decentralization harder than a dense model, a gap none of HexGen, HexGen-2, Helix, or Parallax close).

2026-07-05

• Inference serving, non-colocated GPUs: decentralized and distributed inference across non-colocated GPUs (the serving counterpart to the DiLoCo geo-distributed training recipe: region-replica routing versus cross-WAN model-parallel splitting (Petals, HexGen, HexGen-2, Helix, Parallax), why the KV cache is the hard part, and an executed model of when fetching a cached prefix cross-region beats local recompute).
• Post-training, on-policy distillation: on-policy distillation gained a documented failure mode, prefix failure (dense per-token supervision fragmenting the gradient under a bimodal teacher mixture) and its trajectory-level fix, Trajectory-Refined Distillation, surfaced while checking a research essay against the KB's existing OPSD coverage.

2026-07-03

A six-runbook set for building a managed/serverless RL training API, plus eight pages from a verified source batch: 4-bit precision, GenAI observability, multimodal serving, the harness foundry, the OpenHands SDK, and agentic operations. Later the same day: local inference for DeepSeek V4, the Ornith-1.0 serving cookbook, the KV-cache speedup foundations page, the RL cluster bring-up recipe, DeepSeek's DSpark speculative decoding, platform engineering as a paradigm, provider-API prompt caching, and video KV-cache quantization.

• Inference serving, speculative decoding: DSpark speculative decoding (DeepSeek) (the DeepSpec release, verified against the paper PDF, repo, and model cards: why parallel drafters suffer suffix decay and autoregressive ones cap position-1 capacity, the semi-autoregressive Markov head that fixes it, and confidence-scheduled verification that sizes each request's verify budget against a profiled engine-capacity curve; with executed reproductions of the multi-modal-collision acceptance math and the Appendix A losslessness counterexample, plus the V4 production numbers and their honest reading).
• Post-training & RL, applied: Recipe: RL cluster bring-up (k3s + KubeRay + verl) (the end-to-end assembly this KB previously covered only in parts: GPU nodes into k3s, the device plugin, the KubeRay operator, a gang-shaped RayCluster on the verl image, the GRPO job over the Ray job API, and the colocated-vs-disaggregated weight-sync decision, with a verification gate at every layer boundary and an executed model of the sync knobs proving the worst-case staleness bound, window-delta composition, and anchor recovery).

• Inference serving, KV-cache foundations: how the KV cache speeds up LLM inference (the quadratic-recompute arithmetic the cache eliminates, validated by counting every multiply-accumulate in a cached-vs-uncached decode with corruption and staleness checks; the bandwidth-bound decode regime it creates; and the executed capacity model behind vLLM's growth ladder for production throughput and concurrency, from single-GPU knobs through FP8 KV, TP sized by KV headroom rather than fit, data-parallel replicas, and disaggregation).

• Local serving: DwarfStar (ds4), antirez's single-model engine that runs the 284B DeepSeek V4 Flash on 96-128 GB personal machines: asymmetric 2-bit expert quantization behind an 8-bit skeleton, SSD expert streaming that turns RAM into a speed dial, the disk-first KV cache, and layer-split distributed inference over TCP, with validated quantization and cache-simulation examples.

• Managed RL training API runbooks: RL training-step data-path review (quantify every byte moved per step before shipping a stack), RL checkpoint/resume validation (certify a resumed run is the same experiment), rollout fleet sizing and rebalancing (rate-match the rollout fleet to the trainer), untrusted reward-function onboarding (isolate customer reward code from the trainer), RL training API observability bring-up (run-health alert catalog on top of the platform SLOs), and RL training API contract design review (the platform/ML interface contract with a job-spec linter).

• Precision: NVFP4 across the model lifecycle (E2M1 plus two-level scaling, FP4 training, the NVFP4 KV cache, SageAttention3, and the LongLive-2.0 video case study).

• Observability: GenAI observability with OpenTelemetry (the gen_ai.* span/metric/event conventions, content-capture policy, token-usage rollups).
• Serving: vLLM-Omni, stage-level disaggregation for any-to-any multimodal models (AR plus diffusion pipelines, per-stage pools, the unified connector).
• Agentic systems: HarnessX (a composable harness foundry with trace-driven evolution behind a deterministic gate) and the OpenHands Agent SDK (the V1 re-architecture: event-sourced state, typed tools, measured 61% failure reduction).
• Security evaluation: CTI-REALM in depth (environment construction, the five-checkpoint reward, full results and ablations of the detection-engineering benchmark).
• Operations: agentic incident management (OpsAgent: task-aligned experts, cross-review, dual self-evolution, and a 53-day production deployment) and AIOpsLab (the agent-cloud interface, fault library, and protocol for evaluating AIOps agents end to end).
• Platform architecture: platform engineering for GPU clusters (the internal developer platform paradigm as an architecture: intent compiled into resources under platform policy, plane separation, golden paths, and the validated render/admission/reconcile kernel with its GPU-specific failure modes).
• Agentic systems, cost engineering: prompt caching (provider APIs) (the hosted-API caching contract: cache_control breakpoints and TTL refresh, the 1.25x/2x write versus 0.1x read economics with executed break-even math, tiered invalidation, the 20-block lookback, and the harness discipline that keeps a prefix warm).
• Inference serving, KV-cache depth: video KV cache quantization (autoregressive video generators outgrow their own weights within seconds of 480p; Quant VideoGen's semantic-aware smoothing and progressive residual quantization hold the cache at INT2 for about 7x savings at 1.5-4.3% overhead, completing the video half of NVIDIA's KV-compression analysis alongside KV cache token eviction and the NVFP4 LongLive-2.0 case study).
• Serving: the Ornith-1.0 vLLM cookbook (DeepReinforce's self-scaffolding agentic coding family: RL that trains the model to author the harness driving its own rollouts, with a three-layer reward-hacking defence and staleness-weighted GRPO; a 397B Qwen 3.5 MoE flagship with hybrid linear attention, plus the executed FP8-vs-BF16 fit math showing only the FP8 checkpoint fits the card's 8x80GB quickstart shape).

2026-07-02

Data curation and model merging, an evaluation harness and experiment tracking, more serving, and a nine-page batch spanning loop engineering, harness search, MoE fine-tuning, DRA networking, and time-series forecasting.

• Inference serving, KV-cache depth: KV cache token eviction and compaction (the eviction method families and the two infrastructure walls from NVIDIA's KV-compression analysis: FlashAttention never exposes attention scores, and paged allocators free only fully-empty blocks, with the compaction pass that clears the second wall), plus the companion runbook: KV compression, no savings for triaging an evictor that frees nothing (wrong gauge, eager-attention fallback, or scattered survivors).
• Systems papers, second batch: inference disaggregation in practice (NVIDIA's design-space study: when the prefill/decode split pays and how rate matching sizes the pools), evaluating speculative decoding (SPEED-Bench and the data-dependence of acceptance rates), cuTile Rust (Rust ownership extended to tile-based GPU kernels at zero measured cost), and Legate Sparse (distributed scipy.sparse on the Legion runtime, composing with cuPyNumeric).
• Agentic systems, harness engineering: loop engineering (the layer above the harness: five moves, six parts, generator/evaluator separation, Stripe's Minions case study), automated harness optimization (the Meta-Harness loop on Harvey's Legal Agent Benchmark: freeze the model, search the scaffold), skill optimization (SkillOpt: training the skill document in text space with a held-out acceptance gate), the OpenHands (OpenDevin) agent platform (event-stream architecture, sandboxed runtime, 15-benchmark harness), and multi-agent collaboration (role-specialized LLM teams and bounded adversarial debate, via TradingAgents).
• Fine-tuning: NeMo AutoModel, accelerated MoE fine-tuning on Transformers v5 (expert parallelism on a dedicated mesh, DeepEP dispatch, TransformerEngine kernels).
• Serving: LLM inference efficiency, an orienting convergence map (prefill vs decode economics, the bandwidth ladder, weight and KV compression) distilled from Smola's MLSS 2026 tutorial.
• Cluster platform: Kubernetes network drivers, DRA-based declarative NIC allocation with NRI runtime hooks (DraNet, GPU/NIC PCI alignment).

• Strategy and operations: time-series foundation models, zero-shot telemetry forecasting with TimesFM for capacity planning and alert baselines.

• Post-training knowledge distilled from Thinking Machines' tinker-cookbook: Tinker (training-as-a-service) joins the RL-library pages, alongside the generalized companions chat rendering and token loss masking (the renderer layer, TrainOnWhat policies, round-trip parsing) and LoRA hyperparameter scaling rules (the 10x LR rule, capacity-driven rank selection).

• Security evaluation: evaluating AI agents on cybersecurity tasks (CyberGym vulnerability reproduction, CTI-RealM detection engineering, the Inspect Evals framework, AgentGym), and RLSD now cross-links the SDPG self-distilled policy-gradient sibling.

• Serving: the GLM-5.2 vLLM cookbook, multi-LoRA / adapter serving, and model weight loading in inference engines.

• Data and post-training: model merging (SLERP, TIES, DARE), synthetic data generation, and training-data curation and decontamination.
• Training optimizers: Muon and distributed Muon (DMuon), a matrix-orthogonalization optimizer brought to near-AdamW overhead in sharded distributed training.
• Evaluation and MLOps: the LLM evaluation harness and eval gate and experiment tracking and model registry.
• Performance: software performance engineering for FMware, meeting throughput and latency SLOs across the cognitive architecture, communication, tuning, and deployment of FM-powered software.
• Agents and local serving: running local coding agents (open-weight coders via Ollama/vLLM behind a coding harness), and enriched self-improving harnesses with the 2026 harness papers (Self-Harness, AutoHarness, Meta-Harness, LLM-as-Code, code-as-harness survey).
• Architecture: looped and recurrent-depth transformers, weight-tied iterative latent depth as a scaling axis orthogonal to model size (LoopWM, Universal Transformers, Adaptive Computation Time).
• RL systems: rollout redundancy in RL (prompt deduplication and cascade attention), delta weight sync (sparse, bit-identical trainer-to-rollout weight synchronization), and RLSD (reinforcement learning with self-distillation).
• Evaluation and RL depth: LLM benchmarks (anatomy and metrics), RL scaling laws, and GRPO variants and training tricks.
• Quality: the remaining legacy pages were humanized, and CI now gates every page on pristine prose, coherent structure, and changelog freshness.
• Feeds: the knowledge base now publishes RSS and JSON feeds of new and updated pages, so you can subscribe to changes instead of polling this page (links under References).

2026-07-01

Inference request routing and reinforcement-learning post-training, plus large-model serving cookbooks.

• Inference routing: LLM request routing (Mixture-of-Models) and the vLLM semantic router.
• RL post-training and evaluation: on-policy distillation, RLVR (reinforcement learning with verifiable rewards), autonomous experimentation loops, evaluation integrity and anti-gaming, and learning-curve extrapolation and early stopping.
• vLLM serving cookbooks: DeepSeek-V3.2-Exp, MiniMax-M2, and small models on consumer GPUs.
• Cluster platform: dynamic and fractional GPU sharing.

2026-06-29

The agentic-systems section landed, alongside GPU-platform services and more RL post-training.

• Agentic systems: start at the agentic systems index. Core pages include the agent loop, harness architecture, orchestration control plane, planning and reasoning, tools and function calling, evaluation, and observability, with a security set covering the threat model, sandboxing and isolation, identity and access, and prompt-injection defense.
• GPU platform services: confidential computing, split-plane architecture, the operator for GPU orchestration, remote GPU verification, and container-image provenance.
• RL post-training: agentic RL, async RL systems, PPO, reward-model training, rejection sampling and best-of-N, and reward design.
• Using the KB itself: use as an agent skill.

2026-06-24 to 2026-06-28

Initial knowledge base: the foundational pages across GPU hardware and commissioning, the cluster platform, distributed training, inference serving, RL post-training, observability, and SRE and MLOps, plus the runnable recipes. Reach these through the section tabs, the Start here guide, and the recipes and manifests index.

References

• Per-page update dates appear in each page footer (git revision date localised).
• The authoritative record of changes is the docs/ git history of this knowledge base.
• Subscribe to updates: RSS at https://ai-infrastructure.net/rss.xml and JSON Feed at https://ai-infrastructure.net/feed.json. Feeds of newly created pages are also published as rss-created.xml and feed-created.json. Every page is included.

Related: Start here · Recipes index · Agentic systems index · Glossary