Skip to content
Markdown

slime (THUDM)

Scope: Tsinghua and Z.ai's decoupled, async-first RL post-training framework, including its Megatron plus SGLang architecture and current disk-only delta checkpoint path.

Reference templates use real APIs. The delta path is newer than slime v0.3.0; pin an exact main commit and its bundled SGLang patch before production use.

What it is

slime is an LLM post-training framework that connects Megatron-LM training with SGLang rollout, orchestrated by Ray. Its three main components are a training pool, a rollout pool plus Router, and a central Data Buffer. The pools run asynchronously and scale independently. The framework is used for GLM-family RL and also supports Qwen and DeepSeek-V3-class model layouts; verify the current model list on the repository. See RL libraries for comparison.

Why use it

  • Large dense and MoE RL. Megatron supplies tensor, pipeline, and expert parallel training while SGLang supplies rollout throughput.
  • Async resource isolation. Training and rollout use separate pools, so generation can overlap updates and scale independently.
  • Small extension surface. Megatron and SGLang flags pass through, while custom generation and reward logic load by module path.
  • Checkpoint-byte delta sync. The main branch can publish compressed XOR or overwrite records through shared storage for non-colocated rollouts.

When to use it (and when not)

  • Use slime for large dense or MoE post-training when the target stack is Megatron plus SGLang and separate rollout capacity is required.
  • Prefer verl for the broadest algorithm and backend ecosystem, or SkyRL when switching freely among rollout providers is central.
  • Do not select slime's delta mode for colocated rollout or NCCL transport. The executable argument validator accepts only non-colocated disk transport.

Architecture

flowchart LR
  subgraph Train["Train pool: Megatron-LM"]
    T["Actor update"]
    D["Full checkpoint or byte delta"]
  end
  subgraph Roll["Rollout pool: patched SGLang plus Router"]
    P["Pull and patch local checkpoint"]
    G["Generate and score"]
  end
  BUF["Data Buffer"]
  G -->|"trajectories and rewards"| BUF
  BUF -->|"training batches"| T
  T --> D
  D -->|"NCCL full sync or disk publication"| P
  P --> G
  RAY["Ray orchestration"] -.-> Train
  RAY -.-> Roll

How to use it

slime is distributed as a Docker image (matched Megatron + SGLang). Pin a tagged release rather than latest:

docker pull slimerl/slime:<pinned>     # e.g. a dated release tag; check the repo
docker run --rm --gpus all --ipc=host --shm-size=16g \
  --ulimit memlock=-1 --ulimit stack=67108864 -it slimerl/slime:<pinned> /bin/bash

# inside the container:
cd /root/slime && git pull && pip install -e . --no-deps

# start Ray, then submit a run:
ray start --head --node-ip-address ${MASTER_ADDR} --num-gpus 8 --disable-usage-stats
ray job submit --address="http://127.0.0.1:8265" -- \
  python3 train.py --actor-num-nodes 1 --actor-num-gpus-per-node 4 --rollout-num-gpus 4

The fastest start is an example script, e.g. bash scripts/run-glm4-9B.sh.

How to develop with it

slime's current argument validator permits delta sync only with disk transport and non-colocated rollout. This executed contract test covers the valid tuple and both rejected boundaries:

# slime_delta_contract.py: executed argument-contract model.
def validate_delta(mode, transport, colocate):
    if mode == "delta" and transport != "disk":
        raise ValueError("delta weight sync requires disk transport")
    if mode == "delta" and colocate:
        raise ValueError("delta weight sync requires non-colocated rollout")
    return True


assert validate_delta("delta", "disk", False)
failures = 0
for invalid in [("delta", "nccl", False), ("delta", "disk", True)]:
    try:
        validate_delta(*invalid)
    except ValueError:
        failures += 1
assert failures == 2
print("slime_delta valid=True rejected_invalid=2")

Executed output:

slime_delta valid=True rejected_invalid=2

Runs are shell scripts that group arguments into arrays: MODEL_ARGS (Megatron model, sourced from scripts/models/<model>.sh), CKPT_ARGS, ROLLOUT_ARGS, PERF_ARGS, GRPO_ARGS (algorithm), and SGLANG_ARGS. Megatron flags pass through verbatim; SGLang flags take a --sglang- prefix (so SGLang's --mem-fraction-static becomes --sglang-mem-fraction-static).

Custom generation / reward plug in by path, not by editing core:

python3 train.py \
  --rollout-function-path my_pkg.rollout.generate \
  --custom-generate-function-path my_pkg.gen.multi_turn \
  --rollout-num-gpus 8

The custom function receives prompts from the Data Buffer and returns completions plus rewards/verifier outputs.

How to scale it

Decoupled async means rollout and train pools scale independently across nodes. Add worker nodes to Ray, then size each pool:

# head:
ray start --head --node-ip-address ${MASTER_ADDR} --num-gpus 8 --disable-usage-stats
# each worker:
ray start --address=${MASTER_ADDR}:6379 --num-gpus 8
# submit with separate train/rollout sizing across the cluster:
ray job submit --address="http://127.0.0.1:8265" -- \
  python3 train.py --actor-num-nodes 4 --actor-num-gpus-per-node 8 --rollout-num-gpus 32

MoE training uses Megatron expert, tensor, and pipeline parallelism set through MODEL_ARGS (tensor parallelism, pipeline parallelism); rollout scales by adding SGLang replicas behind the Router.

Weight sync can stream full actor weights or publish disk deltas. The current delta path is non-colocated and disk-only: the trainer diffs raw canonical-checkpoint bytes against its snapshot, compresses XOR or overwrite records with zstd level 1, and writes a version directory. Each rollout host maintains a complete checkpoint on local NVMe. The shipped slime SGLang patch exposes /pull_weights to pull, apply, and verify the version on every host; only the final disk reload uses the ordinary weight loader.

A pinned main-branch reference template for this path is:

# Reference template from slime commit ea9819f88caa5e043eb8aea992b0969ffe79aa8e.
python3 train.py \
  --update-weight-mode delta \
  --update-weight-transport disk \
  --update-weight-disk-dir /shared/fs/delta-updates \
  --update-weight-local-checkpoint-dir /local/nvme/rollout-ckpt \
  --update-weight-delta-encoding overwrite \
  --update-weight-delta-checksum xxh3-128

Supported encodings are xor and overwrite; supported checksums are xxh3-128, blake3, and adler32. Gate object-store or cross-host visibility with --custom-update-weight-post-write-path and --sglang-custom-pull-weights-pre-read-hook. The older --custom-delta-pre-push-path and --custom-delta-pre-read-path names are not current. See delta weight sync for recovery semantics.

Inference

slime does not serve external traffic; rollout uses SGLang internally to generate trajectories (multi-turn capable via the custom generate function). BF16 training with FP8 rollout is supported to cut generation cost. For standalone serving of a trained GLM checkpoint, use the serving stack; for disaggregated serving concepts that mirror slime's train/rollout split, see disaggregated inference.

Fine-tuning

slime is an RL post-training framework (GRPO/PPO-style updates on Megatron), used to RL-tune GLM-class models on top of an SFT/instruct base. For the RL methods see GRPO and the post-training overview; warm-start from SFT/LoRA (SFT and LoRA) before RL.

How to run it in production

Run training and rollout as separate Ray resource pools and export queue depth, policy-version age, rollout throughput, and trainer idle time. For disk delta mode, provision one complete canonical checkpoint on local NVMe per rollout host, verify shared-storage visibility before calling /pull_weights, and record encoded bytes, write time, pull time, checksum failures, and full-version recovery. Prefer overwrite records where notifications or pulls can retry. Seed new hosts from version 0 or an operator-published full version before admitting them to rollout.

Optimised hardware

  • Async weight sync can use the full NCCL path on a fast InfiniBand/RoCE with GDR fabric, or the main-only disk delta path across a shared filesystem or object-store mount. Delta mode still needs the slime SGLang patch for /pull_weights; it is not an engine-independent control path.
  • Megatron MoE training relies on NVLink/NVSwitch for intra-node expert/tensor parallel all-to-all; NCCL env (NCCL_IB_HCA, plus NCCL_NET_GDR_LEVEL only as a measured override since NCCL auto-selects the GDR cutoff from topology, and NCCL_NVLS_ENABLE which already defaults to automatic mode 2) and ACS-off are prerequisites.
  • FP8 rollout / Blackwell: FP8 generation on Blackwell (the Blackwell platform) lowers rollout cost; keep training in BF16 and verify reward parity.

How to maintain it

Pin slime and its bundled SGLang patch as one unit. After either changes, rerun the valid and rejected argument cases, a full checkpoint load, a delta reconstruction, duplicate-notification handling, and forced full recovery. Audit documentation against slime/utils/arguments.py; the current external-rollout recommendation for delta + nccl conflicts with the executable validator. Keep local checkpoint capacity and stale version directories under explicit retention limits.

Cookbook (common use cases)

1) A GLM-class RL run (single node): 

cd /root/slime && bash scripts/run-glm4-9B.sh     # edit ROLLOUT_ARGS / GRPO_ARGS inline

2) Custom rollout generation (multi-turn / agentic): 

python3 train.py \
  --rollout-function-path my_pkg.rollout.generate \
  --custom-generate-function-path my_pkg.agent.loop \
  --sglang-mem-fraction-static 0.8 --rollout-num-gpus 8

3) MoE training (multi-node, separate pools): source a MoE MODEL_ARGS with expert/tensor parallel sizes, start Ray head+workers, then ray job submit ... -- python3 train.py --actor-num-nodes 4 --actor-num-gpus-per-node 8 --rollout-num-gpus 32.

Failure modes

  • SGLang flags need the --sglang- prefix: an un-prefixed SGLang flag is silently ignored or rejected; mirror SGLang's own names underneath.
  • Pool sizing: an under-provisioned rollout pool starves the train pool (and vice versa); tune --rollout-num-gpus against --actor-num-* and watch GPU idle.
  • Async staleness: too-stale rollouts hurt convergence; keep weight-sync cadence tight and monitor reward/entropy/KL for collapse (GRPO).

  • Megatron checkpoint format: model args must match the converted Megatron checkpoint exactly, or load fails. Source the matching scripts/models/<model>.sh.

  • Unsupported delta + nccl. The current executable argument validator rejects this combination even though the external-rollout guide still recommends it. Follow the validator until the upstream documentation is corrected.

  • XOR replay after a partial apply. XOR is non-idempotent; replay can revert sections already applied. Prefer overwrite on retry-heavy storage and force a full version after an ambiguous failure.
  • Missing base checkpoint. slime does not publish scheduled anchors. A fresh host must seed version 0 from the model path or receive an operator-published full version.
  • Assuming dtype support implies compression. The byte codec can reconstruct quantized canonical checkpoints, but a changed shared scale can make their delta dense. Measure encoded bytes.

References

  • slime delta-weight-sync documentation: https://github.com/THUDM/slime/blob/main/docs/en/advanced/delta-weight-sync.md

  • slime PR #2181, delta checkpoint synchronization improvements: https://github.com/THUDM/slime/pull/2181

  • slime repo: https://github.com/THUDM/slime

  • slime docs: https://thudm.github.io/slime/
  • GLM-4.5 (ARC foundation models): https://arxiv.org/abs/2508.06471
  • Anyscale — Open Source RL Libraries for LLMs: https://www.anyscale.com/blog/open-source-rl-libraries-for-llms

Related: RL libraries · OSS models · Post-training · GRPO · verl · SkyRL · Delta weight sync · Glossary