Markdown

Cookbook: serve GLM-5.2 with vLLM¶

Scope: a vLLM reference template for serving zai-org/GLM-5.2-FP8, Z.ai's current flagship long-horizon agentic coding and reasoning model: what GLM-5.2 is (a ~743-753B / ~39B-active MoE with Dynamic Sparse Attention and a 1M-token context), why and when to use it, how to launch the 8-GPU tensor-parallel FP8 shape with MTP speculative decoding and the glm47/glm45 parsers, and how to verify thinking mode and tool calls.

Reference template. Commands and manifests are not executed here. GLM-5.2 needs vLLM 0.23.0 or newer, and the model card notes that using tool calling and MTP speculative decoding at the same time currently requires the vLLM main branch. Pin an exact vLLM image (main-derived if you need both features), DeepGEMM-capable where required, and the model revision, then validate on one node before production.

What it is¶

GLM-5.2 is Z.ai's flagship model for long-horizon tasks, released under MIT on 2026-06-17. It is a Mixture-of-Experts checkpoint of roughly 743B total parameters with ~39B active per token per vLLM's own recipe; the Hugging Face card's safetensors metadata totals 753B (the two sources differ, likely a counting-convention gap such as whether MTP-module weights are included). It succeeds GLM-5 and GLM-5.1 and is a large step up from the GLM-4.7 line for agentic coding.

Two architectural pieces define it:

Dynamic Sparse Attention (DSA) with IndexShare. Every four transformer layers share a lightweight indexer that selects which keys each query attends to, so three of every four layers skip the full indexer dot-product and top-k. Z.ai reports this cuts per-token FLOPs by ~2.9x at 1M-token context, which is what makes the long-context window affordable to serve.
Improved MTP layer. The Multi-Token Prediction module used for speculative decoding shares the indexer and KV cache, raising acceptance length by up to ~20% over the earlier GLM-5 MTP.

Context is native up to 1M tokens, with up to 163,840 output tokens. It exposes thinking mode with effort control (high, max) and structured tool calling, and integrates with coding agents such as Claude Code, OpenCode, and ZCode. It is strongest on long-horizon agentic coding and design/front-end generation; reported scores include SWE-bench Pro 62.1 (GLM-5.1: 58.4), DeepSWE 46.2 (SOTA among open-weight models at release), and Terminal-Bench 2.1 81.0.

Why use it¶

Use GLM-5.2 when the product is a long-running coding or tool-use agent and you want an open-weight (MIT) model that sustains quality over hundreds of tool-call rounds at a very long context. The small ~39B active fraction keeps decode cheap relative to the ~743-753B footprint, DSA keeps long-context attention sub-quadratic, and the MTP layer adds speculative-decoding throughput. It is the highest-capability open-weight agentic coder in this cookbook set at release.

When to use it (and when not)¶

Use it when:

Long-horizon agentic coding, repository editing, terminal tasks, or multi-step tool use dominate.
You need a very long context (well beyond 128K) at a serving cost that a dense model cannot match.
A permissive MIT license and an OpenAI-compatible tool/reasoning path matter.
The platform can run vLLM 0.23.0+ (or main when you need tool calling and MTP together).

Avoid it when:

The serving platform only allows conservative stable images and cannot validate a recent/main vLLM build.
The workload is short chat or minimal-latency completion where a smaller dense model meets the SLO.
You cannot allocate an 8-GPU (H200/H20/B200-class) node; the FP8 checkpoint targets a single 8-GPU node and BF16 is multi-node.

Architecture¶

GLM-5.2 is a Mixture-of-Experts checkpoint of roughly 743B total parameters with ~39B active per token per vLLM's own recipe (the Hugging Face card's safetensors metadata totals 753B; the two sources differ, likely a counting-convention gap). Two pieces define the served architecture: Dynamic Sparse Attention (DSA) with IndexShare, where every four transformer layers share a lightweight indexer that selects which keys each query attends to, so three of every four layers skip the full indexer dot-product and top-k step, cutting per-token FLOPs by ~2.9x at 1M-token context; and an improved Multi-Token Prediction (MTP) layer for speculative decoding that shares the indexer and KV cache with the main model, raising acceptance length by up to ~20% over the earlier GLM-5 MTP. Context runs native up to 1M tokens, with up to 163,840 output tokens. The model is served in FP8 (E4M3) or BF16, with an nvidia/GLM-5.2-NVFP4 variant also published. It exposes thinking mode with effort control (high, max) and structured tool calling, and is the successor to GLM-5 and GLM-5.1 (a step up from the GLM-4.7 line for agentic coding).

flowchart LR
  PROMPT["Long-horizon coding / agent task"] --> VLLM["vLLM server (TP=8)"]
  VLLM --> DSA["Dynamic Sparse Attention + IndexShare (up to 1M ctx)"]
  VLLM --> MTP["MTP speculative decode"]
  DSA --> PARSERS["glm47 tool parser / glm45 reasoning parser"]
  MTP --> PARSERS
  PARSERS --> API["OpenAI-compatible response"]

How to use it¶

1. Size the replica¶

Item	Practical note
Model	`zai-org/GLM-5.2-FP8` (FP8) or `zai-org/GLM-5.2` (BF16)
Params	~743B total / ~39B active MoE per vLLM's recipe; HF safetensors metadata totals 753B
Attention	Dynamic Sparse Attention (DSA) + IndexShare
Context	native up to 1M tokens
Max output	163,840 tokens
Precision	FP8 (E4M3) or BF16; an `nvidia/GLM-5.2-NVFP4` variant also exists
Speculative	improved MTP layer (+~20% acceptance length)
License	MIT
Starting hardware	FP8: 8x H200 / H20 single node (TP=8); BF16: multi-node; full 1M context: 8x B200
vLLM	0.23.0+ stable; `main` for tool calling + MTP simultaneously

The official vLLM recipe uses --tensor-parallel-size 8 for the FP8 checkpoint on a single 8x H200/H20 node. Use that as a smoke baseline, then raise context or lower --max-num-seqs after measuring KV-cache capacity. SGLang (v0.5.13.post1+) and Transformers are also supported per the model card.

2. Bare-metal vLLM server (FP8, 8 GPUs, MTP + tool calling)¶

vllm serve zai-org/GLM-5.2-FP8 \
  --tensor-parallel-size 8 \
  --kv-cache-dtype fp8 \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 5}' \
  --tool-call-parser glm47 \
  --reasoning-parser glm45 \
  --enable-auto-tool-choice \
  --served-model-name glm-5.2-fp8

GLM-5.2 reuses the GLM-4.5/4.7 parsers: --tool-call-parser glm47 and --reasoning-parser glm45. The official vLLM recipe writes the speculative config in the equivalent dotted form (--speculative-config.method mtp --speculative-config.num_speculative_tokens 5); the JSON-string form above is the form shown in the vLLM MTP docs and both are accepted. Tool calling and MTP together currently need the vLLM main branch; if you pin 0.23.0 stable and hit a conflict, drop --speculative-config or move to a main-derived image.

Full 1M-context variant (8x B200) adds an FP8 KV cache, caps concurrency, and skips the DeepGEMM warmup:

VLLM_DEEP_GEMM_WARMUP=skip vllm serve zai-org/GLM-5.2-FP8 \
  --tensor-parallel-size 8 \
  --kv-cache-dtype fp8_e4m3 \
  --max-num-seqs 32 \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 5}' \
  --tool-call-parser glm47 \
  --reasoning-parser glm45 \
  --enable-auto-tool-choice \
  --served-model-name glm-5.2-fp8

3. Thinking-mode smoke test¶

curl -s http://<service-url>:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "glm-5.2-fp8",
    "messages": [{"role": "user", "content": "Design and implement a minimal LRU cache in Python, then explain the eviction invariant."}],
    "temperature": 0.7,
    "top_p": 1.0,
    "max_tokens": 4096
  }'

Thinking is on by default ("think max"). To disable it for a latency-sensitive route, pass the chat-template flag the model card documents:

curl -s http://<service-url>:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "glm-5.2-fp8",
    "messages": [{"role": "user", "content": "Return only the final answer: 2**20 - 1."}],
    "max_tokens": 256,
    "chat_template_kwargs": {"enable_thinking": false}
  }'

4. Tool-call smoke test¶

curl -s http://<service-url>:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "glm-5.2-fp8",
    "messages": [{"role": "user", "content": "Use the search_docs tool to find CUDA driver upgrade instructions."}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "search_docs",
        "description": "Search internal documentation.",
        "parameters": {
          "type": "object",
          "properties": {"query": {"type": "string"}},
          "required": ["query"]
        }
      }
    }],
    "tool_choice": "auto",
    "max_tokens": 1024
  }'

Pass criteria:

The model loads on the pinned vLLM image and /health returns 200 with a non-zero KV-cache size in the logs.
Tool calls are structured, not plain text.
Reasoning metadata is present when thinking is on, and absent when enable_thinking is false.
With MTP enabled, the logs report a non-trivial speculative acceptance rate; measure TTFT/TPOT separately for coding and tool-call routes.

How to run it in production¶

apiVersion: apps/v1
kind: Deployment
metadata:
  name: vllm-glm-5-2
  namespace: serving
spec:
  replicas: 1
  selector:
    matchLabels: { app: vllm-glm-5-2 }
  template:
    metadata:
      labels: { app: vllm-glm-5-2 }
    spec:
      nodeSelector:
        accelerator.nvidia.com/class: h200-8gpu
      containers:
        - name: vllm
          image: vllm/vllm-openai:<tested-0.23+-or-main-tag>
          args:
            - --model=zai-org/GLM-5.2-FP8
            - --tensor-parallel-size=8
            - --kv-cache-dtype=fp8
            - '--speculative-config={"method": "mtp", "num_speculative_tokens": 5}'
            - --tool-call-parser=glm47
            - --reasoning-parser=glm45
            - --enable-auto-tool-choice
            - --served-model-name=glm-5.2-fp8
          ports:
            - { containerPort: 8000, name: http }
          env:
            - { name: NCCL_DEBUG, value: INFO }
          resources:
            limits: { nvidia.com/gpu: 8 }
            requests: { nvidia.com/gpu: 8 }
          readinessProbe:
            httpGet: { path: /health, port: 8000 }
            initialDelaySeconds: 240
          volumeMounts:
            - { name: dshm, mountPath: /dev/shm }
      volumes:
        - name: dshm
          emptyDir: { medium: Memory, sizeLimit: 64Gi }
---
apiVersion: v1
kind: Service
metadata:
  name: vllm-glm-5-2
  namespace: serving
spec:
  selector: { app: vllm-glm-5-2 }
  ports:
    - { name: http, port: 8000, targetPort: 8000 }

How to maintain it¶

Pin an exact vLLM image and the model revision, and validate on one node before production; the FP8 checkpoint and the tool-calling/MTP combination are both sensitive to the exact build.
Track the MTP + tool-calling stable/main gap. Both features together currently require the vLLM main branch; re-check with each vLLM release whether the stable line has closed the gap before moving off a main-derived image.
Pin a digest on main/nightly images and re-run the tool-call and reasoning smoke tests before rollout. Main and nightly images can silently change parser or MTP behavior between builds.
Re-check KV-cache sizing and --max-num-seqs before raising context toward the 1M ceiling. 1M is a ceiling, not a default; --max-num-seqs 32 is only the measured value for the 1M-context 8x B200 shape, not a general default.
Watch the MTP acceptance rate and TTFT/TPOT, not just at smoke-test time; measure them separately for coding and tool-call routes after any image or config change.

How to develop with it¶

This page does not document a GLM-5.2-specific fine-tuning or LoRA recipe: none of the sources cited below (the model card, the Z.ai blog, or the vLLM recipe) publish one for this checkpoint. General SFT/LoRA/DPO/GRPO post-training practice applies to the released checkpoints (fine-tuning, SFT/LoRA); check the current Z.ai Hugging Face org and model card for any recipe they publish before assuming one exists.

Failure modes¶

glm5_moe/GLM-5.2 unsupported: vLLM is older than 0.23.0; upgrade or use a main-derived image.
MTP and tool calling conflict on stable: using both at once currently needs vLLM main; drop --speculative-config or move to a main-derived image.
Tool calls not parsed: missing --enable-auto-tool-choice or --tool-call-parser glm47.
Reasoning content missing: missing --reasoning-parser glm45, or the client ignores the extra field, or enable_thinking was set false.
OOM at long context: 1M is a ceiling, not a default. Cap --max-model-len to the route's real need, lower --max-num-seqs (32 for the 1M B200 shape), and size KV cache accordingly (KV cache management).
FP8 OOM below TP=8 on B200: not independently verified in this KB, but a third-party autotuning report on a GLM-5.2 B200 deployment found FP8 weight re-quantization ran out of memory at TP=4 and required TP=8 as the minimum tensor-parallel degree (Parisio et al., T-Systems AI Foundation Service, 2026); consistent with the TP=8 FP8 starting shape above, but treat the exact OOM boundary as externally reported, not confirmed here.
BF16 on one node: the BF16 checkpoint is a multi-node deployment; use zai-org/GLM-5.2-FP8 for a single 8-GPU node.
Regression after image update: main/nightly images can change parser or MTP behaviour; pin a digest and run a tool-call + reasoning regression test before rollout.

References¶

GLM-5.2 model card: https://huggingface.co/zai-org/GLM-5.2
GLM-5.2-FP8 model card: https://huggingface.co/zai-org/GLM-5.2-FP8
GLM-5.2 blog (DSA/IndexShare, MTP, benchmarks): https://huggingface.co/blog/zai-org/glm-52-blog
vLLM recipe for GLM-5.2 (TP8, MTP, glm47/glm45 parsers, 1M-context B200 shape): https://recipes.vllm.ai/zai-org/GLM-5.2
vLLM MTP speculative decoding (--speculative-config JSON form): https://docs.vllm.ai/en/latest/features/speculative_decoding/mtp/
vLLM tool calling and reasoning parsers: https://docs.vllm.ai/en/latest/features/tool_calling/
Parisio, trongvu, danghungdo, Taofeek (T-Systems AI Foundation Service): An AI-agent cold-tuned our GLM-5.2 serving. Human engineering leveled it up for real production traffic (2026) (third-party B200 deployment report; FP8 OOM at TP=4, TP=8 minimum).