Markdown

NVIDIA DGX Spark playbooks¶

Scope: the DGX Spark Playbooks (NVIDIA/dgx-spark-playbooks, Apache-2.0), NVIDIA's collection of step-by-step guides for running AI/ML workloads on DGX Spark, the Blackwell-architecture desktop AI device. This page covers what the playbook set contains (framework setup, local inference and fine-tuning, agent stacks, and multi-Spark clustering), how the 200GbE interconnect and NCCL bound a multi-node step, and how to use the playbooks as a reference. It is a small-node counterpart to this knowledge base's cluster-scale NCCL and fabric material and sits near cluster bring-up; the local-inference stacks connect to serving open-weight models.

Examined at commit 1fb66f05 (2026-07-29) of NVIDIA/dgx-spark-playbooks, Apache-2.0. The repository holds 66 playbooks under nvidia/, each a README.md with prerequisites, steps, and troubleshooting. The NCCL ring-all-reduce model below is executed and asserted with numpy. No DGX Spark hardware was used: the bandwidth and timing figures are a parameterized model of the ring-all-reduce identity, not a measurement, and the per-playbook commands are read from the repo, not run here. The 200GbE and RoCE details are quoted from the repo's nvidia/connect-two-sparks and nvidia/nccl playbooks.

What it is¶

DGX Spark is a Blackwell-architecture desktop AI device; the playbooks are NVIDIA's curated recipes for getting real workloads running on it. Each playbook is self-contained (prerequisites, step-by-step instructions, troubleshooting, example code) and the set spans the lifecycle:

Framework and runtime setup: llama.cpp, LM Studio, Ollama, vLLM, TRT-LLM, JAX, PyTorch fine-tuning, LLaMA Factory, NeMo, Unsloth.
Inference and quantization: multi-modal inference, speculative decoding, NVFP4 quantization, NIM on Spark, a live VLM WebUI.
Agent stacks: a CLI coding agent, a Hermes agent with local models, NemoClaw agents, and a multi-agent chatbot.
Clustering: connect two Sparks over a direct 200GbE QSFP link, connect three in a ring, connect multiple through a switch, NCCL setup for multiple Sparks, and Tailscale for remote access.
Data science and domain examples: CUDA-X data science, single-cell analysis, topic modeling, portfolio optimization, healthcare agents.

The clustering playbooks are the operationally interesting part, because a single Spark runs local models comfortably but multi-node training and large-model inference depend on the 200GbE interconnect and correct NCCL configuration.

Why use it¶

Vendor-validated recipes. These are NVIDIA's own step-by-step guides for its hardware, so the driver, framework, and interconnect combinations are ones the vendor has run.
Covers the whole device lifecycle. From first network access to fine-tuning to multi-node clustering, the set answers the questions a new Spark owner actually hits.
Local, private inference and fine-tuning. The inference and fine-tuning playbooks target running models on hardware you own, with no cloud dependency, which is the device's reason to exist.
Concrete clustering guidance. The connect-two, connect-three-ring, and NCCL playbooks give the exact interface names, cabling, and test commands for scaling past one node.
Agent stacks included. The CLI coding agent, Hermes, and NemoClaw playbooks show how to run agentic workloads locally on the device.

When to use it (and when not)¶

Use it when you have DGX Spark hardware and want the vendor's validated path for a specific framework, a local model, or a small multi-Spark cluster. It is the right reference for bring-up, for choosing an inference runtime, and for wiring two or three Sparks together correctly.

Do not read it as datacenter-scale guidance: DGX Spark is a desktop device with a 200GbE link and small node counts (two, three, or a small switch fabric), so the clustering advice does not transfer to a rack of DGX systems on InfiniBand (see this knowledge base's networking fabric for that scale). Do not expect the throughput of a multi-GPU server; the value is local, private, Blackwell-class compute at a desk, not peak training throughput. Do not assume a playbook's pinned versions are current; re-check the framework and driver versions against the commit you use.

Architecture¶

flowchart TB
  subgraph SPARK["Single DGX Spark (Blackwell)"]
    RT["Runtime: llama.cpp / vLLM / TRT-LLM / Ollama"]
    FT["Fine-tune: NeMo / LLaMA Factory / Unsloth"]
    AG["Agents: CLI coding, Hermes, NemoClaw"]
  end
  subgraph CLUSTER["Multi-Spark"]
    L200["200GbE QSFP direct or switch"]
    NCCL["NCCL over RoCE interfaces"]
  end
  SPARK --> L200
  L200 --> NCCL
  NCCL --> RING["2-node link / 3-node ring"]

The multi-node step bound, executed¶

When you cluster Sparks, a distributed step's cost is dominated by the collective, and the ring-all-reduce identity is what to reason with: bus bandwidth is busbw = algbw * 2(N-1)/N, and each rank moves 2(N-1)/N * message_bytes over the wire. On a 200GbE fabric that link rate, not the Blackwell compute, bounds the step past two nodes. The numpy model below sizes it for two and three Sparks.

import numpy as np

LINK_GBPS = 200.0                      # per-link 200 GbE QSFP direct connect
LINK_BYTES_PER_S = LINK_GBPS * 1e9 / 8


def ring_allreduce(msg_bytes, n, algbw_bytes_s):
    """Return (busbw, wire_bytes_per_rank, wire_time_s) for a ring all-reduce."""
    factor = 2 * (n - 1) / n
    busbw = algbw_bytes_s * factor
    wire_bytes = factor * msg_bytes
    return busbw, wire_bytes, wire_bytes / LINK_BYTES_PER_S


msg = 1 << 30                          # a 1 GiB gradient buffer
algbw = 0.80 * LINK_BYTES_PER_S        # assume the ring sustains ~80% of line rate

for n in (2, 3):
    busbw, wire, t = ring_allreduce(msg, n, algbw)
    print(f"N={n}: factor={2*(n-1)/n:.3f}  busbw={busbw/1e9*8:6.1f} Gbps  "
          f"wire/rank={wire/2**30:.3f} GiB  time={t*1e3:6.1f} ms")

# 1. The bus-bandwidth identity holds exactly.
busbw2, _, _ = ring_allreduce(msg, 2, algbw)
assert np.isclose(busbw2, algbw * 1.0)             # 2(N-1)/N = 1 at N=2
busbw3, _, _ = ring_allreduce(msg, 3, algbw)
assert np.isclose(busbw3, algbw * (4 / 3))         # = 1.333 at N=3

# 2. Per-rank wire volume grows toward 2x with N: 3 Sparks each move MORE than 2.
_, wire2, _ = ring_allreduce(msg, 2, algbw)
_, wire3, _ = ring_allreduce(msg, 3, algbw)
assert wire3 > wire2
print(f"wire/rank grows {wire2/2**30:.3f} -> {wire3/2**30:.3f} GiB from N=2 to N=3")

# 3. Sanity: at N=2 the wire time equals msg / link_rate exactly.
_, _, t2 = ring_allreduce(msg, 2, LINK_BYTES_PER_S)
assert np.isclose(t2, msg / LINK_BYTES_PER_S)
print(f"N=2 lower-bound wire time at line rate: {t2*1e3:.1f} ms for 1 GiB")
print("OK: ring all-reduce cost scales as 2(N-1)/N; on a 200 GbE Spark ring the "
      "link rate caps step bandwidth, which is why the playbooks tune buffer size")

Executed output:

N=2: factor=1.000  busbw= 160.0 Gbps  wire/rank=1.000 GiB  time=  42.9 ms
N=3: factor=1.333  busbw= 213.3 Gbps  wire/rank=1.333 GiB  time=  57.3 ms
wire/rank grows 1.000 -> 1.333 GiB from N=2 to N=3
N=2 lower-bound wire time at line rate: 42.9 ms for 1 GiB
OK: ring all-reduce cost scales as 2(N-1)/N; on a 200 GbE Spark ring the link rate caps step bandwidth, which is why the playbooks tune buffer size

This is why the NCCL playbook tells you to test with a larger buffer size "to use more of your 200Gbps bandwidth": small messages are latency-bound and never approach line rate, while large messages amortize the per-collective overhead. The nvidia/nccl playbook also shows the RoCE interface mapping (rocep1s0f0 -> enp1s0f0np0) you verify before a run, since a link that comes up Down silently degrades the collective.

How to use it¶

Each playbook is a directory under nvidia/ with a README.md. Start from nvidia/connect-to-your-spark for network access, then pick the workload playbook. To cluster two devices:

# From nvidia/connect-two-sparks: physically connect the QSFP cable, then
# configure the 200GbE interfaces and passwordless SSH between the two nodes,
# and validate the link before running NCCL (nvidia/nccl).

For local inference, the nvidia/llama-cpp, nvidia/ollama, nvidia/lm-studio, and nvidia/vllm-style playbooks each give the install and run steps for that runtime on Spark. For fine-tuning, nvidia/nemo-fine-tune, nvidia/llama-factory, and nvidia/unsloth cover the common paths.

How to develop with it¶

Treat a playbook as a validated baseline you extend. The clustering playbooks are the ones worth internalizing: the interface naming, the RoCE mapping, and the NCCL environment they set are the load-bearing details for multi-node correctness. When adapting a runtime playbook (say, swapping the model in the vLLM or llama.cpp guide), keep the driver and framework versions the playbook pins, since the vendor validated that specific combination on Blackwell.

How to maintain it¶

Pin to a commit; the playbook set grows and the pinned framework and driver versions in each README.md age. Re-check the interconnect playbooks after a system or driver update, since RoCE interface names and NCCL defaults are exactly the things that shift and silently degrade a collective. For a fleet of more than a few Sparks, the switch-based playbook (nvidia/multi-sparks-through-switch) replaces the direct-cable topology.

How to run it in production¶

DGX Spark is a desktop and small-cluster device, so "production" is local inference, local fine-tuning, or a small two-to-three-node job. For inference serving, follow the runtime playbook and this knowledge base's serving open-weight models guidance for the serving layer. For a multi-Spark job, validate the 200GbE link and NCCL busbw before the real run (a low nccl-tests busbw predicts a slow step), size messages large enough to approach line rate, and use the ring topology the connect-three playbook documents. Remote access is via the Tailscale playbook; treat the device's network exposure like any other node and keep it behind your normal controls.

Failure modes¶

Small-scale interconnect. 200GbE and two-or-three-node topologies bound multi-node throughput; the clustering advice does not scale to a rack on InfiniBand.
RoCE link comes up Down. The NCCL playbook shows interfaces that can be Up or Down; a Down link silently halves or worse the collective bandwidth. Verify the mapping before every multi-node run.
Latency-bound small messages. Small collectives never reach line rate; tune buffer size, as the NCCL playbook instructs.
Version pinning ages. Each playbook pins framework and driver versions the vendor validated; they drift, so re-check against your commit.
Not a training server. Expect local, private Blackwell compute, not multi-GPU-server training throughput.

References¶

DGX Spark Playbooks repository (NVIDIA), pinned commit 1fb66f05: https://github.com/NVIDIA/dgx-spark-playbooks
Connect two Sparks (200GbE QSFP): https://github.com/NVIDIA/dgx-spark-playbooks/tree/main/nvidia/connect-two-sparks
NCCL for multiple Sparks: https://github.com/NVIDIA/dgx-spark-playbooks/tree/main/nvidia/nccl
Connect three Sparks in a ring: https://github.com/NVIDIA/dgx-spark-playbooks/tree/main/nvidia/connect-three-sparks
NCCL tests (the busbw benchmark referenced): https://github.com/NVIDIA/nccl-tests