Skip to content
Markdown

BlueField DPUs for AI networking

Scope: NVIDIA BlueField-3 offloading networking, storage, and security off the host CPU: line-rate RDMA/RoCE, NVMe over Fabrics, data-path isolation for secure multitenancy, and the SuperNIC variant that connects GPUs in Spectrum-X AI Ethernet fabrics; when a DPU or SuperNIC earns its slot in a GPU cluster, and when a plain NIC suffices.

What it is

A BlueField is a programmable network device that fuses an RDMA-capable NIC with an on-board Arm CPU complex plus fixed-function accelerators for networking, storage, and crypto. It runs the data-plane work that would otherwise burn host CPU cycles (packet processing, RDMA, NVMe-oF, encryption, telemetry) directly on the card.113 NVIDIA positions DOCA as the BlueField software framework, "like CUDA for GPUs": a single SDK plus runtime (DPDK, P4, SPDK drivers and DOCA libraries) targeting every BlueField generation.16

The line ships in two roles built on the same BlueField-3 silicon:

  • BlueField-3 DPU (P-series, e.g. B3240), full infrastructure offload: up to 400 Gb/s Ethernet or NDR InfiniBand, 16 Arm cores, and a programmable control plane that can host services such as a firewall, virtual switch, the InfiniBand Subnet Manager, or the SHARP Aggregation Manager.241516
  • BlueField-3 SuperNIC (E-series, e.g. B3140H), a leaner accelerator tuned for GPU-to-GPU AI traffic: up to 400 Gb/s with RDMA/RoCE acceleration, GPUDirect, and GPUDirect Storage, 8 Arm cores, delivering deterministic, isolated throughput with secure cloud multitenancy. It is the endpoint building block of the Spectrum-X Ethernet platform (paired with Spectrum-4 switches).1415

In the GB200/GB300 NVL72 design, each compute node carries high-speed NICs plus a BlueField-3 DPU. The DPU handles line-rate packet processing, RDMA, and NVMe over Fabrics (NVMe-oF, the NVMe protocol extended across a network fabric), moving data directly between network, storage, and GPU memory with no CPU in the path.1 Inside the rack NVLink/NVSwitch carries all GPU-to-GPU traffic; the DPU and NICs own everything that leaves the rack (other NVL72 racks and external storage) over Quantum-X800 InfiniBand or Spectrum-X800 Ethernet.5

flowchart LR
  subgraph NODE["NVL72 compute node"]
    CPU["Grace CPU"]
    GPU["Blackwell GPU<br/>(HBM)"]
    BF["BlueField-3 DPU<br/>(Arm + accelerators)"]
  end
  STORE["Storage server<br/>(NVMe-oF)"]
  FAB["Spectrum-X / Quantum-X800<br/>fabric"]
  BF -->|"GPUDirect RDMA<br/>(no host staging)"| GPU
  BF -.->|"control / preprocessing only"| CPU
  BF <-->|"line-rate RDMA / RoCE"| FAB
  FAB <--> STORE

Why it matters

Three offloads pay for the card in an AI cluster:

  • Data path off the CPU. The DPU moves bytes between NIC, NVMe-oF storage, and GPU memory without staging through host RAM or spending host cycles, so the Grace CPU is not bogged down servicing network interrupts and is free for data preprocessing while the DPU streams a dataset straight into GPU memory.13 The transfer uses GPUDirect RDMA, NVIDIA's RDMA implementation that lets an RDMA-capable NIC DMA directly to and from GPU device memory across two servers, bypassing host CPU and system RAM entirely.6
  • Line-rate AI fabric. A BlueField-3 SuperNIC drives up to 400 Gb/s per port (the book cites ConnectX-8 SuperNICs at the 800 GbE class delivering up to 800 Gb/s per port with GPUDirect RDMA) and the recommendation is 1 NIC per GPU to optimize prefill/decode disaggregation and MoE all-to-all performance.714
  • Isolation for multitenancy. The DPU acts as a smart firewall / virtual switch on the node, isolating per-job and per-user network traffic so different teams (or external clients) can share partitions of an NVL72 without interfering, the same model cloud providers use to multi-tenant a large server.4

The failure mode this avoids is silent fallback. If a container lacks direct access to the host InfiniBand devices, NCCL can silently drop from GPUDirect RDMA to TCP sockets with no error, collapsing throughput from tens of GB/s to a few Gb/s; a GID mismatch can likewise force CPU-driven RDMA copies instead of true GPUDirect.8 A DPU does not by itself prevent this. You still verify the data path (see below).

When it is needed (and when not)

Needed:

  • Multinode training or disaggregated inference where storage and inter-rack traffic must reach GPU memory at line rate without taxing the host CPU, exactly the NVL72 role.17
  • Secure multitenancy: multiple teams or external clients sharing a rack and needing isolated network domains enforced below the host OS.4
  • NVMe-oF storage streaming for large-scale training, where the DPU deposits data directly into GPU memory while the CPU preprocesses.3
  • DOCA-offloaded fabric services: hosting the Subnet Manager or SHARP Aggregation Manager on the card rather than a separate management host.10

Not the right tool:

  • Single-node / intra-rack-only workloads. Inside an NVL72, NVLink and NVSwitch carry all GPU-to-GPU traffic; a DPU adds nothing to intra-rack collectives.5
  • In-network reduction. A DPU offloads transport and can host the SHARP Aggregation Manager, but the SHARP reduction arithmetic happens in the switch ASIC, not on the DPU. For collective offload see SHARP: In-Network Reduction.10
  • When a plain RDMA NIC suffices. If you do not need programmable on-card services (firewall, virtual switch, NVMe-oF target, tenant isolation), a ConnectX-class NIC delivers GPUDirect RDMA without the Arm complex. The DPU/SuperNIC justifies itself through offload, isolation, and the DOCA control plane, not raw RDMA alone.514

How: implement, integrate, maintain

Provisioning. BlueField is programmed through the DOCA framework, an SDK (DPDK, P4, SPDK, DOCA libraries and APIs) plus a runtime with services, reference applications, and provisioning tools. Prebuilt DOCA microservices ship as containers on NGC and deploy with standard orchestration; custom offloads build against the DOCA SDK.16

Verify the GPUDirect RDMA data path before trusting any throughput number. The kernel module nvidia_peermem registers GPU memory with the InfiniBand subsystem so the NIC can DMA to/from GPU memory; it ships with the standard NVIDIA driver (R470 / CUDA 11.4 and later) but is not auto-loaded.179 Confirm it is resident:

lsmod | grep nvidia_peermem

If absent, load it (and check dmesg for initialization):179

sudo modprobe nvidia_peermem

Then confirm NCCL actually selects the InfiniBand/RDMA path rather than falling back to sockets:89

export NCCL_DEBUG=INFO   # look for NET/IB (not NET/Socket) in the logs

For an end-to-end GPU-to-GPU check, run the RDMA perftests with CUDA buffers (the book's recommended validation):9

# server node
ib_write_bw --use_cuda=<gpu_index> -d <ib_device>
# client node
ib_write_bw --use_cuda=<gpu_index> -d <ib_device> <server_host>

A drop to CPU-staged copies or TCP shows up as an order-of-magnitude throughput reduction in the profiler. Continuously monitor for these stealthy fallbacks rather than assuming the RDMA path stays active.8

Container access. Give the container direct access to the host InfiniBand devices (e.g. /dev/infiniband); without it NCCL falls back to TCP sockets with no obvious error, and a GID mismatch versus the host prevents GPUDirect registration.8

Storage and isolation. For NVMe-oF, the DPU terminates the storage transport on-card and DMAs into GPU memory, keeping the host CPU on preprocessing.13 For multitenancy, configure the DPU's firewall/virtual-switch role to isolate per-tenant traffic; combine with MIG and SLURM/Kubernetes partitioning to carve a rack across teams.4

Telemetry. BlueField DPUs and NICs expose their own counters; watch them alongside NVLink utilization so a saturated storage link or NIC does not masquerade as a GPU stall. NVL72-class systems surface this telemetry; fabric-wide, NVIDIA NetQ and Unified Fabric Manager (UFM) provide I/O-fabric telemetry and lifecycle management.1112

Reference-template guidance only. The roles, port speeds, core counts, flags, and module names above are drawn from the cited book chapter and NVIDIA documentation; they have not been hardware-validated here. Confirm exact SKU specifications against the BlueField-3 datasheet and your DOCA / driver versions before relying on them.

References

Related: SHARP: In-Network Reduction · RDMA and RoCE Performance Tuning · NVSHMEM: GPU-Initiated Communication · NCCL Collectives and Algorithm Selection · Communication-Computation Overlap · HPC Networking Fabric · Fabric Bring-Up, Validation and Benchmarking · Fabric Manager · NVSwitch and NVLink · Ansible Role: rdma_fabric · Distributed Training Platform · Disaggregated Inference · GPU Memory Hierarchy · NCCL Hang / Collective Stall · Glossary

  1. Fregly, Ch. 2, "Multirack and Storage Communication": each NVL72 node has high-speed NICs plus a BlueField-3 DPU; the DPU offloads, accelerates, and isolates networking/storage/security from the host CPU, handling line-rate packet processing, RDMA, and NVMe-oF, moving data between network, storage, and GPU memory without CPU involvement. ↩↩↩↩↩

  2. Fregly, Ch. 2: NVL72 trays commonly use four ConnectX-8 800 Gb/s NICs per node for external bandwidth; BlueField-3 DPUs are used where in-network acceleration or offload is required for storage, security, and control-plane tasks; integrate with Quantum-X800 InfiniBand or Spectrum-X800 Ethernet. ↩

  3. Fregly, Ch. 2: BlueField DPUs avoid CPU involvement when streaming large datasets from a storage server; the DPU handles the transfer and deposits data directly into GPU memory while the CPU focuses on preprocessing. ↩↩↩

  4. Fregly, Ch. 2, "Sharing and Scheduling": the BlueField DPU enables secure multitenancy, acting as a firewall and virtual switch to isolate network traffic for different jobs and users — letting departments or external clients use partitions safely. ↩↩↩↩

  5. Fregly, Ch. 2: inside the NVL72, NVLink/NVSwitch carry all GPU-to-GPU traffic; outside the rack it relies on InfiniBand/Ethernet NICs and BlueField DPUs alongside NVLink for inter-rack and storage connectivity. ↩↩↩

  6. Fregly, Ch. 4, "High-Speed, Low-Overhead Data Transfers with RDMA": GPUDirect RDMA lets an RDMA-capable NIC (InfiniBand or RoCE) DMA to/from GPU device memory across two servers, bypassing host CPU and system RAM; registers GPU buffers with the NIC for one-sided reads/writes. ↩

  7. Fregly, Ch. 15: multinode clusters with ConnectX-8 SuperNICs (800 GbE-class) provide up to 800 Gb/s per port with GPUDirect RDMA; deploy 1 NIC per GPU to optimize prefill-decode disaggregation and improve MoE all-to-all performance. ↩↩

  8. Fregly, Ch. 4: without direct access to host InfiniBand devices (e.g. /dev/infiniband), NCCL may silently fall back from GPUDirect RDMA to TCP sockets (tens of GB/s -> a few Gb/s) with no obvious error; GID mismatches force CPU-driven RDMA copies instead of true GPUDirect. ↩↩↩↩

  9. Fregly, Ch. 4: verify true GPUDirect RDMA — confirm the kernel module with lsmod | grep nvidia_peermem, check dmesg, run NCCL with NCCL_DEBUG=INFO to confirm NET/IB paths, and use RDMA perftests with --use_cuda to validate GPU-to-GPU transfers. ↩↩↩↩

  10. Fregly, Ch. 4, Magnum IO in-network compute: SHARP reduction arithmetic happens in the switch silicon; BlueField DPUs offload networking and can host control services such as the Subnet Manager and the SHARP Aggregation Manager. ↩↩

  11. Fregly, Ch. 2: BlueField DPUs and NICs have their own statistics monitored to ensure storage links are not saturated; modern systems like the NVL72 expose this telemetry alongside NVLink usage. ↩

  12. Fregly, Ch. 4, Magnum IO I/O management: NVIDIA NetQ and Unified Fabric Manager (UFM) provide real-time telemetry, diagnostics, and lifecycle management for the data-center I/O fabric. ↩

  13. NVIDIA, "BlueField Networking Platform": BlueField-3 is a 400 Gb/s infrastructure compute platform with line-rate processing of software-defined networking, storage, and cybersecurity, programmable through DOCA. ↩

  14. NVIDIA, "High-Performance AI Networking — Ethernet SuperNICs": the BlueField-3 SuperNIC is a network accelerator up to 400 Gb/s for hyperscale AI with RDMA/RoCE acceleration, GPUDirect and GPUDirect Storage, deterministic isolated performance and secure cloud multitenancy; a central part of the Spectrum-X platform with Spectrum-4 switches. ↩↩↩

  15. NVIDIA BlueField-3 Networking Platform User Guide, "Specifications": E-series SuperNICs 8 Arm cores, P-series DPUs 16 Arm cores; Ethernet 400/200/100/50/25/10 Gb/s; InfiniBand NDR/NDR200/HDR/HDR100/EDR/FDR/SDR on B3140H/B3240. ↩↩

  16. NVIDIA, "DOCA Software Framework": DOCA for DPUs and SuperNICs is "like CUDA for GPUs" — a consistent SDK plus runtime (DPDK, P4, SPDK drivers, DOCA libraries/APIs) across BlueField generations; prebuilt microservices on NGC, custom services built on the SDK. ↩↩↩

  17. NVIDIA GPU Operator docs, "GPUDirect RDMA and GPUDirect Storage": the nvidia-peermem module registers the NVIDIA GPU with the InfiniBand subsystem for peer-to-peer access; included in the standard driver since CUDA 11.4 / R470; not auto-loaded — load manually via sudo modprobe nvidia-peermem; if the GPU driver is installed before MLNX_OFED it must be reinstalled. ↩↩