Skip to content
Markdown

Prima.cpp: heterogeneous home-cluster inference

Scope: how to run a 30-70B model across a handful of ordinary, unequal devices (laptops, a desktop with a consumer GPU, a Mac, a phone) whose combined RAM and VRAM is less than the model needs, without out-of-memory failures and without the disk-loading stalls that sink a naive offload. Prima.cpp (arXiv 2504.08791) is the worked example: pipelined-ring parallelism overlaps memory-mapped weight loading with computation, and the Halda scheduler assigns layers to devices under each one's RAM and VRAM limits. This is the low-resource, cross-device counterpart to the datacenter techniques in disaggregated inference and cross-WAN model-parallel inference, and the multi-device extension of running small models on a single consumer GPU; the pipeline-parallel base it builds on is pipeline parallelism.

Primary source: Li, Li, Feng, Xiao, She, Huang, Guizani, Yu, Ho, Xiang, and Liu, "Prima.cpp: Fast 30-70B LLM Inference on Heterogeneous and Low-Resource Home Clusters", arXiv:2504.08791v3 [cs.DC], 2026-07-04, published as a conference paper at ICLR 2026. This page is anchored on v3.

Version history matters here. v1 (2025-04) was titled "Speeding Up 70B-Scale LLM Inference on Low-Resource Everyday Home Clusters", had five authors, and called the method "piped-ring parallelism". v3 has eleven authors across MBZUAI, UESTC, City University of Hong Kong, and La Trobe, renames the method "pipelined-ring parallelism", and adds eleven appendices that did not exist in v1: speculative decoding, energy accounting, concurrency, context-length limits, network sensitivity, background-load resilience, a ten-node testbed, and a comparison against heuristic schedulers. An earlier revision of this page cited v1 tables; the numbers below are v3's, and the places where v3 restated a v1 figure are called out.

What this page adds. Two executed Python blocks (stdlib only, both run and asserted). The first models the prefetch-release conflict and reproduces the paper's Appendix A.1 example. The second audits v3's own published tables and finds that the abstract's two headline numbers come from two different clusters. The prima.cpp invocations are reference templates from the repository README, not executed here.

flowchart TB
  subgraph HALDA["Halda scheduler (offline): layer-to-device assignment"]
    PROF["Device + model profiler:<br/>compute, memory, disk, comms, OS behavior"] --> ILP["Enumerate rounds k -> ILPs (HiGHS)<br/>solve window w_m and GPU layers n_m<br/>under RAM/VRAM limits"]
  end
  ILP --> RING
  subgraph RING["Pipelined-ring parallelism (runtime): a ring of unequal devices"]
    D1["Mac M1 (head)<br/>window w1, prefetch next"] --> D2["Laptop i9 + RTX 3070<br/>window w2"]
    D2 --> D3["Desktop i9 + RTX 2080Ti<br/>window w3"]
    D3 --> D4["Phone (Kirin 9000)<br/>window w4"]
    D4 -->|"next round (multiple rounds per token)"| D1
  end
  RING --> OUT["Head device emits the token<br/>disk load hidden behind other devices' compute"]
  ILP -.->|"drops devices whose compute<br/>gain is under one ring hop"| DROP["Removed from ring"]

What it is

Prima.cpp is a distributed inference system, built on llama.cpp, that runs a model too big for any one device by splitting its layers across a ring of heterogeneous machines and streaming weights from disk with memory mapping. The paper's target is deliberately modest hardware: its default cluster is four everyday devices (a Mac M1, an Intel i9 laptop with an RTX 3070, an i9 desktop with an RTX 2080 Ti, and a Kirin 9000 phone) whose total available RAM plus VRAM is 37 GiB, "not enough for a Q4K-quantized 70B model."1 It reports serving Llama 3-70B at 674 ms per token with a time-to-first-token under two seconds on that cluster, while keeping per-device memory pressure below 6%.5

Two ideas make that possible. Pipelined-ring parallelism (PRP) is pipeline parallelism arranged as a ring, run over multiple rounds per token, with each device handling a small window of layers per round so that memory-mapped prefetching of the next window overlaps the other devices' computation instead of stalling on disk. Halda is the scheduler that decides, for each device, how many layers it handles per round and how many of those run on its GPU, under that device's RAM and VRAM limits, by solving what the paper frames as a layer-to-device assignment problem. Halda also decides which devices to use at all, and v3 makes that device-selection ability a first-class contribution: a weak device assigned only one layer is dropped from the ring, unless it is the only reachable hop, in which case it stays as a relay carrying no work.23

The hard problem underneath is offloading. When a model does not fit in RAM, llama.cpp uses mmap to load weights lazily and lets the OS reclaim inactive pages, but under memory pressure that reclaim reloads pages from disk repeatedly. The paper measures this cliff: at 45B "only a few pages are released, so efficiency loss is small," but at 60B "more active mmap-ed pages are labeled as inactive earlier and then released, leading to a sharp increase in token latency and TTFT."4 Prima.cpp's job is to hide that disk latency rather than pay it.

Why use it

  • It runs models everyday hardware otherwise cannot. v3's motivating claim is that on-device inference struggles "to run models beyond 8B" while "reliable long-term planning and tool use require 32B or more", and that Qwen 2.5-14B (Q4K) on an 8 GiB Mac M1 takes 10 s/token under llama.cpp.4 Prima.cpp brings 70B into the same home cluster at sub-second per-token latency.
  • It is fast against the alternatives, and it does not OOM where they do. On the four-device cluster, prima.cpp reduces TPOT by up to 17x and TTFT by up to 8x against llama.cpp, and against exo and dllama it reports at least 18x lower TPOT and 42x lower TTFT; at 70B both exo and dllama run out of memory entirely while prima.cpp serves it. Read the baseline comparison with its caveats: llama.cpp is a single-device system run on the strongest device rather than on the cluster, exo could not run on the phone because it needs root, and dllama supports only Llama 3, so several cells are structurally absent rather than measured.5
  • It keeps devices usable. Because it respects each device's available (not total) memory, per-device memory pressure stays below 6%, versus request schedulers that grab enough RAM to freeze the machine; the paper measures exo and dllama pushing a device into critical pressure (over 50%) even on an 8B model.5
  • It is genuinely cross-platform. The same system spans macOS (both unified-memory and NUMA), Linux, and Android or HarmonyOS phones (via a Termux-simulated Linux), mixing CPU and GPU and RAM and VRAM across those devices simultaneously.1

When to use it (and when not)

Use it when:

  • You have several ordinary devices and want to run a model larger than any one of them holds, and you value privacy or cost enough to keep inference local rather than calling a hosted API.
  • The devices are unequal (a fast desktop, a slow laptop, a phone) and a uniform split would bottleneck on the weakest link; Halda's whole point is assigning more work to stronger devices under each one's memory limit.
  • The model does not fit in aggregate RAM, so disk offloading is unavoidable and hiding its latency is the deciding factor. PRP's benefit is largest exactly here: v3 reports that "for large models, PRP reduces PP's TPOT by about 50%".2

Do not reach for it when:

  • The model fits comfortably in one device's memory. With sufficient RAM, k=1 (a single round, the whole model as one window) is best, because PRP at k=1 simply is pipeline parallelism; on small models PRP "converges to PP with similar TPOT" and its extra kernel launches make it slightly worse, so multi-device ring parallelism adds coordination cost for no gain.2 Halda reaches the same conclusion by itself and collapses the cluster to one device.
  • You need cloud-grade latency. v1 stated plainly that "70B models remain much slower than those on the cloud"; this is for when local execution is the constraint, not when throughput is the goal. v3 drops the sentence but nothing in it measures otherwise.12
  • The cluster is low-RAM with no SSD and no GPU. v1 warned that in that regime "larger models will be extremely slow," because there is nothing to hide the disk latency behind.12
  • You need long context. This is the sharpest practical limit v3 adds and it is easy to miss, because it does not appear in the abstract. The KV cache competes with model weights for the same page cache, so a longer context makes prefetch-release conflicts more likely. On Llama-3-70B Q4K, TPOT stays flat only up to roughly 4K tokens on a low-memory testbed and roughly 16K on a medium-memory one; beyond that it "rises sharply or causes OOM."7 A home cluster is therefore a short-context deployment unless you size memory for the KV cache explicitly. Nothing in the paper's headline numbers, which are short-context, reflects this.
  • The links are high-latency. Prima.cpp is insensitive to bandwidth above roughly 50 Mbps but sensitive to round-trip time: raising RTT from 100 ms to 200 ms costs 68% more TPOT.8 The ring pays a hop per device per round, so latency, not bandwidth, is the network property to check first. This is the same conclusion the WAN split-inference work reaches from the opposite direction, where RTT is 63-64% of per-token time; see privacy-aware split inference over a WAN.

Architecture

Piped-ring parallelism starts from ordinary pipeline parallelism (split the model into layer segments along a chain of devices) and closes the chain into a ring so I/O and the emitted token stay on the head device, which matters for privacy. The departure from prior ring systems is that a token is predicted over multiple rounds, and each device processes only a small layer window per round. The paper's example: six devices, a 36-layer model, window size two, gives eighteen segments so each device does three rounds per token.2 The window size is the load-bearing knob: "By setting the layer window size small, we ensure the model layers stays within memory limits, avoiding prefetch-release during prefetching," so a device can mark its next window's layers WILLNEED and the OS loads them from disk in the background while the other devices compute, fully overlapping disk with compute on a fast disk.2

The failure this avoids is the prefetch-release conflict. Vanilla prefetching marks a whole set of upcoming layers WILLNEED; if that set exceeds available memory, the later prefetched layers evict the earlier ones, so when computation finally reaches those earlier layers they fault back in from disk, and "all layers being loaded twice, adding unnecessary disk overhead without any benefit from prefetching."2 A small window guarantees the prefetched layers fit; k=1 (the whole model as a single window, one round) is the most extreme case where they do not, which is why on a model too large for RAM, k=1 "offers no benefit due to the prefetch-release effect": the single giant window still exceeds the device's resident capacity, so it pays the same doubled-load tax as any oversized window, just at the largest possible scale.2

Halda solves the assignment. For each device it chooses a per-round window w_m and how many of those layers run on the GPU n_m, minimizing token latency subject to RAM and VRAM constraints, given a profiler's measurements of each device's compute, memory-access, disk-loading, and communication latency plus its OS memory-management behavior. The raw problem is NP-hard because the total window size appears in both the objective and the constraints, so Halda breaks it into tractable pieces: since the layer count L is under 100 in typical LLMs, the number of rounds k has few valid values ("at most 11 valid factors for any k <= 100"), so Halda enumerates k, which turns each case into a standard integer linear program solved by HiGHS, then iterates a device-set assignment (which devices must overload their RAM versus which must not) until stable, with a calibration step that reassigns layers to a GPU sitting idle while another device is overloaded. The result "find[s] the optimal solution in polynomial time," O(T(M + K(2M)^3.5)).3

How to use it

Prima.cpp is built on llama.cpp, so the binaries are familiar, but the launch is not: a plain make plus a bare llama-cli is single-device llama.cpp, not prima.cpp. Every device must be given the ring's shape on the command line, and rank 0 must be built against HiGHS or Halda cannot solve the assignment at all. The commands below follow the repository README (the project has moved to the OpenCPIL organization); check them against it before a real deployment, because the flag set is young.

# Build. Rank 0 (the head) needs USE_HIGHS=1: HiGHS is the integer-linear-program solver
# Halda calls to assign layer windows automatically. Without it, rank 0 cannot compute the
# assignment. Worker ranks may build plain, and add backend flags as the device requires.
git clone https://github.com/OpenCPIL/prima.cpp && cd prima.cpp
make USE_HIGHS=1 -j$(nproc)          # rank 0 (head)
make -j$(nproc)                      # worker ranks; GGML_CUDA=1 on an NVIDIA worker

# Launch the ring. EVERY rank needs the same --world and model, its own --rank, the head's
# --master, and --next pointing at the following device in the ring. --prefetch is what
# overlaps the next window's disk load with this round's compute (the whole point of PRP).
# Rank 0, the head: it holds the prompt, runs the scheduler, and emits the tokens.
./llama-cli -m download/qwq-32b-q4_k_m.gguf -c 1024 -n 256 -p "what is edge AI?" \
  --world 4 --rank 0 --master 192.168.1.2 --next 192.168.1.3 --prefetch

# Ranks 1..3, one per worker device. --gpu-mem caps the VRAM (GiB) Halda may allocate there.
./llama-cli -m download/qwq-32b-q4_k_m.gguf \
  --world 4 --rank 1 --master 192.168.1.2 --next 192.168.1.4 --prefetch --gpu-mem 8

# Server mode: rank 0 runs llama-server instead, and the workers stay on llama-cli.
./llama-server -m download/qwq-32b-q4_k_m.gguf --world 2 --rank 0 --prefetch \
  --host 127.0.0.1 --port 8080

The practical inputs you control are the model and its quantization (which sets the total memory footprint the cluster must cover), the ring order and each device's reachability over Wi-Fi or Ethernet, and each device's memory ceiling (--gpu-mem for VRAM). Halda takes it from there: you do not hand-assign layers, and that is precisely the step that silently does not happen if rank 0 was built without HiGHS.

How to develop with it

The mechanism worth understanding before trusting a cross-device offload is why a small layer window turns a doubled disk cost into a hidden one. This executed model implements the prefetch-release conflict as an LRU page-reclaim simulation (Part A) and the resulting per-token latency with and without overlap (Part B), reproducing the paper's Appendix A.1 example (six layers, three fit, loaded twice) and its finding that k=1 gives no benefit:

# prima_ring.py -- validated: the two mechanisms that let prima.cpp keep a
# 70B model's disk loading overlapped instead of paying it twice. Pure stdlib.
#
# Part A models the "prefetch-release" conflict (paper Appendix A.1): a device
# can hold only `capacity` layers resident (mem_available), and mmap prefetch
# marks a whole window WILLNEED. If the window exceeds capacity, later prefetched
# layers evict earlier ones, so computation page-faults on the evicted layers and
# every layer is loaded TWICE. A small layer window keeps the window within
# capacity, so each layer is loaded exactly once.
#
# Part B models piped-ring overlap: with each layer loaded once, the next round's
# prefetch overlaps this round's compute, so per-round latency collapses from
# (compute + disk) to max(compute, disk). With prefetch-release it cannot overlap.

from collections import OrderedDict


def disk_loads_per_pass(num_layers: int, capacity: int, window: int) -> int:
    """Count disk loads to prefetch-then-compute one window of `window` layers on
    a device that can hold `capacity` layers resident. LRU page reclaim models
    the OS freeing inactive mmap-ed pages. Prefetch loads the window in order;
    compute then walks the same layers in order, reloading any the prefetch phase
    evicted. Returns total loads (prefetch + compute) for one window."""
    assert 1 <= window <= num_layers and capacity >= 1
    cache: "OrderedDict[int, bool]" = OrderedDict()
    loads = 0

    def touch(layer: int) -> None:
        nonlocal loads
        if layer in cache:
            cache.move_to_end(layer)
            return
        loads += 1                       # disk load (page fault or prefetch miss)
        cache[layer] = True
        cache.move_to_end(layer)
        while len(cache) > capacity:
            cache.popitem(last=False)    # OS reclaims the least-recently-used page

    for layer in range(window):          # prefetch phase: mark window WILLNEED
        touch(layer)
    for layer in range(window):          # compute phase: walk layers in order
        touch(layer)
    return loads


# --- Part A: prefetch-release is exactly a 2x disk tax when window > capacity ---
CAP = 3   # a device that can hold 3 layers resident (the A.1 example)

# Small window that fits: prefetch 3, compute 3 all resident -> 3 loads, loaded ONCE.
assert disk_loads_per_pass(num_layers=6, capacity=CAP, window=3) == 3

# The paper's A.1 case: window 6 > capacity 3. Prefetch loads 6 (only the last 3
# survive), compute reloads all 6 -> 12 loads = every layer loaded TWICE.
assert disk_loads_per_pass(num_layers=6, capacity=CAP, window=6) == 12

# General rule this exposes: any window strictly above capacity pays 2x; any
# window at or below capacity pays 1x. Checked across a sweep.
for w in range(1, 13):
    loads = disk_loads_per_pass(num_layers=12, capacity=CAP, window=w)
    expected = w if w <= CAP else 2 * w
    assert loads == expected, (w, loads, expected)

# Boundary: window == capacity is the largest window that still loads once.
assert disk_loads_per_pass(12, CAP, CAP) == CAP
assert disk_loads_per_pass(12, CAP, CAP + 1) == 2 * (CAP + 1)


# --- Part B: with layers loaded once, ring prefetch overlaps disk with compute ---
def per_token_latency(num_layers, capacity, window, t_compute_layer, t_disk_layer):
    """Latency to predict one token across a ring, one device's view. A token needs
    num_layers/window rounds; each round computes `window` layers. When the window
    fits (loaded once), the next round's prefetch overlaps this round's compute, so
    a round costs max(compute, disk). When it does not fit, prefetch-release forces
    a synchronous reload, so a round costs compute + the reload disk time."""
    assert num_layers % window == 0
    rounds = num_layers // window
    compute = window * t_compute_layer
    if window <= capacity:                       # overlapped
        disk = window * t_disk_layer             # prefetch the same count, hidden
        return rounds * max(compute, disk)
    reload_disk = window * t_disk_layer          # the extra (2nd) load is NOT hidden
    return rounds * (compute + reload_disk)


# A device where disk load per layer dominates compute (the >60B regime, where the
# model does not fit in RAM). t_disk > t_compute, so hiding disk is the whole game.
tc, td = 1.0, 4.0
N = 12

fits = per_token_latency(N, capacity=3, window=3, t_compute_layer=tc, t_disk_layer=td)
release = per_token_latency(N, capacity=3, window=6, t_compute_layer=tc, t_disk_layer=td)

# Overlapped: 4 rounds * max(3*1, 3*4)=12 -> 48. Prefetch-release: 2 rounds *
# (6*1 + 6*4)=30 -> 60. The small window is faster AND the paper's own finding
# that k=1 (one round, whole model at once) "offers no benefit" falls out: a
# single round of the full model cannot overlap anything.
assert fits == 48 and release == 60

# k=1 means window == num_layers: one giant round, no overlap possible, and here
# it also busts capacity so it pays the reload tax too -- worst of both.
k1 = per_token_latency(N, capacity=3, window=N, t_compute_layer=tc, t_disk_layer=td)
assert k1 == (N * tc + N * td)      # 12 + 48 = 60, no rounds to overlap across
assert fits < k1, "a small layer window must beat the single-round (k=1) layout"

# Sanity: when the model DOES fit in RAM (capacity >= num_layers), there is no
# disk to hide, so a bigger window (fewer rounds, less ring overhead) is fine --
# matching the paper's note that k=1 is best when memory is sufficient.
resident = per_token_latency(N, capacity=N, window=N, t_compute_layer=tc, t_disk_layer=0.0)
assert resident == N * tc      # pure compute, 12

print(f"OK Part A: window<=cap loads each layer once (3); window>cap loads twice "
      f"(A.1: 6 layers, cap 3 -> 12 loads). "
      f"OK Part B: small-window ring={fits} < prefetch-release={release} < "
      f"k=1 single round={k1}; fully-resident={resident} (no disk to hide).")

Run output:

OK Part A: window<=cap loads each layer once (3); window>cap loads twice (A.1: 6 layers, cap 3 -> 12 loads). OK Part B: small-window ring=48.0 < prefetch-release=60.0 < k=1 single round=60.0; fully-resident=12.0 (no disk to hide).

The model makes the design rule concrete: the window must stay at or below a device's resident-layer capacity, and the win only exists when disk time is comparable to or larger than compute time (the offloaded regime). When the model fits in RAM there is no disk to hide, and a small window just adds ring rounds, which is why Halda picks the window per device rather than using a fixed value.

Speculative decoding, and which cluster the 26 tok/s comes from

v3 adds speculative decoding: a 0.5-3B draft model runs as a standalone process on the head device, proposes five tokens per step, and the target model verifies them in a batch.6 The gain is real and well scoped in the paper's own body text, which claims a 25-45% latency reduction across 14-70B models. The published per-model table supports that range exactly, and it also shows the technique doing nothing at the small end: Qwen-2.5-7B gains 10% and Llama 3.2-8B gains nothing at all.

The trap is in the abstract. It reads: "On four consumer home devices, a 70B model reaches 674 ms/token TPOT with <6% memory pressure, and a 32B model with speculative decoding achieves 26 tokens/s." The two clauses come from two different clusters. The 674 ms/token figure is the four-device home cluster of Table 3, whose weakest member is a phone with 1.9 GiB of RAM. The 26 tok/s figure is from a separate testbed of four Linux machines with 8-core CPUs, 8 GiB of RAM each, 600 MB/s disks, and two RTX 4090s. On the actual home cluster, the same 32B model posts 89 ms/token, which is 11.2 tok/s.

The feature is shipped, and one apparent contradiction between the repository and the paper is not one. The README advertises speculative decoding as a speedup "by up to 80%" while the paper's body says 25-45%. Both describe the same measurement in different units: the best row, Llama 3.3-70B at 803 to 442 ms/token, is a 45.0% latency reduction and an 81.7% throughput increase. Quote whichever you like, but do not add them together or treat them as two results.

Executed audit of the v3 tables

# prima_v3_audit.py -- audit of the prima.cpp v3 (ICLR 2026) published tables.
# Pure stdlib. Every PUB_* value is transcribed from the v3 PDF.

tps = lambda ms: 1000.0 / ms

# ---- Part A: the abstract's two headline numbers come from two different clusters ----
# Table 4, cluster D1-D4 of Table 3 (Mac M1 / i9+3070 / i9+2080Ti / Mate40Pro), 37 GiB.
PUB_T4_HOME = {8: 15, 14: 20, 30: 72, 45: 233, 60: 468, 65: 569, 70: 674}  # ms/token
PUB_T9_HOME_32B = 89        # Table 9, Qwen-2.5-32B and QwQ-32B on the same D1-D4 cluster

# Table 10 testbed: 4 Linux devices, 8-core, 8 GiB RAM, 600 MB/s disk, TWO with a 4090
# capped at 11 GiB VRAM. Not the Table 3 home cluster.
PUB_T10 = {                       # (prima.cpp, prima.cpp + speculative) ms/token
    "Qwen-2.5-7B":    (20, 18),  "Llama 3.2-8B":    (20, 20),
    "Qwen-2.5-14B":   (36, 27),  "DS-R1-Qwen-14B":  (32, 22),
    "Qwen-2.5-32B":   (55, 38),  "DS-R1-Llama-70B": (859, 593),
    "Llama 3.3-70B":  (803, 442), "Qwen-2.5-72B":   (963, 544),
}

# The abstract reads: "On four consumer home devices, a 70B model reaches 674 ms/token
# TPOT with <6% memory pressure, and a 32B model with speculative decoding achieves 26
# tokens/s." The 70B half is the home cluster; the 26 tok/s half is Table 10's testbed.
assert PUB_T4_HOME[70] == 674
assert round(tps(PUB_T10["Qwen-2.5-32B"][1])) == 26
# On the home cluster the same 32B model posts 89 ms/token, a 62% higher TPOT than the
# 55 ms baseline of the testbed that produced the 26 tok/s figure.
assert round(tps(PUB_T9_HOME_32B), 1) == 11.2
assert round(PUB_T9_HOME_32B / PUB_T10["Qwen-2.5-32B"][0] - 1, 2) == 0.62

# Apply the paper's own best measured speculative gain to the home cluster's 32B.
gains = {k: (b - s) / b for k, (b, s) in PUB_T10.items()}
best = max(gains.values())
assert round(best, 4) == 0.4496 and max(gains, key=gains.get) == "Llama 3.3-70B"
home_32b_best_case = PUB_T9_HOME_32B * (1 - best)
assert 20.0 < tps(home_32b_best_case) < 20.6   # ~20.4 tok/s, short of the 26 headline

# The "25-45% reduction across 14-70B models" claim holds, and is honestly scoped:
# the 7B and 8B rows it excludes gain 10% and 0%.
big = {k: v for k, v in gains.items() if "7B" not in k and "8B" not in k}
assert all(0.25 <= g <= 0.45 for g in big.values()) and len(big) == 6
assert gains["Qwen-2.5-7B"] == 0.10 and gains["Llama 3.2-8B"] == 0.0

# Section 4.1 writes "674 ms/token (442 ms/token with speculative decoding)". The 442 is
# Table 10's Llama 3.3-70B, whose own non-speculative baseline is 803 ms, not 674:
# a different cluster and a different model from the 674 it is parenthesised against.
assert PUB_T10["Llama 3.3-70B"] == (803, 442) and PUB_T4_HOME[70] == 674


# ---- Part B: the ring-hop rule behind Halda's counterintuitive device drop ----
# A.19.2, 10-node testbed, Llama-3-70B Q4K, 80 layers. "Fill-Fastest-GPUs-First" spreads
# 13/46/19/2 layers over D2/D0/D6/D5 for 156 ms/token. Halda drops D5 and moves its 2
# layers onto D2's CPU for 146 ms/token, though D5's 1080Ti has ~30x D2-CPU's FLOPS.
PUB_FFGF_MS, PUB_HALDA_MS, D5_LAYERS, FLOPS_RATIO = 156, 146, 2, 30
assert PUB_FFGF_MS - PUB_HALDA_MS == 10


def ring_tpot(layer_times_ms, hop_ms):
    """TPOT of a ring: each device's layer compute plus one hop per device."""
    assert layer_times_ms and hop_ms >= 0
    return sum(layer_times_ms) + len(layer_times_ms) * hop_ms


def worth_adding(n_layers, t_gpu_per_layer, t_fallback_per_layer, hop_ms):
    """A device earns its ring slot iff the compute it lifts off the critical path
    exceeds the one hop it adds. Independent of how fast that device is in isolation."""
    return n_layers * (t_fallback_per_layer - t_gpu_per_layer) > hop_ms


# Keeping D5 costs one hop and saves 2 layers' worth of CPU-versus-GPU difference:
#     hop_ms - D5_LAYERS * (t_cpu - t_gpu) = +10 ms
# The paper publishes neither the hop time nor the per-layer times, so this pins a
# relation, not a reconstruction. Both consistent points below reach the same verdict.
for t_gpu in (0.05, 0.10):
    t_cpu = FLOPS_RATIO * t_gpu
    hop = 10 + D5_LAYERS * (t_cpu - t_gpu)
    assert not worth_adding(D5_LAYERS, t_gpu, t_cpu, hop)
    # The same GPU would earn its slot if it carried enough layers to beat the hop.
    break_even = hop / (t_cpu - t_gpu)
    assert break_even > D5_LAYERS
    assert worth_adding(int(break_even) + 1, t_gpu, t_cpu, hop)

# The general shape: on a high-latency ring, more devices is not monotonically better.
# Eight layers at 1 ms/layer, hop 6 ms: one device beats four thin ones.
assert ring_tpot([8.0], 6.0) == 14.0
assert ring_tpot([2.0] * 4, 6.0) == 32.0
assert ring_tpot([8.0], 6.0) < ring_tpot([2.0] * 4, 6.0)

print("prima.cpp v3 audit: all assertions passed")
print(f"  home cluster, 32B        : {PUB_T9_HOME_32B} ms/token = {tps(PUB_T9_HOME_32B):.1f} tok/s")
print(f"  Table 10 testbed, 32B    : {PUB_T10['Qwen-2.5-32B'][0]} -> "
      f"{PUB_T10['Qwen-2.5-32B'][1]} ms/token = {tps(PUB_T10['Qwen-2.5-32B'][1]):.1f} tok/s")
print(f"  home 32B + best-case spec: {home_32b_best_case:.0f} ms/token = "
      f"{tps(home_32b_best_case):.1f} tok/s (best measured gain {best*100:.1f}%)")

Run output:

prima.cpp v3 audit: all assertions passed
  home cluster, 32B        : 89 ms/token = 11.2 tok/s
  Table 10 testbed, 32B    : 55 -> 38 ms/token = 26.3 tok/s
  home 32B + best-case spec: 49 ms/token = 20.4 tok/s (best measured gain 45.0%)

Two things fall out.

The 26 tok/s target is not reachable on the paper's own home cluster. Applying the best speculative gain the paper measures anywhere (45%, on Llama 3.3-70B) to the home cluster's measured 89 ms/token gives about 49 ms/token, or 20.4 tok/s. That is a good result and it still clears the "20-50 tok/s for LLM agents" bar the paper cites, but it is not 26, and it assumes a gain measured on a different model on better hardware. If you are sizing a cluster from the abstract, size it from 11 tok/s measured and 20 tok/s optimistic, not from 26.

Adding a faster GPU can make the ring slower. In the ten-node study, a "fill the fastest GPUs first" heuristic produced a four-device plan at 156 ms/token. Halda instead removed a device holding a GTX 1080Ti and moved its two layers onto another node's CPU, reaching 146 ms/token, even though that GPU has roughly 30 times the CPU's FLOPS. The reason is the ring: every device in it adds a round trip per token, and two layers of GPU speedup did not cover one hop of Wi-Fi latency. The worth_adding predicate is the general form of that rule. A device earns its slot only when the compute it lifts off the critical path exceeds the hop it adds, which means the threshold rises with network latency and falls with how many layers the device can take. This is why the paper concludes that a candidate pool of about five devices is enough for a household, and why "more devices is faster" is wrong on a high-latency ring.

How to maintain it

  • Re-profile when the cluster changes. Halda's assignment is only as good as the profiler's latency measurements; adding a device, swapping a disk, or changing which processes compete for RAM shifts the optimal window and GPU-layer split, so re-run profiling rather than reusing an old assignment.
  • Expect OS-specific disk behavior. The paper notes macOS with Metal "reclaims memory more aggressively than Linux, causing more reloads, while Linux optimizes sequential reads, making reloading faster," which makes disk latency hard to estimate; a mixed-OS cluster needs the per-OS device cases Halda models, not a single disk-latency constant.4
  • Pin the quantization to the memory budget. Q4K is the default; IQ1 exists for the tightest budgets. Moving quantization changes the total footprint the cluster must cover and therefore the whole assignment, so treat it as a re-planning event.
  • Re-read the paper on each revision, and keep v1 for the caveats. This source has drifted more than most: between v1 and v3 the title, the author list, the method's name, and the headline speedup against exo and dllama all changed, and v3 removed the limitations section v1 carried. Reviewed publication did not make the paper more cautious about itself; every remaining mention of "limitation" in v3 is about a competing system. When a page cites a preprint, pin the version in the citation, because a later revision can quietly retire the sentence you relied on.
  • Track the context budget as an operational metric, not a model property. The safe region depends on free memory across the cluster, so it moves when a user opens other apps. A cluster that handled 16K yesterday can OOM on it today.

Running it in production

The realistic deployment is a home or small-office cluster serving latency-tolerant local inference (the paper's framing is embodied AI and always-listening assistants), not a throughput-oriented service. v3 adds four operational results that were absent from v1 and that decide whether a deployment survives contact with a real household.

Concurrency peaks early, then declines. Prima.cpp does implement continuous batching, with an independent KV cache per request, and PRP and Halda work unchanged under it. Throughput does not scale indefinitely: when the model is large or memory is tight, it grows sublinearly, peaks around 8-16 concurrent requests, and then falls, because KV caches and compute buffers evict model weights and trigger reloads. Only when resources are ample does it scale near-linearly and saturate around 32.9 Size for the peak rather than assuming monotonic scaling, and note the paper positions 1-8 concurrent requests as typical household load.

Everyday background apps are survivable; games are not. Running TikTok, YouTube, a work suite, Zoom, or a large download alongside inference costs 5-18% more TPOT. A realtime 3D game costs 66%, because it contends for the GPU directly.10 Prima.cpp is designed to yield rather than fight: it lowers RAM priority and releases memory when other apps start, on the reasoning that a user whose game stutters will kill the inference process. Treat that as the intended behaviour, not a regression.

Energy is a reason to distribute, not just a cost. Moving from single-device llama.cpp to distributed prima.cpp cut per-device energy by 91-99% and total energy by 57-90% per 1K tokens, largely because time-to-completion collapses even where average power rises. Against a datacenter server with eight RTX 6000 Ada GPUs, the local cluster used about 28% more total energy, a gap the paper argues narrows once datacenter cooling and residential electricity tariffs are counted.11 The distribution of that energy is lopsided and useful: in the distributed run the one GPU laptop drew 86% of the total, so adding low-power helpers to shorten the critical path is cheap.

Two further cautions from v1 carry over directly. First, more devices is not always faster, and fitting in aggregate RAM is not the deciding factor: the authors' own device-count study found a 3-device cluster (38 GiB, which does not fit the 40 GiB Llama-3-70B Q4K model) beat a 6-device cluster (50 GiB, which does fit it) on token latency, because fast SSDs on the 3-device cluster let mmap swap layers in fast enough to win anyway, while two of the six devices had weak CPUs and slow disks that bottlenecked the larger cluster despite its memory headroom; add devices to hold a bigger model, not by default to speed up one that already fits, and check the specific devices you add, not just the aggregate memory total.5 Second, latency is sensitive to memory competition: because prima.cpp respects available memory, other processes grabbing RAM force it to slow down to free space, so the paper reports "a stable value from multiple runs instead of an error bound," meaning a device doing other work will degrade inference in a way a single benchmark number hides.12

Failure modes

  • Launching plain llama.cpp and believing it is prima.cpp. A bare make and an unranked ./llama-cli build and run single-device llama.cpp: no ring, no Halda, no cross-device assignment, and none of the behavior this page describes. The distributed system exists only when every rank is given --world, --rank, --master, and --next, and --prefetch is what buys the disk/compute overlap. Rank 0 additionally needs a USE_HIGHS=1 build, because HiGHS is the solver Halda calls; without it the automatic layer assignment is not available. A cluster misconfigured this way does not error, it just quietly performs like one machine.
  • Prefetch-release from too large a window. If a device's window exceeds its resident capacity (a mis-profiled memory limit, or another process shrinking available RAM), prefetched layers evict each other and every layer loads twice, erasing the overlap; the executed model above quantifies the exact 2x tax. This is why the window must be sized against available, not total, memory.
  • A single-round (k=1) layout under memory pressure. Running the whole model in one round cannot overlap disk with anything and, when it overflows RAM, also pays the reload tax; PRP's benefit requires multiple rounds with a fitting window.
  • Idle GPUs from a bad assignment. Without Halda's calibration step, a GPU can sit with free VRAM while another device is overloaded; the scheduler must reassign lagging-disk devices to use that GPU, or the cluster underperforms its hardware.
  • Context growth walks off a cliff rather than degrading gracefully. TPOT is flat inside a memory-dependent safe region and then rises sharply or OOMs, at roughly 4K tokens on a low-memory cluster and 16K on a medium-memory one for a 70B Q4K model. A deployment sized on short prompts will fail on the first long one.7
  • Adding a fast device slows the ring. Every device costs a round trip per token. A GPU that carries too few layers to cover that hop makes TPOT worse, which is why Halda removed a 1080Ti node and gained 10 ms/token. Never hand-add devices to a ring on the assumption that more hardware helps.
  • Sizing from the abstract's 26 tok/s. That figure is from a four-node Linux testbed with two RTX 4090s, not from the four consumer home devices the same sentence names. The home cluster's measured 32B rate is 11.2 tok/s.
  • Cloud-latency expectations. A 70B model at 674 ms per token is usable for local, latency-tolerant interaction, not for a service with strict SLOs; the authors say so directly.12
  • Unfiltered content on user devices. The paper flags a safety caveat: running larger open-source models locally means "malicious content may not be filtered," a deployment concern for any home-served model.12

References

  • Li, Li, Feng, Xiao, She, Huang, Guizani, Yu, Ho, Xiang, and Liu, "Prima.cpp: Fast 30-70B LLM Inference on Heterogeneous and Low-Resource Home Clusters", ICLR 2026 (arXiv 2504.08791): https://arxiv.org/abs/2504.08791
  • Prima.cpp v3 PDF (the version this page cites for exact tables): https://arxiv.org/pdf/2504.08791v3
  • Prima.cpp v1 PDF (the 2025 revision, five authors, "piped-ring parallelism"; superseded): https://arxiv.org/pdf/2504.08791v1
  • Leviathan, Kalman, and Matias, "Fast Inference from Transformers via Speculative Decoding" (the technique v3's Appendix A.9 applies): https://arxiv.org/abs/2211.17192
  • Prima.cpp repository (official, MBZUAI CPIL; MIT): https://github.com/OpenCPIL/prima.cpp
  • Prima.cpp README, multi-device deployment (the USE_HIGHS=1 rank-0 build and the --world/--rank/--master/--next/--prefetch launch flags reproduced in "How to use it"): https://github.com/OpenCPIL/prima.cpp#run-on-multiple-devices
  • llama.cpp (the base engine prima.cpp extends, mmap weight loading): https://github.com/ggml-org/llama.cpp
  • HiGHS (the linear/integer programming solver Halda uses): https://github.com/ERGO-Code/HiGHS
  • exo (a distributed home-cluster inference baseline): https://github.com/exo-explore/exo

Related: Disaggregated inference · Cross-WAN model-parallel inference · Privacy-aware split inference over a WAN · Non-colocated inference: which pattern? · vLLM: small models on consumer GPUs · DwarfStar (ds4): DeepSeek V4 local · Pipeline parallelism · Speculative decoding · Quantization for inference · KV cache management · Local coding agents · Inference serving and optimization · Glossary

  1. Prima.cpp (arXiv 2504.08791v3), Section 4 and Table 3. Default cluster D1-D4: Mac M1 (macOS, unified memory, 2.4 GiB available RAM, 0.7 GB/s disk, Apple Silicon GPU); Intel i9 laptop (Linux, 8 cores, 4.1 GiB RAM, 3.0 GB/s disk, RTX 3070, 8 GiB VRAM); Intel i9 desktop (Linux, 16 cores, 9.7 GiB RAM, 3.0 GB/s disk, RTX 2080 Ti, 11 GiB VRAM); Mate40Pro phone (Kirin 9000, HarmonyOS with Termux, 1.9 GiB RAM, 1.4 GB/s disk, no GPU). Total available RAM plus VRAM 37 GiB, "not enough for a Q4K-quantized 70B model." Devices connect over a Wi-Fi router at 320-610 Mbps with 3-7 ms link latency. Table 3 also lists D5 (Honor Pad, Dimensity 8100, Android) and D6 (Mac Air, Intel i5), used in the device-selection study. Table 1 positions consumer local deployment at 2-26 tok/s for 32-70B models against 10-70 tok/s for cloud APIs. ↩↩

  2. Prima.cpp v3, Section 3.1 and Appendices A.1-A.2. v3 renames the mechanism from v1's "piped-ring parallelism" to "pipelined-ring parallelism" and rewrites the section; the quotations here are v3's. PRP "connects devices end-to-end in a ring and runs multiple rounds to predict one token", and "since only a small segment (whose size is referred to as the layer window size) is loaded per round, memory overflow can be avoided, and prefetched layers are less likely to be evicted, thus mitigating the prefetch-release conflict." Note the verb: v3 says mitigating, not eliminating, and states that on slow disks PRP leaves "residual page-fault loading latency, since required layers may not be fully loaded when computation begins." PRP "processes both input and output on the head device, providing enhanced interaction privacy." The worked example (Fig. 1) is six devices, a 36-layer model, window size two, giving eighteen segments and three rounds per token. On the prefetch-release conflict itself (Appendix A.1): the OS prefetches to the memory limit, keeps going, evicts the earliest layers, and "finally, all layers are loaded twice, incurring unnecessary disk I/O without any benefit from prefetching." Fig. 2: "for large models, PRP reduces PP's TPOT by about 50% (PP is equivalent to PRP at k = 1), while for small models, PRP converges to PP with similar TPOT." Appendix A.16 reproduces this on a heterogeneous testbed and notes PRP adds kernel-launch overhead that slightly raises TPOT on small models. ↩↩↩↩↩↩↩

  3. Prima.cpp v3, Sections 3.2-3.3 (unchanged in structure from v1, whose wording some quotations retain). Halda solves the layer-to-device assignment: per device, a window w_m and GPU-layer count n_m minimizing token latency (objective Eqs. 1-5) subject to RAM and VRAM constraints, using a device profiler (compute, memory access, disk loading, communication, OS memory behavior) and a model profiler (FLOPs per layer). NP-hard because the total window appears in objective and constraints; Halda enumerates the rounds k ("at most 11 valid factors for any k <= 100"), reducing each case to an integer linear program solved by HiGHS, iterates a device-set assignment (device cases M1-M4 by OS and memory sufficiency), and calibrates GPU underutilization. "Halda breaks the NP-hard problem into a set of simple ILPs, so we can find the optimal solution in polynomial time," complexity O(T(M + K(2M)^3.5)). ↩↩

  4. Prima.cpp Sections 1-2 and 4.1. v1's phrasing: user devices "struggle to run anything beyond 10B, even with 4-bit quantization"; running Qwen 2.5-14B (Q4K) on a Mac M1 with 8 GiB RAM "takes a staggering 10 seconds per token." v3 rewrites both, saying on-device inference struggles "to run models beyond 8B" while "reliable long-term planning and tool use require 32B or more", and giving the same Mac M1 measurement as "10 s/token with llama.cpp". High-memory alternatives (an Apple M2 Ultra with 192 GiB, or kTransformers needing about 75 GiB for a 70B) are inaccessible to most users. llama.cpp uses mmap for lazy loading; under memory pressure the OS reclaims inactive pages and reloads them, so at 45B "only a few pages are released" but at 60B pages are "released, leading to a sharp increase in token latency and TTFT." OS heterogeneity: macOS with Metal "reclaims memory more aggressively than Linux ... while Linux optimizes sequential reads," making disk latency hard to estimate. ↩↩↩

  5. Prima.cpp v3, Table 4 and Sections 4.1-4.2. Llama 3-70B at 674 ms/token and 1793 ms TTFT (Q4K, D1-D4), versus llama.cpp at 10120 ms/token and 10806 ms TTFT, a 15.0x TPOT ratio; exo and dllama OOM at 70B. Section 4.1: "Compared with llama.cpp, it reduces TPOT by up to 17x and TTFT by up to 8x; against exo and dllama, it achieves at least 18x lower TPOT and 42x lower TTFT, without OOMs." This restates v1, which had claimed 5-8x latency and 12-24x TTFT against exo and dllama. Baseline caveats stated in Section 4: llama.cpp is standalone and was run on D3 alone (largest RAM and VRAM); exo ran on D1-D4 but "D4 failed because it requires root access on the phone"; dllama supports only Llama 3, and exo only Llama 8B and 70B, so unavailable cells are marked "-". vLLM and SGLang "failed to run on our home cluster". Ablations from the same table: prima.cpp without Halda is 20848 ms/token at 70B (30.9x worse), without prefetching 755 ms/token. Per-device memory pressure stays below 6%. Device-count study (v3 Appendix A.7, v1 Appendix A.5): a 3-device cluster (38 GiB, insufficient for the 40 GiB Llama-3-70B Q4K model) beat a 6-device cluster (50 GiB, sufficient) on token latency, because fast SSDs let mmap swap effectively while two of the six devices had weak CPUs and slow disks; "more devices do not always result in faster inference." Ten-node testbed (Appendix A.19, Table 12): at Llama-3.3-70B, prima.cpp 146 ms/token against llama.cpp 24037 and dllama 8415, with exo OOM. Appendix A.19.2: the "Fill-Fastest-GPUs-First" heuristic gives 156 +/- 7 ms/token over four devices; Halda drops the GTX 1080Ti node D5, moves its two layers to D2's CPU, and reaches 146 +/- 5 ms/token, because "adding D5 introduces an extra RTT per token on the high-latency Wi-Fi ring, which costs more than D5 GPU's gain." ↩↩↩↩

  6. Prima.cpp v3, Appendix A.9 and Table 10. A 0.5-3B draft model runs as a standalone process on the head device (set to the most powerful device), predicting 5 tokens per step for batch verification by the target model. "With speculative decoding, prima.cpp delivers an additional 25-45% latency reduction across 14-70B models. For example, Qwen 2.5-32B speeds up from 18 to 26 tokens/s, and Llama 3.3-70B from 1.2 to 2.3 tokens/s." The A.9 testbed is stated in that appendix and is not the Table 3 home cluster: "4 Linux devices, each with an 8-core CPU, 8 GiB RAM, and a sequential disk read throughput of 600 MB/s. Two nodes are equipped with a 4090 GPU, but each is limited to 11 GiB VRAM." Table 10's own non-speculative baseline for Qwen-2.5-32B is 55 ms/token, against 89 ms/token for the same model on the home cluster in Table 9. Section 4.1 quotes "674 ms/token (442 ms/token with speculative decoding)" in a single sentence, but 442 is Table 10's Llama 3.3-70B on the A.9 testbed, whose non-speculative baseline is 803 ms/token, not the home cluster's 674. ↩

  7. Prima.cpp v3, Appendix A.15. Llama-3-70B Q4K at context lengths from 1K to 32K on two heterogeneous testbeds. "Increasing the context length has a safe region where TPOT remains flat. Larger memory provides a wider flat region, e.g., about 1-4K tokens on the low-memory testbed and 1-16K tokens on the medium-memory testbed. Beyond this range, TPOT rises sharply or causes OOM, as the expanding KV cache exhausts available memory." The mechanism is competition for the same page cache: KV-cache growth leaves less room for weights, making prefetch-release conflicts more likely. Note that the figure carrying this result (Fig. 17) has a caption describing a different experiment, the MemSched/PerfSched/Halda breakdown. ↩↩

  8. Prima.cpp v3, Appendix A.17. Four-device heterogeneous testbed over Wi-Fi (Mac M1 Pro, Mac M3 Pro, Linux laptop with a 4090, Honor Pad), bandwidth and latency shaped with Network Link Conditioner and tc, baseline RTT 94 ms, five repeats. "Prima.cpp is insensitive to bandwidth changes, except under extremely low bandwidth (e.g., lower than 50 Mbps)" and "when RTT increases from 100 ms to 200 ms, TPOT rises by 68%, but still far outperforms the baseline." Conclusion: "bandwidth is no longer the primary bottleneck, but link latency has become the main limiting factor." ↩

  9. Prima.cpp v3, Appendix A.11. "This paper focuses on single-request inference" for two stated reasons: home deployments have low concurrency, and continuous batching is not the paper's contribution. Prima.cpp nonetheless implements continuous batching, merging active requests at the token step with an independent KV cache per request; because PRP also operates on token-step batches, "PRP and Halda work unchanged in multi-user/request settings." Llama-3-14B and 70B at 1-40 concurrent requests: throughput "grows sublinearly, peaks around 8-16 concurrent requests, then declines" for the 70B on the medium-memory testbed and the 14B on the low-memory one, because concurrency consumes memory for KV caches and compute buffers and triggers weight eviction; with ample resources (14B, medium memory) it scales near-linearly, saturates around 32, then drops. The paper states typical household usage is 1-8 concurrent requests. ↩

  10. Prima.cpp v3, Appendix A.18. Six background workloads on the head device (TikTok, a work suite of Chrome/Teams/Slack/ChatGPT/VSCode/Outlook/WhatsApp, YouTube, Zoom with screen share and camera, a large App Store download, and the realtime 3D game Asphalt), five repeats each. "Everyday apps such as TikTok, YouTube, downloads, work suites, and video conferencing increase TPOT by only 5-18% ... The sharpest slowdown appears with large realtime 3D games, which heavily compete for GPU time and increase TPOT by 66%." Prima.cpp "adopts a lower RAM priority: it reduces memory use when other apps start and reclaims it when they stop", so that it degrades rather than being killed by the user. ↩

  11. Prima.cpp v3, Appendix A.13 and Fig. 13. Local testbed of four mobile devices (Mac M1 Pro, Mac M3 Pro, Linux laptop with a 4090, Honor Pad), power recorded with PowerMetrics, NVIDIA Power Counters plus powercap, and Ludashi; cloud comparison on a datacenter server with eight RTX 6000 Ada GPUs. Energy per 1K output tokens: switching from single-device llama.cpp to distributed prima.cpp "reduces per-device energy by 91-99% and total energy by 57-90%". For the GPU laptop, average power falls 37% and time-to-1K-tokens falls 86%, so total energy falls 91%; in the distributed run that laptop "contributes 86% of total energy, while others account for only 14%". Against the cloud server, total local energy is higher by 28% (12.6 versus 9.1 Wh per 1K tokens in Fig. 13), a gap the paper argues narrows once datacenter cooling and residential-versus-commercial electricity tariffs are counted. These are the paper's measurements on its own hardware, not a general result. ↩

  12. These limitations are stated in v1 (Section 5) and were removed in v3. v1's list: limited device types and quantity restrict the exploration; "70B models remain much slower than those on the cloud"; in low-RAM clusters without SSDs or GPUs "larger models will be extremely slow"; token latency "is heavily affected by memory competition," so the paper reports "a stable value from multiple runs instead of an error bound" (no error bars) because other processes force prima.cpp to slow down to free RAM; and a safety caveat that larger open-source models on user devices run "where malicious content may not be filtered." A design caveat in v1: the implementation assigns every device at least one layer, which the authors planned to relax so idle devices drop out; v3 delivers exactly that as Halda's device selection. v3 replaces Section 5 with a Conclusion carrying no self-directed limitations: each of the five occurrences of "limitation" in v3 describes a competing system (existing distributed systems, exo, or dllama), never prima.cpp. The v1 cautions are retained on this page because they remain true of the system and nothing in v3 retracts them; v3 does supply measurements for two of them, in Appendix A.18 (memory competition from background apps) and Appendix A.15 (context length). ↩↩↩↩↩