Markdown

DFloat11: lossless BFloat16 compression¶

Scope: entropy-coding the BFloat16 exponent field so an LLM occupies roughly 70% of its BF16 size while producing bit-for-bit identical outputs, and the GPU kernel that decompresses weights inline on every forward pass. Covers the compression format, the hierarchical lookup tables and the two-phase decode kernel, the dfloat11 package, and the central trade this buys: capacity at the cost of speed. Proposed in "70% Size, 100% Accuracy" (Zhang et al., NeurIPS 2025).¹ Contrast with quantization, which is lossy and much faster.

The numpy block below is self-contained, was executed with system python3, and asserts every claim it prints, including the adversarial and failure cases. The dfloat11 package install and its import boundary were executed on this page's CPU-only host on 2026-07-17, re-verified the same day against both the resolver-default CUDA 13 torch stack and a deliberately pinned CUDA 12.6 torch stack (see "The install boundary" below); both reach the identical cudaErrorNoDevice import failure, because that failure comes from the missing GPU, not from either CUDA version. The GPU decode path, the health gate, and the rollback template need a CUDA GPU and are reference templates, not executed here.

What it is¶

DFloat11 (DF11) is a lossless compression format for BF16 model weights, plus a CUDA kernel that decompresses them on the GPU while the model runs. It is not quantization. The decompressed weights are the same bits that went in, so the model's outputs are unchanged, not approximately unchanged.¹

The idea rests on one measurement. A BF16 number is 1 sign bit, 8 exponent bits, and 7 mantissa bits. Across the linear projections of Llama 3.1 8B, Gemma 2 9B, Qwen 2.5 14B, Mistral Small 24B and Llama 3.3 70B, the paper measures the Shannon entropy of each field: the sign carries a full 1.0 bits, the mantissa carries about 6.9 to 7.0 bits of its 7, and the exponent carries about 2.6 bits of its 8.¹ Trained weights cluster tightly around zero, so only about 40 of the 256 possible exponent values ever appear, and a handful of those dominate. The sign and mantissa look like noise and cannot be compressed; the exponent is a nearly-degenerate distribution stored in a fixed-width field.

So DF11 Huffman-codes the exponent byte and leaves everything else alone. Each weight becomes a variable-length exponent code plus a raw byte holding the sign and mantissa. The result averages 10.8 to 11.1 bits per weight, which is where the name comes from, and lands the checkpoint at 67.6% to 69.5% of BF16.¹

Model	BF16	DF11	Ratio	Avg bits
Llama 3.1 8B Instruct	16.06 GB	10.90 GB	67.84%	10.85
Llama 3.3 70B Instruct	141.11 GB	95.40 GB	67.61%	10.82
Llama 3.1 405B Instruct	811.71 GB	551.22 GB	67.91%	10.87
QwQ 32B	65.53 GB	44.65 GB	68.14%	10.90
Mistral Small 3	47.14 GB	31.86 GB	67.58%	10.81
FLUX.1 dev	23.80 GB	16.33 GB	68.61%	10.98
Stable Diffusion 3.5 Large	16.29 GB	11.33 GB	69.52%	11.12

Figures are from Table 1 of arXiv v3.¹ Version drift is worth knowing about if you cite this: for the four LLMs v1 also evaluated, v1's table reported worse ratios than v3's except for the 405B, whose row is unchanged (Llama 3.1 8B went from 69.98% and 11.20 bits in v1 to 67.84% and 10.85 in v3); FLUX.1 dev and Stable Diffusion 3.5 Large are not "worsened" in v1, they simply are not in it, since v1 never evaluated diffusion models at all. v1 carried Gemma rows that v3 silently drops. Gemma compresses worst of any model in either table, at 71.81% and 11.49 bits for gemma-2-9b-it.² Quote v3 for Llama, Qwen and Mistral, quote v1 for Gemma, and do not mix the two.

Why use it¶

One reason, and it is a good one: a model that does not fit, fits. Cutting 32% off the weights moves a checkpoint across a GPU memory boundary without touching its outputs, so no evaluation has to be re-run and no accuracy regression has to be argued about. Llama 3.1 405B is 811.71 GB in BF16, which does not fit an 8x80GB node; at 551.22 GB it does.¹

The alternative when a model overflows is to offload layers to CPU and pay PCIe on every token. Against that baseline (Hugging Face Accelerate offloading), DF11 is 2.31x to 46.24x faster.¹ The spread is not a property of DF11; it is a measure of how much of the model spilled to the host. The 46.24x is Mistral Small 3 (48 GB) on a 40 GB A100 at batch 1, where the BF16 baseline is crawling at 11.56 s/token. Within a fixed memory budget, the freed space also buys KV cache: 5.70x to 14.86x more generated tokens before OOM, measured at batch 1.¹

Against quantization, the argument is narrower and it is about risk, not speed. INT4 or FP8 shrinks the same model by 4x or 2x rather than 1.45x, and runs faster rather than slower. What DF11 offers is that you never have to prove the compressed model is still good, because it is the same model. The paper's own quantization datapoint is a fair statement of the risk it avoids: Llama 3.1 8B at INT8 loses 4.0 points on MATH Hard and 1.12 on GPQA CoT.¹ If your workload can absorb that, quantize; DF11 is for when it cannot.

When to use it (and when not)¶

Use it when the model sits just over a memory line and the next GPU is not available or not worth its price; when a lossy format is unacceptable for contractual, regulatory or evaluation-churn reasons; when the workload is a diffusion model, where the compute per forward pass is large enough to hide the decode (Stable Diffusion 3.5 pays +4.1% latency, FLUX.1 dev +5.5%, for about 28% less peak memory);¹ or when the honest comparison is against CPU offloading rather than against a model that fits.

Do not use it when the model already fits. This is the fact the abstract's "negligible overhead" framing obscures, and it is the single most important thing on this page. Appendix G of the paper measures DF11 against BF16 on the same GPUs with both models resident, and DF11 loses every time:¹

Model / GPU	Batch	BF16 tok/s	DF11 tok/s	DF11 slowdown
Llama 3.1 8B / A100-40GB	1	16.6	9.4	1.77x
Llama 3.1 8B / A100-40GB	128	2582.5	1146.3	2.25x
Llama 3.1 8B / A100-40GB	512	4422.2	3190.1	1.39x
Qwen 3 14B / A100-40GB	32	443.7	180.9	2.45x
Llama 3.3 70B / 4xA100-40GB	1	5.4	1.5	3.60x
Llama 3.3 70B / 4xA100-40GB	8	53.5	12.2	4.39x
Llama 3.3 70B / 4xA100-40GB	128	265.8	154.7	1.72x

The slowdown ratios are this page's arithmetic over the paper's throughput figures. The upstream README states the same thing in one line: "At batch size = 1, inference is approximately 2× slower than the original BF16 model."³ A user reproduced 31.91 tok/s BF16 against 9.51 tok/s DF11 on Qwen2.5-14B, and the maintainer confirmed the behaviour is expected.⁷

Also skip it when the checkpoint is already quantized. DF11 compresses the exponent field and nothing else, so an INT8 or INT4 model has no exponent redundancy left to harvest. The maintainer declined to support this for exactly that reason, noting that an INT8 model "might already use more than 7 bits of entropy."¹⁰ The code block below reproduces that argument from first principles.

Finally, do not reach for it if you need LoRA, fine-tuning, or tensor parallelism. None are supported (see Failure modes).

Architecture¶

flowchart TB
  subgraph OFF["Offline, once, CPU only"]
    A["BF16 checkpoint"] --> B["Split each weight:<br/>1 sign + 8 exponent + 7 mantissa"]
    B --> C["Huffman-code the exponent<br/>(~2.6 bits of entropy)"]
    B --> D["PackedSignMantissa<br/>1 raw byte per weight"]
    C --> E["EncodedExponent bitstream<br/>+ Gaps + BlockOutputPos"]
  end
  subgraph GPU["Online, every forward pass"]
    E --> F["DF11 weights resident in HBM"]
    D --> F
    F --> G["Per transformer block:<br/>one batched decode kernel"]
    G --> H["Phase 1: count elements,<br/>no HBM writes, prefix-sum"]
    H --> I["Phase 2: re-decode,<br/>reassemble BF16, coalesced write"]
    I --> J["BF16 weights in buffer"]
    J --> K["Block matmuls"]
    K --> L["Discard buffer, next block"]
  end

Three design choices carry the kernel, and each exists to work around a way that Huffman decoding is hostile to a GPU.

Hierarchical lookup tables. Decoding a Huffman code by walking the tree bit by bit is a serial pointer chase with a data-dependent branch at every step, which is close to a worst case for a GPU. The standard fix is a lookup table indexed by the next L bits, where L is the longest code, turning the walk into one load. But for LLM exponent trees L runs 24 to 32 bits, so that table wants up to 2^32 entries, about 4.29 billion.¹² Shared memory is around 100 KB per block by the paper's own working figure, and 228 KB per SM on an H100.¹⁸ So DF11 splits the tree into subtrees of height 8 and gives each one a 256-entry, one-byte-per-entry table. Four to eight such tables cover a real model, and the whole structure fits in at most (8+1) x 256 = 2304 bytes of SRAM.¹ The pointers between tables are free: exponent values 240 to 255 never occur in real weights (they would encode magnitudes around 2^113 and up), so those slots are repurposed as "descend into table n" markers.¹

A two-phase kernel. Variable-length codes mean a thread cannot know where its output belongs until every thread before it has finished decoding, which is a write-position dependency across the whole stream. DF11 breaks it by decoding twice. Each thread takes a contiguous 8 bytes of the encoded stream; in phase 1 every thread decodes its bytes and only counts how many weights it produced, writing nothing; a block-wide Blelloch prefix sum then converts those counts into output offsets; in phase 2 every thread decodes the same bytes again and writes to the offset it now knows.¹ Two small auxiliary arrays make this work: Gaps holds, for each thread, the bit offset of the first code that actually starts inside its 8-byte slice (a value in [0,31], so 5 bits), and BlockOutputPos holds one 32-bit integer per block rather than per thread, since per-thread offsets would eat the compression gain.¹

Block-level batching. A single weight matrix is too small to saturate a GPU, so decompression throughput climbs with matrix size. DF11 therefore decompresses all the matrices of one transformer block in a single kernel launch, issued just before that block runs.¹ Crucially, the decompressed BF16 weights are discarded right after the block's matmuls, so every weight is re-decompressed on every forward pass. There is no cache. This is the source of the entire latency cost.

How it works: the core algorithm¶

The block below builds DF11 end to end in numpy: split the BF16 fields, measure their entropy, Huffman-code the exponent, build both a flat and a hierarchical decoder, and prove the roundtrip is bit-exact. It then attacks the method: the case where the compression evaporates, the hard floor, what a single corrupted bit does, and why a quantized checkpoint gains nothing.

# dfloat11_core.py -- executed, numpy only. BF16 = 1 sign | 8 exponent | 7 mantissa.
# DFloat11 Huffman-codes the exponent byte and stores sign+mantissa raw.
import heapq
import numpy as np

RNG = np.random.default_rng(0)


def to_bf16(x: np.ndarray) -> np.ndarray:
    """float32 -> bf16 bit patterns (uint16), round-to-nearest-even."""
    u = x.astype(np.float32).view(np.uint32)
    rounded = (u + 0x7FFF + ((u >> 16) & 1)) >> 16
    return rounded.astype(np.uint16)


def split_bf16(w: np.ndarray) -> tuple[np.ndarray, np.ndarray]:
    """bf16 -> (exponent byte, sign|mantissa byte)."""
    exponent = (w >> 7).astype(np.uint8)
    sign = ((w >> 15) & 1).astype(np.uint8)
    mantissa = (w & 0x7F).astype(np.uint8)
    return exponent, ((sign << 7) | mantissa).astype(np.uint8)


def join_bf16(exponent: np.ndarray, signmant: np.ndarray) -> np.ndarray:
    sign = (signmant >> 7).astype(np.uint16)
    mantissa = (signmant & 0x7F).astype(np.uint16)
    return ((sign << 15) | (exponent.astype(np.uint16) << 7) | mantissa).astype(np.uint16)


def entropy_bits(counts: np.ndarray) -> float:
    p = counts[counts > 0] / counts.sum()
    return float(-(p * np.log2(p)).sum())


def huffman_lengths(counts: np.ndarray) -> dict[int, int]:
    """Canonical Huffman code lengths per symbol."""
    live = [(int(c), [int(s)]) for s, c in enumerate(counts) if c > 0]
    if len(live) == 1:
        return {live[0][1][0]: 1}
    depth = {s: 0 for _, syms in live for s in syms}
    heap = [(c, i, syms) for i, (c, syms) in enumerate(live)]
    heapq.heapify(heap)
    tie = len(heap)
    while len(heap) > 1:
        c0, _, s0 = heapq.heappop(heap)
        c1, _, s1 = heapq.heappop(heap)
        for s in s0 + s1:
            depth[s] += 1
        heapq.heappush(heap, (c0 + c1, tie, s0 + s1))
        tie += 1
    return depth


def canonical_codes(lengths: dict[int, int]) -> dict[int, tuple[int, int]]:
    codes: dict[int, tuple[int, int]] = {}
    code, prev_len = 0, 0
    for sym in sorted(lengths, key=lambda s: (lengths[s], s)):
        code <<= lengths[sym] - prev_len
        codes[sym] = (code, lengths[sym])
        code += 1
        prev_len = lengths[sym]
    return codes


def encode(exponent: np.ndarray, codes: dict[int, tuple[int, int]]) -> np.ndarray:
    bits: list[int] = []
    for s in exponent.tolist():
        code, ln = codes[s]
        bits.extend((code >> (ln - 1 - i)) & 1 for i in range(ln))
    return np.array(bits, dtype=np.uint8)


def build_lut(codes: dict[int, tuple[int, int]], max_len: int) -> tuple[np.ndarray, np.ndarray]:
    """Flat LUT: index by the next max_len bits -> (symbol, code length)."""
    sym = np.zeros(1 << max_len, dtype=np.uint8)
    ln = np.zeros(1 << max_len, dtype=np.uint8)
    for s, (code, cl) in codes.items():
        base = code << (max_len - cl)
        sym[base: base + (1 << (max_len - cl))] = s
        ln[base: base + (1 << (max_len - cl))] = cl
    return sym, ln


def build_hierarchy(codes: dict[int, tuple[int, int]], chunk: int = 8) -> list[tuple]:
    """Decompose one 2**max_len LUT into a tree of 2**chunk tables (the DF11 trick).

    Each table holds sym[256], len[256], next[256]. next >= 0 means "consume
    `chunk` bits and descend"; otherwise len > 0 marks a leaf. DF11 itself avoids
    the separate next array by reusing exponent values 240-255 as pointers.
    """
    tables: list[tuple[np.ndarray, np.ndarray, np.ndarray]] = []

    def make(sub: dict[int, tuple[int, int]]) -> int:
        idx = len(tables)
        sym = np.zeros(1 << chunk, dtype=np.uint8)
        ln = np.zeros(1 << chunk, dtype=np.uint8)
        nxt = np.full(1 << chunk, -1, dtype=np.int16)
        tables.append((sym, ln, nxt))
        deeper: dict[int, dict[int, tuple[int, int]]] = {}
        for s, (code, cl) in sub.items():
            if cl <= chunk:
                base = code << (chunk - cl)
                sym[base: base + (1 << (chunk - cl))] = s
                ln[base: base + (1 << (chunk - cl))] = cl
            else:
                prefix = code >> (cl - chunk)
                deeper.setdefault(prefix, {})[s] = (code & ((1 << (cl - chunk)) - 1), cl - chunk)
        for prefix, child in deeper.items():
            nxt[prefix] = make(child)
        return idx

    make(codes)
    return tables


def _window(padded: np.ndarray, pos: int, chunk: int) -> int:
    w = 0
    for b in padded[pos: pos + chunk]:
        w = (w << 1) | int(b)
    return w


def decode_hier(bits: np.ndarray, n: int, tables: list[tuple], chunk: int = 8) -> np.ndarray:
    """Decode by walking the LUT hierarchy, `chunk` bits per hop."""
    out = np.zeros(n, dtype=np.uint8)
    padded = np.concatenate([bits, np.zeros(64, dtype=np.uint8)])
    pos = 0
    for i in range(n):
        t, used = 0, 0
        while True:
            sym, ln, nxt = tables[t]
            w = _window(padded, pos + used, chunk)
            if nxt[w] >= 0:
                t, used = int(nxt[w]), used + chunk
                continue
            out[i] = sym[w]
            used += int(ln[w])
            break
        pos += used
    return out


def decode(bits: np.ndarray, n: int, sym: np.ndarray, ln: np.ndarray, max_len: int) -> np.ndarray:
    """Flat-LUT decode of n symbols starting at bit 0."""
    out = np.zeros(n, dtype=np.uint8)
    padded = np.concatenate([bits, np.zeros(max_len, dtype=np.uint8)])
    pos = 0
    for i in range(n):
        w = _window(padded, pos, max_len)
        out[i] = sym[w]
        pos += int(ln[w])
    return out


def stats(w_bf16: np.ndarray) -> tuple[float, float, float]:
    exponent, _ = split_bf16(w_bf16)
    counts = np.bincount(exponent, minlength=256)
    lengths = huffman_lengths(counts)
    avg = sum(counts[s] * l for s, l in lengths.items()) / counts.sum()
    return entropy_bits(counts), avg, 8.0 + avg


# LLM weights cluster near zero, so model them as normal. This is a synthetic
# stand-in for a real tensor, not a measurement of Llama's weights.
w = to_bf16(RNG.normal(0.0, 0.02, 200_000).astype(np.float32))
exponent, signmant = split_bf16(w)
counts = np.bincount(exponent, minlength=256)
H = entropy_bits(counts)
lengths = huffman_lengths(counts)
codes = canonical_codes(lengths)
avg_len = sum(counts[s] * l for s, l in lengths.items()) / counts.sum()
bits_per_weight = 8.0 + avg_len
MAXLEN = max(lengths.values())

print(f"distinct exponents used     : {int((counts > 0).sum())} of 256")
print(f"exponent entropy            : {H:.3f} bits")
print(f"huffman avg code length     : {avg_len:.3f} bits  (max {MAXLEN})")
print(f"sign+mantissa (stored raw)  : 8.000 bits")
print(f"bits per weight             : {bits_per_weight:.3f}  -> DFloat11")
print(f"compressed size             : {bits_per_weight / 16:.1%} of BF16")

# 1) Huffman lands within one bit of the Shannon limit.
assert H <= avg_len < H + 1.0

# 2) the other 8 bits are noise: their entropy is already ~8, so coding them is
#    wasted work. This is why DF11 leaves sign and mantissa raw.
h_raw = entropy_bits(np.bincount(signmant, minlength=256))
print(f"entropy of sign|mantissa    : {h_raw:.3f} bits (incompressible)")
assert h_raw > 7.9

# 3) lossless: the roundtrip returns the identical BF16 bit patterns.
bits = encode(exponent, codes)
sym_lut, len_lut = build_lut(codes, MAXLEN)
dec = decode(bits, exponent.size, sym_lut, len_lut, MAXLEN)
assert np.array_equal(dec, exponent)
rt = join_bf16(dec, signmant)
assert np.array_equal(rt, w)
print(f"roundtrip bit-exact         : {bool(np.array_equal(rt, w))} ({w.size} weights)")

# 4) the two-phase kernel's premise: given a per-chunk start offset, independent
#    workers decode disjoint chunks and land byte-identical to a serial decode.
CHUNK = 4096
offsets = np.concatenate([[0], np.cumsum([lengths[s] for s in exponent.tolist()])])
starts = offsets[::CHUNK][:-1] if exponent.size % CHUNK == 0 else offsets[::CHUNK]
parts = []
for i, start in enumerate(starts.tolist()):
    n = min(CHUNK, exponent.size - i * CHUNK)
    parts.append(decode(bits[start:], n, sym_lut, len_lut, MAXLEN))
parallel = np.concatenate(parts)
assert np.array_equal(parallel, exponent)
print(f"two-phase chunk decode == serial: {bool(np.array_equal(parallel, exponent))} "
      f"({len(starts)} chunks, block-offset array {len(starts) * 4} B)")

# 5) why one flat LUT will not do: it is indexed by the LONGEST code, so it costs
#    2**MAXLEN entries. The hierarchy of 8-bit tables decodes identically in a
#    fraction of the SRAM. Real models have 24-32 bit codes, far worse than this.
flat_kb = (sym_lut.nbytes + len_lut.nbytes) / 1024
hier = build_hierarchy(codes)
hier_kb = sum(t.nbytes for level in hier for t in level) / 1024
dec_h = decode_hier(bits, exponent.size, hier)
assert np.array_equal(dec_h, exponent)
print(f"flat LUT ({MAXLEN}-bit code)     : {flat_kb:.0f} KB "
      f"(H100 SM shared memory: 228 KB)")
print(f"hierarchical 8-bit LUTs     : {hier_kb:.2f} KB in {len(hier)} tables, "
      f"decode identical: {bool(np.array_equal(dec_h, exponent))}")
assert hier_kb < flat_kb / 20

# 6) adversarial: the win is a property of the weights, not of the code. On
#    uniform exponents Huffman degenerates to a fixed 8-bit code, the tensor does
#    not shrink at all, and the auxiliary array makes it a net loss. (This models
#    DF11's per-block BlockOutputPos, one uint32 per block; its per-thread Gaps
#    array costs a further 5 bits per thread.)
u = RNG.integers(0, 256, 200_000).astype(np.uint8)
u_counts = np.bincount(u, minlength=256)
u_len = sum(u_counts[s] * l for s, l in huffman_lengths(u_counts).items()) / u.size
overhead = 32.0 / CHUNK
print(f"uniform-exponent tensor     : {8.0 + u_len:.3f} bits/weight, "
      f"{8.0 + u_len + overhead:.3f} with the offset array (BF16 is 16.000)")
assert 8.0 + u_len == 16.0
assert 8.0 + u_len + overhead > 16.0

# 7) floor: even a tensor whose weights share one exponent pays 1 bit for it, so
#    9 bits per weight is the best DF11 can ever do.
d = np.full(1000, 0x3E, dtype=np.uint8)
d_counts = np.bincount(d, minlength=256)
d_len = sum(d_counts[s] * l for s, l in huffman_lengths(d_counts).items()) / d.size
print(f"single-exponent tensor      : {8.0 + d_len:.3f} bits/weight (hard floor)")
assert 8.0 + d_len == 9.0

# 8) corruption: the stream is not self-synchronising. One flipped bit desyncs the
#    decoder and silently rewrites most of the tensor. Checksum the artifact.
bad = bits.copy()
bad[5000] ^= 1
dec_bad = decode(bad, exponent.size, sym_lut, len_lut, MAXLEN)
wrong = int((dec_bad != exponent).sum())
print(f"1 flipped bit corrupts      : {wrong} of {exponent.size} weights")
assert wrong > 100
assert int(bits.sum()) != int(bad.sum())

# 9) DF11 exploits ONE redundancy, in the exponent. Round a tensor to 255 INT8
#    levels and the whole 16-bit word carries under 8 bits, yet DF11 still spends
#    ~10.4, because it never touches the mantissa. Ship that tensor as INT8.
q = to_bf16((RNG.integers(-127, 128, 200_000) * (0.02 / 127)).astype(np.float32))
_, _, q_bits = stats(q)
_, word_counts = np.unique(q, return_counts=True)
word_h = entropy_bits(word_counts)
print(f"int8-rounded tensor in bf16 : {q_bits:.3f} bits/weight from DFloat11, "
      f"but the word carries {word_h:.3f} bits")
assert q_bits > word_h + 2.0

Executed output:

distinct exponents used     : 20 of 256
exponent entropy            : 2.544 bits
huffman avg code length     : 2.590 bits  (max 16)
sign+mantissa (stored raw)  : 8.000 bits
bits per weight             : 10.590  -> DFloat11
compressed size             : 66.2% of BF16
entropy of sign|mantissa    : 7.971 bits (incompressible)
roundtrip bit-exact         : True (200000 weights)
two-phase chunk decode == serial: True (49 chunks, block-offset array 196 B)
flat LUT (16-bit code)     : 128 KB (H100 SM shared memory: 228 KB)
hierarchical 8-bit LUTs     : 3.00 KB in 3 tables, decode identical: True
uniform-exponent tensor     : 16.000 bits/weight, 16.008 with the offset array (BF16 is 16.000)
single-exponent tensor      : 9.000 bits/weight (hard floor)
1 flipped bit corrupts      : 156083 of 200000 weights
int8-rounded tensor in bf16 : 10.375 bits/weight from DFloat11, but the word carries 7.994 bits

Read the numbers carefully, because two of them are the whole method and two of them are its boundary.

The 2.544-bit exponent entropy against 7.971 bits for the sign-mantissa byte is the entire thesis, reproduced on a synthetic normal tensor: it recovers the paper's ~2.6-bit measurement without having its weights. The 10.590 bits per weight is slightly better than the paper's 10.81 to 11.12, which is expected: a pure normal is a touch more concentrated than a real tensor, and this model does not carry the auxiliary arrays. Treat this block as a proof of mechanism, not a substitute for the paper's measurements.

The uniform-exponent line is the adversarial case, and it fails more subtly than expected. Huffman does not expand a uniform distribution; it degenerates to a flat 8-bit code and returns exactly 16.000 bits per weight, at which point the auxiliary offset array tips it to a net loss. So a tensor with high exponent entropy does not blow up, it just quietly buys nothing while costing a decompression kernel. The int8-rounded line is the same trap wearing different clothes, and it independently reproduces the maintainer's stated reason for not supporting quantized models.¹⁰

The corruption line is an operational warning. A Huffman stream is not self-synchronising, so one flipped bit resynchronises the decoder into garbage and silently rewrote 156,083 of 200,000 weights here. Nothing raises. Checksum DF11 artifacts on download and after any copy.

How to use it¶

Install the package and load a pre-compressed checkpoint. The API wraps a Hugging Face model rather than replacing it, so generation is unchanged.

# Reference template from the upstream README (needs a CUDA GPU). Not executed here.
# Install with the pins verified on 2026-07-17 (see the install boundary below).
# Pin a cu12x torch build FIRST, or the resolver defaults to an unsupported CUDA 13 stack:
# pip install "torch==2.13.0+cu126" --index-url https://download.pytorch.org/whl/cu126
# pip install "dfloat11[cuda12]==0.5.0" "setuptools==80.9.0"
import torch
from dfloat11 import DFloat11Model
from transformers import AutoTokenizer

model_id = "DFloat11/Qwen3-8B-DF11"

model = DFloat11Model.from_pretrained(model_id, device_map="auto")

tokenizer = AutoTokenizer.from_pretrained(model_id)
tokenizer.pad_token = tokenizer.eos_token

prompt = "Question: What is a binary tree and its applications? Answer:"
inputs = tokenizer(prompt, return_tensors="pt", padding=True).to(model.device)

with torch.no_grad():
    output = model.generate(**inputs, max_new_tokens=256, do_sample=True)

print(tokenizer.batch_decode(output, skip_special_tokens=True))

The public surface is two names, DFloat11Model and compress_model. The installed package exposes no console-script entry point, so there is no dfloat11 command; the repo instead ships argparse scripts you run directly (inference.py for benchmarking, examples/compress_flux1/compress_flux.py for compression). Multi-GPU means Accelerate's device_map="auto" layer sharding, not tensor parallelism. The DFloat11 org on Hugging Face publishes 45 pre-compressed models, including Llama-3.1-405B-Instruct-DF11 at 547 GB, Qwen3-8B-DF11 at 11.17 GB, and FLUX.1-dev-DF11 at 16.33 GB.⁶

Check your CUDA version before anything else. The wheel ships a precompiled PTX file built against CUDA 12.2 (PTX ISA 8.2) and JIT-compiles it through the driver, so CUDA 12.0 and 12.1 fail with CUDA_ERROR_UNSUPPORTED_PTX_VERSION, and CUDA 13 is not supported upstream.¹¹¹⁴ There is no GPU architecture floor: the PTX targets sm_52 and the kernel uses no arch-gated intrinsics. torch is not in install_requires (the 0.5.0 metadata declares accelerate, dahuffman==0.4.2, huggingface-hub, safetensors, transformers and tqdm, with cupy behind the cuda11/cuda12 extras)⁵ and no version floor is documented anywhere, so pin it yourself.

The install boundary, executed on a CPU-only host¶

The pins above are not decorative. This page ran the install in a fresh venv on a CUDA-less x86_64 Linux host (Python 3.12.3, no NVIDIA driver) on 2026-07-17. pip install dfloat11==0.5.0 succeeds, and the resulting package cannot be imported, first for two reasons that have nothing to do with the missing GPU, then for one that does. The three failures, captured verbatim and trimmed to the load-bearing lines:

$ pip install dfloat11==0.5.0     # succeeds; resolver picks setuptools 83.0.0
$ python -c "import dfloat11"
  File ".../site-packages/dfloat11/dfloat11.py", line 19, in <module>
    import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'

dfloat11/dfloat11.py:19 imports pkg_resources, which current setuptools no longer ships; the deprecation warning that fires once it is back says, verbatim, "Refrain from using this package or pin to Setuptools<81." Pinning setuptools==80.9.0 restores it (verified) and exposes the next layer:

$ pip install setuptools==80.9.0 && python -c "import dfloat11"
  File ".../site-packages/dfloat11/dfloat11.py", line 27, in <module>
    import cupy as cp
ModuleNotFoundError: No module named 'cupy'

cupy only arrives through the [cuda11] or [cuda12] extra, so the extra is mandatory for import, not an optional accelerator path. With dfloat11[cuda12]==0.5.0 installed (cupy-cuda12x 14.1.1), the real GPU boundary appears:

$ python -c "import dfloat11"
  File ".../site-packages/dfloat11/dfloat11.py", line 40, in <module>
    _decode = cp.RawModule(path=ptx_path).get_function('decode')
cupy_backends.cuda.api.runtime.CUDARuntimeError: cudaErrorNoDevice:
    no CUDA-capable device is detected

The module loads its PTX decode kernel at import time, so a host with no CUDA device cannot even import dfloat11. There is no CPU mode of any kind.

What the resolver produced on 2026-07-17, and what a deployment should therefore pin:

dfloat11==0.5.0          # latest on PyPI, uploaded 2025-08-24; no release since
setuptools==80.9.0       # unpinned resolves 83.0.0, which lacks pkg_resources
cupy-cuda12x==14.1.1     # via the [cuda12] extra; mandatory for import
torch==2.13.0            # NOT declared by dfloat11; arrives transitively via accelerate
transformers==5.14.1  accelerate==1.14.0  dahuffman==0.4.2
safetensors==0.8.0    huggingface_hub==1.23.0  numpy==2.5.1

One more trap sits in that resolution. The default PyPI torch 2.13.0 wheel bundles the CUDA 13 runtime (this install pulled nvidia-*-cu13 wheels and cuda-toolkit 13.0.3.0 alongside it), while upstream's own issue tracker says CUDA 13 is not supported.¹⁴ An unpinned install today therefore hands you a CUDA 13 torch stack under a kernel documented as broken on CUDA 13. Pin a cu12x build instead of relying on the resolver default:

$ pip install "torch==2.13.0+cu126" --index-url https://download.pytorch.org/whl/cu126
$ pip install "dfloat11[cuda12]==0.5.0" "setuptools==80.9.0"

Verified on this same CPU-only host, 2026-07-17: PyTorch's wheel index publishes cu126, cu129 and cu130 builds of torch 2.13.0 for cp312/manylinux, and no cu12x build older than cu126.¹⁵ Installing the cu126 wheel first, then dfloat11[cuda12]==0.5.0 with setuptools==80.9.0, leaves torch.__version__ at 2.13.0+cu126 and torch.version.cuda at 12.6: pip's resolver sees accelerate's unpinned torch>=2.0.0 already satisfied and leaves it alone, so none of the nvidia-*-cu13 or cuda-toolkit-13.x packages get pulled in, and the CUDA 12.6 stack (cuda-toolkit-12.6.3, nvidia-cudnn-cu12-9.10.2.21, nvidia-nccl-cu12-2.29.3, and so on) lands instead. import dfloat11 under this cu12.6 stack fails at the exact same line as under the resolver-default cu13 stack, with the exact same error: cudaErrorNoDevice: no CUDA-capable device is detected at dfloat11.py:40. That is expected and not a regression: cupy's device probe fails on any CPU-only host regardless of which CUDA version torch was built against, so this host cannot distinguish a working cu12.6 kernel from a broken one by import alone. What the pinned install does establish, first-hand, is that a CUDA 12.x torch build inside upstream's supported range is a real, obtainable wheel rather than a hypothetical one to "pick deliberately." Whether the DF11 PTX kernel actually decodes correctly under it still needs a real GPU, and that step remains genuinely unexecuted here.

How to develop with it: compressing your own checkpoint¶

compress_model compresses a BF16 checkpoint. Compression is CPU-only Huffman work; the GPU is touched only by the optional correctness check, so a small accelerator suffices. It cannot be no accelerator: as the executed boundary above shows, import dfloat11 dies with cudaErrorNoDevice on a GPU-less machine, so compress_model is unreachable without a CUDA device even though the compression math never uses one.

# Reference template from examples/compress_flux1/. Not executed on a GPU here;
# the pinned packages install cleanly with pip (verified 2026-07-17), but
# `import dfloat11` itself does NOT import cleanly on this CPU-only host --
# it dies with cudaErrorNoDevice, per "The install boundary" above. This
# script is therefore unreachable end to end without a CUDA device, exactly
# like compress_model() itself, even though nothing below this comment
# actually needs a GPU to run.
pip install "diffusers==0.39.0" "dfloat11[cuda12]==0.5.0" "setuptools==80.9.0"
python compress_flux.py \
    --model_name_or_path black-forest-labs/FLUX.1-dev \
    --save_path ./FLUX.1-dev-DF11 \
    --save_single_file \
    --check_correctness

# dfloat11/dfloat11.py:496 -- the real signature.
compress_model(model, pattern_dict: dict[str, list[str]], save_path: str,
               block_range: list[int] = [0, 10000],
               save_single_file: bool = True, check_correctness: bool = True)

The wall here is pattern_dict: a hand-written map from a regex over module paths to the list of linear submodules inside each block, and you must write one per architecture. The shipped example only covers FLUX; users report spending days adapting it to T5-XXL and Gemma-2.¹² Always leave check_correctness=True, which is the step that verifies the decompressed weights match the originals. compress_model asserts the input dtype is torch.bfloat16 and hard-fails on anything else.

Budget the time. The paper reports single-CPU-thread compression per transformer block of 191 s for Llama 3.1 8B, 547 s for Llama 3.3 70B, and 2133 s for Llama 3.1 405B.¹ Blocks are independent, so this parallelises across cores almost perfectly, and the repo ships a taskset-based parallel script. Compression is one-time; the artifact is then reusable forever.

How to run it in production¶

Size the memory honestly. Compressed weights are not the whole story: the kernel must materialise one transformer block's weights in BF16 before it can run the block's matmuls, and that buffer is real. The paper never quantifies it, which is a genuine gap; the only proxy it offers is Stable Diffusion 3.5, where DF11 peak memory is 11.78 GB against 11.33 GB of compressed weights.¹ So plan for compressed weights plus roughly one block of BF16 plus KV cache, and treat the 32% saving as an upper bound on what you actually get back.

Push batch size up. The decompression cost is constant per forward pass and independent of batch size,³ so it is pure overhead at batch 1 and amortises as the batch grows. That single fact should drive the whole deployment: DF11 is a poor fit for latency-sensitive, low-concurrency serving and a much better fit for throughput-oriented batch work. It is a worse fit than the numbers first suggest for the interactive case, because decode is exactly where batches are smallest.

Do not expect the rest of the serving stack to help. DF11 is not integrated into vLLM, SGLang, TensorRT-LLM, llama.cpp, or transformers proper; the vLLM feature request was closed as not_planned by a stale bot with no human reply.¹⁶ You get Hugging Face generate(), which means you also give up continuous batching, paged attention, and prefix caching. Weigh that against the memory saved, because a serving engine with paged KV may well recover more memory than DF11 does, and go faster.

Checksum every artifact. The corruption test above is not hypothetical: a single bit flip in the encoded stream silently corrupts most of a tensor and raises nothing.

Health gate before traffic (reference template, not executed)¶

Losslessness turns the usual fuzzy canary into a binary one. Record the uncompressed BF16 model's greedy output for a fixed prompt once, on the exact stack the deployment will run (same GPU model, driver, torch, transformers and batch shape, because BF16 matmul kernels are not bit-stable across hardware or library versions; DF11 guarantees identical weights, and that only propagates to identical outputs when everything else is held fixed). After that, the DF11 service must reproduce those token IDs exactly before it takes traffic. Identity is the pass condition and there is no tolerance band: any mismatch means a corrupted artifact, a broken fork, or stack drift, and the gate fails immediately. Greedy decoding only; sampling makes identity meaningless. Pair it with a latency assertion against your deployment's own SLO, taken from config rather than from this page, because DF11 is slower than BF16 by construction (1.4x to 4.4x in Appendix G) and a correctness-only gate will happily pass a deployment that is far outside its budget.

# df11_health_gate.py -- reference template, NOT executed here (needs a CUDA GPU).
import json
import time
from pathlib import Path

import torch

PROBE_PROMPT = "Fixed health-probe prompt. Never change it between runs."
MAX_NEW_TOKENS = 64


def generate_greedy(model, tokenizer) -> list[int]:
    inputs = tokenizer(PROBE_PROMPT, return_tensors="pt").to(model.device)
    with torch.no_grad():
        out = model.generate(**inputs, max_new_tokens=MAX_NEW_TOKENS, do_sample=False)
    return out[0].tolist()


def record_bf16_reference(bf16_model, tokenizer, path: Path) -> None:
    """Run ONCE against the uncompressed model on the production stack."""
    path.write_text(json.dumps({"token_ids": generate_greedy(bf16_model, tokenizer)}))


def health_gate(df11_model, tokenizer, reference_path: Path, token_slo_s: float) -> None:
    """Fail fast: exact token identity, then this deployment's own latency SLO."""
    reference: list[int] = json.loads(reference_path.read_text())["token_ids"]
    t0 = time.monotonic()
    produced = generate_greedy(df11_model, tokenizer)
    elapsed = time.monotonic() - t0
    assert produced == reference, (
        "DF11 output diverged from the BF16 reference. The format is lossless, "
        "so ANY mismatch means a corrupt artifact or stack drift. Do not serve.")
    per_token = elapsed / MAX_NEW_TOKENS
    assert per_token <= token_slo_s, (
        f"{per_token:.3f}s/token exceeds this deployment's SLO of {token_slo_s}s/token")

Rollback artifact (reference template, not executed)¶

Because DF11 is derived losslessly from the BF16 checkpoint, the BF16 original is the rollback target and flipping back reopens no accuracy questions. Pin both artifacts by Hugging Face revision hash and make the switch one config value. One executed caveat shapes the template: DFloat11Model.from_pretrained at 0.5.0 does not accept a revision argument (its **kwargs go to AutoModelForCausalLM.from_config, and its hub fallback calls snapshot_download with no revision), so the only way to pin the DF11 artifact is to pre-download the exact revision to a local directory and pass the path; the loader uses a local path directly when it exists. That signature fact was read from the installed 0.5.0 source on this host; the loading itself is not executed here.

# serving.yaml -- reference template, not executed here.
model_mode: df11        # the ONLY value that changes on rollback: df11 | bf16
df11:
  local_dir: /models/Qwen3-8B-DF11        # hf download DFloat11/Qwen3-8B-DF11 \
  revision: "<exact HF commit hash>"      #   --revision <hash> --local-dir <dir>
bf16:
  local_dir: /models/Qwen3-8B-bf16        # hf download Qwen/Qwen3-8B \
  revision: "<exact HF commit hash>"      #   --revision <hash> --local-dir <dir>

# loader.py -- reference template, not executed here.
import torch


def load(cfg: dict):
    entry = cfg[cfg["model_mode"]]
    if cfg["model_mode"] == "bf16":
        from transformers import AutoModelForCausalLM
        return AutoModelForCausalLM.from_pretrained(
            entry["local_dir"], torch_dtype=torch.bfloat16, device_map="auto")
    from dfloat11 import DFloat11Model
    return DFloat11Model.from_pretrained(entry["local_dir"], device_map="auto")

Two operational conditions make the switch real rather than decorative. Pre-download both snapshots to local disk so a rollback is a process restart, not a multi-hundred-gigabyte download during an incident. And confirm the rollback host actually fits the BF16 model: if you chose DF11 because the model did not fit, the same GPUs cannot serve the uncompressed checkpoint, and the honest rollback plan names a larger host shape or accepts CPU-offload latency, in writing, before the incident.

How to maintain it¶

Watch the upstream, because it is close to dormant. LeanModels/DFloat11 last saw a commit on 2025-11-24, the PyPI package has sat at 0.5.0 since 2025-08-24, and the Hugging Face org has published nothing since 2025-09-30; all three were re-verified on 2026-07-17 and none had moved.⁴⁶ Twenty-two issues are open, including the CUDA 12.0/12.1 breakage and CUDA 13 support; the CUDA 13 request has sat without a maintainer reply since 2026-05-05, and the newest activity anywhere in the tracker is a user comment from 2026-06-08.⁴ Asked about serving frameworks in April 2025, the maintainer said DF11 "can definitely be made compatible" but that "it would require some work to adapt the model format and modify the code." That was a feasibility statement rather than a commitment, and nothing has shipped since.⁸

The live ecosystem has moved to a community fork, mingyi456/ComfyUI-DFloat11-Extended, which covers models the official org never did and adds LoRA support for a few architectures.¹⁷ Treat it with care: its own README reports that FLUX.2-klein outputs are "slightly different from BF16 output," which means losslessness, the entire reason to choose DF11, is currently broken there. If you use a fork, verify bit-exactness yourself on your model before trusting it.

Watch the wider landscape too, not just this one repository. DFloat11 sits in a small family of lossless exponent or sign entropy coders for BF16 weights. ZipNN¹⁹ is an earlier lossless AI-model compressor in the same size range, claiming 33% to over 50% savings on popular checkpoints. ZipServ²⁰ is a more recent, architecturally different design: rather than DF11's decompress-a-whole-block-then-matmul split, it fuses a fixed-length bitmap code into the GEMM kernel itself, decompressing weights directly into Tensor Core registers. ZipServ's own abstract claims up to 2.21x kernel-level speedup over cuBLAS and a 1.22x average end-to-end speedup over vLLM alongside up to 30% smaller weights, which, if it holds up under independent scrutiny, would be a fused lossless design that actually beats BF16 rather than trailing it the way DF11 does; this page has not independently verified those numbers beyond the abstract.

A third, much smaller project worth knowing about, and reading skeptically, is brianbell-x/weight-compression,²¹ a 47-star, single-human-authored repository (one contributor by commit count; a second listed GitHub contributor is Anthropic's Claude account, credited only via a co-authored-by trailer on one commit) (first commit 2026-07-01) that ran two lossless BF16 codecs against GLM-5.2, a real 753B-parameter model on Hugging Face.²² Its byte-split codec, a raw sign-and-exponent high byte reconstructed from a codebook plus an escape stream, with the mantissa low byte kept verbatim, is real and reproducible: the author's verify.py script streams all 282 shards of the checkpoint from Hugging Face, and the page reports a bit-exact match on all 59,509 tensors at 24.967% smaller than BF16 (12.005 bits per weight). Two other numbers on the same page need their caveats attached whenever cited. The headline 30.168% "K15" ratio (11.173 bits per weight, a 4-bit code over a 15-entry joint sign-and-exponent table) is, in the author's own words, "separately charged accounting" that "was not independently decoded at GLM scale": no working decoder has actually reconstructed the 753B model from it. And the GPU number, a dense 12-bit prototype kernel clocked at "0.733 times BF16 GEMV time" on an A40, that is, faster than BF16, is explicitly not fused with the sparse escape-stream correction the format needs to stay lossless (the author's own words: "not fused into or included in that timing"), and it has not been validated at GLM scale or end to end. Do not read that number as evidence that lossless compression has beaten BF16 throughput: DFloat11's own decompress-then-matmul design remains 1.4x to 4.4x slower under the comparable, lossless, end-to-end conditions of Appendix G above, and an unfused, dense-only, single-kernel microbenchmark on a narrower path is not a like-for-like rebuttal of that result.

Re-verify losslessness after any upgrade. It is cheap and it is the one property you are paying for: decompress, compare weight tensors bit-for-bit against the BF16 original, and run a fixed-seed generation against both.

Failure modes¶

Symptom	Cause	Response
Throughput 1.4x to 4.4x worse than BF16	The model fits in BF16, so you are paying decompression for nothing	Do not use DF11 when the model fits. It is a capacity tool, not a speed tool¹³
Slow at batch 1, fine at batch 128	Decompression cost is constant per forward pass	Raise batch size, or accept it. Do not use DF11 for low-latency single-stream decode³
`CUDA_ERROR_UNSUPPORTED_PTX_VERSION`	Driver older than CUDA 12.2; the shipped PTX is ISA 8.2	Upgrade to CUDA 12.2 or newer. CUDA 13 is not supported upstream¹¹¹⁴
`ModuleNotFoundError: No module named 'pkg_resources'` at import	Current setuptools (83.0.0 resolved here) no longer ships `pkg_resources`; `dfloat11/dfloat11.py:19` still imports it	Pin `setuptools==80.9.0` next to the package. Executed on this page, 2026-07-17
`ModuleNotFoundError: No module named 'cupy'` at import	cupy is only installed via the `[cuda11]`/`[cuda12]` extra, which is mandatory for import	Install `dfloat11[cuda12]==0.5.0`, never the bare package. Executed on this page
`cudaErrorNoDevice` at `import dfloat11`	The module loads its PTX kernel at import time; there is no CPU mode, not even for `compress_model`	Use a host with a CUDA device. Executed on this page (CPU-only host), for both the cu13 and cu126 torch stacks
Resolver silently installs a CUDA 13 torch stack	`dfloat11` does not pin `torch`; the default PyPI wheel for `torch` 2.13.0 bundles CUDA 13, which upstream's tracker says is unsupported	Pin a cu12x wheel explicitly, e.g. `torch==2.13.0+cu126` from `https://download.pytorch.org/whl/cu126`, before installing `dfloat11`. Install verified on this page; GPU decode under it is not¹⁴¹⁵
LoRA adapter fails to load, "no weight attribute"	DF11 deletes `module.weight` and rebinds it per forward	Not supported. Fine-tune in BF16, then compress the result⁹¹³
Compressing a quantized checkpoint saves almost nothing	The exponent redundancy DF11 harvests is already gone	Do not stack DF11 on INT8/INT4. Pick one¹⁰
Compression fails on a new architecture	`pattern_dict` is hand-written per model family	Write the regex map for your architecture; expect a slow first attempt¹²
Outputs differ from BF16	Losslessness is broken, not "approximate"	Stop. Verify bit-exactness. A community fork ships this bug today¹⁷
Model silently degrades after a copy	One flipped bit desyncs the Huffman stream and rewrites most of a tensor	Checksum artifacts on download and after every copy
Memory saving smaller than 32%	The decompression buffer holds a block of BF16 weights	Budget compressed weights + one block of BF16 + KV cache¹

References¶

DFloat11 paper (v3, NeurIPS 2025): 70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float. https://arxiv.org/abs/2504.11651
DFloat11 paper v1 (carries the Gemma rows and kernel details dropped from v3). https://arxiv.org/abs/2504.11651v1
Upstream repository, LeanModels/DFloat11 (Apache-2.0). https://github.com/LeanModels/DFloat11
PyPI package dfloat11 (latest 0.5.0, released 2025-08-24; re-verified 2026-07-17). https://pypi.org/project/dfloat11/
Pre-compressed models, Hugging Face DFloat11 org. https://huggingface.co/DFloat11
Community fork with wider model coverage, mingyi456/ComfyUI-DFloat11-Extended. https://github.com/mingyi456/ComfyUI-DFloat11-Extended
NVIDIA Hopper Tuning Guide (shared memory per SM). https://docs.nvidia.com/cuda/hopper-tuning-guide/index.html
NeuZip: Memory-Efficient Training and Inference with Dynamic Compression of Neural Networks, a related weight-compression scheme "based on the entropy of floating-point numbers", with lossless and near-lossless modes. DFloat11 discusses it as related work and benchmarks against the nvCOMP ANS library. https://arxiv.org/abs/2410.20650

Zhang, Hariri, Zhong, Chaudhary, Sui, Hu, Shrivastava, 70% Size, 100% Accuracy (arXiv 2504.11651v3, 1 Jan 2026; NeurIPS 2025). Entropy of the BF16 fields and the ~40 occupied exponent values: §2.2 and Figure 1. Compression table: Table 1. Hierarchical LUTs, the 24-32 bit maximum code length, and the 2304-byte SRAM budget: §2.3.1. Two-phase kernel, Gaps, and BlockOutputPos: §2.3.2 and Algorithm 1. Block-level decompression and per-forward-pass discard: §2.3.3. CPU-offloading comparison (2.31-46.24x): Figure 4. Generation length (5.70-14.86x): Figure 5. BF16-fits throughput comparison: Appendix G, Figure 10; the slowdown ratios in this page's table are computed from those throughputs. Diffusion latency and memory: Table 3. Compression time per block: Appendix F, Table 5. INT8 accuracy drop: Appendix H. Hardware: Appendix E, Table 4 (A5000, A100-40GB, Quadro RTX 8000; no H100 and no 8x80GB node appears in any benchmarked experiment, so the 405B claim is arithmetic, 551.22 GB against 640 GB, and is not benchmarked in the paper. The word "H100" does appear once in the paper's introduction, as an illustrative example of a large-server capacity ("DGX A100/H100 with 8x80GB GPUs"), not tied to any measurement). ↩↩↩↩↩↩↩↩↩↩↩↩↩↩↩↩↩↩↩↩↩
arXiv 2504.11651v1 (15 Apr 2025), Table 2: gemma-2-9b-it 20.32 GB to 14.59 GB, 71.81%, 11.49 bits; gemma-3-12b-it 71.27%; gemma-3-27b-it 70.28%. v1 §3.3.1 also documents the maximum code length being forced to 32 by rebuilding the tree, and a LUT-pointer ambiguity case, both absent from v3. ↩↩
LeanModels/DFloat11 README, master branch, retrieved 2026-07-13. It states that decompression overhead is "constant" per forward pass and "independent of batch size" (the original sets those two spans in bold, dropped here), and, verbatim: "At batch size = 1, inference is approximately 2× slower than the original BF16 model, but the performance gap narrows significantly with larger batches." https://github.com/LeanModels/DFloat11 ↩↩↩↩
gh api repos/LeanModels/DFloat11, first retrieved 2026-07-13, re-verified 2026-07-17: Apache-2.0, 642 stars, default branch master, last push 2025-11-24T09:46:56Z (commit 4577338, "Add CUDA kernel"), 22 open issues, not archived. Issue tracker sorted by recent activity, 2026-07-17: the newest event is a comment on closed issue #12 (2026-06-08); the CUDA 13 request #35 was last touched 2026-05-05 and remains open. PyPI dfloat11 0.5.0, released 2025-08-24, is still the latest. ↩↩
PyPI JSON API for dfloat11, https://pypi.org/pypi/dfloat11/json, retrieved 2026-07-17. Releases: 0.1.0 (2025-04-17), 0.2.0 (2025-05-06), 0.3.0 (2025-07-29), 0.3.1 (2025-07-31), 0.3.2 (2025-08-07), 0.5.0 (2025-08-24); there is no 0.4.x. requires_dist: accelerate, dahuffman==0.4.2, huggingface-hub, safetensors, transformers, tqdm, plus extras cuda11 (cupy-cuda11x) and cuda12 (cupy-cuda12x). requires_python: >=3.9. torch is absent from the metadata. ↩
https://huggingface.co/api/models?author=DFloat11, retrieved 2026-07-13: 45 models, newest Qwen-Image-Edit-2509-DF11 last modified 2025-09-30. Sizes summed from repo blobs. ↩↩
Issue #7, Qwen2.5-14B on an A40: 31.91 tok/s BF16 against 9.51 tok/s DF11; maintainer confirms the decompression overhead is expected. https://github.com/LeanModels/DFloat11/issues/7 ↩
Issue #8, framework integration, open since April 2025. https://github.com/LeanModels/DFloat11/issues/8 ↩
Issue #10, maintainer: "LoRA fine-tuning is not currently supported ... the full weight matrices are deleted." https://github.com/LeanModels/DFloat11/issues/10 ↩
Issue #15, maintainer on quantized models: "the gain is often minimal for quantized models ... an INT8 model might already use more than 7 bits of entropy." https://github.com/LeanModels/DFloat11/issues/15 ↩↩↩
Issue #19, CUDA_ERROR_UNSUPPORTED_PTX_VERSION on CUDA 12.0/12.1. https://github.com/LeanModels/DFloat11/issues/19 ↩↩
Issue #28, adapting pattern_dict to new architectures. https://github.com/LeanModels/DFloat11/issues/28 ↩↩
Issue #31, delattr(module, 'weight') breaks LoRA loading. https://github.com/LeanModels/DFloat11/issues/31 ↩
Issue #35, CUDA 13 unsupported. https://github.com/LeanModels/DFloat11/issues/35 ↩↩↩↩
PyTorch wheel index for torch, https://download.pytorch.org/whl/torch/, retrieved 2026-07-17: for cp312/manylinux, torch 2.13.0 publishes cu126, cu129 and cu130 builds (no cu127, cu128, or a cu13x-named build under the +cuXXX local-version scheme). pip install "torch==2.13.0+cu126" --index-url https://download.pytorch.org/whl/cu126 installed cleanly on this page's CPU-only host, 2026-07-17, and left torch.version.cuda at 12.6; import dfloat11 on top of it still fails with cudaErrorNoDevice, identically to the resolver-default cu13 stack, because this host has no GPU. ↩↩
vLLM issue #20003, "[Feature]: support dfloat11", opened 2025-06-24, closed 2025-10-23 as not_planned by the stale bot with no human reply. https://github.com/vllm-project/vllm/issues/20003 ↩
mingyi456/ComfyUI-DFloat11-Extended README, retrieved 2026-07-13, on FLUX.2-klein: "there is currently an unexplainable bug causing outputs to be slightly different from BF16 output." https://github.com/mingyi456/ComfyUI-DFloat11-Extended ↩↩
NVIDIA Hopper Tuning Guide: 228 KB of shared memory per SM on compute capability 9.0, with a 227 KB per-thread-block ceiling. https://docs.nvidia.com/cuda/hopper-tuning-guide/index.html ↩
Hershcovitch, Wood, Choshen, Girmonsky, Leibovitz, Ennmouri, Malka, Chin, Sundararaman, Harnik, ZipNN: Lossless Compression for AI Models (arXiv 2411.05239, v1 7 Nov 2024, v2 4 Jun 2025). Abstract: "often saving 33% and at times reducing over 50% of the model size." https://arxiv.org/abs/2411.05239 ↩
Fan, Yu, Pan, Li, Luo, Wang, Wang, Chu, ZipServ: Fast and Memory-Efficient LLM Inference with Hardware-Aware Lossless Compression (arXiv 2603.17435, v1 18 Mar 2026). Abstract: "reduces the model size by up to 30%, achieves up to 2.21x kernel-level speedup over NVIDIA's cuBLAS, and expedites end-to-end inference by an average of 1.22x over vLLM." Not independently reproduced on this page. https://arxiv.org/abs/2603.17435 ↩
brianbell-x/weight-compression, retrieved 2026-07-14 via gh api: 47 GitHub stars, created 2026-07-01, last push 2026-07-13. Quotes are from the project page, https://brianbell-x.github.io/weight-compression/; source and verify.py at https://github.com/brianbell-x/weight-compression. ↩
zai-org/GLM-5.2 on Hugging Face, retrieved 2026-07-14: 753,329,940,480 parameters per the repo's safetensors metadata. https://huggingface.co/zai-org/GLM-5.2 ↩