Skip to content
Markdown

Covariant obfuscation for private LLM inference

Scope: the pattern where a client transforms an open-weight model offline, ships the transformed weights to an untrusted operator, and then talks to that model in a private token alphabet. This page covers what "covariant" means precisely, the key-matrix construction that hides the secret permutation, which model components commute exactly and which only approximately, the multi-tenancy cost of per-client weights, and where the published privacy numbers hold and where they do not. The approach that keeps the secret in the activations and needs a confidential GPU is activation obfuscation on untrusted accelerators; the hardware-attested alternative is GPU confidential computing; the layer-split alternative is privacy-aware split inference.

Primary source: Yu Lin, Qizhi Zhang, Wenqiang Ruan, Daode Zhang, Jue Hong, Ye Wu (ByteDance) and Hanning Xia, Yunlong Mao, Sheng Zhong (Nanjing University), "Towards Privacy-Preserving LLM Inference via Covariant Obfuscation (Technical Report)", arXiv:2603.01499v2 [cs.CR], 30 March 2026. Technical report, no venue stated, no code released. The system is named AloePri.

What this page adds. Both Python blocks are executed and asserted (Python 3.13, numpy 2.4.6). The first implements the paper's Algorithm 1 key matrices and confirms the feed-forward network commutes exactly, then reproduces from first principles the mechanism behind the paper's own Table 4 ablation: key matrices leave attention scores bit-identical, and only the rotary block permutation moves them. It also quantifies the one component that never commutes exactly. The second bounds the paper's Theorem 4 privacy guarantee using only the definition of its own terms, and prices the per-tenant model copy the design requires. Figures with a table number are the paper's.

The efficiency headline is measured on a small workload. Table 7 uses 17-token average prompts, 100 generated tokens, and request concurrency of 1 and 4. There is no throughput, batch-size, or sustained-load measurement anywhere in the paper.

What it is

A data-only obfuscation scheme perturbs the input and hopes the model still works. A covariant obfuscation transforms the input and the weights so that the obfuscated model computes exactly the obfuscated answer. Formally the paper defines the quintuple (phi_X, phi_Theta, phi_Y, psi_Y, f~): a data map, a weight map, an output map, an output inverse, and inference in the obfuscated space, satisfying a commutation condition f~(phi_X(x), phi_Theta(theta)) approximately equals phi_Y(f(x, theta)) within a stated error, and a de-obfuscation condition psi_Y . phi_Y = id. Data-only obfuscation is the degenerate case where the weight map is the identity.

The concrete system works in two phases.

Offline, on the client. The client holds the open-weight checkpoint. It samples a secret token permutation and applies it to the embedding table and the LM head, which is what makes the private alphabet work. On its own that permutation is trivially recoverable by comparing public and published weights, so it is buried under a stack of further transforms: Gaussian noise added to the embedding and head, and a set of key matrices applied to every weight matrix in the model. The key matrices come in pairs P of shape d x (d+2h) and Q of shape (d+2h) x d with P Q = I. Adjacent layers get matched pairs, so each transform is cancelled by the next layer's inverse as activations flow through the network. Attention picks up rotary-aware rotations and scalings, plus permutations of attention heads and of the rotary blocks; the FFN picks up a diagonal scaling and a permutation chosen so SiLU and the Hadamard product pass through; a mixture-of-experts router picks up an expert permutation.

Online. The client tokenizes the prompt locally, maps each token through the secret vocabulary mapping, detokenizes, and sends what looks like ordinary text. The server tokenizes it and runs a completely standard inference stack. The response comes back in the private alphabet and the client maps it home.

The property that matters for infrastructure is what is absent. There is no cryptographic protocol at inference time, no TEE, no per-token round trip, no engine modification. The server runs vLLM on weights that have the same shapes and the same dtypes as any other checkpoint, one dimension wider.

Why use it

The paper's framing is three industrial constraints, and they are the right ones for a serving fleet.

Hardware compatibility. Confidential computing exists on recent NVIDIA parts and nowhere else in a typical fleet. A cluster of mixed accelerators, older GPUs, NPUs, or FPGAs cannot run a TEE-anchored scheme at all. A weight transformation runs anywhere a matrix multiply runs.

Software compatibility. Paged attention, prefix caching, prefill and decode disaggregation, continuous batching, and speculative decoding are all engineering that a privacy scheme built on a custom protocol has to reimplement. Because the obfuscated model is structurally a normal model, the paper reports deploying it under vLLM 0.9.1 with no engine changes.

Accuracy and efficiency. Table 2 on Qwen2.5-14B-Instruct puts plaintext at 97.25 SST2, 81.95 MMLU, 77.45 PIQA and 77.09 IFEval, against 97.13, 80.61, 75.23 and 79.49 obfuscated. The obfuscation-based baselines it compares against are far behind: RANTEXT drops PIQA from 77.45 to 18.30, and SGT drops MMLU from 81.95 to 30.30 while still leaking over 90% of tokens to an inversion-model attack. Against those baselines the result is not close.

The scale of the demonstration is also unusual for this literature. Most privacy-preserving inference papers stop at a 7B model; this one obfuscates DeepSeek-V3.1-Terminus at 671B parameters and serves it.

When to use it (and when not)

The decisive cost is that every client needs its own copy of the model. The client generates the secret and performs the obfuscation, so the weights deployed on the server are specific to that client. Two tenants cannot share a set of weights, because sharing them would mean sharing the secret. Everything that makes multi-tenant serving economic depends on the opposite: one resident copy of the weights amortised over many tenants. The paper never states this consequence, and it is the first thing to price. For DeepSeek-V3.1-Terminus at 671B, ten tenants means ten obfuscated checkpoints, roughly 6.7 TB at one byte per parameter, and 80 hours of offline obfuscation. The executed block below tabulates it.

Privacy is not uniform across models, and the abstract quotes the best row. The abstract says attackers recover "less than 5% of tokens", and the conclusion repeats it for DeepSeek-V3.1-Terminus (4.80%). Table 3 also reports 25.05% for Qwen3-14B and 19.64% for Qwen3-32B under the same vocabulary-matching attack.1 A quarter of tokens recovered is a different security posture from one in twenty. Run the attack against your own model rather than inheriting a number.

The formal guarantee does not carry the weight it appears to. Theorem 4 bounds a Renyi-metric differential privacy budget, and Remark 1 concludes AloePri's budget is strictly smaller than that of the token-perturbation mechanism it composes with. Both statements are true. The executed audit below shows the reduction is smaller than one part in a billion for any usable budget, and that the token-perturbation mechanism is optional and not used in the experiments that produce the accuracy numbers. The honest description of the security is the paper's own remark, that it "aims to provide adjustable security for constrained attackers in real-world scenarios, rather than offering ideal-world security guarantees against worst-case attackers."

Do not read "identical efficiency" as a throughput result. Table 7's largest measurement is concurrency 4 on 17-token prompts. At concurrency 1 the obfuscated model posts a lower time per output token than plaintext (6.72 ms against 6.93 ms), which cannot be a real effect given it does strictly more work on a wider residual stream, so the measurement noise exceeds the quantity being measured. The genuine signal is in the 671B rows at concurrency 4: TTFT 185.11 ms to 199.97 ms and TPOT 21.42 ms to 22.41 ms, so roughly 5 to 8%.

Do not use it where the client cannot hold the plaintext model. The client must download the full checkpoint, obfuscate it locally, and upload the result. The paper's client is a CPU-only host with two Xeon 8457C processors. That is a heavy client for what is sold as a lightweight scheme, and it inverts the usual reason for using a hosted service.

Do not use it on an architecture without rotary embeddings without re-testing. The executed block below shows that key matrices, rotations, and scalings all cancel inside the attention score, leaving it bit-identical to plaintext. Only the rotary block permutation perturbs the score, and it does so only because rotary embeddings tie a fixed frequency to each position in the head vector. This is precisely why the paper's own Table 4 shows attention-score recovery stuck at 87.14% until head and block permutations are added.

Architecture

flowchart TB
  subgraph CLIENT["Client, offline (CPU host)"]
    OPEN["Open-weight checkpoint"] --> PERM["Secret token permutation tau<br/>applied to embedding + LM head"]
    PERM --> HIDE["Hide tau: Gaussian noise,<br/>key matrices P/Q with P Q = I,<br/>RoPE rotations, head + block perms"]
    HIDE --> OBF["Obfuscated checkpoint<br/>residual stream d -> d + 2h"]
  end
  subgraph SERVER["Untrusted operator"]
    ENGINE["Stock vLLM / SGLang<br/>no engine changes"]
  end
  subgraph ONLINE["Online, per request"]
    MAP["Client maps tokens<br/>through secret vocabulary"]
    BACK["Client maps response back"]
  end
  OBF -->|"one-time upload, per tenant"| ENGINE
  MAP -->|"obfuscated text"| ENGINE
  ENGINE -->|"obfuscated text"| BACK

The arrow to watch is the one from the client's obfuscated checkpoint to the server. It is per tenant, it carries the whole model, and it is what buys the absence of any protocol on the request path.

How to use it

The paper's defaults are matrix coefficient lambda = 0.3, expansion size h = 128, embedding noise alpha_e = 1.0, head noise alpha_h = 0.2, and rotary block parameters beta = 8, gamma = 1e3. Three of those have measured cliffs on either side.

Embedding noise is the one that fails open. At alpha_e = 0.5 the vocabulary-matching attack recovers over 30% of tokens, because with small noise the plaintext and obfuscated weight pairs differ almost only by row and column permutations, and sorting then matching recovers the secret permutation directly. The noise is not a refinement; it is what makes the permutation unrecoverable.

The matrix coefficient fails closed, and it fails in bfloat16 first. At lambda = 3.0 accuracy collapses under bfloat16 while float32 survives, because larger lambda widens the distribution of internal states until they overflow the format's range. The failure is numeric, not cryptographic, and it will present as a quality regression rather than as an error.

The expansion size is a straightforward compute tax. It widens the residual stream from d to d + 2h, which at the default is 5% on a 14B and 3.6% on DeepSeek-V3.1. Raising it to 512 costs under 10% additional time per output token, per Figure 5.

Validate on the model you intend to ship. Table 4's ablation is the template for how to do this: obfuscate incrementally, and run an internal-state inversion attack against attention scores and hidden states separately after each step. Noise alone leaves attention scores at 87.14% recovery and hidden states at 40.0%. Adding key matrices takes hidden states to 0.82% and leaves attention scores untouched at 87.14%. Only head and block permutation takes both to 0.0%.

How to develop with it

The mechanism is short enough to build and check directly. What follows constructs the paper's Algorithm 1 key matrices, verifies the FFN commutes, and then isolates why the attention score needs its own defence.

# Runnable on system python3 (numpy only). Covariant obfuscation (AloePri,
# arXiv:2603.01499v2 Sec 5.2): transform weights and data together so the obfuscated
# model computes the obfuscated answer exactly. We build the rectangular key matrices
# of Algorithm 1, assert the FFN commutes exactly, and then reproduce the mechanism
# behind the paper's own Table 4 ablation: key matrices leave attention scores
# untouched, and only the RoPE block permutation moves them.

import numpy as np

rng = np.random.default_rng(7)
d, h, lam = 64, 16, 0.3  # hidden size, expansion size, matrix coefficient


def haar(m, rng):
    Q, R = np.linalg.qr(rng.standard_normal((m, m)))
    return Q * np.sign(np.diag(R))


def key_matrices(d, h, lam, rng):
    """Algorithm 1. P is d x (d+2h) and Q is (d+2h) x d with P @ Q == I_d.
    The cancellation needs C F == 0 and E D == 0, so C's rows live in the left
    null space of F and D's columns live in the null space of E. Both null spaces
    are non-trivial because E and F are built through a width-h/2 bottleneck."""
    B = haar(d, rng) + lam * rng.standard_normal((d, d)) / np.sqrt(d)
    E = (rng.standard_normal((d, h // 2)) @ rng.standard_normal((h // 2, h))) / np.sqrt(d)
    F = (rng.standard_normal((h, h // 2)) @ rng.standard_normal((h // 2, d))) / np.sqrt(d)
    Z = haar(d + 2 * h, rng)

    def null(M):  # orthonormal basis of the null space of M
        _, s, Vt = np.linalg.svd(M)
        return Vt[np.sum(s > 1e-10 * max(M.shape) * s.max()):].T

    nF, nE = null(F.T).T, null(E)  # rows for C, columns for D
    C = rng.standard_normal((d, nF.shape[0])) @ nF
    D = nE @ rng.standard_normal((nE.shape[1], d))
    P = np.hstack([B, C, E]) @ Z
    Q = Z.T @ np.vstack([np.linalg.inv(B), F, D])
    return P, Q


P, Q = key_matrices(d, h, lam, rng)
assert P.shape == (d, d + 2 * h) and Q.shape == (d + 2 * h, d)
assert np.allclose(P @ Q, np.eye(d), atol=1e-9), "P Q must be the identity on R^d"
# The reverse product is deliberately not the identity: Q P is a rank-d projector
# inside the widened space, which is what hides the secret permutation.
assert not np.allclose(Q @ P, np.eye(d + 2 * h), atol=1e-3)
print(f"1. key matrices  P {P.shape} Q {Q.shape}  "
      f"||PQ - I|| = {np.linalg.norm(P @ Q - np.eye(d)):.2e}   "
      f"rank(QP) = {np.linalg.matrix_rank(Q @ P)} of {d + 2*h}")

# 2. FFN covariance. Weights absorb the transforms; the output is the plaintext
# output right-multiplied by the next key matrix. Z is a permutation and H a
# diagonal scaling, both chosen so SiLU and the Hadamard product pass through.
silu = lambda x: x / (1.0 + np.exp(-x))
d_ffn = 128
Wg, Wu = rng.standard_normal((d, d_ffn)), rng.standard_normal((d, d_ffn))
Wd = rng.standard_normal((d_ffn, d))
Zf = np.eye(d_ffn)[rng.permutation(d_ffn)]
Hf = np.diag(np.exp(rng.uniform(-0.5, 0.5, d_ffn)))
P_dn, Q_dn = key_matrices(d, h, lam, rng)

Wg_o, Wu_o = Q @ Wg @ Zf, Q @ Wu @ Hf @ Zf
Wd_o = np.linalg.inv(Zf) @ np.linalg.inv(Hf) @ Wd @ P_dn

x = rng.standard_normal((8, d))
plain = (silu(x @ Wg) * (x @ Wu)) @ Wd
obf = (silu((x @ P) @ Wg_o) * ((x @ P) @ Wu_o)) @ Wd_o
assert np.allclose(obf, plain @ P_dn, atol=1e-8), "FFN must commute exactly"
print(f"2. FFN covariance          max abs err {np.abs(obf - plain @ P_dn).max():.2e}")

# 3. Attention scores. With Q_q/Q_k cancelling the input key matrix, an orthogonal
# rotation R, and a diagonal scaling H applied inversely to q and k, every factor
# cancels inside q k^T. Without RoPE the obfuscated scores are bit-for-bit the
# plaintext scores, so an attacker reading attention scores learns everything.
d_head, n_tok = 32, 12
Wq, Wk = rng.standard_normal((d, d_head)), rng.standard_normal((d, d_head))
n_blk = d_head // 2
ang = rng.uniform(0, 2 * np.pi, n_blk)
R = np.zeros((d_head, d_head))
for i, a in enumerate(ang):  # block-diagonal 2x2 rotations
    R[2*i:2*i+2, 2*i:2*i+2] = [[np.cos(a), -np.sin(a)], [np.sin(a), np.cos(a)]]
s = np.exp(rng.uniform(-0.5, 0.5, n_blk))
Hqk = np.diag(np.repeat(s, 2))

perm = rng.permutation(n_blk)
Zb = np.zeros((d_head, d_head))  # permute the 2x2 RoPE blocks
for j, pj in enumerate(perm):
    Zb[2*pj:2*pj+2, 2*j:2*j+2] = np.eye(2)

Wq_o = Q @ Wq @ R @ Hqk @ Zb
Wk_o = Q @ Wk @ R @ np.linalg.inv(Hqk) @ Zb
xs = rng.standard_normal((n_tok, d))
q_p, k_p = xs @ Wq, xs @ Wk
q_o, k_o = (xs @ P) @ Wq_o, (xs @ P) @ Wk_o
assert np.allclose(q_o @ k_o.T, q_p @ k_p.T, atol=1e-7), "scores leak without RoPE perm"
print(f"3. scores, no RoPE         max abs err {np.abs(q_o @ k_o.T - q_p @ k_p.T).max():.2e}"
      "  (identical: fully leaked)")


def rope(v, base=10000.0):
    """Apply RoPE in place over 2-dim blocks; block i gets frequency base^(-2i/d)."""
    pos = np.arange(v.shape[0])[:, None]
    theta = base ** (-2.0 * np.arange(v.shape[1] // 2) / v.shape[1])
    ang, out = pos * theta, np.empty_like(v)
    ev, od = v[:, 0::2], v[:, 1::2]
    out[:, 0::2] = ev * np.cos(ang) - od * np.sin(ang)
    out[:, 1::2] = ev * np.sin(ang) + od * np.cos(ang)
    return out


# 4. RoPE is what makes the block permutation bite. Because RoPE assigns a fixed
# frequency to each position in the head vector, shuffling the blocks changes which
# frequency each pair receives, and the scores move. The permutation is inert
# without RoPE (step 3) and load-bearing with it.
sc_p = rope(q_p) @ rope(k_p).T
sc_o = rope(q_o) @ rope(k_o).T
drift = np.abs(sc_o - sc_p).max() / np.abs(sc_p).max()
assert drift > 0.1, "RoPE plus block permutation must perturb the scores"
print(f"4. scores, RoPE + blockperm relative drift {drift:.2f}  (perturbed: protected)")

# Control: with the identity block permutation the scores return to plaintext even
# under RoPE, confirming the drift comes from the permutation and not from R or H.
Wq_id, Wk_id = Q @ Wq @ R @ Hqk, Q @ Wk @ R @ np.linalg.inv(Hqk)
sc_id = rope((xs @ P) @ Wq_id) @ rope((xs @ P) @ Wk_id).T
assert np.allclose(sc_id, sc_p, atol=1e-6), "R and H alone never move the scores"
print(f"5. control, no blockperm   max abs err {np.abs(sc_id - sc_p).max():.2e}")

# 6. RMSNorm is the one component that does not commute exactly. RMSNorm divides by
# a norm, and ||x P|| is not proportional to ||x|| unless P is orthogonal, so no
# reweighting can fix it per token. Matching c * RMSNorm_{d+2h}(x P) to RMSNorm_d(x) P
# gives c = r * sqrt(d / (d+2h)) with r = ||x P|| / ||x||, and the best a constant can
# do is use E[r]. The residual is then set by how much r varies from token to token,
# which is exactly the obfuscation error the paper carries through its composition
# theorems.
rms = lambda v: v / np.sqrt((v ** 2).mean(axis=-1, keepdims=True))
big = rng.standard_normal((4096, d))
r = np.linalg.norm(big @ P, axis=1) / np.linalg.norm(big, axis=1)
kappa = r.mean() * np.sqrt(d / (d + 2 * h))
target = rms(x) @ P
naive = np.linalg.norm(rms(x @ P) - target) / np.linalg.norm(target)
fixed = np.linalg.norm(kappa * rms(x @ P) - target) / np.linalg.norm(target)
cv = r.std() / r.mean()
print(f"6. RMSNorm  kappa {kappa:.4f}  spread of ||xP||/||x|| {100*cv:.1f}%")
print(f"   uncorrected rel err {naive:.4f}   corrected {fixed:.4f}")
assert fixed < 0.25 * naive, "the kappa correction must remove most of the mismatch"
# The residual is not arbitrary: it is set by the per-token spread of r, so it should
# land within a small factor of that spread rather than at some tuned value.
assert 0.5 * cv < fixed < 2.0 * cv, "residual must track the spread of ||xP||/||x||"

# The residual tracks the spread of r, not the size of the network: an orthogonal P
# would make r constant and the layer exactly covariant. Confirm with lambda = 0
# and no expansion, where B is orthogonal.
P_ortho = haar(d, rng)
err_o = np.linalg.norm(rms(x @ P_ortho) - rms(x) @ P_ortho) / np.linalg.norm(rms(x) @ P_ortho)
assert err_o < 1e-12, "an orthogonal key matrix makes RMSNorm exactly covariant"
print(f"   orthogonal P control: rel err {err_o:.2e} (exact, but leaks the norm)")
print("all assertions passed")

Executed output:

1. key matrices  P (64, 96) Q (96, 64)  ||PQ - I|| = 6.63e-14   rank(QP) = 64 of 96
2. FFN covariance          max abs err 8.82e-11
3. scores, no RoPE         max abs err 1.49e-11  (identical: fully leaked)
4. scores, RoPE + blockperm relative drift 1.33  (perturbed: protected)
5. control, no blockperm   max abs err 1.22e-11
6. RMSNorm  kappa 2.7202  spread of ||xP||/||x|| 18.2%
   uncorrected rel err 0.6498   corrected 0.1368
   orthogonal P control: rel err 4.10e-16 (exact, but leaks the norm)
all assertions passed

Steps 3 through 5 are the useful part. The attention score is an inner product between the query and key projections of the same input, so any transform applied to both sides cancels: the input key matrix cancels against its inverse, the rotation is orthogonal, and the diagonal scaling appears with opposite exponents. The result is bit-identical to the plaintext score, which is exactly what the paper's Table 4 measures when it reports attention-score recovery unchanged at 87.14% after key matrices are added. What breaks the identity is that rotary embeddings bind a specific frequency to a specific pair of coordinates, so permuting the pairs changes the frequency each one receives, and the control in step 5 confirms nothing else in the construction contributes.

Step 6 isolates the only component that cannot be made exact. RMSNorm divides by a norm, and a non-orthogonal key matrix changes each token's norm by a different factor, so a single rescaling constant can only correct the mean. The residual, 13.7% here against an 18.2% spread in ||xP||/||x||, is the obfuscation error that the paper's composition theorems then propagate through every layer. Choosing an orthogonal key matrix would make the layer exact, and would also make the norm of every hidden state directly observable.

Auditing the privacy bound and the tenancy cost

# Runnable on system python3 (numpy only). Two audits of AloePri
# (arXiv:2603.01499v2): what its Theorem 4 privacy bound is actually worth, and what
# per-client model obfuscation costs a serving fleet.

import numpy as np

# --- Audit 1: Theorem 4 -------------------------------------------------------
# Theorem 4 gives eps = eps1 - eps1^2 / (4 (n-1) eps2) when eps1 <= 2 (n-1) eps2,
# where n is the vocabulary size, eps1 is the budget of an OPTIONAL token-perturbation
# mechanism, and eps2 = pi^2 (eps_e + eps_h) with
#     eps_e = alpha (lam1(We)^2 + lam2(We)^2) / (4 sigma_e^2),  alpha = 2.
# Remark 1 concludes AloePri's budget is "strictly smaller" than eps1. It is. The
# question is by how much, and that is decided by the size of (n-1) eps2.
#
# We do not need the real embedding matrix. sigma_e is the entrywise standard
# deviation of We, so for a mean-zero We, ||We||_F^2 = n d sigma_e^2. At most
# min(n,d) = d singular values are non-zero, so lam1^2 >= ||We||_F^2 / d = n sigma_e^2.
# That is a bound, not an estimate: any real embedding matrix is more concentrated
# than the equality case, making eps_e larger still.

ALPHA = 2.0


def eps2_lower_bound(vocab, d):
    """Lower bound on eps2 from Theorem 4, using only lam1^2 >= n sigma_e^2."""
    eps_e_min = ALPHA * vocab / 4.0  # (lam1^2 + lam2^2)/(4 sigma^2) >= n/4 * alpha
    return np.pi ** 2 * eps_e_min  # eps_h >= 0, so drop it


MODELS = {"Qwen2.5-14B / R1-Distill-14B": (152064, 5120),
          "DeepSeek-V3.1-Terminus": (129280, 7168)}

print("Theorem 4: how much does model obfuscation reduce the token budget eps1?")
print(f"{'model':<30} {'(n-1)*eps2 >=':>16} {'eps1=1':>10} {'eps1=10':>10}")
print("-" * 70)
for name, (vocab, d) in MODELS.items():
    floor = (vocab - 1) * eps2_lower_bound(vocab, d)
    red = [e / (4.0 * floor) for e in (1.0, 10.0)]  # relative reduction eps1/(4(n-1)eps2)
    print(f"{name:<30} {floor:>16.3e} {red[0]:>10.2e} {red[1]:>10.2e}")
    # The regime switch at eps1 > 2(n-1)eps2 is unreachable for any usable budget.
    assert 2 * floor > 1e9, "the piecewise branch never activates in practice"
    assert red[1] < 1e-9, "the improvement over eps1 alone is numerically negligible"

print("\nRemark 1 holds (eps < eps1) but the gap is below one part in a billion,")
print("so the theorem does not make a weak token mechanism meaningfully stronger.")
# And with no token mechanism at all, eps1 is unbounded and eps saturates at the cap,
# which is astronomically large: a formally vacuous guarantee.
vocab, d = MODELS["Qwen2.5-14B / R1-Distill-14B"]
cap = (vocab - 1) * eps2_lower_bound(vocab, d)
print(f"With no token mechanism, eps saturates at (n-1)eps2 >= {cap:.2e} "
      "(no usable guarantee).")
assert cap > 1e10

# --- Audit 2: what per-client obfuscation costs -------------------------------
# The client generates the secret and obfuscates the weights, so each client needs
# its own obfuscated copy of the model deployed on the server. Weight sharing across
# tenants, which is what makes multi-tenant serving economic, is gone.
print("\nPer-tenant cost of client-side model obfuscation")
print(f"{'model':<26} {'params':>8} {'offline':>9} {'10 tenants':>12} {'weights @FP8':>14}")
print("-" * 74)
FLEET = [("R1-Distill-14B", 14e9, 3.43), ("R1-Distill-32B", 32e9, 9.28),
         ("Qwen3-MoE-30B-A3B", 30e9, 4.58), ("DeepSeek-V3.1", 671e9, 482.38)]
for name, params, minutes in FLEET:
    gb = params * 1.0 / 1e9  # 1 byte/param at FP8
    print(f"{name:<26} {params/1e9:>7.0f}B {minutes/60:>8.1f}h "
          f"{10*minutes/60:>11.1f}h {10*gb:>13.0f} GB")

# The expansion parameter widens the residual stream from d to d+2h, which inflates
# every matrix that reads it. This is a real cost the "identical efficiency" claim
# is measured around, at the default h=128.
print(f"\nResidual-stream widening at the paper's defaults (h=128, and h=512):")
for name, (vocab, d) in MODELS.items():
    for h in (128, 512):
        print(f"  {name:<30} h={h:<4} d {d} -> {d+2*h} (+{100*2*h/d:.1f}%)")
assert (5120 + 256) / 5120 < 1.06, "h=128 is a few percent on a 14B"
assert (5120 + 1024) / 5120 > 1.19, "h=512 is a fifth more width"
print("all assertions passed")

Executed output:

Theorem 4: how much does model obfuscation reduce the token budget eps1?
model                             (n-1)*eps2 >=     eps1=1    eps1=10
----------------------------------------------------------------------
Qwen2.5-14B / R1-Distill-14B          1.141e+11   2.19e-12   2.19e-11
DeepSeek-V3.1-Terminus                8.248e+10   3.03e-12   3.03e-11

Remark 1 holds (eps < eps1) but the gap is below one part in a billion,
so the theorem does not make a weak token mechanism meaningfully stronger.
With no token mechanism, eps saturates at (n-1)eps2 >= 1.14e+11 (no usable guarantee).

Per-tenant cost of client-side model obfuscation
model                        params   offline   10 tenants   weights @FP8
--------------------------------------------------------------------------
R1-Distill-14B                  14B      0.1h         0.6h           140 GB
R1-Distill-32B                  32B      0.2h         1.5h           320 GB
Qwen3-MoE-30B-A3B               30B      0.1h         0.8h           300 GB
DeepSeek-V3.1                  671B      8.0h        80.4h          6710 GB

Residual-stream widening at the paper's defaults (h=128, and h=512):
  Qwen2.5-14B / R1-Distill-14B   h=128  d 5120 -> 5376 (+5.0%)
  Qwen2.5-14B / R1-Distill-14B   h=512  d 5120 -> 6144 (+20.0%)
  DeepSeek-V3.1-Terminus         h=128  d 7168 -> 7424 (+3.6%)
  DeepSeek-V3.1-Terminus         h=512  d 7168 -> 8192 (+14.3%)
all assertions passed

The theorem's improvement over the token mechanism alone is real and negligible, because the vocabulary size multiplies the denominator. Since the token mechanism is optional and switching it on is what costs accuracy in the very baselines this work outperforms, the configuration that produced the published accuracy numbers has no meaningful formal budget at all. Treat the empirical attack results, which are extensive, as the actual evidence.

How to run it in production

Plan the fleet around per-tenant weights rather than per-tenant requests. The consequences follow directly: no cross-tenant KV cache or prefix reuse, no shared model server, and a GPU memory budget that scales with tenant count instead of with concurrency. A mixture-of-experts model helps here, since active parameters stay low even as resident parameters multiply.

Treat obfuscation as a build step with a real service-level objective. Eight hours for a 671B model is a deployment window, and a key rotation is a full rebuild plus a full redeploy, not a config change. There is no partial re-obfuscation.

Keep the plaintext and obfuscated checkpoints strictly separated. The vocabulary-matching attack works by comparing published weights against obfuscated ones, so an operator that can obtain both, or a client that leaks its noise parameters, hands over the permutation. Noise coefficients are as secret as the permutation itself.

Instrument accuracy continuously against a plaintext reference. Both the numerical failure mode (large lambda under bfloat16) and the accuracy drift from the RMSNorm residual present as quality regressions with no error signal. A canary evaluation against the unobfuscated model is the only detector.

How to maintain it

Re-obfuscate on every checkpoint change; the transform is bound to specific weights. Re-run the attack suite after any architecture change, particularly to attention: the protection of attention scores depends on rotary block permutation, so a model with different positional encoding needs its own analysis rather than an inherited result. Re-tune alpha_e if the embedding table is resized or retrained, since the attack that it defeats works on the statistics of the weight matrix. Pin the serving engine version alongside the obfuscated checkpoint; the compatibility claim is measured against vLLM 0.9.1 and nothing else.

Failure modes

  • Embedding noise set too low. At alpha_e = 0.5 the permutation is recoverable by sorting and matching, and over 30% of tokens fall out. Fails open, silently.
  • Matrix coefficient set too high under bfloat16. Internal state norms grow until they overflow; accuracy collapses while float32 is fine.
  • Model without rotary embeddings. Attention scores stay bit-identical to plaintext and the internal-state attack recovers them at the unmitigated rate.
  • Tenant count grows. Weights, not requests, drive memory. The failure is a capacity planning surprise rather than an incident.
  • Plaintext checkpoint co-resident with the obfuscated one. Supplies exactly the comparison the vocabulary-matching attack needs.
  • Client-side obfuscation host under-provisioned. The offline step is a dense linear-algebra job over the whole checkpoint on a CPU host.
  • Assuming the token mapping hides structure. The mapping is a permutation of the vocabulary, so token frequency survives it. The paper measures this and finds recovery low, but the distribution-aware attacker is its strongest case: 16.51% top-100 recovery when the attacker knows the client's data distribution.

Open questions and validation

  • No code is released, so the construction can only be checked against the paper's text. The block above is this page's independent reimplementation of Algorithm 1 and the component transforms, not the authors' code.
  • Whether the approach survives a serving-scale workload is untested. The published latency work stops at concurrency 4 with 17-token prompts, and paged attention over a widened residual stream is exactly where a compatibility claim would be stressed.
  • The gap between 4.80% token recovery on DeepSeek-V3.1 and 25.05% on Qwen3-14B is unexplained. Without a mechanism, neither number generalises to an untested model.
  • Model confidentiality runs the wrong way for a hosted service: the client must hold plaintext weights. The paper sketches an FHE or CPU-TEE variant that would let a client obfuscate weights it cannot read, but does not build it.
  • The interaction with quantisation is unaddressed. The key matrices are dense and randomly generated, and the bfloat16 failure at lambda = 3.0 suggests the transformed weights have a wider dynamic range than the originals, which is the thing post-training quantisation is most sensitive to.

References

Related: Activation obfuscation on untrusted accelerators · GPU confidential computing · Privacy-aware split inference · Membership inference against fine-tuned LLMs · Security, isolation and multi-tenancy · KV cache management · Serving OSS models · Disaggregated inference · Gradient leakage in distributed training · Glossary

  1. arXiv:2603.01499v2, Table 3, "AloePri privacy against VMA", text-token recovery success ratio: R1-Distill-14B 12.36%, R1-Distill-32B 10.10%, Qwen3-14B 25.05%, Qwen3-32B 19.64%, Llama3-8B 2.57%, Qwen3-MoE-30B-A3B 5.91%, DeepSeek-V3.1-Terminus 4.80%. The abstract's "less than 5% of tokens recovered" and the conclusion's identical phrasing are both scoped to DeepSeek-V3.1-Terminus in context, but the abstract states it as a property of the method. Table 2 separately reports 13.51% under the same attack on Qwen2.5-14B-Instruct. Personally-identifiable-information recovery is uniformly low across all seven models (0% to 2.63%), so the spread is in ordinary tokens rather than in the sensitive spans. ↩