Skip to content
Markdown

Split training and security

Scope: the security of splitting a model across parties that do not fully trust each other, for training and fine-tuning rather than for inference alone. This page maps the trust boundary, names the four channels that cross it, catalogs what the published attacks recover and what the published defenses cost, and routes to the focused pages that carry implementable detail. The inference-side cousins already have their own pages and are not repeated here: privacy-aware split inference over a WAN, covariant obfuscation, and activation obfuscation on untrusted accelerators. The hardware-attested alternative to all of it is GPU confidential computing.

What this page adds. Two Python blocks, both executed and asserted (Python 3.12.3, numpy 2.4.6). The first checks the algebra that three published obfuscation schemes rely on and locates precisely what each one still exposes, including that KV-Shield's permutation cancels inside the attention score product so the untrusted GPU computes the true attention map in the clear. The second quantifies the "Not-too-far" property that every semi-white-box split attack is built on, and finds that nearest-neighbour token inversion survives additive noise as large as the signal itself at 99.6% recovery. Results labelled "derived" are this page's; results carrying a table or section number belong to the cited paper.

Read every number as a research result, not a product spec. The split-learning experiments here top out around eight billion parameters and run on single machines simulating multiple parties; BiSR's, the most-cited of them, use batch size 2 with LoRA on V100s. None of the defenses below has a production implementation this page has verified, and no attack was reproduced here.

Focused pages

The seam appears in several places in this KB. Open the page that matches the boundary you actually have:

What the split is, and what crosses it

Split learning cuts a network at one or more layer boundaries and gives the pieces to different owners. The dominant shape for LLM fine-tuning is three slices: a head and a tail held by the party that owns the private data, and a trunk (also called the body) hosted on a server with the GPUs. The data holder embeds its tokens, runs them through the head, and ships the resulting activations, conventionally called smashed data, to the server. The server runs the trunk and returns activations to the tail. Backpropagation reverses the path, so gradients flow back across the same seam.

The claim being made is that raw data never leaves. The claim being tested by the literature below is whether that is the same thing as privacy.

flowchart LR
  subgraph DP["Data party (private)"]
    X["Input tokens X"] --> Hd["Head M_h"]
    Tl["Tail M_t"] --> Y["Response Y"]
  end
  subgraph MP["Model party (untrusted server)"]
    Tr["Trunk M_b"]
  end
  Hd -- "smashed data H" --> Tr
  Tr -- "activations T" --> Tl
  Tl -. "gradients dL/dT" .-> Tr
  Tr -. "gradients dL/dH" .-> Hd
  Tr --> A1["Channel 1: invert H to recover X"]
  Tr --> A2["Channel 2: match gradients to recover labels"]
  Tr --> A3["Channel 3: complete M_t to recover Y"]
  Y --> A4["Channel 4: invert outputs to steal capability"]

Four channels, not one

Most split-learning threat modelling stops at the first channel. All four are live.

Channel What the untrusted party observes What it recovers Anchor
Forward Smashed data H The private input prompt SIP / BiSR, PIDI
Backward Gradients returned to the head and tail Labels, and via the autoregressive overlap, the input again TAG, LAMP, BiSR
Response The tail's parameters being close to a public checkpoint The generated response, never transmitted PIDI
Capability Final answers and reasoning summaries only The model owner's reasoning ability Trace Inversion

The third and fourth deserve emphasis because they invert the usual framing. Channel 3 leaks the output, which the split was never designed to protect and which in a finance or medical setting is as sensitive as the input. Channel 4 runs the other way: the split boundary is also what protects the server's asset, and hiding chains of thought does not protect it.

Two structural properties make all of this work, and both are named explicitly in the literature:

  • Not-too-far. Fine-tuning does not move weights far from the pre-trained checkpoint, so an attacker who downloads the public base model holds an accurate prior for the private head and tail. This turns a black-box attack into a semi-white-box one. BiSR states it as Assumption 1 and builds its whole method on it; DualGuard and ADMI both target it directly as the thing to break.
  • Autoregressive overlap. For a generative task the input and the label overlap heavily, so a defense that only blocks one direction leaves the other one able to reconstruct the same content.

What the attacks actually recover

Attack Setting Threat model Reported result
SIP (CCS '24) Split fine-tuning, forward channel Semi-white-box, attacker trains a single-layer GRU inverter on an auxiliary corpus ROUGE-L F1 above 0.9 at shallow split points on LLaMA2-chat-7B and ChatGLM3-6B; encoder-only BERT and RoBERTa leak measurably less than decoder-only models
BiSR (CCS '24) Split fine-tuning, both directions Semi-white-box, adds optimization over smashed data and gradients on top of SIP Reconstructs private data through Embedding-dxP, Smashed-Data-DP and NoPeek; its NaMoE mixture-of-experts inverter adapts when the defender's noise scale is unknown
Prompt inference on distributed inference Petals-style split inference Black-box and passive, no shadow model, no knowledge of the client's layers Above 90% reconstruction accuracy for the two attacks with unlimited queries; above 50% in most cases with limited queries and no auxiliary data. Early layers leak far more than late ones
PIDI Split-LLM open-ended generation Model party, 50-sample auxiliary set Reconstructs input prompts and generated responses. Its blended attack-performance metric (AP at alpha = 0.5, weighting input and response privacy equally) reaches 0.868 and 0.901 undefended on two of its settings, against 0.017 and 0.074 under its own ADMI defense
Trace Inversion Black-box commercial API Only inputs, final answers and reasoning summaries 81% token-length recovery and 52.76 token-overlap F1 against DeepSeek-R1 traces. Fine-tuning Llama-3.1-8B-Instruct on inverted traces reaches 52.4% on MATH500 against 16.4% when fine-tuned on the summaries and answers alone

The black-box result is the one to internalise. The Petals-style attack needs no auxiliary model and no knowledge of the client's layers, because intermediate embeddings from different layers form distinct clusters in embedding space and that geometry alone is enough. A defense that hides which vector is which, but preserves the distances between them, does not touch this attack. The executed block below shows exactly that failure.

Where the secret lives: three shapes, three different leaks

Every non-cryptographic obfuscation scheme in this corpus applies an invertible transform and relies on it cancelling somewhere. The axis it acts on decides what the untrusted side still sees.

"""Which axis the secret transform acts on decides what the untrusted side still sees.

Executed against the algebra the three obfuscation schemes publish:
KV-Shield (arXiv:2409.04040 Eq. 2-6), GELO (arXiv:2603.05035v2), and the
covariant construction (arXiv:2603.01499). No model is run; this is the
linear algebra those papers state, checked for what it does and does not hide.
"""
import numpy as np

rng = np.random.default_rng(0)
m, d = 6, 32                      # m tokens, d features
H = rng.standard_normal((m, d))   # hidden states entering one attention block
Wq, Wk, Wv = (rng.standard_normal((d, d)) / np.sqrt(d) for _ in range(3))


def softmax(S):
    S = S - S.max(axis=-1, keepdims=True)
    E = np.exp(S)
    return E / E.sum(axis=-1, keepdims=True)


def attention(Q, K, V):
    return softmax(Q @ K.T / np.sqrt(d)) @ V


Q, K, V = H @ Wq, H @ Wk, H @ Wv
A_true = attention(Q, K, V)

# ---------------------------------------------------------------- feature axis
# KV-Shield: permute the FEATURE axis by folding a permutation into the weights.
P = np.eye(d)[:, rng.permutation(d)]          # d x d permutation, held in the TEE
assert np.allclose(P @ P.T, np.eye(d))
Qp, Kp, Vp = H @ (Wq @ P), H @ (Wk @ P), H @ (Wv @ P)

# 1. Correctness: the TEE recovers the true output by un-permuting (Eq. 6).
Ap = attention(Qp, Kp, Vp)
assert np.allclose(Ap @ P.T, A_true, atol=1e-12)

# 2. What the untrusted GPU still computes in the clear: the permutation is
#    orthogonal, so it cancels inside the score bilinear form. The two score
#    matrices agree to summation order, i.e. to floating-point rounding.
S_obf, S_ref = Qp @ Kp.T, Q @ K.T
scores_leak = float(np.abs(S_obf - S_ref).max() / np.abs(S_ref).max())
assert scores_leak < 1e-14

# 3. Secrecy of P rests entirely on the permuted weights being unreadable.
#    For an open-weight model the reference W is public, so P falls out directly.
P_rec = np.linalg.pinv(Wq) @ (Wq @ P)
assert np.array_equal(np.rint(P_rec).astype(int), P.astype(int))

# ------------------------------------------------------------------ token axis
# GELO: mix the TOKEN axis with a fresh invertible A per batch.
Amix = rng.standard_normal((m, m))
assert abs(np.linalg.det(Amix)) > 1e-6

# 4. A projection GEMM offloads exactly: inv(A) @ ((A H) W) == H W.
offloaded = np.linalg.inv(Amix) @ ((Amix @ H) @ Wv)
assert np.allclose(offloaded, H @ Wv, atol=1e-10)

# 5. Attention does NOT offload the same way: token mixing conjugates the score
#    matrix, and softmax does not commute with that conjugation.
S_mixed = (Amix @ Q) @ (Amix @ K).T / np.sqrt(d)
conjugated = float(np.abs(S_mixed - Amix @ (Q @ K.T / np.sqrt(d)) @ Amix.T).max())
naive = np.linalg.inv(Amix) @ (softmax(S_mixed) @ (Amix @ V))
assert conjugated < 1e-10                      # scores are exactly conjugated
assert not np.allclose(naive, A_true, atol=1e-3)   # but softmax breaks the undo

print(f"feature-axis  scores leaked verbatim, max relative delta = {scores_leak:.1e}")
print(f"feature-axis  permutation recovered from public weights: exact")
print(f"token-axis    projection GEMM offloads exactly")
print(f"token-axis    attention does not: max|A_hat - A| = {np.abs(naive - A_true).max():.3f}")

Executed output:

feature-axis  scores leaked verbatim, max relative delta = 2.1e-16
feature-axis  permutation recovered from public weights: exact
token-axis    projection GEMM offloads exactly
token-axis    attention does not: max|A_hat - A| = 5.003

Three things follow, and the first two are this page's observations rather than claims the papers make.

A feature-axis permutation does not hide the attention map. KV-Shield's own Equation 5 computes the softmax on the untrusted GPU from q_p K_p^T, and because the permutation matrix is orthogonal it cancels there exactly. The permuted key-value cache is protected; the token-by-token attention pattern computed over it is not. That is a real coverage gap for a scheme whose stated goal is preventing conversation reconstruction, since the attention map alone carries sequence structure.

Its secrecy is one memory-read deep. For an open-weight model running on the user's own device, which is exactly KV-Shield's setting, the reference weight matrix is public. Recovering the permutation from the permuted copy is a pseudo-inverse away, and the executed check recovers it exactly. The scheme therefore depends entirely on its stated threat model holding: the attacker reads shared and local GPU memory but never the weights in global memory. Treat that assumption as load-bearing, not incidental. This is a recognised attack class rather than a hypothetical: AloePri's evaluation runs a Vocabulary-Matching Attack and an Invariant Attack for exactly this purpose, to check whether an adversary can "recover the secret mapping based on the relationship between plaintext and obfuscated mode weights". KV-Shield's evaluation contains no such test.

And its own cost table understates the model it targets. The paper projects "for an entire model with over 20 layers, the latency can reach 5 minutes", which matches its d_model = 768 row over twenty layers (15.75 * 20 = 315s). Applied to LLaMA2-7B, the model in its own Table 4, the same Table 5 gives 84.22 * 32 = 2695s, roughly 45 minutes of one-time initialization, and the runtime path costs 4.3 * 32 = 138s of TEE permutation per generated token. The authors do concede the runtime latency is too high for real-time generation; the arithmetic is this page's.

Token-axis mixing is structurally different. It offloads a projection GEMM exactly, which is why GELO can rent out the dominant matrix multiplications, but it conjugates the score matrix rather than cancelling in it, and softmax does not commute with the conjugation. That is the mathematical reason GELO keeps attention inside the TEE instead of offloading it, and why its offloadable fraction is bounded by construction.

The Not-too-far property, and what each defense lever actually moves

Split defenses reduce to three levers: move the parameters, add noise, or change what the representation carries. The block below prices the first two against a nearest-neighbour inversion attack.

"""The 'not-too-far' property as a measurable quantity, and what each defense lever moves.

Executed derivation for this page. It models one split-learning client head as
the linear map an inversion attacker actually inverts, then measures token
recovery under the three levers the literature uses: fine-tuning drift
(arXiv:2409.00960's Not-too-far assumption), parameter-space transformation
(ACL 2025 DualGuard, arXiv:2606.14210 ADMI), and additive noise (DP-Forward).
It is a linear stand-in for one client head, not a transformer.
"""
import numpy as np

rng = np.random.default_rng(7)
V, d, n = 512, 64, 256            # vocab, width, tokens in the private batch
E = rng.standard_normal((V, d)) / np.sqrt(d)     # public pre-trained embedding
W0 = rng.standard_normal((d, d)) / np.sqrt(d)    # public pre-trained head
ids = rng.integers(0, V, n)                      # the private input
chance = 1.0 / V


def smashed(W, noise=0.0):
    Z = E[ids] @ W
    return Z + noise * rng.standard_normal(Z.shape) * Z.std()


def recover(Z, W_prior):
    """Attacker inverts with whatever head it believes the client is using."""
    table = E @ W_prior                                   # V x d reference
    dist = ((Z[:, None, :] - table[None, :, :]) ** 2).sum(-1)
    return float((dist.argmin(1) == ids).mean())


assert recover(smashed(W0), W0) == 1.0                    # exact prior, exact inversion

# ------------------------------------------------- lever 1: fine-tuning drift
# W = W0 + eps*G with ||G||_F == ||W0||_F, so eps is drift relative to the head's
# own norm. This is the "Not-too-far" assumption, measured.
G = rng.standard_normal((d, d))
G *= np.linalg.norm(W0) / np.linalg.norm(G)
drift = [(eps, recover(smashed(W0 + eps * G), W0))
         for eps in (0.0, 0.2, 0.8, 1.2, 2.0, 4.0, 8.0)]
assert dict(drift)[1.2] > 0.98        # drift larger than the head itself still inverts
assert dict(drift)[8.0] < 0.10        # only an order-of-magnitude move breaks it

# ------------------------------------------------- lever 2: additive DP noise
noise = [(s, recover(smashed(W0, noise=s), W0))
         for s in (0.0, 0.5, 1.0, 2.0, 4.0, 8.0)]
assert dict(noise)[1.0] > 0.98        # noise as large as the signal: no protection at all
killed = next(s for s, acc in noise if acc <= 0.05)
distortion = float(np.linalg.norm(smashed(W0, noise=killed) - smashed(W0))
                   / np.linalg.norm(smashed(W0)))

# ------------------------------- lever 3: parameter-space transform, no noise
# Fold an orthogonal R into the head; the trunk absorbs R.T, so the composed
# function is unchanged and the utility cost is exactly zero.
R, _ = np.linalg.qr(rng.standard_normal((d, d)))
Zt, Zref = E[ids] @ (W0 @ R), E[ids] @ W0
assert np.allclose(Zt @ R.T, Zref, atol=1e-12)            # trunk recovers exactly
prior_attack = recover(Zt, W0)
assert prior_attack <= chance                             # the pre-trained prior is dead
assert recover(Zt, W0 @ R) == 1.0                         # an attacker who relearns is not

# ...but an orthogonal map is an isometry, so the geometry a clustering or
# nearest-neighbour attack relies on (arXiv:2503.09291) survives it untouched.
def pdist(Z):
    return np.sqrt(((Z[:, None, :] - Z[None, :, :]) ** 2).sum(-1))
geom = float(np.abs(pdist(Zt) - pdist(Zref)).max())
assert geom < 1e-12

print("drift  eps -> recovery:", "  ".join(f"{e:g}:{a:.1%}" for e, a in drift))
print("noise  sigma -> recovery:", "  ".join(f"{s:g}:{a:.1%}" for s, a in noise))
print(f"noise to reach <=5% recovery: sigma={killed:g}, "
      f"distorting the smashed data by {distortion:.0%}")
print(f"orthogonal transform: prior attack {prior_attack:.2%} (chance {chance:.2%}), "
      f"relearned attack 100%, geometry shift {geom:.0e}, utility cost 0")

Executed output:

drift  eps -> recovery: 0:100.0%  0.2:100.0%  0.8:100.0%  1.2:100.0%  2:80.9%  4:19.1%  8:2.3%
noise  sigma -> recovery: 0:100.0%  0.5:100.0%  1:99.6%  2:71.5%  4:16.0%  8:3.1%
noise to reach <=5% recovery: sigma=8, distorting the smashed data by 797%
orthogonal transform: prior attack 0.00% (chance 0.20%), relearned attack 100%, geometry shift 7e-16, utility cost 0

Four derived results, each of which explains something the papers report but do not mechanise:

  • Not-too-far is a wide basin, not a tight one. Token recovery stays at 100% even when the head has drifted by 1.2 times its own Frobenius norm, and only collapses past roughly an eight-fold move. A defense that hopes fine-tuning will naturally carry the client model far enough from the public checkpoint is hoping for a move far larger than task adaptation produces, which is why BiSR's assumption holds so comfortably in practice. It is also why DualGuard and ADMI both jump the parameter space deliberately in a warm-up phase rather than waiting for training to drift there.
  • Noise as large as the signal buys nothing. At sigma = 1, where the injected noise matches the standard deviation of the smashed data, recovery is still 99.6%. Reaching 5% requires distorting the transmitted representation by roughly 800%, which is the privacy-utility cliff DP-Forward and dxP sit on. It is the executed counterpart of DualGuard's measured task cost: 15.2% average ROUGE-L loss for DP-forward and 21.6% for dxP, against 2.4% for the parameter-space approach.
  • A parameter-space transform is free in utility and total against a prior-based attacker. Folding an orthogonal matrix into the head and letting the next stage absorb its transpose leaves the composed function bit-identical, and drops the pre-trained-prior attack below chance. That is the mechanism behind DualGuard's warm-up and ADMI's model-distance regularizer.
  • And it does not touch a geometry-based attacker. The pairwise distance matrix of the transformed activations is identical to the original at 7e-16. An attacker who clusters embeddings rather than matching them against a public table, which is precisely the Petals-style black-box attack, sees no change at all. An attacker who simply relearns the head from auxiliary data recovers 100%. Parameter-space transformation is a defense against prior reuse, not against inversion in general, which is why both DualGuard and ADMI pair it with a second mechanism.

The defense catalog, and where each one stops

Defense Lever Covers Measured cost Caveat
DP-Forward, Smashed-Data-DP Noise on activations Forward only 15.2% average task ROUGE-L loss (DualGuard Table 2) BiSR reconstructs through it; noise-aware inverters adapt
Embedding dxP Noise on the embedding Forward only 21.6% average task loss Same
DP-SGD Noise on gradients Backward only 4.2% average task loss Leaves the forward channel completely open
NoPeek Correlation regularizer Forward Not isolated in these papers BiSR reconstructs through it
DualGuard (ACL 2025) Parameter-space transform plus retention Both directions 2.4% average task loss Geometry-preserving; an attacker who relearns the head is unaffected
ADMI Adapter warm-up, mutual-information and model-distance regularizers Both directions plus response side Reported as best privacy-utility trade-off in its own benchmark Evaluated only against the attacks in VFLAIR-LLM
DarkneTZ TEE holds the most sensitive layers Membership inference on the model 3% overhead for the last layer, up to 10% fully utilised CNN image classification on Arm TrustZone, 2020. Not shown for LLMs
KV-Shield Feature-axis permutation, TEE holds the permutation KV cache readable in device memory 84.22s to permute one layer's weights and 4.3s per attention-vector permutation at d_model = 4096, on an Intel 11800H under QEMU with 16MB of TEE memory (its Table 5) The authors call the runtime latency too high for real-time generation. Attention scores are also still computed in the clear; see the block above
GELO Fresh per-batch token-axis mixing Hidden states on rented accelerators 20-30% latency overhead claimed Two-process microbenchmark, no inference engine. See its page
AloePri Covariant transform of weights and data Input and output for LMaaS inference Claimed near-plaintext speed Per-tenant model copy. See its page
Preempt Sanitize the prompt before it is sent Token-derivable sensitive fields Reported as utility-preserving across four task types Only protects information derivable from individual tokens, by construction
Confidential GPUs Hardware TEE plus attestation Everything in device memory See the page Requires Hopper or Blackwell and an attesting operator

The single most useful row-to-row comparison in this corpus is DualGuard's Table 1, which runs five attacks against four defenses. On its first model-dataset column (GPT2-large on GSM8k, as recovered from the PDF text), DP-SGD drives the two gradient-matching attacks to 0.00 and 0.35 ROUGE-L while leaving the forward SIP attack at 87.37. DP-forward does the mirror image: it drops SIP to 36.37 and leaves TAG at 86.84. Under each defense's best attack, the undefended score of 87.63 becomes 86.84, 87.73 and 87.37 for the three DP baselines, against 25.35 for DualGuard. Choosing a unidirectional defense for a bidirectional threat buys close to nothing.

A numerical caution on that paper. Section 4.2 reports the optimal-attack averages as 1.67% for DualGuard against 8.21%, 8.67% and 7.52% for the three baselines, but Table 1's own optimal-attack rows put the DP baselines in the 63 to 90 range and DualGuard in the low twenties. The stated averages are consistent with the table only if read as 16.7%, 82.1%, 86.7% and 75.2%, a factor of ten. The contribution bullet's headline, "the average RougeL-F score decreasing from 0.752 to 0.167", then names 0.752 as the pre-defense number, but 75.2% is the DP-SGD baseline, not the undefended one; Table 1's undefended optimal-attack entries are 87.63 and 89.78 in the two columns recoverable from the PDF text. Read the table, not the prose. This is this page's reconciliation, not an erratum the authors have published.

Trusting the participants, not only the channel

Splitting a model across owners creates a second problem that obfuscation does not address: whether the other party did the work it claims. Two anchors in this corpus sit outside the activation channel.

Permissionless contribution. Gauntlet is an incentive and verification scheme deployed on the bittensor blockchain for a live 1.2B pre-training run with no control over who registers or what hardware they bring. Its defenses are behavioural rather than cryptographic: a two-stage filter for uptime, format and synchronization, a loss-delta score estimating the model loss before and after each peer's pseudo-gradient, an OpenSkill rank-based rating to track competitiveness over time, and a proof-of-computation that assigns each peer a unique data subset and checks whether that peer's loss is lower on its assigned data than on a random subset. Peer copying and sybil duplication are the named attacks. A failed fast check multiplies the peer's running score by 0.75, so repeated failures degrade it quickly, and the aggregation weight is 1/G for the top G peers and zero elsewhere. This is the trust model for geo-distributed and DiLoCo-style training with contributors you do not employ.

Transport-level exposure. Even a perfectly obfuscated payload reveals who is talking to whom and when. The Bitswap work is the clearest small example: in a content-addressed peer-to-peer network the request itself leaks a peer's interest to every neighbour, and adding request forwarding with trickle-spreading reduces a passive observer's source prediction to 40% at the cost of retrieval time. The same shape applies to any overlay mesh carrying split activations: metadata is a channel.

How to judge an obfuscation claim

Balsa's framework is the most useful lens in this corpus for reading the rest of it, because nearly every scheme above is obfuscation in his sense and nearly every evaluation makes an implicit choice he makes explicit.

  • Mechanism-centred versus attack-centred evaluation. A mechanism-centred measure describes what the transform does. An attack-centred measure describes what a specific adversary achieves against it. They sit at different points in a hierarchy of assumptions, and a scheme can look strong under the first while failing the second. Every ROUGE-L number in the tables above is attack-centred, so it is a statement about the attacks that were run, not a security property. DualGuard's own results show why this matters: DP-SGD looks excellent until a forward attack is added to the suite.
  • Utility-degrading versus utility-preserving obfuscation. Arbitrarily reducing privacy loss under a public utility requirement forces utility-degrading obfuscation. A personal utility requirement does not impose that trade-off in theory, and chaff is the illustrative utility-preserving mechanism. Mapped onto this page: additive noise on smashed data is utility-degrading, and its cost is the 15.2% and 21.6% figures. A parameter-space transform is utility-preserving, and its cost is 2.4%. The transform is not strictly better, though; it defends against a narrower attacker, as the executed geometry result shows.

Applied as a checklist, the question to ask any of these papers is: which attacks were in the suite, what would a slightly stronger adversary do, and is the reported cost the cost of the mechanism or the cost of the parameter setting that happened to be chosen?

Don't-miss checklist

  • Decide which of the four channels your split actually exposes before choosing a defense. A unidirectional defense against a bidirectional threat measurably buys close to nothing.
  • Split deeper if you split at all. Both the split fine-tuning and the distributed inference results show early layers leaking far more than late ones, and the client cost of holding more layers is usually the cheapest privacy you can buy.
  • Do not treat encoder-only intuitions as transferable. Decoder-only models leak more from intermediate activations than BERT and RoBERTa in the same experiment.
  • Assume the attacker holds the public base checkpoint. If your model is a fine-tune of something on Hugging Face, the semi-white-box threat model is the correct one.
  • Price any noise-based defense in task loss, not in epsilon. The published task costs are 15.2% and 21.6% average ROUGE-L for the two forward DP mechanisms.
  • If a scheme's security rests on an attacker not being able to read a particular region of memory, verify that boundary is enforced by hardware, not by convention.
  • Protect the response as well as the prompt. In finance and medical settings the generated answer is often the more sensitive of the two, and it is the channel the split was never designed to cover.
  • If you serve a reasoning model, treat hidden chains of thought as a speed bump rather than a control. Inverted traces distil close to the real thing.
  • For permissionless or rented contributors, add a proof-of-computation and a synchronization filter. Obfuscation says nothing about whether the peer did the work.
  • Prefer confidential GPUs when you can require the hardware. Every scheme on this page exists because that requirement is sometimes impossible.

Failure modes

  • Believing "raw data never leaves" is a privacy property. It is a data-flow statement. SIP reaches above 0.9 ROUGE-L on shallow splits without ever seeing raw data.
  • Choosing a defense from a paper whose attack suite excludes your adversary. Attack-centred numbers do not generalise past the attacks that produced them.
  • Tuning noise by epsilon alone. Epsilon is not comparable across dxP, DP-Forward and DP-SGD, and none of the three maps cleanly to reconstruction risk.
  • Assuming a permutation or rotation hides structure. Orthogonal transforms are isometries, so anything an attacker derives from distances or angles survives them exactly.
  • Assuming fine-tuning drift protects you. The executed measurement puts the collapse point at roughly eight times the head's own norm, far beyond what fine-tuning produces.
  • Splitting for privacy while sharing gradients unprotected. The backward channel is a separate attack surface with its own literature; see gradient leakage.
  • Assuming a TEE-based split scales from CNNs to LLMs. DarkneTZ's 3% figure is a last-layer CNN result on Arm TrustZone. KV cache for a 1000-token LLaMA2-7B conversation runs to hundreds of MiB or several GiB, well past a TrustZone secure-memory budget.
  • Treating FHE as a fallback. The measured self-attention latency for a single token at d_model = 4096 is 0.00018s in PyTorch, 25.60s under TenSeal and 866.81s under ConcreteML.

Open questions and validation

  • No defense here has been evaluated against an adaptive attacker who knows the defense. BiSR's NaMoE is the closest, and it is an attacker adapting to a defense rather than the reverse.
  • The attack results run at batch size 2 with LoRA on models up to about 8B. Whether inversion quality holds at production batch sizes, full fine-tuning, or 70B-plus models is untested in this corpus.
  • ADMI and DualGuard are evaluated on their own benchmark suites. Neither has been run against the other's strongest attack by an independent party.
  • The parameter-space transform's geometry-preservation gap identified above is this page's derivation on a linear model. Whether a clustering attack in the style of arXiv:2503.09291 actually defeats DualGuard on a real transformer has not been tested here and would be the single most useful experiment to run.
  • KV-Shield's attention-score exposure is derived from the paper's stated equations, not from an implementation. There is no released code to check.
  • The DualGuard scaling discrepancy is reconciled from the PDF text of Table 1 and Section 4.2. Confirm against the published tables before citing either figure.

References

  • Chen et al., "Unveiling the Vulnerability of Private Fine-Tuning in Split-Based Frameworks for Large Language Models: A Bidirectionally Enhanced Attack" (BiSR/SIP), ACM CCS 2024: https://arxiv.org/abs/2409.00960
  • Liu, Wang, Wang, Wu, "DualGuard: A Parameter Space Transformation Approach for Bidirectional Defense in Split-Based LLM Fine-Tuning", ACL 2025 Long Papers, pp. 17065-17080: https://aclanthology.org/2025.acl-long.835/
  • Gu, Ye, Liu, "From Prompts to Responses: Dual-Sided Data Leakage and Defense in Split Large Language Models" (PIDI/ADMI), arXiv:2606.14210v1: https://arxiv.org/abs/2606.14210 · code: https://github.com/FLAIR-THU/VFLAIR-LLM
  • Luo, Yu, Xiao, "Prompt Inference Attack on Distributed Large Language Model Inference Frameworks", arXiv:2503.09291v2: https://arxiv.org/abs/2503.09291
  • Zhang, Morris, Shmatikov, "How to Steal Reasoning Without Reasoning Traces", arXiv:2603.07267v2, Cornell Tech: https://arxiv.org/abs/2603.07267
  • Yang et al., "A First Look At Efficient And Secure On-Device LLM Inference Against KV Leakage" (KV-Shield), MobiArch '24: https://arxiv.org/abs/2409.04040
  • Mo et al., "DarkneTZ: Towards Model Privacy at the Edge using Trusted Execution Environments", arXiv:2004.05703, MobiSys 2020: https://arxiv.org/abs/2004.05703
  • Belikov, Fedotov, "Good-Enough LLM Obfuscation (GELO)", arXiv:2603.05035v2: https://arxiv.org/abs/2603.05035
  • Lin et al., "Towards Privacy-Preserving LLM Inference via Covariant Obfuscation" (AloePri), arXiv:2603.01499v2: https://arxiv.org/abs/2603.01499
  • Roy Chowdhury et al., "Preempt: Sanitizing Sensitive Prompts for LLMs", arXiv:2504.05147v2: https://arxiv.org/abs/2504.05147
  • Balsa, "Privacy engineering through obfuscation", arXiv:2308.12514: https://arxiv.org/abs/2308.12514
  • Lidin, Sarfi, Pappas, Dare, Belilovsky, Steeves, "Incentivizing Permissionless Distributed Learning of LLMs" (Gauntlet), arXiv:2505.21684: https://arxiv.org/abs/2505.21684
  • Daniel, Ebert, Tschorsch, "Improving Bitswap Privacy with Forwarding and Source Obfuscation", arXiv:2307.03480: https://arxiv.org/abs/2307.03480
  • Petals (the distributed inference framework the prompt-inference attack targets): https://github.com/bigscience-workshop/petals
  • OP-TEE (the TrustZone stack DarkneTZ builds on): https://www.op-tee.org/

Related: Split inference over a WAN · Gradient leakage · Gradient inversion under FedAvg · Covariant obfuscation · Activation obfuscation · GPU confidential computing · Membership inference · Security & multi-tenancy · Remote GPU verification · Geo-distributed training · DiLoCo recipe · Fine-tuning & post-training · Glossary