Gradient leakage in distributed training¶
Scope: the gradient tensor as a data channel. This page covers what an attacker who sees one worker's gradient can reconstruct from it, the difference between the optimization attack and the closed form that underlies it, the specific and often-omitted constraints under which the original results were produced, what each published defense costs in model accuracy, and which cluster topologies actually put a gradient across a trust boundary. The hardware-attested alternative is GPU confidential computing; the tenancy controls that decide who can observe the wire at all are in security, isolation and multi-tenancy; the closest existing threat analysis is privacy-aware split inference, which is the inference-side counterpart.
Primary sources: Ligeng Zhu, Zhijian Liu, Song Han, "Deep Leakage from Gradients", arXiv:1906.08935v2 [cs.LG], NeurIPS 2019, with the released implementation at
mit-han-lab/dlgcommitd21007f; and Bo Zhao, Konda Reddy Mopuri, Hakan Bilen, "iDLG: Improved Deep Leakage from Gradients", arXiv:2001.02610v1 [cs.LG], 8 Jan 2020. Later measurements are quoted from Jonas Geiping et al., "Inverting Gradients", arXiv:2003.14053. No CNN attack was reproduced here and no model was trained.What this page adds. The Python block is executed and asserted (Python 3.12.3, numpy 2.4.6). It derives the closed form that makes gradient inversion possible, reproduces iDLG's label rule at 5000/5000, quantifies exactly what batching does to a reconstruction, and runs the paper's own defense list against the same gradient. That last sweep is the reason this page exists: in a setting where a closed form is available, every defense DLG's Table 3 marks as successful still leaks the sample, and magnitude pruning has to reach 99% rather than the paper's 20%. Results labelled "derived" or "executed" are this page's; results with a section or table number are the source's.
This is a trust-boundary threat, not a universal one. Single-tenant training inside one datacenter, where every rank belongs to the same owner, is not the setting these attacks target. Read the topology table before acting on any of it.
Overview¶
The assumption under test is the one that makes collaborative and federated training viable at all: raw samples stay on the node, only gradients are exchanged, therefore the data is protected. DLG shows the exchange is invertible. An attacker holding the model F, the weights W, and one worker's gradient ∇W initializes a dummy sample and label, computes its dummy gradient, and descends on the dummy inputs rather than the weights, minimizing ||∇W' - ∇W||². When the gradients match, the inputs match. The paper reports pixel-accurate image recovery and token-level text recovery, with no generative model and no prior over the data.
iDLG then removes half the problem. For any model trained with cross-entropy over one-hot labels, the ground-truth label is not something you have to search for: it is readable directly from the sign of the last-layer gradient, analytically, at 100% accuracy, at any training stage.
flowchart LR
DATA["Private batch x and y"] --> BWD["Forward and backward"]
BWD --> GRAD["Gradient tensor"]
GRAD --> DEF{"Transform before send"}
DEF -->|"noise, quantize, prune"| WIRE["Aggregation wire"]
WIRE --> OBS["Parameter server or peer rank"]
OBS --> ATK["Gradient matching or closed form"]
ATK --> OUT["Recovered sample and label"]The operational question is not "is this possible" but "does a gradient produced by data I care about ever become visible to a party I do not control". That question has a different answer per topology, and the table further down is the part worth acting on.
Core knowledge¶
Why a gradient inverts at all¶
The optimization attack gets the attention, but it is standing on an algebraic fact that needs no optimizer. For any layer computing z = Wx + b, the chain rule gives ∂L/∂W = (∂L/∂z) xᵀ and ∂L/∂b = ∂L/∂z. Every row of the weight gradient is the layer's input scaled by one scalar, and the bias gradient hands over that exact scalar. So for any row i where ∂L/∂b_i is nonzero:
One division per element. This is why gradients leak: a weight gradient is not a lossy summary of the input, it is the input times a scalar, stored in full. The optimization in DLG exists to invert the layers above the point where this closed form applies, not to perform the inversion itself.
The block below establishes that identity, checks the hand-derived gradients against central differences, reproduces iDLG's label rule, measures what batching does, and then runs DLG's defense list against the same gradient.
"""Gradient leakage: closed-form inversion of one linear layer, and what defenses cost."""
from __future__ import annotations
import numpy as np
Array = np.ndarray
def softmax(z: Array) -> Array:
e = np.exp(z - z.max(axis=-1, keepdims=True))
return e / e.sum(axis=-1, keepdims=True)
def grads(x: Array, c: Array, W: Array, b: Array) -> tuple[Array, Array, Array]:
"""Cross-entropy grads for z = Wx + b. Returns (dW, db, dz), averaged over the batch."""
z = x @ W.T + b
dz = (softmax(z) - np.eye(W.shape[0])[c]) / len(x)
return dz.T @ x, dz.sum(axis=0), dz
def numeric_dW(x: Array, c: Array, W: Array, b: Array, eps: float = 1e-6) -> Array:
"""Central-difference dL/dW, to prove the analytic form above is right."""
out = np.zeros_like(W)
for i in range(W.shape[0]):
for j in range(W.shape[1]):
hi, lo = W.copy(), W.copy()
hi[i, j] += eps
lo[i, j] -= eps
out[i, j] = (loss(x, c, hi, b) - loss(x, c, lo, b)) / (2 * eps)
return out
def loss(x: Array, c: Array, W: Array, b: Array) -> float:
p = softmax(x @ W.T + b)
return float(-np.log(p[np.arange(len(c)), c]).mean())
def invert(dW: Array, db: Array) -> tuple[Array, int]:
"""Recover the layer input in closed form. dW = dz x^T and db = dz, so any row
i with db[i] != 0 gives x = dW[i] / db[i]. Pick the row with the most signal."""
i = int(np.argmax(np.abs(db)))
return dW[i] / db[i], i
def idlg_label(dW: Array) -> int:
"""iDLG eq. 4: the ground-truth class is the row of dW whose sign opposes the rest."""
return int(np.argmin(dW.sum(axis=1)))
def to_bf16(a: Array) -> Array:
u = a.astype(np.float32).view(np.uint32)
u = (u + (0x7FFF + ((u >> 16) & 1))) & 0xFFFF0000
return u.view(np.float32)
def to_int8(a: Array) -> Array:
s = np.abs(a).max() / 127.0
return np.round(a / s).clip(-127, 127) * s
def prune(a: Array, ratio: float, thresh: float) -> Array:
return np.where(np.abs(a) >= thresh, a, 0.0)
def nmse(a: Array, b: Array) -> float:
return float(np.mean((a - b) ** 2))
def cosine(a: Array, b: Array) -> float:
return float(a @ b / (np.linalg.norm(a) * np.linalg.norm(b) + 1e-30))
rng = np.random.default_rng(0)
D, C = 64, 10 # 8x8 sample, 10 classes
x1 = rng.random((1, D)) # one private sample, pixels in [0, 1]
c1 = np.array([7])
W = rng.normal(0, 0.5, (C, D))
b = rng.normal(0, 0.5, C)
dW, db, _ = grads(x1, c1, W, b)
print("== 1. analytic gradient == numeric gradient ==")
err = np.abs(dW - numeric_dW(x1, c1, W, b)).max()
print(f"max |analytic - central difference| = {err:.3e}")
assert err < 1e-8, err
print("\n== 2. closed-form recovery of the private sample, batch size 1 ==")
xr, row = invert(dW, db)
print(f"attacker used row {row}; true label {c1[0]}")
print(f"recovery MSE = {nmse(xr, x1[0]):.3e} max abs pixel error = {np.abs(xr - x1[0]).max():.3e}")
assert nmse(xr, x1[0]) < 1e-20
print("no optimizer, no priors, no iterations: one division per pixel.")
print("\n== 3. iDLG label rule, 5000 random trials ==")
hits = 0
for _ in range(5000):
Wt = rng.normal(0, 0.5, (C, D))
bt = rng.normal(0, 0.5, C)
xt = rng.random((1, D))
ct = int(rng.integers(C))
dWt, _, _ = grads(xt, np.array([ct]), Wt, bt)
hits += idlg_label(dWt) == ct
print(f"label recovered exactly in {hits}/5000 trials = {100 * hits / 5000:.1f}%")
assert hits == 5000
print("DLG's own optimizer scored 89.9 / 83.3 / 79.1% on MNIST / CIFAR-100 / LFW (iDLG Table 1).")
print("\n== 4. what batching actually does ==")
print(" B best per-sample MSE MSE vs dz-weighted mean negative rows / distinct labels")
for B in (1, 2, 4, 8):
xb = rng.random((B, D))
cb = rng.integers(C, size=B)
dWb, dbb, dzb = grads(xb, cb, W, b)
xrb, i = invert(dWb, dbb)
mixture = (dzb[:, i] @ xb) / dzb[:, i].sum()
best = min(nmse(xrb, xb[k]) for k in range(B))
neg = int((dWb.sum(axis=1) < 0).sum())
print(f"{B:2d} {best:.4e} {nmse(xrb, mixture):.3e} {neg} / {len(set(cb.tolist()))}")
print("the row is exact for the mixture, never for a sample: batching mixes, it does not hide.")
print("\n== 5. defenses, applied to the same shared gradient ==")
print("(paper's Fig. 6 calls a reconstruction recovered below MSE 0.03 and unrecognizable above 0.2)")
flat = np.abs(np.concatenate([dW.ravel(), db]))
print(f"{'defense':<22}{'MSE':>12}{'cosine':>10} verdict")
def report(name: str, gW: Array, gb: Array) -> None:
xr, _ = invert(gW, gb)
m, cs = nmse(xr, x1[0]), cosine(xr, x1[0])
verdict = "recovered" if m < 0.03 else ("partial" if m < 0.2 else "defended")
print(f"{name:<22}{m:>12.3e}{cs:>10.4f} {verdict}")
report("none", dW, db)
report("fp16", dW.astype(np.float16).astype(np.float64), db.astype(np.float16).astype(np.float64))
report("bfloat16", to_bf16(dW).astype(np.float64), to_bf16(db).astype(np.float64))
report("int8 per-tensor", to_int8(dW), to_int8(db))
for s in (1e-4, 1e-3, 1e-2, 1e-1):
report(f"gaussian sigma={s:g}", dW + rng.normal(0, s, dW.shape), db + rng.normal(0, s, db.shape))
for r in (0.01, 0.10, 0.20, 0.30, 0.50, 0.70, 0.99):
t = np.quantile(flat, r)
report(f"prune {r:.0%}", prune(dW, r, t), prune(db, r, t))
print("\n== 6. why magnitude pruning is uneven ==")
print("dW row i scales with |dz_i|, so pruning is not spread evenly over rows.")
print(" ratio survivors in attacker's row survivors overall")
for r in (0.20, 0.50, 0.70, 0.90, 0.99):
t = np.quantile(flat, r)
keep_row = float(np.mean(np.abs(dW[row]) >= t))
keep_all = float(np.mean(np.abs(dW) >= t))
print(f" {r:5.0%} {keep_row:>22.1%} {keep_all:>16.1%}")
print("the attacker reads the row pruning protects least.")
Executed output:
== 1. analytic gradient == numeric gradient ==
max |analytic - central difference| = 3.624e-10
== 2. closed-form recovery of the private sample, batch size 1 ==
attacker used row 7; true label 7
recovery MSE = 5.093e-34 max abs pixel error = 1.110e-16
no optimizer, no priors, no iterations: one division per pixel.
== 3. iDLG label rule, 5000 random trials ==
label recovered exactly in 5000/5000 trials = 100.0%
DLG's own optimizer scored 89.9 / 83.3 / 79.1% on MNIST / CIFAR-100 / LFW (iDLG Table 1).
== 4. what batching actually does ==
B best per-sample MSE MSE vs dz-weighted mean negative rows / distinct labels
1 1.1676e-33 0.000e+00 1 / 1
2 3.1316e-02 1.830e-33 2 / 2
4 4.3722e-02 4.385e-33 2 / 3
8 4.9630e-02 3.659e-33 3 / 4
the row is exact for the mixture, never for a sample: batching mixes, it does not hide.
== 5. defenses, applied to the same shared gradient ==
(paper's Fig. 6 calls a reconstruction recovered below MSE 0.03 and unrecognizable above 0.2)
defense MSE cosine verdict
none 5.093e-34 1.0000 recovered
fp16 1.147e-08 1.0000 recovered
bfloat16 1.035e-06 1.0000 recovered
int8 per-tensor 4.900e-06 1.0000 recovered
gaussian sigma=0.0001 1.597e-08 1.0000 recovered
gaussian sigma=0.001 1.749e-06 1.0000 recovered
gaussian sigma=0.01 2.313e-04 0.9998 recovered
gaussian sigma=0.1 1.271e-02 0.9834 recovered
prune 1% 5.093e-34 1.0000 recovered
prune 10% 5.093e-34 1.0000 recovered
prune 20% 5.093e-34 1.0000 recovered
prune 30% 1.172e-07 1.0000 recovered
prune 50% 1.172e-07 1.0000 recovered
prune 70% 2.669e-04 0.9996 recovered
prune 99% 2.472e-01 0.5020 defended
== 6. why magnitude pruning is uneven ==
dW row i scales with |dz_i|, so pruning is not spread evenly over rows.
ratio survivors in attacker's row survivors overall
20% 100.0% 79.8%
50% 98.4% 49.7%
70% 87.5% 29.8%
90% 71.9% 9.7%
99% 9.4% 0.9%
the attacker reads the row pruning protects least.
Four things fall out of that run.
Recovery at batch size 1 is exact to machine precision, at MSE 5.1e-34, with no optimizer involved. The row the attacker picks by largest bias-gradient magnitude is also the true-class row, which is the same signal iDLG uses for labels.
Batching mixes rather than hides. The recovered row is exact for the ∂L/∂z-weighted mean of the batch (MSE around 1e-33 at every batch size tested) and never matches any individual sample: best per-sample error climbs to 3.1e-2 at B=2 and 5.0e-2 at B=8. A mean of two images is not a private image. The label signature also survives partially, with 3 rows carrying the negative signature against 4 distinct labels at B=8, which is why iDLG's own limitation section restricts its 100% claim to per-sample gradients.
Every defense in DLG's Table 3 that the paper marks as successful still leaks here. Int-8 quantization, which the paper marks as defending at a cost of 22.6 accuracy points, returns the sample at MSE 4.9e-6. Gaussian noise at σ=1e-2, marked as defending at a cost of 31 accuracy points, returns it at 2.3e-4. Even σ=1e-1, which the paper reports drives CIFAR-100 accuracy to 1% or below, leaves MSE at 1.3e-2 and cosine similarity at 0.983, still inside the paper's own "recovered" band. The mechanism is unremarkable: the closed form reads the layer with the largest gradient magnitude, where ∂L/∂z is order 1, so a fixed absolute noise is relatively small exactly where the attacker is looking.
Magnitude pruning is not uniform, and its non-uniformity favours the attacker. Because row i of the weight gradient scales with |∂L/∂z_i|, and the true-class row carries the largest such scalar, global magnitude pruning removes that row last. At a 20% global prune ratio the attacker's row keeps 100% of its entries; at 70%, when the tensor overall retains 29.8%, that row still retains 87.5%. Only at 99% does recovery fail, and the paper's stated tolerance of "around 20%" does not transfer.
The honest reading is not that the papers are wrong. It is that defendability in Table 3 is a property of DLG's L-BFGS attack on a multi-layer CNN, not a property of gradient sharing, and a cheaper attack in a setting with a closed form moves every threshold. Anyone using that table to size a production noise level is calibrating against one attack from 2019.
The constraints the results were produced under¶
These are stated in the sources but routinely dropped when DLG is cited. They matter because they decide whether the result applies to your training run.
| Constraint | What the source says |
|---|---|
| Activation | ReLU replaced with Sigmoid, and strides removed, because "our algorithm requires the model to be twice-differentiable" (DLG §4.1) |
| Weights | "all our experiments are using randomly initialized weights" (DLG §4) |
| Batch size | "DLG currently only works for batch size up to 8" (DLG §5.3) |
| Resolution | "and image resolution up to 64×64" (DLG §5.3) |
| Label recovery | 89.9% / 83.3% / 79.1% on MNIST / CIFAR-100 / LFW (iDLG Table 1) |
| iDLG's own scope | labels recoverable "only if gradients w.r.t. every sample in a training batch are provided" (iDLG §4) |
The randomly-initialized-weights constraint is the load-bearing one, and it was later measured. DLG §4 asserts that "the attack can happen anytime during the training", but every reported experiment uses random initialization, so the claim is asserted rather than tested. Geiping et al. tested it directly, with 16 L-BFGS restarts to avoid under-tuning the baseline, and report mean PSNR over 100 CIFAR-10 images (Table 1):
| Attack | LeNet untrained | LeNet trained | ResNet20-4 untrained | ResNet20-4 trained |
|---|---|---|---|---|
| Euclidean loss with L-BFGS (the DLG formulation) | 46.25 ± 12.66 | 13.24 ± 5.44 | 10.29 ± 5.38 | 6.90 ± 2.80 |
| Cosine-similarity objective (proposed there) | 18.00 ± 3.33 | 18.08 ± 4.27 | 19.83 ± 2.96 | 13.95 ± 3.38 |
DLG's own formulation is near-perfect on the untrained shallow smooth network it was demonstrated on and, in that paper's words, "completely fails on the trained ResNet". This cuts both ways and both directions are operationally relevant. It means the 2019 attack does not straightforwardly apply to a mid-training ResNet, and it means the fix was published within a year: an objective based on gradient direction rather than magnitude recovers from trained ReLU networks, ImageNet resolution, and batches of up to 100 under federated averaging. Treating "we use ReLU", "our model is already trained", or "our batches are large" as defenses is reasoning against the 2019 attack only.
Paper against released code¶
The released implementation at mit-han-lab/dlg commit d21007f disagrees with the paper text in ways worth knowing before citing either.
main.pyinstantiatesLeNet, not ResNet-56. DLG §4.1 states experiments on "modern CNN architectures ResNet-56", while the released reproduction script uses a four-layer network (three stride-2/1 Sigmoid convolutions into a single 768-to-100 linear layer). iDLG §3 says "Following the settings in [1], we use the randomly initialized LeNet", so the widely reproduced result follows the code rather than the paper.- The repository's ResNet cannot run.
models/vision.pycallsF.Sigmoid(...)insideBasicBlock.forward.torch.nn.functionaldefines only lowercasesigmoid, so that path raisesAttributeErroron any PyTorch version. It is dead code, which is consistent withmain.pynever importing it. - Iteration count differs. The paper states 1200 iterations for image tasks;
main.pyruns 300. The L-BFGS settings the paper lists (learning rate 1, history size 100, max iterations 20) are PyTorch's defaults, and the script passes none of them explicitly. - Initialization is non-standard.
weights_initappliesuniform_(-0.5, 0.5)to the weight and bias of every module that has them, which is a much wider distribution than PyTorch defaults and is not the initialization a real training run would use.
Where a gradient actually crosses a trust boundary¶
This is the part to act on. The attack needs an observer who sees a gradient and does not own the data that produced it.
| Topology | Who sees a per-worker gradient | Exposure |
|---|---|---|
| Single-tenant DDP or FSDP inside one datacenter | Ranks you own, on a fabric you own | Not the threat model. The attacker would already have the dataset. |
| Rented or shared parameter server | The server operator | Direct. DLG Fig. 1a is exactly this: a server that stores no training data can reconstruct every participant's. |
| Decentralized all-reduce with untrusted peers | Neighbouring ranks | Direct. DLG Fig. 1b: any participant can target its neighbours. Ring reductions expose partial sums, not just your own tensor. |
| Cross-organization federated or collaborative training | Aggregator, and peers under some protocols | The original motivating case (hospitals, keyboards). Highest value, weakest default protection. |
| Geo-distributed training over a WAN | Anyone on the path without transport encryption | Direct. Also the case where compression is already in use, which changes the calculus (below). |
| Gradients written to shared storage, checkpoints, or debug logs | Anyone with read access to that path | Frequently overlooked. A gradient dump in an object store is the same disclosure as one on the wire, with a longer retention period. |
If your answer to every row is "we own all of it", this page is background reading. If any row is a rented server, a partner organization, or a bucket with broad read access, it is a live issue.
What actually helps¶
Ordered by whether the protection survives a stronger attack, rather than by the order the paper presents.
- Do not let the gradient cross the boundary in plaintext. Secure aggregation means the aggregator sees only the sum over many participants, never a single contribution, which removes the per-worker tensor the attack needs. DLG §5.3 calls cryptography "the most secured one" among the defenses it considers, while objecting that the 2016 protocol it cites requires gradients to be integers and so is "not compatible with most CNNs". That objection is about representation, not security: the constraint means float gradients have to be encoded into a finite field before aggregation, which costs precision rather than ruling the approach out. The assessment is from 2019, so check the current state of whatever library you would actually deploy rather than treating the paper's objection as settled. Homomorphic encryption is the other option cited there, and DLG notes it addresses the parameter-server case only.
- Aggregate before anyone untrusted sees anything. A local trusted aggregator that sums a site's ranks before shipping one tensor upstream converts a per-sample exposure into a large-batch one. This is the cheapest structural fix and it composes with everything else.
- Hardware attestation, if the boundary is a rented machine rather than a peer. Confidential computing with remote verification keeps the gradient encrypted in use, which is a different and stronger guarantee than perturbing it.
- Aggressive gradient compression, as a side effect rather than a plan. DLG §5.2 measures a tolerance around 20% sparsity and then argues that Deep Gradient Compression exceeds 99%, so compression is a practical defense. The extrapolation is reasonable but was not run end to end: the paper tests fixed-ratio magnitude pruning, not DGC. DGC also carries error feedback, so pruned components are accumulated locally and transmitted later rather than discarded, which defers information instead of destroying it. The executed sweep above is a further reason for caution, since pruning protected the attacker's chosen row far more slowly than the global ratio suggests. Treat high sparsity as raising cost, not as a guarantee.
- Differential privacy noise, if you are willing to pay the accuracy. DLG Table 3 puts the price plainly: of the four noise levels it marks as defending, the cheapest costs 30.1 accuracy points on CIFAR-100 (76.3% to 46.2% at Laplacian σ=1e-2), the Gaussian equivalent costs 31.0 (to 45.3%), and both σ=1e-1 settings drive accuracy to 1% or below. Int-8 defends there at 53.7%, a 22.6-point drop. If you want a privacy guarantee rather than an empirical threshold, use a DP-SGD implementation with an accounted budget; a hand-picked σ borrowed from this table is neither a guarantee nor, per the sweep above, reliably effective.
Two things that are not defenses. Reduced precision is explicitly measured as failing in DLG §5.1 for both IEEE float16 and bfloat16, and fails in the executed sweep above, so mixed-precision communication buys nothing here. Large batches raise cost without closing the channel: the executed run shows exact recovery of the batch mean at every size, and Geiping et al. recover several images from federated averaging over 100.
Don't-miss checklist¶
- Enumerate every path where a gradient leaves a trust boundary, including object storage, checkpoint directories, and debug logs, not just the collective wire.
- Encrypt the transport for any geo-distributed or cross-site training run. This is table stakes and is separate from everything above.
- If gradients cross an organizational boundary, use secure aggregation or a trusted local aggregator, and confirm the aggregator cannot observe a single participant's contribution.
- Never treat fp16, bf16, or a large batch size as a privacy control.
- If you deploy noise, derive it from an accounted DP budget and record the measured accuracy cost, rather than copying a σ from a 2019 table.
- Check whether per-sample gradients are exposed anywhere. Per-sample gradients are the strongest possible case for the attacker and are the precondition for iDLG's 100% label recovery.
- When citing DLG, state the constraints it ran under. A claim that "gradients leak training data" is well supported; a claim that "DLG recovers data from a trained ResNet" contradicts the measurement in Geiping et al. Table 1.
Failure modes¶
- Citing the 20% pruning tolerance as a configuration target. It is specific to DLG's attack and its architecture. The executed sweep recovers the sample at 70% global pruning in a setting with a closed form.
- Assuming ReLU protects. It defeats the 2019 attack's second-derivative requirement and does not defeat a direction-based objective.
- Assuming a trained model protects. Training substantially weakens DLG on both architectures Geiping et al. tested, from 46.25 to 13.24 PSNR on LeNet and from 10.29 to 6.90 on ResNet20-4, and does not stop the cosine-similarity attack, which holds at 13.95 on the trained ResNet20-4 cell.
- Deploying DP noise without accounting. A σ chosen to make reconstructions look bad is an empirical observation against one attack, not a bound, and buys a large accuracy loss for it.
- Believing the parameter server is trusted because it holds no data. DLG Fig. 1a is precisely the case of a server that stores no training data and can reconstruct all of it.
- Aggregating only at the end. In a ring reduction, partial sums traverse peers. Whether that is a problem depends on who owns the ranks, which is why the topology table matters more than the algorithm.
Open questions & validation¶
- The executed sweep uses a single linear layer, where a closed form exists. That is the strongest case for the attacker and the weakest test of a defense, and it is deliberately so: a defense that fails there cannot be relied on elsewhere. It does not establish that these defenses fail against a deep CNN, which is the setting DLG measured. Both readings are needed and neither substitutes for the other.
- No CNN attack was reproduced here. Reproducing DLG requires PyTorch and was out of scope for this environment, which has numpy only. The published CNN numbers are quoted, not verified.
- Whether DGC with error feedback resists reconstruction over a multi-step window is, as far as the sources here go, untested. The deferred-transmission argument cuts against the paper's extrapolation and the multi-step aggregation argument cuts for it. It would take an experiment against a real DGC run to settle.
- The attacks in scope target supervised training with cross-entropy. Whether comparable reconstruction applies to the gradients in RL post-training, where the loss and batch structure differ, is not covered by these sources.
References¶
- Deep Leakage from Gradients (arXiv:1906.08935) - Zhu, Liu, Han. NeurIPS 2019. The attack, defense sweep, and Table 3 accuracy/defendability trade-off.
- mit-han-lab/dlg - the released implementation. Commit
d21007fa1540ba2303ebc034976aa331814727c7inspected for the code claims above. - iDLG: Improved Deep Leakage from Gradients (arXiv:2001.02610) - Zhao, Mopuri, Bilen. Analytic label extraction and the DLG label-accuracy measurements.
- PatrickZH/Improved-Deep-Leakage-from-Gradients - the iDLG implementation, as cited in that paper.
- Inverting Gradients: How easy is it to break privacy in federated learning? (arXiv:2003.14053) - Geiping et al. Cosine-similarity objective, trained-network and ImageNet results, Table 1.
- See through Gradients: Image Batch Recovery via GradInversion (arXiv:2104.07586) - Yin et al. Batch recovery at larger scale.
- Practical Secure Aggregation for Federated Learning on User-Held Data (arXiv:1611.04482) - Bonawitz et al. The protocol DLG §5.3 cites as the most secure defense.
- Deep Gradient Compression (arXiv:1712.01887) - Lin et al. The compression work DLG extrapolates its pruning defense from.
- Exploiting Unintended Feature Leakage in Collaborative Learning (arXiv:1805.04049) - Melis et al. The "shallow" leakage baseline DLG compares against.
Related: security, isolation and multi-tenancy | GPU confidential computing | remote GPU verification | privacy-aware split inference | distributed training | geo-distributed training placement | DiLoCo | delta weight sync