Skip to content
Markdown

Gradient inversion under federated averaging

Scope: the protocol-level question that follows once you accept that a single gradient reconstructs its input. Does averaging help? This page covers what federated averaging over multiple local steps, mini-batches, and epochs actually buys, the attack that made reconstruction work on trained non-smooth networks at ImageNet resolution, the analytic result that a fully-connected stack can be walked backwards with no optimizer at all, and which architecture choices measurably change exposure. The mechanism, the DLG baseline, the defense-cost table, and the map of which cluster topologies expose a gradient at all live in gradient leakage in distributed training; this page assumes them and does not repeat them.

Primary source: Jonas Geiping, Hartmut Bauermeister, Hannah Dröge, Michael Moeller, "Inverting Gradients: How easy is it to break privacy in federated learning?", arXiv:2003.14053v2 [cs.CV], 11 Sep 2020, published at NeurIPS 2020 (per dblp; the v2 PDF itself is still footered "Preprint. Under review."). The first three authors contributed equally. Implementation at JonasGeiping/invertinggradients. No attack was reproduced here: every PSNR figure quoted is the paper's.

What this page adds. The Python block is executed and asserted (Python 3.12.3, numpy 2.4.6). It implements the paper's Proposition 3.1 as running code, recovering a network input through three stacked fully-connected layers with ReLU between them, exactly, with no optimizer. It then demonstrates numerically why the cosine objective replaced the Euclidean one, shows that the proposition's technical condition can only be violated by a network that has also stopped training, and derives a mechanism for an effect the paper reports but does not explain: that batching distorts some images far more than others.

Read the PSNR numbers as CIFAR-10 and ImageNet image classification. Every result is vision. Nothing here has been shown for language models, and the paper says image classification is "possibly especially vulnerable" given the structure of image data.

Overview

Federated learning's privacy argument has two layers. The first is that raw data never leaves the device. The second, when the first is shown to be insufficient, is that what does leave is aggregated: averaged over several local gradient steps, over a mini-batch, and over epochs, so no individual sample survives. This paper attacks the second layer.

The finding is that aggregation degrades reconstruction quality without removing it, and that the degradation is uneven in a way that matters more than the average. Out of a batch of 100 images, several remain recognizable. Across 100 local gradient descent steps, reconstruction quality is essentially unchanged. The only configuration that broke the attack was a learning rate so large it would have diverged during training anyway.

flowchart LR
  subgraph USER["User device"]
    DATA["n local images"] --> STEPS["E epochs, minibatch B<br/>multiple local SGD steps"]
    STEPS --> DELTA["Parameter update sent"]
  end
  DELTA --> SRV["Honest-but-curious server"]
  SRV --> ATK["Cosine matching + TV prior<br/>signed Adam"]
  SRV --> ANA["Analytic walk-back<br/>through FC layers"]
  ATK --> REC["Recovered images"]
  ANA --> REC

Core knowledge

The change that made the attack realistic

The DLG-era objective minimizes the Euclidean distance between dummy and observed gradients with L-BFGS. Two things break that on a realistic network. Differentiating a gradient with respect to the input needs a second derivative, and L-BFGS then builds a third-order approximation on top, which is ill-behaved for ReLU units whose higher derivatives are discontinuous. Separately, a gradient's magnitude mostly encodes how well the model already fits the point, which shrinks as training progresses, so a magnitude-sensitive objective spends its effort matching a quantity that carries little information about the image.

The replacement decomposes the gradient into magnitude and direction and keeps only direction, minimizing

1 - <grad_dummy, grad_real> / (||grad_dummy|| ||grad_real||)  +  alpha * TV(x)

over x constrained to [0,1]^n, with total variation as the only image prior. It is optimized with Adam on the sign of the gradient, a trick borrowed from adversarial-example generation, with step size decay.

This is a trade, not a strict improvement, and the paper is straightforward about it. On the untrained shallow smooth network that DLG was demonstrated on, the old Euclidean attack scores 46.25 PSNR against the new method's 18.00. Everywhere else the ordering reverses, and on the trained ResNet the Euclidean attack "completely fails".

Proposition 3.1: no optimizer required for fully-connected stacks

The paper's analytic result is stronger than the single-layer identity most people know. For a biased fully-connected layer preceded solely by fully-connected layers, where the loss derivative with respect to each such layer's output has at least one non-zero entry, the input to the network can be reconstructed uniquely from the gradients, independent of the layer's position and of what surrounds it.

The recursion works because each step recovers both a layer's input and enough information to continue: the biased layer hands over its pre-activation sensitivity directly, a row division recovers that layer's input, the ReLU mask is readable from the recovered activation itself (an entry is live exactly when the recovered activation is positive), and the chain rule through the known weight matrix produces the next layer's sensitivity. The block below implements exactly that.

"""Proposition 3.1 executed: recover a network input through stacked FC layers from
gradients alone, then show why magnitude-invariance and uneven batch weighting matter."""

from __future__ import annotations

import numpy as np

Array = np.ndarray


def softmax(z: Array) -> Array:
    e = np.exp(z - z.max(axis=-1, keepdims=True))
    return e / e.sum(axis=-1, keepdims=True)


def forward(x: Array, W1: Array, W2: Array, W3: Array, b3: Array) -> tuple[Array, ...]:
    """W1, W2 unbiased; W3 biased. ReLU between. Returns activations."""
    z1 = x @ W1.T
    x2 = np.maximum(z1, 0)
    z2 = x2 @ W2.T
    x3 = np.maximum(z2, 0)
    z3 = x3 @ W3.T + b3
    return z1, x2, z2, x3, z3


def loss(x: Array, c: Array, W1: Array, W2: Array, W3: Array, b3: Array) -> float:
    p = softmax(forward(x, W1, W2, W3, b3)[-1])
    return float(-np.log(p[np.arange(len(c)), c]).mean())


def backward(x: Array, c: Array, W1: Array, W2: Array, W3: Array, b3: Array) -> dict:
    """Analytic grads. dz_l are the per-layer pre-activation sensitivities."""
    z1, x2, z2, x3, z3 = forward(x, W1, W2, W3, b3)
    n, C = len(x), W3.shape[0]
    dz3 = (softmax(z3) - np.eye(C)[c]) / n
    dz2 = (dz3 @ W3) * (z2 > 0)
    dz1 = (dz2 @ W2) * (z1 > 0)
    return {"W3": dz3.T @ x3, "b3": dz3.sum(0), "W2": dz2.T @ x2, "W1": dz1.T @ x,
            "dz3": dz3, "dz2": dz2, "dz1": dz1}


def numeric_dW1(x: Array, c: Array, W1: Array, W2: Array, W3: Array, b3: Array,
                eps: float = 1e-6) -> Array:
    out = np.zeros_like(W1)
    for i in range(W1.shape[0]):
        for j in range(W1.shape[1]):
            hi, lo = W1.copy(), W1.copy()
            hi[i, j] += eps
            lo[i, j] -= eps
            out[i, j] = (loss(x, c, hi, W2, W3, b3) - loss(x, c, lo, W2, W3, b3)) / (2 * eps)
    return out


def best_row(v: Array) -> int:
    return int(np.argmax(np.abs(v)))


def attack(g: dict, W2: Array, W3: Array) -> tuple[Array, list[int]]:
    """Proposition 3.1, executed. The attacker knows the weights and the shared
    gradients, and nothing else. Walk back layer by layer, no optimizer."""
    dz3 = g["b3"]                                   # biased layer hands over dL/dz directly
    i = best_row(dz3)
    x3 = g["W3"][i] / dz3[i]                        # input to layer 3

    dz2 = (dz3 @ W3) * (x3 > 0)                     # ReLU mask readable from x3 itself
    j = best_row(dz2)
    x2 = g["W2"][j] / dz2[j]                        # input to layer 2

    dz1 = (dz2 @ W2) * (x2 > 0)
    k = best_row(dz1)
    x0 = g["W1"][k] / dz1[k]                        # the network input
    return x0, [int((np.abs(v) > 0).sum()) for v in (dz3, dz2, dz1)]


rng = np.random.default_rng(7)
D, H1, H2, C = 24, 16, 12, 8
W1 = rng.normal(0, 0.6, (H1, D))
W2 = rng.normal(0, 0.6, (H2, H1))
W3 = rng.normal(0, 0.6, (C, H2))
b3 = rng.normal(0, 0.3, C)
x = rng.random((1, D))
c = np.array([3])

g = backward(x, c, W1, W2, W3, b3)

print("== 1. analytic gradients match central differences ==")
err = np.abs(g["W1"] - numeric_dW1(x, c, W1, W2, W3, b3)).max()
print(f"max |analytic dW1 - central difference| = {err:.3e}")
assert err < 1e-8, err

print("\n== 2. Proposition 3.1: walk the input out through 3 FC layers, no optimizer ==")
xr, live = attack(g, W2, W3)
print(f"live rows available per layer (3, 2, 1): {live} of ({C}, {H2}, {H1})")
print(f"input recovery MSE = {np.mean((xr - x[0]) ** 2):.3e}   max abs error = {np.abs(xr - x[0]).max():.3e}")
assert np.mean((xr - x[0]) ** 2) < 1e-20
print("recovered the network input, not just the last layer's input.")

print("\n== 3. magnitude invariance: why cosine replaced euclidean ==")
print("a trained network emits far smaller gradients. scale the whole gradient and compare.")
print(f"{'scale':>8}{'analytic MSE':>16}{'euclid ||g_s - g||':>22}{'cosine(g_s, g)':>18}")
ref = np.concatenate([g["W1"].ravel(), g["W2"].ravel(), g["W3"].ravel(), g["b3"]])
for s in (1.0, 1e-2, 1e-4, 1e-6):
    gs = {k: (v * s if k in ("W1", "W2", "W3", "b3") else v) for k, v in g.items()}
    xs, _ = attack(gs, W2, W3)
    flat = ref * s
    eu = float(np.linalg.norm(flat - ref))
    cs = float(flat @ ref / (np.linalg.norm(flat) * np.linalg.norm(ref)))
    print(f"{s:>8.0e}{np.mean((xs - x[0]) ** 2):>16.3e}{eu:>22.6f}{cs:>18.10f}")
print("the ratio cancels the scale exactly; euclidean distance does not.")

print("\n== 4. the technical condition, and what it costs to violate it ==")
blocked = 0
for t in range(2000):
    Wa = rng.normal(0, 0.6, (H1, D))
    Wb = rng.normal(0, 0.6, (H2, H1))
    Wc = rng.normal(0, 0.6, (C, H2))
    bc = rng.normal(0, 0.3, C)
    xt = rng.random((1, D))
    ct = np.array([int(rng.integers(C))])
    gt = backward(xt, ct, Wa, Wb, Wc, bc)
    xrt, livet = attack(gt, Wb, Wc)
    if min(livet) == 0:
        blocked += 1
    else:
        assert np.mean((xrt - xt[0]) ** 2) < 1e-18, np.mean((xrt - xt[0]) ** 2)
print(f"random networks with a fully dead layer (attack blocked): {blocked}/2000")

W2_dead = -np.abs(rng.normal(0, 0.6, (H2, H1))) - 5.0    # forces every relu in layer 2 off
g_dead = backward(x, c, W1, W2_dead, W3, b3)
with np.errstate(invalid="ignore", divide="ignore"):     # the division is meant to fail here
    _, live_dead = attack(g_dead, W2_dead, W3)
print(f"hand-built dead-ReLU layer: live rows per layer = {live_dead}, attack blocked = {min(live_dead) == 0}")
print(f"but that same network has dL/dW1 identically zero: {np.abs(g_dead['W1']).max():.1e}")
print("the only thing that blocks the walk-back also stops every layer below it from learning.")

print("\n== 5. why a batch average is not equal protection ==")
print("mixture weight for sample k at the attacker's row i is exactly the model's residual")
print("on class i: p_k[i] - 1[c_k == i]. Assert that identity, then look at its spread.")
B = 8
xb = rng.random((B, D))
cb = np.arange(B) % C
for scale, tag in ((0.6, "confident (large weights)"), (0.05, "near-uniform (small weights)")):
    Wa, Wb = W1 * scale / 0.6, W2 * scale / 0.6
    Wc, bc = W3 * scale / 0.6, b3 * scale / 0.6
    gb = backward(xb, cb, Wa, Wb, Wc, bc)
    i = best_row(gb["b3"])
    p = softmax(forward(xb, Wa, Wb, Wc, bc)[-1])
    residual = p[:, i] - (cb == i)
    assert np.abs(gb["dz3"][:, i] * B - residual).max() < 1e-12   # the identity
    share = np.abs(residual) / np.abs(residual).sum()
    print(f"\n  {tag}, attacker's row {i}")
    print("   share: " + " ".join(f"{v:.4f}" for v in np.sort(share)[::-1]))
    print(f"   most exposed sample holds {share.max():.1%}; least holds {share.min():.2%}"
          f"; ratio {share.max() / share.min():.1f}x")
print("\nexposure is set by the model's error on each sample, which the data owner does not")
print("control and cannot observe. averaging equalises nothing.")

Executed output:

== 1. analytic gradients match central differences ==
max |analytic dW1 - central difference| = 2.660e-10

== 2. Proposition 3.1: walk the input out through 3 FC layers, no optimizer ==
live rows available per layer (3, 2, 1): [8, 6, 6] of (8, 12, 16)
input recovery MSE = 1.091e-33   max abs error = 1.110e-16
recovered the network input, not just the last layer's input.

== 3. magnitude invariance: why cosine replaced euclidean ==
a trained network emits far smaller gradients. scale the whole gradient and compare.
   scale    analytic MSE    euclid ||g_s - g||    cosine(g_s, g)
   1e+00       1.091e-33              0.000000      1.0000000000
   1e-02       2.568e-33              4.479250      1.0000000000
   1e-04       3.667e-33              4.524042      1.0000000000
   1e-06       1.807e-32              4.524490      1.0000000000
the ratio cancels the scale exactly; euclidean distance does not.

== 4. the technical condition, and what it costs to violate it ==
random networks with a fully dead layer (attack blocked): 0/2000
hand-built dead-ReLU layer: live rows per layer = [8, 0, 0], attack blocked = True
but that same network has dL/dW1 identically zero: 0.0e+00
the only thing that blocks the walk-back also stops every layer below it from learning.

== 5. why a batch average is not equal protection ==
mixture weight for sample k at the attacker's row i is exactly the model's residual
on class i: p_k[i] - 1[c_k == i]. Assert that identity, then look at its spread.

  confident (large weights), attacker's row 7
   share: 0.2271 0.2245 0.1452 0.1412 0.1132 0.0566 0.0483 0.0440
   most exposed sample holds 22.7%; least holds 4.40%; ratio 5.2x

  near-uniform (small weights), attacker's row 2
   share: 0.5089 0.0702 0.0702 0.0702 0.0701 0.0701 0.0701 0.0701
   most exposed sample holds 50.9%; least holds 7.01%; ratio 7.3x

exposure is set by the model's error on each sample, which the data owner does not
control and cannot observe. averaging equalises nothing.

Three consequences follow.

Depth is not protection for a fully-connected stack. The walk-back recovered the network input, not merely the last layer's input, exactly, through three layers. The paper's phrasing is that there is "little defense-in-depth", and its empirical section extends the point to a ResNet-152 on ImageNet.

The analytic attack is magnitude-invariant for free. Scaling the entire gradient by 1e-6, which is roughly what training does to gradient norms, leaves reconstruction at machine precision because the recovery is a ratio, while the Euclidean distance between scaled and unscaled gradients saturates at the norm of the original. That is the same property the cosine objective was engineered to give the optimization attack. The magnitude carries the training state; the direction carries the image.

The proposition's escape hatch is not usable as a defense. Across 2000 random networks the condition never failed. A hand-built network with an entire ReLU layer forced off does block the walk-back, but the same construction drives the gradient of every layer below it to exactly zero, so nothing underneath can train. Blocking the attack this way and training the network are mutually exclusive.

Batching protects samples unequally, and the mechanism is arithmetic. The paper observes this empirically and calls it its most surprising finding: "the distortions arising from batching are not uniform ... some images are highly distorted and others only to an extend at which the pictured object can still be recognized easily". The block above supplies a mechanism. The recovered row of a batched gradient is a weighted mean whose weight for sample k is exactly the model's residual on the attacker's chosen class, p_k[i] - 1[c_k == i], asserted to 1e-12. That residual is the prediction error, so a sample the model already fits contributes almost nothing and a sample it gets wrong dominates. In the near-uniform-prediction regime one sample of eight holds 50.9% of the row. Batch size is therefore an average protection with no per-user floor, and the samples that leak most are the ones the model handles worst, which is not a property the data owner can see or control. Read as a mechanism for the paper's observation, not as a reproduction of its CIFAR-100 batch-of-100 experiment.

What federated averaging actually buys

All figures below are the paper's, on CIFAR-10 with an untrained ConvNet unless stated.

Multiple local steps do essentially nothing. At a learning rate of 1e-4, one local step gives 19.77 dB and one hundred local steps give 19.39 dB. The single failure case the authors could produce was a learning rate of 1e-1 at 5 steps, giving 4.96 dB, and they note that this step size "would lead to a divergent training update, and as such does not provide useful model updates". At 1e-2 with 5 steps the reconstruction is 23.74 dB, better than the single-step baseline.

Averaging over images degrades quality but does not stop leakage. Table 2, over the first 100 CIFAR-10 validation images:

Epochs Images Batch size PSNR
1 4 2 16.92 ± 2.10
1 8 2 14.66 ± 1.12
1 8 8 16.49 ± 1.02
5 1 1 25.05 ± 3.28
5 8 8 16.58 ± 0.96

Two things to read off this. Single-image reconstruction (25.05) is far ahead of every multi-image setting, so the batch is doing the work, not the epoch count. And comparing the two 8-image, batch-size-8 rows, 1 epoch at 16.49 and 5 epochs at 16.58, confirms that adding epochs does not help the defender.

At the extreme, a batch of 100 averaged gradients on CIFAR-100 with a ResNet32-10 still leaks: most recovered images are unrecognizable, but the paper publishes the five most recognizable and treats privacy as broken.

Architecture choices that move exposure

These are the levers the paper actually measured, and two of them point the opposite way from intuition.

  • Width increases exposure. Going from base width 16 to 128 on ResNet-18 raises average PSNR from 19.02 to 22.94. In the paper's words, greater width "increases the computational effort of the attacker, but does not provide greater security".
  • Depth barely matters. Average PSNR is 22.04 for ResNet-18 at width 64, 21.59 for ResNet-34, and 20.98 for ResNet-50, while ImageNet reconstruction through a ResNet-152 still succeeds.
  • Zero padding is a privacy risk. A conventional CNN using zero-padded convolutions allows high-quality recovery; a provably translation-invariant CNN using circular padding "makes the localization of objects impossible" because the object is separated from its position. This is the closest thing to an architectural mitigation in the paper, and it is a qualitative result shown in one inset figure, not a measured sweep.
  • Data augmentation costs the attacker location information. Networks trained with augmentation produce reconstructions where objects move or duplicate.
  • Training biases reconstructions toward class-typical features, which obscures fine detail and background while leaving the subject recoverable.

Treat the per-architecture PSNR values as noisy. The standard deviations run from 2.84 to 6.83 dB, and the single displayed image in the width comparison scores 17.24 at width 16 and 25.25 at width 128, an 8 dB gap where the corresponding averages differ by 3.9 dB. The direction of the width effect is the finding; the size of it is not well pinned down by five configurations.

The threat model is narrower than the result sounds

The attacker is honest-but-curious: it may store and process each user's update separately, but may not modify the architecture to suit the attack and may not send malicious global parameters. That restriction is what makes the result meaningful, and the paper flags in the same breath that "the attack is near-trivial under weaker constraints on the attacker". A server willing to send crafted weights is a materially different and easier adversary, so a deployment whose only defense is that the server is contractually well-behaved has no technical defense at all.

Don't-miss checklist

  • Do not count local steps as a privacy control. One hundred local steps scored within 0.4 dB of one step.
  • Do not quote a batch size as a per-user privacy guarantee. Protection is uneven by construction, and the worst-fit samples leak most.
  • If a configuration appears to defend, check that it is still a trainable configuration. The learning rate that broke the attack also diverges, and the dead-layer construction that blocks the analytic walk-back also zeroes the gradients below it.
  • Decide explicitly whether your threat model is honest-but-curious or malicious. The published results assume the former and the authors say the latter is much easier.
  • If you are choosing an architecture under this threat, prefer narrower over wider, and know that depth buys nothing.
  • Treat differential privacy with an accounted budget as the only mechanism the paper endorses: it concludes that "provable differential privacy remains the only way to guarantee security".
  • For where a gradient crosses a trust boundary in a cluster at all, and what each defense costs in accuracy, use gradient leakage in distributed training.

Failure modes

  • Assuming aggregation composes into safety. Averaging over steps, images, and epochs each reduce quality somewhat; none of them removed leakage in any published configuration.
  • Reading an average PSNR as a per-sample guarantee. The executed block shows one sample of eight holding half the mixture weight. Batch-of-100 leaks its five most recognizable images.
  • Assuming a wider network is a harder target. Measured the other way: wider is easier.
  • Assuming the cosine attack strictly dominates the older one. On untrained shallow smooth networks the Euclidean attack scores 46.25 PSNR against 18.00. The new attack wins everywhere realistic, not everywhere.
  • Citing circular padding as a fix. It defeats object localization in one inset figure. It is not a measured defense across architectures, and it changes the network.
  • Treating an honest-but-curious result as the worst case. It is the best case for the defender among realistic servers.

Open questions & validation

  • The executed block covers fully-connected stacks, which is exactly where the closed form exists. It says nothing about convolutional trunks, where the paper needs its optimizer. Both halves are needed and neither substitutes for the other.
  • The batch-weight mechanism is derived for the final classification layer under cross-entropy. Whether it explains the specific pattern of which images survived in the paper's batch-of-100 ResNet32-10 experiment was not tested here, and would need that experiment re-run.
  • No attack was reproduced. Reproducing the CNN and ImageNet results needs PyTorch and a GPU; this environment has numpy only.
  • Every result is image classification. The paper's own caution that image data may be especially vulnerable is worth taking literally, and gradient inversion against language-model training is not covered by this source.
  • The paper predates practical secure aggregation deployments and modern DP-SGD tooling. Its conclusion that differential privacy is the only guarantee stands, but the cost figures it cites for that guarantee should be re-checked against current implementations.

References

Related: gradient leakage in distributed training | security, isolation and multi-tenancy | GPU confidential computing | geo-distributed training placement | DiLoCo | distributed training | privacy-aware split inference