Differential Private Stochastic Gradient Descent (DP-SGD) is the most important privacy technique most ML engineers have never shipped. Here's how it works, what it costs, and when to use it.
The Problem We're Actually Solving
The last article covered model inversion, how an attacker can reconstruct training data by querying your model's outputs. That attack exists because of something more fundamental: models memorize.
Not just statistically, literally. Language models can be prompted to recite verbatim text from their training data. Classification models trained on medical records encode patient-level signals in their weights. The deeper the model trains, the more it overfits to individual examples. This is what enables membership inference: given a record and a model, can you determine whether that record was in the training set? On many production models, yes, with well above random accuracy.
Model inversion defences are applicable at inference time. It limits what information leaks through the API. DP-SGD defends at training time. It limits what information the model was ever allowed to learn.
What Differential Privacy Actually Guarantees
Differential privacy is a mathematical property of an algorithm, not a product claim.
A training algorithm is (ε, δ) differentially private if, for any two datasets (D1 and D2) that differ by a single record, the guarantee is that the probability of the algorithm producing different output is bounded by e^ε.
P(Algorithm(D1) = output) / P(Algorithm(D2) = output) ≤ e^ε
The probability of failure of the above guarantee is δ.
ε controls the strength of the guarantee. Smaller ε means stronger privacy, at the cost of more noise and lower utility. In practice, ε values of 1 to 10 are common for production systems. Below 1 is considered strong. Above 10 is closer to noise-in-the-wind than formal privacy.
δ is a small failure probability, typically set to 1/n, where n is the training set size.
This guarantee is about the training process, not the outputs. A DP-trained model may still make confident predictions. The change is that no individual training example has disproportionate influence on what the model learns.
Put simply, adding or removing any single person's data changes what the model learns by a bounded, measurable amount.
How DP-SGD Works
Standard SGD accumulates gradients across a mini-batch and updates the model. DP-SGD makes two modifications before that update:
Step 1: Per-sample gradient clipping.
Instead of accumulating a batch gradient, compute a separate gradient per training example. Clip each per-sample gradient to a maximum L2 (the straight-line distance/length of a vector from the origin) norm C. This bounds the influence any single example can have on the update.
Step 2: Gaussian noise injection.
Add noise sampled from N(0, σ²C²I) to the clipped gradient sum. σ is the noise multiplier, the primary parameter controlling how much privacy you buy per step.
The update then proceeds normally.
Pseudocode — see Opacus for the production implementation:
for x, y in dataloader:
per_sample_grads = compute_per_sample_gradients(model, x, y)
# Clip
norms = [g.norm() for g in per_sample_grads]
clipped = [g / max(1, norm / C) for g, norm in zip(per_sample_grads, norms)]
# Aggregate and add noise
summed = sum(clipped)
noisy = summed + torch.normal(0, sigma * C, size=summed.shape)
# Standard gradient step
model.update(noisy / batch_size)
The key insight is that the noise is calibrated to the clipping bound C, not the raw gradient magnitudes. Clipping first bounds the sensitivity of the computation; noise then masks that bounded sensitivity. The Gaussian mechanism guarantees that the noisy gradient satisfies (ε, δ) DP for a given (σ, C, batch_size) combination.
Privacy Accounting - Law of Diminishing Returns
A single noisy gradient step gives you some privacy. Training runs for thousands of steps. Privacy costs compose.
The naive composition theorem says privacy degrades linearly with the number of steps — which would require enormous noise to achieve meaningful guarantees over a full training run. The moments accountant and its successor, Rényi Differential Privacy (RDP), give a much tighter bound through a mathematical technique called privacy amplification by subsampling.
Because each mini-batch is a random sample of the full training set, any given example is only included in a fraction:
Sampling rate q = batch_size / dataset_size
This probabilistic inclusion significantly reduces the per-step privacy cost. The moments accountant tracks how privacy degrades across the full training run using this tighter bound, allowing you to achieve meaningful ε values at practical noise levels.
In practice, you specify target ε and δ, set your batch size and number of epochs, and use Opacus or Google's DP-accounting library to compute the σ required. This replaces trial and error with a calibrated calculation.
The Utility Cost Is Real But Manageable
Adding noise to gradients makes training harder. Loss curves are noisier. Convergence is slower. Final accuracy is lower. This is not a bug you can engineer around; it is the mechanism by which privacy is purchased.
The severity of this tradeoff depends heavily on two things:
- Model scale: Large models absorb noise better than small ones. A model with millions of parameters is less sensitive to noisy gradient updates than a shallow network. This is counterintuitive but consistently observed: more capacity means more redundancy, and redundancy tolerates noise. Google's work training DP language models found that larger models achieved meaningful ε values at much smaller accuracy gaps than smaller models trained on the same data.
- Pretraining: The most practically important insight in private fine-tuning research: train your feature representations on public data, then fine-tune on private data with DP-SGD. The private training run starts from a strong initialization, so the noisy gradient steps have much less work to do. In practice, this recovers most of the utility lost to DP noise. Fine-tuning a pretrained transformer on private medical records with ε = 3 is feasible. Training one from scratch the same way is not.
Shipping It in Practice
Opacus (Meta) is the standard PyTorch implementation. It handles per-sample gradient computation, clipping, noise injection, and privacy accounting in a wrapper around your existing training loop.
from opacus import PrivacyEngine
model = MyModel()
optimizer = torch.optim.Adam(model.parameters())
data_loader = DataLoader(dataset, batch_size=256)
privacy_engine = PrivacyEngine()
model, optimizer, data_loader = privacy_engine.make_private_with_epsilon(
module=model,
optimizer=optimizer,
data_loader=data_loader,
epochs=20,
target_epsilon=3.0,
target_delta=1e-5,
max_grad_norm=1.0, # this is C
)
# Your training loop is unchanged from here
for x, y in data_loader:
loss = criterion(model(x), y)
loss.backward()
optimizer.step()
epsilon = privacy_engine.get_epsilon(delta=1e-5)
print(f"Training completed with ε = {epsilon:.2f}")
The main decision points are max_grad_norm (clipping bound), target_epsilon, and batch size. Larger batches improve the signal-to-noise ratio per step. Tighter ε requires larger σ, which requires larger batches or fewer epochs to hit a target accuracy.
When to Use DP-SGD
DP-SGD is not the right tool for every problem. It adds implementation complexity, increases compute requirements (per-sample gradients are more expensive), and degrades model utility. You reach for it when the threat model justifies it:
- Training on medical, financial, or behavioral records where membership inference is a meaningful risk
- Regulatory environments where formal privacy guarantees are required (GDPR, HIPAA, COPPA)
- Federated learning deployments where the aggregation server must not learn individual client updates
- Any system where the model will be made public or where the API is queryable at scale
Google has used DP-SGD in production since at least 2017 for training models on sensitive user data in Gboard and Chrome. Apple uses local differential privacy for on-device data collection. These are not experimental deployments. They are production systems serving hundreds of millions of users under formal privacy guarantees.
Conclusion
DP-SGD is the mechanism by which the privacy guarantees in differential privacy are actually applied to machine learning. It is not magic, and it is not free. It requires you to specify, upfront, exactly how much privacy you are providing, which is uncomfortable because it forces honesty about tradeoffs that most teams prefer to leave vague.
That discomfort is part of the point. "We anonymized the data" is an informal claim. "We trained with ε = 3 differential privacy" is a mathematical statement with defined, auditable properties. If your model is trained on data that carries real privacy expectations, the second kind of claim is the one you should be prepared to make.
The model does not have to remember. You can choose how much it forgets.