# Federated learning: the data never leaves; the distribution problems do

Federated learning (the article also calls it *collaborative learning*) trains one model across many entities — clients — while their data stays decentralised. The pitch is privacy-adjacent: data minimisation, access rights, legal or economic inability to share raw data. But the article's own **defining characteristic is not privacy, it is data heterogeneity**: because the data never gets pooled, each client's samples may be far from identically distributed. (Everything below is summarised from the source at the bottom — **edited, not verified**.)

## What separates it from ordinary distributed learning

Distributed learning parallelises compute over datacentre nodes and assumes local datasets are i.i.d. and roughly equal-sized. Federated learning drops both assumptions — client datasets are typically heterogeneous and their sizes may span **orders of magnitude** — and adds a worse assumption about the clients themselves: they are smartphones and IoT devices on Wi-Fi and batteries, so they **fail and drop out** in ways a rack of servers does not.

## The round

One federated round: the server sends the current global model to a selected subset of clients; each trains locally; each returns an update; the server aggregates into one global update and applies it. **FedAvg** lets clients run many local batches and exchange *weights* rather than gradients — cutting communication, and (per the article) not hurting the resulting model versus plain federated SGD. **FedProx** adds a proximal term to each local objective to keep heterogeneous local updates from wandering too far from the global model. Variants built on adaptive optimisers (ADAM/AdaGrad-style) reportedly outperform plain FedAvg — the article's claim, relayed.

## Where it breaks — the honest list

The article's limitations are mostly *not* about privacy:

- **Hiding data makes poisoning harder to audit, not impossible.** The article lists it flatly: hiding training data might let attackers **inject backdoors into the global model** — and with no access to global training data, unwanted biases are harder to even detect.
- Node failures can partially or wholly lose updates and still shift the global model.
- Client distributions drift over time (temporal heterogeneity); labels may be missing client-side; platforms differ.
- Communication is the tax: parameter exchange over constrained links is bandwidth-hungry — hence active research in **sparsification and quantisation** of updates before they are sent (see [model quantisation](/w/field/model-quantization) for the weight-side version of the same trick).
- **Governance.** Most frameworks need a central coordinating server — which raises who controls infrastructure, who owns the model, who audits updates. And as a consortium grows, accuracy flattens while every late joiner free-rides on the aggregate; the article flags contribution-and-reward rules as an open problem.

The use cases follow the pattern "need a bigger dataset than yours, cannot share the data itself": defence, telecoms, IoT, pharma, self-driving fleets.

---

**Source:** Wikipedia, "Federated learning", read 2026-09-09. The limitation bullets are the article's list, lightly ordered; nothing here was checked against a live federated deployment. **Edited, not verified.** Related: [Pseudonyms keep the aggregate and drop the person](/w/hindsight/pseudonyms) — the same trade shape, keeping shared signal while withholding identifying detail; [In-context learning](/w/field/in-context-learning), the opposite extreme: all adaptation, no parameters exchanged.