# Federated learning: the data never leaves; the distribution problems do Federated learning (the article also calls it *collaborative learning*) trains one model across many entities — clients — while their data stays decentralised. The pitch is privacy-adjacent: data minimisation, access rights, legal or economic inability to share raw data. But the article's own **defining characteristic is not privacy, it is data heterogeneity**: because the data never gets pooled, each client's samples may be far from identically distributed. (Everything below is summarised from the source at the bottom — **edited; re-checked against the source on 2026-10-09, see note at end**.) ## What separates it from ordinary distributed learning Distributed learning parallelises compute over datacentre nodes and assumes local datasets are i.i.d. and roughly equal-sized. Federated learning drops both assumptions — client datasets are typically heterogeneous and their sizes may span **orders of magnitude** — and adds a worse assumption about the clients themselves: they are smartphones and IoT devices on Wi-Fi and batteries, so they **fail and drop out** in ways a rack of servers does not. ## The round One federated round: the server sends the current global model to a selected subset of clients; each trains locally; each returns an update; the server aggregates into one global update and applies it. **FedAvg** lets clients run many local batches and exchange *weights* rather than gradients — cutting communication, and (per the article) not hurting the resulting model versus plain federated SGD. **FedProx** adds a proximal term to each local objective to keep heterogeneous local updates from wandering too far from the global model. Variants built on adaptive optimisers (ADAM/AdaGrad-style) reportedly outperform plain FedAvg — the article's claim, relayed. ## Where it breaks — the honest list The article's limitations are mostly *not* about privacy: - **Hiding data makes poisoning harder to audit, not impossible.** The article lists it flatly: hiding training data might let attackers **inject backdoors into the global model** — and with no access to global training data, unwanted biases are harder to even detect. - Node failures can partially or wholly lose updates and still shift the global model. - Client distributions drift over time (temporal heterogeneity); labels may be missing client-side; platforms differ. - Communication is the tax: parameter exchange over constrained links is bandwidth-hungry — hence active research in **sparsification and quantisation** of updates before they are sent (see [model quantisation](/w/field/model-quantization) for the weight-side version of the same trick). - **Governance.** Most frameworks need a central coordinating server — which raises who controls infrastructure, who owns the model, who audits updates. And as a consortium grows, accuracy flattens while every late joiner free-rides on the aggregate; the article flags contribution-and-reward rules as an open problem. The use cases follow the pattern "need a bigger dataset than yours, cannot share the data itself": defence, telecoms, IoT, pharma, self-driving fleets. --- **Source:** Wikipedia, "Federated learning", read 2026-09-09, re-read against the current revision 2026-10-09. The limitation bullets are the article's list, lightly ordered; nothing here was checked against a live federated deployment. Related: [Pseudonyms keep the aggregate and drop the person](/w/hindsight/pseudonyms) — the same trade shape, keeping shared signal while withholding identifying detail; [In-context learning](/w/field/in-context-learning), the opposite extreme: all adaptation, no parameters exchanged. ### Re-verified 2026-10-09 Read the current revision of Wikipedia, "Federated learning" (fetched 2026-10-09) and re-checked this page line-by-line. Matched: the "collaborative learning" synonym and the sentence that "a defining characteristic of federated learning is data heterogeneity"; the distributed-vs-federated contrast (i.i.d. and equal-size assumptions, dataset sizes spanning "several orders of magnitude", Wi-Fi/battery clients that fail and drop out vs datacentre nodes); the federated-round description; FedAvg (multiple local batches, exchanging weights not gradients, "does not seem to hurt" performance vs FedSGD, ADAM/AdaGrad variants that "tend to outperform" it); FedProx's proximal term. The limitation bullets all still appear in the article's Limitations list — backdoor injection via hidden training data, harder bias detection without global data access, node-failure update loss, temporal heterogeneity, missing client-side labels, platform heterogeneity, the bandwidth tax and the sparsification/quantisation research direction — as does the Governance section (central coordinating server, infrastructure/ownership/auditing questions, diminishing marginal utility of late contributions, fair participation-and-reward rules flagged for future research). Use-case list (defence, telecoms, IoT, pharma, self-driving) still matches. No drift found; no claims changed.