Federated learning: the data never leaves; the distribution problems do

Federated learning (the article also calls it collaborative learning) trains one model across many entities — clients — while their data stays decentralised. The pitch is privacy-adjacent: data minimisation, access rights, legal or economic inability to share raw data. But the article's own defining characteristic is not privacy, it is data heterogeneity: because the data never gets pooled, each client's samples may be far from identically distributed. (Everything below is summarised from the source at the bottom — edited, not verified.)

What separates it from ordinary distributed learning

Distributed learning parallelises compute over datacentre nodes and assumes local datasets are i.i.d. and roughly equal-sized. Federated learning drops both assumptions — client datasets are typically heterogeneous and their sizes may span orders of magnitude — and adds a worse assumption about the clients themselves: they are smartphones and IoT devices on Wi-Fi and batteries, so they fail and drop out in ways a rack of servers does not.

The round

One federated round: the server sends the current global model to a selected subset of clients; each trains locally; each returns an update; the server aggregates into one global update and applies it. FedAvg lets clients run many local batches and exchange weights rather than gradients — cutting communication, and (per the article) not hurting the resulting model versus plain federated SGD. FedProx adds a proximal term to each local objective to keep heterogeneous local updates from wandering too far from the global model. Variants built on adaptive optimisers (ADAM/AdaGrad-style) reportedly outperform plain FedAvg — the article's claim, relayed.

Where it breaks — the honest list

The article's limitations are mostly not about privacy:

• Hiding data makes poisoning harder to audit, not impossible. The article lists it flatly: hiding training data might let attackers inject backdoors into the global model — and with no access to global training data, unwanted biases are harder to even detect.
• Node failures can partially or wholly lose updates and still shift the global model.
• Client distributions drift over time (temporal heterogeneity); labels may be missing client-side; platforms differ.
• Communication is the tax: parameter exchange over constrained links is bandwidth-hungry — hence active research in sparsification and quantisation of updates before they are sent (see model quantisation for the weight-side version of the same trick).
• Governance. Most frameworks need a central coordinating server — which raises who controls infrastructure, who owns the model, who audits updates. And as a consortium grows, accuracy flattens while every late joiner free-rides on the aggregate; the article flags contribution-and-reward rules as an open problem.

The use cases follow the pattern "need a bigger dataset than yours, cannot share the data itself": defence, telecoms, IoT, pharma, self-driving fleets.

Source: Wikipedia, "Federated learning", read 2026-09-09. The limitation bullets are the article's list, lightly ordered; nothing here was checked against a live federated deployment. Edited, not verified. Related: Pseudonyms keep the aggregate and drop the person — the same trade shape, keeping shared signal while withholding identifying detail; In-context learning, the opposite extreme: all adaptation, no parameters exchanged.