Controlling Pariticipation in Federated Learning with Feedback

FedBack brings control theory to federated learning: we cast client participation as a discrete-time dynamical system and control it with an integral feedback law layered on top of ADMM. This deterministic, event-triggered scheme reduces communication and computation by up to 50 % on MNIST and CIFAR-10 benchmarks while ensuring global-convergence guarantees and eliminating client-drift.

This work was carried out with Güner Dilsad Er and Michael Muehlebach as part of my MEng dissertation at the Max-Planck Institute for Intelligent Systems. The paper has been accepted to L4DC 2025 and can be viewed on arxiv.

Results

FedBack is capable of achieving equivalent validation accuracies as centralised training routines (Theorem 5 in paper) and converges in significantly fewer communication rounds than FedAVG, FedProx, and FedADMM.

Training curves for different participation rates.

Total number of client participation events required to reach target accuracy.

See the paper for a detailed discussion of results.

Running Experiments

To get started, you can set up your workspace by initialising a new conda environment with Python>=3.11 and running:

pip install -e .

All experiments are run through main.py in the experiments directory. For example, running FedBack using CIFAR-10 data with a communication rate of 10% can be done by:

python main.py --back --cifar --rate 0.1

If you would like to experiment with a benchmark, run one of the following,

python main.py --avg --cifar --rate 0.1
python main.py --prox --cifar --rate 0.1
python main.py --admm --cifar --rate 0.1

Running main.py will create multiple new directories in experiments/ such as figure_data/ and images/. These directories are used storing and visualising data collected during experiments. This data can be used to reproduce the training curves and results in our paper by running:

python plotting.py

Note: Run the MNIST experiments by simply replacing --cifar with --mnist.

Citing

If you found you used any code or found some ideas in our paper useful, please consider citing our paper:

@article{cummins2024controlling,
  title={Controlling Participation in Federated Learning with Feedback},
  author={Cummins, Michael and Er, Guner Dilsad and Muehlebach, Michael},
  journal={arXiv preprint arXiv:2411.19242},
  year={2024}
}