Decentralized personalized federated learning: Lower bounds and optimal algorithm for all personalization modes

Authors

Abdurakhmon Sadiev, Moscow Institute of Physics and Technology (MIPT), Russian Federation & Mohamed bin Zayed University of Artificial Intelligence
Ekaterina Borodich, Moscow Institute of Physics and Technology (MIPT), Russian Federation & HSE University, Russian FederationFollow
Aleksandr Beznosikov, Moscow Institute of Physics and Technology (MIPT), Russian Federation & Mohamed bin Zayed University of Artificial Intelligence & HSE University, Russian FederationFollow
Darina Dvinskikh, HSE University, Russian Federation
Saveliy Chezhegov, Moscow Institute of Physics and Technology (MIPT), Russian Federation
Rachael Tappenden, University of Canterbury, New Zealand
Martin Takac, Mohamed bin Zayed University of Artificial IntelligenceFollow
Alexander Gasnikov, Moscow Institute of Physics and Technology (MIPT), Russian Federation & Mohamed bin Zayed University of Artificial Intelligence & HSE University, Russian Federation & Institute for Information Transmission Problems RAS, Russian Federation

Document Type

Article

Publication Title

EURO Journal on Computational Optimization

Abstract

This paper considers the problem of decentralized, personalized federated learning. For centralized personalized federated learning, a penalty that measures the deviation from the local model and its average, is often added to the objective function. However, in a decentralized setting this penalty is expensive in terms of communication costs, so here, a different penalty — one that is built to respect the structure of the underlying computational network — is used instead. We present lower bounds on the communication and local computation costs for this problem formulation and we also present provably optimal methods for decentralized personalized federated learning. Numerical experiments are presented to demonstrate the practical performance of our methods. © 2022 The Authors

DOI

10.1016/j.ejco.2022.100041

Publication Date

9-13-2022

Keywords

Accelerated algorithms, Decentralized optimization, Distributed optimization, Federated learning, Lower and upper bounds

Comments

Archived with thanks to Elsevier ScienceDirect

License: CC BY-NC-ND 4.0

Uploaded 01 February 2023

Recommended Citation

A. Sadiev, et al, "Decentralized personalized federated learning: Lower bounds and optimal algorithm for all personalization modes", EURO Journal on Computational Optimization, vol. 10, September 2022, doi:10.1016/j.ejco.2022.100041

Additional Links

Publisher's link: https://www.sciencedirect.com/science/article/pii/S219244062200017X?via%3Dihub