Adaptive Regularization and Resilient Estimation in Federated Learning
Article
Article Title | Adaptive Regularization and Resilient Estimation in Federated Learning |
---|---|
ERA Journal ID | 36548 |
Article Category | Article |
Authors | Uddin, Md Palash, Xiang, Yong, Zhao, Yao, Ali, Mumtaz, Zhang, Yushu and Gao, Longxiang |
Journal Title | IEEE Transactions on Services Computing |
Journal Citation | 17 (4), pp. 1369-1381 |
Number of Pages | 14 |
Year | 2024 |
Publisher | IEEE (Institute of Electrical and Electronics Engineers) |
Place of Publication | United States |
ISSN | 1939-1374 |
Digital Object Identifier (DOI) | https://doi.org/10.1109/TSC.2023.3332703 |
Web Address (URL) | https://ieeexplore.ieee.org/document/10316585 |
Abstract | Federated Learning (FL) is an emerging research area that produces a globally trained model using numerous local users' data and maintains their privacy. Heterogeneous or non-Independent and Identically Distributed ( non-IID) data affect the global model's convergence and, therefore, cause high communication costs. These are because traditional FL approaches often disregard an adaptive regularized objective for the user-side training and utilize conventional arithmetic mean on the locally trained models for the server-side aggregation. To alleviate these issues, we propose a novel FL scheme in this paper. In particular, we propose an adaptive regularization approach to add to the classical objective function of the users' local models during training and a resilient estimation approach to the locally trained models during aggregation. The adaptive regularization approach is derived using the users' local and global performance diversification while the resilient estimation scheme uses a modified geometric mean aggregation over the local models' parameters. We provide consolidated theoretical results and perform extensive experiments on the IID and non-IID settings of MNIST, CIFAR-10, and Shakespeare datasets with various deep networks. The results manifest that our FL scheme outperforms the state-of-the-art approaches in terms of communication speedup, test-set performance, training convergence stability, and resiliency against attacks. |
Keywords | Training; Adaptation models; Convergence; Data models; Linear programming; Servers; Computational modeling |
Contains Sensitive Content | Does not contain sensitive content |
ANZSRC Field of Research 2020 | 461199. Machine learning not elsewhere classified |
Public Notes | The accessible file is the accepted version of the paper. Please refer to the URL for the published version. |
Byline Affiliations | Deakin University |
UniSQ College (Pathways) | |
Centre for Sustainable Agricultural Systems | |
Centre for Applied Climate Sciences | |
Al-Ayen University, Iraq | |
Nanjing University of Aeronautics and Astronautics, China | |
Qilu University of Technology, China |
https://research.usq.edu.au/item/z3024/adaptive-regularization-and-resilient-estimation-in-federated-learning
Download files
Accepted Version
Adaptive_Regularization_and_Resilient_Estimation_in_Federated_Learning.pdf | ||
File access level: Anyone |
71
total views17
total downloads1
views this month1
downloads this month