Thumbnail Image
Item

A Remedy for Heterogeneous Data: Clustered Federated Learning with Gradient Trajectory

Liu, Ruiqi
; 
Yu, Songcan
; 
Lan, Linsi
; 
Wang, Junbo
; 
Kant, Krishna
; 
Calleja, Neville
Citations
Altmetric:
Genre
Journal article
Date
2024-12-04
Advisor
Committee member
Group
Department
Computer and Information Sciences
Subject
Federated learning (FL)
 Clustering
 Heterogeneous data
 Distributed system
Permanent link to this record
https://scholarshare.temple.edu/handle/20.500.12613/11658
Collections
Faculty/Researcher Works
Show all metadata
Files
Thumbnail Image
Kant-JournalArticle-2024-12.pdf
Adobe PDF, 8.76 MB
Research Projects
Organizational Units
Journal Issue
DOI
https://doi.org/10.26599/bdma.2024.9020065
Abstract
Federated Learning (FL) has recently attracted a lot of attention due to its ability to train a machine learning model using data from multiple clients without divulging their privacy. However, the training data across clients can be very heterogeneous in terms of quality, amount, occurrences of specific features, etc. In this paper, we demonstrate how the server can observe data heterogeneity by mining gradient trajectories that the clients compute from a two-dimensional mapping of high-dimensional gradients computed by each client from its bottom layer. Based on these ideas, we propose a new clustered federated learning with gradient trajectory method, called CFLGT, which dynamically clusters clients together based on the gradient trajectories. We analyze CFLGT both theoretically and experimentally to show that it overcomes several drawbacks of mainstream Clustered Federated Learning (CFL) methods and outperforms other baselines.
Description
Citation
R. Liu, S. Yu, L. Lan, J. Wang, K. Kant and N. Calleja, "A Remedy for Heterogeneous Data: Clustered Federated Learning with Gradient Trajectory," in Big Data Mining and Analytics, vol. 7, no. 4, pp. 1050-1064, December 2024, doi: 10.26599/BDMA.2024.9020065. keywords: {Data privacy;Federated learning;Computational modeling;Training data;Big Data;Data models;Trajectory;Servers;Federated Learning (FL);clustering;heterogeneous data;distributed system},
Citation to related work
Tsinghua University Press
Has part
Big Data Mining and Analytics, Vol. 7, Iss. 4
ADA compliance
Embedded videos
License
Attribution CC BY
cb