Dynamic Adaptive Collaboration for Personalized Federated Learning

Abstract

With the rapid development of AI-driven personalized services, model training increasingly depends on highly sensitive user-side data, such as location, social behaviour, and biometric information. These data not only exhibit pronounced non-independent and identically distributed (non-IID) characteristics but also pose serious privacy risks when processed centrally. Achieving efficient personalized modelling while preserving data locality and privacy has thus become a critical challenge in the evolution of personalized AI. In recent years, personalized federated learning (PFL) has gained significant attention for its strong performance in addressing non-IID data challenges. However, existing approaches often fall short in effectively balancing collaborative efficiency with personalization. To overcome this limitation, we propose FedDAC, a dynamically adaptive, collaboration-enhanced personalized federated learning method. By quantitatively assessing the responsiveness of each parameter to non-IID data, FedDAC dynamically selects collaborative clients, ensuring effective cooperation while retaining personalized feature information. Extensive experiments on four benchmark datasets (EMNIST, CIFAR-10, CIFAR-100, and Tiny ImageNet) under two pathological non-IID settings show that FedDAC consistently outperforms strong baselines. It improves accuracy by 1.5–3.2% on average, reaching 5.9% on highly heterogeneous tasks. The source code is publicly available at https://github.com/Lixinqin9/FedDAC-MAIN.

Keywords

Personalized federated learning non-independent and identically distributed statistical heterogeneity collaborative learning dynamic adaptation

Introduction

Federated learning (FL) is an emerging paradigm of distributed machine learning that enables decentralized clients to collaboratively train models without exposing their private data (McMahan et al., 2017). This approach has been widely applied in various domains such as computer vision (Himeur et al., 2023), finance (Long et al., 2020; Pei et al., 2024), and healthcare (Rieke et al., 2020; Sadilek et al., 2021). Despite its potential, FL faces significant challenges due to heterogeneity in client data (Li et al., 2021), which adversely affects global model training (Gong et al., 2021). In particular, non-independent and identically distributed (non-IID) data (Zhang et al., 2021) lead to differences in local tasks across clients, making it difficult for the global model to perform well on all clients (Huang et al., 2021).

To address the issue of data heterogeneity, personalized federated learning (PFL) (Arivazhagan et al., 2019; Fallah et al., 2020; Li et al., 2021; Tang et al., 2021) has emerged as a promising solution. PFL allows each client to train a personalized model that better adapts to its local data distribution, mitigating the impact of non-IID data while still enabling knowledge sharing among clients (Pei et al., 2024). Existing PFL methods can be broadly categorized into two groups: one line of work focuses on extracting personalized features during local model updates to better fit each client’s data distribution, while the other achieves inter-client collaboration by sharing a subset of global model parameters (Collins et al., 2021; Dinh et al., 2020). Although these approaches have improved personalization, they still face challenges such as high communication overhead and privacy leakage risks (Chen et al., 2024; Hu et al., 2021; Li et al., 2021).

At the same time, over-reliance on personalization can slow global model convergence or hinder its adaptability across clients. Conversely, placing too much emphasis on collaboration may reduce the model’s ability to handle data heterogeneity, thereby compromising personalization. Many current PFL methods prioritize providing independent personalized models for each client while overlooking the importance of effective collaboration in global model training.

To address the imbalance between collaboration and personalization in existing PFL methods, this paper proposes a novel framework called FedDAC (federated learning with dynamic adaptive collaboration). This framework introduces a dynamic collaboration strategy based on a responsiveness evaluation mechanism, which assesses each client’s collaboration potential by analyzing key changes in parameter updates. A collaboration criterion is then derived from these changes to dynamically select a subset of clients for participation in global model sharing. This strategy allows for efficient control over which clients require personalized customization and optimization during aggregation.

To measure the collaboration standard among clients, a mask-based marking approach is employed, which not only reduces communication overhead but also helps mitigate privacy risks. Furthermore, an adjustable weighted aggregation strategy is integrated. By considering differences in data distributions and parameter responsiveness across clients, this strategy assigns similarity-based weights to optimize the aggregation process. This ensures that each client benefits from global collaboration while retaining the ability to build a personalized model tailored to its own data.

We propose a novel dynamic collaboration strategy for PFL, which considers both the data distribution differences among clients and the responsiveness of parameters to non-IID data. This strategy dynamically adjusts client participation, enabling more active collaboration while avoiding the negative effects of non-IID data.

We design a flexible and personalized adjustable weighting strategy under highly heterogeneous data settings. This strategy enables client-specific adaptation and demonstrates that dynamic weight adjustment can significantly enhance personalization performance.

We conduct extensive experiments on four real-world datasets, demonstrating that FedDAC consistently outperforms existing methods under two heterogeneous FL settings.

Related Work

Personalized Federated Learning (Arivazhagan et al., 2019; Fallah et al., 2020; Li et al., 2021; Tang et al., 2021) has emerged as an effective solution to address the data heterogeneity challenges in Federated Learning. PFL aims to train a personalized model for each client to better fit the local data distribution (Zhang et al., 2020). The core idea of PFL is to encourage collaboration among clients while preserving privacy, thereby improving personalization performance for each individual client. Current approaches in PFL can be broadly categorized into two groups: data-based PFL and model-based PFL.

Data-based PFL focuses on reducing inter-client data heterogeneity to achieve personalization. These methods typically involve sharing a global dataset or exchanging private statistical information (Collins et al., 2021) across clients to minimize distributional differences. However, this comes with a significant risk of privacy leakage (Hu et al., 2021), especially in sensitive applications where such data sharing may violate privacy policies. Therefore, while data-based PFL methods can be effective, they are limited in their ability to ensure privacy. In contrast, model-based PFL (Marfoq et al., 2021; Yoon et al., 2021) addresses heterogeneity by customizing the models of individual clients without sharing raw data. This category can be further divided into single-model PFL and multi-model PFL (Tang et al., 2021).

Single-model PFL extends traditional FL algorithms to achieve personalized training. For example, FedProx (Li et al., 2020) introduces a proximal term in the client’s optimization objective to alleviate the impact of non-IID data. FedAvg FT and FedProx FT fine-tune global models generated by FedAvg (McMahan et al., 2017; Wang et al., 2019) and FedProx, respectively, to obtain personalized models for each client. These methods aim to tackle the challenges posed by non-IID data and enable clients to retain a customized model. However, single-model approaches are inherently limited in personalization capability, as a single global model often struggles to adapt to the diverse and heterogeneous nature of client data.

Multi-model PFL (Li et al., 2023), on the other hand, trains multiple models across clients to better meet personalization needs. For instance, FedPer (Arivazhagan et al., 2019) is specifically designed for non-IID settings and is particularly effective when client data distributions differ significantly. It assigns an independent personalized model to each client. ClusterFL (Ghosh et al., 2020; Li et al., 2021) groups clients into clusters based on loss values or gradients and trains a shared model within each cluster. This clustering strategy allows targeted optimization based on client similarity but may suffer when the clustering method is ineffective, particularly in highly heterogeneous settings. To improve this, FedEM (Marfoq et al., 2021) proposes a soft clustering approach that flexibly groups clients, enhancing personalization performance. However, FedEM requires each client to download multiple models, leading to increased communication costs and posing practical challenges.

While single-model PFL strikes a balance between global consistency and personalization, it often struggles to fully accommodate client heterogeneity. Multi-model PFL better fulfills personalization requirements, though it faces challenges such as communication overhead and the difficulty of effective clustering. In practice, balancing communication efficiency and personalization performance remains a major issue in multi-model PFL. Additional approaches include additive combinations of local and global models, such as FedVKD (Tao et al., 2022), APFL (Deng et al., 2020), and multi-task learning methods with model similarity regularization, such as pFedMe (Dinh et al., 2020). FedProto (Tan et al., 2022) decomposes the FL model into a shared feature extractor and a personalized component to realize PFL. Furthermore, FedALA (Zhang et al., 2023) proposes an adaptive local aggregation module that effectively combines global and local models to address client heterogeneity and improve performance in personalized FL. These approaches are particularly useful in non-IID scenarios where client data distributions differ greatly, but they may introduce privacy risks due to increased sensitivity in modelling.

Recently, some methods have introduced customized aggregation weights for personalization, such as FedAMP (Huang et al., 2021), FedRep (Collins et al., 2021), FedDWA (Li et al., 2023), and FedBN (Li et al., 2021). These methods assign client-specific aggregation weights based on similarity, thereby promoting collaboration among similar clients. FedAMP uses an adaptive model propagation mechanism to dynamically adjust model updates for each client. However, because it requires all clients to participate in each training round, it lacks flexibility. Additionally, FedCAC evaluates the sensitivity of each parameter to non-IID data and dynamically selects clients with similar data distributions for collaboration, allowing more parameters to be shared and improving performance. However, although collaboration improves model performance, it may increase computational load and communication overhead in scenarios with significant data differences. FedAS (Chen et al., 2024), a framework for PFL, addresses inconsistencies within and between clients through federated parameter alignment and client synchronization. However, it faces the challenge of reduced communication efficiency due to over-reliance on personalization, while neglecting effective global collaboration.

While these methods are effective in certain scenarios, relying solely on customized aggregation weights to identify clients with similar data distributions remains challenging in practical applications, especially in cases of extreme data heterogeneity. Existing PFL methods often consider only partial factors affecting collaboration and lack a comprehensive collaboration guideline, making them difficult to adapt to complex non-IID environments. Although some methods mitigate non-IID effects by localizing sensitive model layers, they often underestimate the potential of collaboration, adopting overly conservative strategies that fail to fully leverage collaboration opportunities among similar clients.

To address these limitations, this paper introduces a novel collaboration guideline featuring a dynamic responsiveness evaluation mechanism. By analyzing critical changes in model parameter updates, the method assesses each client’s collaboration potential and selects appropriate participants for global collaboration. This effectively controls which clients require personalized customization, reducing communication overhead and mitigating privacy risks. Furthermore, an adjustable weighted aggregation strategy is introduced to refine client selection and aggregation based on data distribution differences and parameter responsiveness. By integrating personalized collaboration and adaptive weighting, the proposed method enables efficient information sharing while preserving data privacy, lowering communication costs, and enhancing personalization performance in heterogeneous federated learning scenarios.

Method

Overview of FedDAC

To address the challenge that traditional federated learning struggles to reconcile global model generalization with personalized needs under non-independent and identically distributed (non-IID) data settings, we propose the FedDAC framework in Figure 1.

Figure 1.

An Overview of the System Architecture in FedDAC.

First, each client updates its local model $W_{i}$ along with critical parameters $M_{i}$ . Based on a dynamic responsiveness mechanism, the system stores masks, computes collaboration criteria, and dynamically synthesizes stable selections of important parameters. Clients are dynamically selected for collaboration based on the similarity of parameter masks, defining the scope of collaboration.

Important parameters are aggregated through localized collaboration, where only clients with similar data distributions are allowed to participate in updates. This helps preserve key personalized features and enables the distinction between important $W_{i}^{t}$ and less important parameters $u_{i}^{t}$ . The less important parameters are aggregated using an adjustable weighted strategy, where each client allocates weights dynamically through local guidance models, preferentially selecting clients with similar distributions for global averaging–thus balancing efficiency and generalization.

Finally, an adaptive personalized model is initialized and returned to the local client.

Problem Description

Traditional federated learning methods typically train a single global model that is shared among all clients. However, in real-world applications, due to significant differences in data distributions across clients, a single global model often fails to meet the personalized needs of each individual client. To address the challenges of traditional federated learning, personalized federated learning (PFL) introduces a new optimization objective that aims to personalize the model for each client while maintaining coherence with the global model. Specifically, the objective of personalized federated learning can be formulated as:

min_{{w_{i}}} \frac{1}{N} \sum_{i = 1}^{N} [E_{D_{i}} [L (f (x; w_{i}), y)]]

(1)

The variable

w_{i}

represents the model parameters of the

i

-th client.

E_{D_{i}}

denotes the expectation over the local data distribution

D_{i}

of the

i

-th client.

L

is the loss function, and

f (x; w_{i})

represents the model’s prediction on input

x

using the parameters

w_{i}

y

is the corresponding ground truth label. Due to the significant differences in data distributions across clients, directly using a global model may result in a decline in the performance of the personalized model on local tasks. Specifically, during training, the model is prone to overfitting on

D_{i}

, leading to poor generalization on the local data

D_{i}

. Despite this, collaboration among clients remains necessary during the training process. However, in non-IID scenarios, due to the distinct data distributions across clients, directly collaborating with other clients’ personalized models could negatively impact the performance of

w_{i}

on local tasks. Therefore, minimizing the impact of non-IID data distributions and performing dynamic adaptive collaboration among clients during the training process is a key challenge in personalized federated learning.

Dynamic Responsiveness Evaluation in FedDAC

In personalized federated learning scenarios, the contributions of model parameter updates vary significantly across clients due to task heterogeneity. To facilitate efficient personalization, FedDAC introduces a dynamic responsiveness evaluation mechanism that analyzes key parameter changes during local updates to control collaborative updates more effectively. This mechanism enables clients to customize their models while still participating in global knowledge sharing.

In designing the collaboration strategy, identifying parameters with high responsiveness to non-IID data helps optimize training and mitigate the negative effects of data heterogeneity on performance. However, responsiveness can vary across both training stages and parameters, making it essential to evaluate these dynamics comprehensively when selecting which clients and parameters to involve in collaboration.

First, FedDAC evaluates the responsiveness of each parameter on the client side to identify those most sensitive to non-IID data. It then considers the influence of training stages on responsiveness to refine client and parameter selection. The responsiveness of a parameter refers to the degree of change in the loss function when the parameter is zeroed out. Given a model $w$ with a parameter set $θ = {θ_{1}, θ_{2}, \dots, θ_{n}}$ , the responsiveness of the $i$ -th parameter, denoted as $r_{i}$ , is defined as follows:

r_{i} = | L (Θ) - L (θ_{1}, \dots, θ_{i} = 0, \dots, θ_{n}) |

(2)

Here,

L

represents the loss function, and

Θ

denotes the complete set of model parameters. This metric quantifies the impact of each parameter on the local task. Especially in non-IID data environments, changes in important parameters may significantly affect the model’s accuracy.

To reduce the computational cost of evaluating each parameter individually via forward passes, we approximate responsiveness using the first-order Taylor expansion. Specifically, for a parameter $θ_{i}$ with a change $Δ θ_{i}$ , the Taylor approximation of the loss difference is:

L (Θ) \approx | \nabla_{θ_{i}} L (Θ) \cdot Δ θ_{i} |

(3)

In practice, we further reduce computation by replacing the gradient term with the observed parameter change during training:

Δ θ_{i}^{t} = θ_{i}^{t, E} - θ_{i}^{t, 0}

(4)

Through this Taylor approximation-based method, we efficiently estimate the contribution of each parameter without incurring high computation costs. This allows FedDAC to dynamically assess how parameter updates influence local tasks and identify the most relevant ones under non-IID conditions. Dynamic responsiveness refers to the extent to which a parameter’s change affects the local task during each training round. Its computation is approximated by:

r_{dynamic} \approx | \nabla_{θ_{i}} L (Θ) \cdot Δ θ_{i} |

(5)

Based on this dynamic responsiveness metric, each client performs an internal ranking of its parameters and selects those with the highest values to be involved in global collaboration. The remaining parameters are updated locally. This forms the basis for generating a binary importance mask for each layer’s parameters. Specifically, a parameter is marked important if its dynamic responsiveness falls within the top

τ_{adjusted} (l)

highest values in its layer

l

. The mask generation is defined as:

M_{i}^{t} (j) = I (j \in top- τ_{adjusted} (l))

(6)

Here,

M_{i}^{t} (j)

is the binary mask applied by client

i

at round

t

, and

I (\cdot)

is the indicator function. The parameter selection and mask generation strategy is heuristic, based on dynamic responsiveness ranking at each training round. The threshold

τ_{adjusted} (l)

may be varied over training to ensure stage-aware parameter selection.

During aggregation, only parameters marked as important (i.e., where $M_{i}^{t} (j) = 1$ ) are uploaded and aggregated globally. Parameters with $M_{i}^{t} (j) = 0$ remain locally updated, allowing clients to preserve personalized components of their models. This mechanism ensures the stability of parameter selection and adapts to changes at different training stages, allowing the model to maintain a balance between global collaboration and personalized optimization.

To further elucidate this dynamic workflow, FedDAC implements a multi-step process that facilitates fine-grained and adaptive parameter collaboration in heterogeneous environments. During local training, each client computes updates for its parameters, denoted as $Δ θ_{i}$ . These updates are subsequently used to approximate the dynamic responsiveness of each parameter through a precise gradient-weighted formulation (as detailed in Eq. 5). Parameters are then ranked on a per-layer basis according to their responsiveness scores. A binary mask is applied to select the top $τ_{adjusted} %$ most responsive parameters within each layer. Only these masked parameters, identified as the most critical for effective global knowledge sharing, are transmitted to the central server for aggregation, while the remaining parameters are retained locally and updated independently. This design empowers FedDAC to selectively share the most pertinent model components, thereby effectively mitigating the challenges posed by data heterogeneity in federated learning scenarios. The full procedure for estimating parameter responsiveness and constructing the binary mask is formalized in Algorithm 1.

Parameter Collaboration Strategy Aggregation Design

In traditional federated learning methods, the heterogeneity of client data significantly impacts the collaboration effectiveness of the model. To effectively address this challenge, FedDAC introduces a flexible dynamic collaboration mechanism, conducting the second-stage aggregation filtering to better adapt to changes in the data distribution across different clients, thereby optimizing the collaboration process.

First, the previous section identifies a subset of parameters with high responsiveness to non-IID data. We then introduce a dynamic collaboration standard that divides model parameters into two categories: important parameters and less important parameters.

Based on the selected parameters, different aggregation methods are designed, assigning weightings to the less important parameters, which were identified in the first filtering step. This allows us to fully leverage cross-client collaborative information while also accounting for personalization needs. The clients then receive the aggregated results and combine them with their own masks to generate the new personalized model for the next round.

Important Parameter Aggregation Stage

To enable fine-grained collaboration, FedDAC introduces a similarity metric to quantify the similarity between client data distributions. By calculating the similarity between each pair of clients based on important parameters, it allows for more targeted and efficient collaboration.

S_{i, j}^{t} = | M_{t, i} - M_{t, j} | \frac{1}{2 n}

(7)

Here,

M_{t, i}

and

M_{t, j}

represent the parameter positions of client

i

and client

j

in training round

t

, respectively, and

n

is the total number of parameters. We can measure the similarity of data distributions between two clients. By calculating the average similarity

S_{avg}^{t}

and the maximum similarity

S_{max}^{t}

for each round, we can dynamically adjust the collaboration criteria. The specific formulas are as follows:

S^{t} = S_{avg}^{t} + \sqrt{\frac{t}{α}} (S_{max}^{t} - S_{avg}^{t})

(8)

Here,

α \in [1, T]

controls the rate at which the collaboration criteria change as training progresses. As the training rounds increase, the collaboration criteria gradually rise, ensuring that, in the later stages of training, only clients with the most similar data distributions can participate in collaboration. When the data distribution differences between clients are large, the collaboration threshold will adjust the collaboration criteria accordingly, allowing only clients with similar data distributions to collaborate.

The collaboration set $A_{i (t)}$ is the set of clients selected based on the dynamic collaboration criteria, and it is defined as follows:

A_{i (t)} = {j ∣ S_{{i, j}}^{t} \geq S_{t}}

(9)

In FedDAC, we design a three-stage collaboration strategy that gradually adjusts the collaboration criteria based on different training stages. In the early stage (

t < α

), the collaboration criteria are set low to allow more clients to participate in collaboration, accelerating the model’s initial convergence. Although there is significant data heterogeneity between clients, the low collaboration threshold at this stage helps to speed up the learning process. In the middle stage (

t \approx α

), as training progresses, the collaboration criteria are gradually increased, and only clients with similar data distributions are allowed to collaborate. The similarity between clients converges, and both collaboration efficiency and accuracy are significantly improved. In the later stage (

t \geq α

), the collaboration threshold becomes more stringent, allowing only the most similar clients to continue collaborating. At this stage, the model stabilizes and reaches the strictest level of collaboration. This gradual tightening of the collaboration range focuses on clients whose important parameter distributions are similar.

For important parameters, the aggregation method is more refined and complex. During aggregation, not only the influence of the global model must be considered, but also a weighting based on the similarity of client data distributions. By introducing the collaboration set $A_{i}^{t}$ , it ensures that only clients with similar data distributions participate in the update of important parameters. The aggregation formula is as follows:

u_{i} (t) = \frac{1}{| A_{i} (t) | + 1} \sum_{j \in A_{i} (t)} w_{j} (t)

(10)

Here,

u_{i} (t)

is the important parameter update for client

i

at round

t

, calculated by aggregating the model parameters from clients with similar data distributions.

Less Important Parameter Aggregation Stage: Adjustable Weighting Strategy

For the aggregation of less important parameters, we propose a flexible and client-specific weighting scheme to enhance model personalization. This stage introduces an adjustable weighting rule, where each client $i$ learns an adaptive weight vector $p_{i} = [p_{i, 1}, \dots, p_{i, N}]^{T}$ to linearly combine multiple candidate models received from all other clients. The goal is to construct a personalized model $w_{i}^{⋆}$ by minimizing the distance between the local guidance model and other clients’ updates.

To measure the similarity between the local model of client $i$ and that of client $j$ , we use the Euclidean ( $ℓ_{2}$ ) distance in parameter space. The similarity is asymmetric–that is, client $i$ ’s evaluation of others is tailored to its own local objective and data distribution. The optimal weights $p_{i}$ are obtained by solving the following optimization problem:

min_{p_{i}} \sum_{j = 1}^{N} p_{i, j}^{2} {‖ \hat{w_{i}^{⋆}} - \hat{w^{j}} (t) ‖}^{2} s . t . 1_{N}^{T} p_{i} = 1, p_{i, j} \geq 0

(11)

Here,

\hat{w_{i}^{⋆}}

is a guidance model locally computed by client

i

, and

\hat{w^{j}} (t)

denotes the received parameter set from client

j

at round

t

. By retaining only diagonal terms of the parameter distance matrix, we avoid computationally intensive matrix inversion. A closed-form solution for

p_{i, j}

is derived as:

p_{i, j} = \frac{{‖ \hat{w_{i}^{⋆}} - \hat{w^{j}} (t) ‖}^{- 2}}{\sum_{k = 1}^{N} {‖ \hat{w_{i}^{⋆}} - \hat{w^{k}} (t) ‖}^{- 2}}

(12)

This ensures that greater similarity (i.e., smaller

ℓ_{2}

-distance) leads to a higher weight in the aggregation.

To further reduce computational costs and improve aggregation efficiency, only the top- $k$ most similar clients are selected for aggregation. Specifically, for client $i$ , we compute the $ℓ_{2}$ distances to all other clients, retain the $k$ lowest-distance clients, zero out the rest of the entries in $p_{i}$ , and renormalize the retained weights.

The personalized guidance model $\hat{w_{i}^{⋆}}$ is obtained via a local gradient descent step prior to aggregation, which reflects the local data distribution of client $i$ :

\hat{w_{i}^{⋆}} = w_{i} (t) - η_{i}^{(t - 1)} \nabla f_{i} (w_{i} (t))

(13)

This locally adapted vector guides the weighting strategy, enabling the client to identify others with similar learning objectives or data distributions. The final update for each client then integrates the aggregated important and less important parameters as:

w_{i} (t + 1) = u_{i} (t) ⊙ M_{i} (t) + w_{i} (t) ⊙ (J - M_{i} (t))

(14)

Here,

J

is an all-ones matrix of the same shape as the model parameters, and

M_{i} (t)

is a binary mask indicating the important parameters identified by client

i

The complete training procedure of the FedDAC algorithm is detailed in Algorithm 2.

Experiments

Baseline Coverage and Result Interpretation

We conducted extensive experiments on four widely used image classification benchmark datasets: EMNIST (Cohen et al., 2017), CIFAR-10 (Wu et al., 2023), CIFAR-100 (Krizhevsky et al., 2010), and Tiny ImageNet (Krizhevsky & Hinton, 2009). To comprehensively evaluate the robustness of FedDAC in heterogeneous data scenarios, we adopted two different non-independent and identically distributed (non-IID) settings.

Pathological Heterogeneity Setting 1: In this setup, each client is randomly assigned data from 2 classes in CIFAR-10/100 and 4 classes in EMNIST, respectively. For Tiny ImageNet, each client is randomly assigned data from 20 classes. Data samples from each class are uniformly distributed across the selected clients, simulating extreme class imbalance and data distribution differences among clients.

Pathological Heterogeneity Setting 2: In this setup, each client contains most of the classes, but the data distribution within each class is uneven. We create a joint version of the dataset by partitioning it randomly among $N$ clients using the symmetric Dirichlet distribution (Chen & Chao, 2021), $Dir (α)$ (with $α = 0.5$ by default). For each class $c$ , we sample a vector $p_{c}$ from $Dir (α)$ and assign scores $p_{c}$ to all training instances of class $c$ , which are then allocated to client $m$ .

To highlight the effectiveness of client collaboration, we assign a small amount of data to each client. Each client has 500 training samples and 100 test samples, with the test data partitioned in the same non-IID manner as the training data to reflect real-world application scenarios.

To validate the adaptability of FedDAC to different model architectures, we employed different neural networks on the EMNIST, CIFAR-10, CIFAR-100, and Tiny ImageNet datasets. For EMNIST and CIFAR-10, we used a lightweight CNN model (McMahan et al., 2017) widely adopted in federated learning. It includes two convolutional layers with ReLU activation and max pooling, followed by two fully connected layers. The input channels are 1 for EMNIST and 3 for CIFAR-10, with the feature dimensions set to 1024 and 1600. For CIFAR-100 and Tiny ImageNet, we used the ResNet-10 and ResNet-18 models (Le & Yang, 2015), respectively, with pre-trained backbone networks that enhance feature extraction capabilities.

In the FedDAC training configuration, we set appropriate parameters for different datasets as shown in Table 1. Specifically, the local training epochs for each client were set to 10 for EMNIST and CIFAR-10, and 5 for Tiny ImageNet and CIFAR-100. The batch size was fixed at 100 to ensure training stability and computational efficiency. The optimizer used was SGD with momentum, and the initial learning rate was set to 0.01. For the global federated training, 40 clients were selected in each round. CIFAR-10 and Tiny ImageNet were trained for 300 rounds each to guarantee sufficient optimization and convergence across datasets.

Table 1.

FedDAC Experiment Configuration Table.

Dataset	Model	Local Epochs	Batch Size	Learning Rate	Number of Clients	Global Epochs
EMNIST	CNN	10	100	0.001	40	300
CIFAR-10	CNN	10	100	0.01	40	300
CIFAR-100	ResNet-10	5	100	0.01	40	300
Tiny ImageNet	ResNet-18	5	100	0.01	40	300

To comprehensively evaluate the effectiveness of FedDAC, we conducted extensive comparisons against eight representative PFL baselines, including FedAvg McMahan et al. (2017), FedProx Li et al. (2020), FedAMP Huang et al. (2021), FedPer Arivazhagan et al. (2019), FedRep Collins et al. (2021), pFedMe Dinh et al. (2020), and FedBN Li et al. (2021). These baselines were carefully selected to reflect a broad spectrum of personalization strategies within federated learning: from classical global aggregation (FedAvg), regularization-based methods addressing non-IID challenges (FedProx), and adaptive collaboration schemes (FedAMP), to partial model personalization (FedPer, FedRep), local optimization with proximal constraints (pFedMe), and normalization-aware approaches (FedBN). While some very recent methods were not included, our selection focuses on well-established and widely adopted algorithms to ensure meaningful, reproducible, and fair comparisons that accurately reflect the current landscape of personalized FL research.

The experimental results, summarized in Table 2, demonstrate that FedDAC consistently outperforms all compared methods across various datasets and non-IID scenarios. Under both pathological and Dirichlet heterogeneity settings, FedDAC achieves the best performance across all four datasets–EMNIST, CIFAR-10, CIFAR-100, and Tiny ImageNet–surpassing mainstream methods such as FedAvg, FedProx, FedRep, and FedAMP. Notably, on the more challenging and heterogeneous datasets CIFAR-100 and Tiny ImageNet, FedDAC reaches accuracies of 57.94% and 37.53%, respectively, showing a significant improvement over the second-best methods. This performance advantage stems from FedDAC’s integration of dynamic anonymous credentials and a contribution-aware aggregation mechanism within the federated learning process. By protecting client identity and data privacy while dynamically adjusting aggregation weights based on each client’s actual contribution to the global model, FedDAC effectively mitigates the negative impact of data heterogeneity. Furthermore, its design balances personalized local learning with shared global knowledge, allowing the model to better adapt to scenarios with substantial statistical differences across clients. Overall, the robustness and adaptability demonstrated by FedDAC validate its practical value and potential in heterogeneous federated learning tasks.

Table 2.

Average Test Accuracy (%) for Three Different Datasets under Pathological and Dirichlet Heterogeneity Settings with 40 Clients Participating 100%.

Method	Pathological Heterogeneity Setting 1				Pathological Heterogeneity Setting 2
	EMNIST	CIFAR-10	CIFAR-100	Tiny	EMNIST	CIFAR-10	CIFAR-100	Tiny
FedAvg	71.78	58.28	23.29	12.93	63.57	54.36	19.16	11.26
FedProx	69.95	57.42	31.12	8.46	62.89	70.03	41.53	6.40
FedRep	86.28	82.68	45.26	35.28	75.42	84.59	40.25	20.83
FedAMP	96.65	85.60	49.97	34.73	91.17	84.99	46.68	22.85
FedPer	96.31	86.65	55.59	33.09	89.21	84.43	47.38	22.33
pFedMe	92.95	82.12	53.12	33.65	90.91	73.32	43.21	19.04
FedBN	90.42	87.35	55.75	34.32	90.42	58.55	48.35	21.33
FedDWA	96.80	87.60	56.97	35.73	93.80	85.99	46.68	25.85
FedDAC	97.38	88.41	57.94	37.53	94.38	86.32	49.33	28.19

The Impact of Collaboration Strategy on the Experiment

The Impact of Dynamic Responsiveness Strategy

To investigate the impact of the dynamic responsiveness strategy on the experiment, we conducted experiments using three different methods: “Dynamic collaboration”, “Static collaboration” and “No collaboration” to select client sets for training. During the training process, the collaboration standard was first used to locate all important parameters, excluding the influence of key parameter collaboration. The results are shown in Figure 2.

Figure 2.

Comparison of Collaboration Strategies. From Left to Right: CIFAR-10, CIFAR-100, and Tiny ImageNet.

As we can see, the accuracy of the “Static collaboration” strategy exhibited oscillations in the early stages, which suggests that it randomly selected some important parameters, failing to ensure model stability. Furthermore, as training progressed, its accuracy did not reach a satisfactory level, indicating that the impact of non-IID data was not effectively handled.

The “No collaboration” strategy, which does not involve collaboration, led to slow convergence and poor final accuracy, as the impact of each client’s local task during collaboration was significant.

The “Dynamic collaboration” strategy outperformed all other strategies in terms of convergence speed and accuracy across all datasets, indicating that our strategy can efficiently and accurately select parameters relevant to non-IID data.

The Impact of the Adjustable Weighting Strategy

In this paper, for the “less important parameter stage” we designed a collaboration-based aggregation strategy based on similarity. For the less important parameter stage, we introduce a more flexible and personalized weighting scheme, further utilizing the selected clients. We conducted training experiments based on the optimal parameters from the first stage.

The “No weighted strategy” adopts a simple strategy for aggregating important parameters, showing rapid initial improvement but poor performance in the later stages, failing to achieve a good result. In contrast, with the “Weighted strategy” we apply the adjustable weighting strategy to further aggregate the less important parameters, resulting in more stable model convergence, and the final accuracy is better than the “No weighted strategy”. The experiments show in Figure 3 that our strategy can better perform model aggregation and achieve good results in non-IID data scenarios.

Figure 3.

Comparison of Weighting Strategies. From Left to Right: CIFAR-10, CIFAR-100, and Tiny ImageNet.

The Impact of Parameters on Experimental Results

The Impact of $α$

In the FedDAC mechanism, $α$ is primarily used to regulate the growth rate of the similarity-based collaboration standard, which in turn affects the collaboration pattern among clients. Specifically, $α$ controls the rate at which the standard increases, thereby determining the degree of collaboration between different clients. A smaller $α$ promotes a rapid increase in the collaboration standard during the early stages of training. However, this rapid expansion of collaboration may weaken the uniqueness of individual data, leading to premature convergence of the model, especially in non-independent and identically distributed (non-IID) data environments, which may impact personalization performance.We conducted related experiments under pathological heterogeneity settings, and as shown in Figure 4, a larger $α$ results in a more gradual growth of the similarity-based collaboration standard. This makes the client more cautious when selecting collaboration partners, helping to preserve the personalized features of the data and enhancing the generalization ability of individual models.

Figure 4.

Model Accuracy (%) with Different Parameters of α Under the Pathological Heterogeneity Setting Scenario. From Left to Right: CIFAR-10, CIFAR-100, and Tiny ImageNet.

The Impact of $τ$

$τ$ controls the proportion of important parameters selected in the first step and influences client collaboration. In non-IID scenarios, a small $τ$ may miss many important parameters, affecting collaboration. If $τ$ is too large, less important parameters are overlooked, reducing aggregation effectiveness. Experiments in the pathological heterogeneity setup, as shown in Figure 5, indicate that smaller $τ$ leads to lower accuracy, and as $τ$ increases, accuracy improves until it reaches the optimal value. Further increases in $τ$ reduce accuracy. The best result is achieved at $τ = 0.7$ , with further adjustments possible.

Figure 5.

Model Accuracy (%) with Different Values of τ Under the Pathological Heterogeneity Setting Scenario. From Left to Right: CIFAR-10, CIFAR-100, and Tiny ImageNet.

The Impact of $k$

The value of $k$ adjusts the number of relevant clients selected during the aggregation of less important parameters, complementing the important parameters chosen in the first step. It ensures that each client selects the closest clients in terms of data distribution, allowing for quicker and more personalized parameter aggregation.Experiments in the Dirichlet non-IID scenario ( $α = 0.5$ ) show in Figure 6 that a smaller $k$ reduces computational burden but may cause overly restricted aggregation, limiting the use of global information. This affects model adaptation, especially with significant data differences, leading to weaker personalization. In practice, we prioritize $k = 0.5$ and adjust based on specific non-IID scenarios.

Figure 6.

Model Accuracy (%) with Different Values of k Under the Dirichlet Non-IID (α = 0.5) Scenario. From Left to Right: CIFAR-10, CIFAR-100, and Tiny ImageNet.

Discussion

Summary of Findings and Contributions

This paper introduces FedDAC, a novel personalized federated learning framework that dynamically balances collaboration and personalization by evaluating client responsiveness and data heterogeneity. Unlike existing approaches that either overly emphasize collaboration or personalization, FedDAC adaptively selects clients for global model aggregation based on parameter update patterns. It further incorporates a mask-based marking mechanism to reduce communication overhead and mitigate privacy risks, along with an adjustable weighted aggregation strategy to optimize the contribution of each client according to their data distribution and responsiveness. This combination enhances both the generalization ability of the global model and the personalization quality on each client, addressing critical challenges in non-IID federated learning scenarios.

Extensive experiments on four widely-used benchmark datasets–EMNIST, CIFAR-10, CIFAR-100, and Tiny ImageNet–demonstrate that FedDAC consistently outperforms strong baselines, achieving an average accuracy improvement between 1.5% and 3.2%. On more heterogeneous datasets like Tiny ImageNet, the performance gain reaches up to 5.9%. These results confirm the effectiveness and robustness of the proposed dynamic adaptive collaboration strategy in diverse non-IID environments, highlighting FedDAC’s potential for practical deployment in real-world personalized federated learning applications.

Limitations and Future Work

Despite its strong performance, FedDAC still has several areas for improvement. Firstly, the current client selection mechanism heavily relies on historical performance, which may lead to frequent selection of specific clients, potentially limiting generalization to underrepresented data. Secondly, the framework assumes relatively uniform computational capabilities across clients, which may not hold in real-world scenarios, where device heterogeneity can impact training fairness and efficiency.

To address these issues, future work will explore more balanced and representative client scheduling strategies to improve fairness and robustness. We also plan to integrate privacy-preserving techniques such as differential privacy and secure multi-party computation to strengthen data protection. Furthermore, investigating FedDAC’s compatibility with heterogeneous model architectures will be essential for broader edge deployment scenarios.

Conclusions

This paper addresses the challenges of non-independent and identically distributed (non-IID) data in personalized federated learning (PFL) by proposing a dynamic adaptive collaboration framework, FedDAC. Additionally, we introduce a comprehensive PFL collaboration guideline that simultaneously considers data distribution differences between clients and the responsiveness of each parameter. Based on this guideline, the method uses quantitative metrics to evaluate the responsiveness of each parameter and dynamically selects clients with similar data distributions for collaboration on important parameters. Our experimental results demonstrate that under the guidance of the proposed guideline, our FedDAC method effectively enables each client to gain more support from other clients, leading to superior performance in a wide range of complex non-IID scenarios.

Footnotes

ORCID iD

ChunLong Yao

Funding

The author(s) received no financial support for the research, authorship and/or publication of this article.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

Arivazhagan

M. G.

Aggarwal

Singh

A. K.

Choudhary

(2019). Federated learning with personalization layers. arXiv preprint arXiv:1912.00818.

Chen

H.-Y.

Chao

W.-L.

(2021). On bridging generic and personalized federated learning for image classification. arXiv preprint arXiv:2107.00778.

Chen

J.-X.

Yan

Liu

Z.-Y.

Zhang

Xiong

(2024). When federated learning meets privacy-preserving computation. ACM Computing Surveys, ACM New York, NY.

Cohen

Afshar

Tapson

Van Schaik

(2017). EMNIST: Extending MNIST to handwritten letters. In 2017 International joint conference on neural networks (IJCNN) (pp. 2921–2926). IEEE.

Collins

Hassani

Mokhtari

Shakkottai

(2021). Exploiting shared representations for personalized federated learning. In International conference on machine learning (pp. 2089–2099).

Deng

Kamani

M. M.

Mahdavi

(2020). Adaptive personalized federated learning. arXiv preprint arXiv:2003.13461.

Dinh

Tran

Nguyen

(2020). Personalized federated learning with moreau envelopes. Advances in Neural Information Processing Systems, 33, 21394–21405.

Fallah

Mokhtari

Ozdaglar

(2020). Personalized federated learning with theoretical guarantees: A model-agnostic meta-learning approach. Advances in Neural Information Processing Systems, 33, 3557–3568.

Ghosh

Chung

Yin

Ramchandran

(2020). An efficient framework for clustered federated learning. Advances in Neural Information Processing Systems, 33, 19586–19597.

10.

Gong

Sharma

Karanam

Chen

Doermann

Innanje

(2021). Ensemble attention distillation for privacy-preserving federated learning. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 15076–15086).

11.

Himeur

Varlamis

Kheddar

Amira

Atalla

Singh

Bensaali

Mansoor

(2023). Federated learning for computer vision. arXiv preprint arXiv:2308.13558.

12.

Salcic

Sun

Dobbie

Zhang

(2021). Source inference attacks in federated learning. In 2021 IEEE international conference on data mining (ICDM) (pp. 1102–1107).

13.

Huang

Chu

Zhou

Wang

Liu

Pei

Zhang

(2021). Personalized cross-silo federated learning on non-iid data. In Proceedings of the AAAI conference on artificial intelligence (Vol. 35, No. 9, pp. 7865–7873).

14.

Krizhevsky

Hinton

(2009). Learning multiple layers of features from tiny images. In Toronto, ON, Canada.

15.

Krizhevsky

Nair

Hinton

(2010). CIFAR-10 (Canadian institute for advanced research). In URL: http://www.cs.toronto.edu/kriz/cifar.html (Vol. 5, no. 4, pp. 1).

16.

Yang

(2015). Tiny ImageNet Visual Recognition Challenge. In CS 231N (Vol. 7, no. 7, pp. 3).

17.

Varshney

P. K.

(2021). Federated learning with soft clustering. IEEE Internet of Things Journal, 9(10), 7773–7782.

18.

Chen

Zhou

(2023). FedDWA: Personalized federated learning with dynamic weight adjustment. arXiv preprint arXiv:2305.06124.

19.

Song

(2021). Model comparison federated learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 10713–10722).

20.

Beirami

Smith

(2021). Ditto: Fair and robust federated learning through personalization. In International conference on machine learning (pp. 6357–6368).

21.

Sahu

A. K.

Zaheer

Sanjabi

Talwalkar

Smith

(2020). Federated optimization in heterogeneous networks. Proceedings of Machine Learning and Systems, 2, 429–450.

22.

Wang

Cheng

Song

. (2023). Optimum-statistical collaboration towards general and efficient black-box optimization. arXiv preprint arXiv:2106.09215.

23.

Jiang

Zhang

Kamp

Dou

(2021). Fedbn: Federated learning on non-iid features via local batch normalization. arXiv preprint arXiv:2102.07623.

24.

Long

Tan

Jiang

Zhang

(2020). Federated learning for open banking. In Federated Learning: Privacy and incentive (pp. 240–254). Springer.

25.

Marfoq

Neglia

Bellet

Kameni

Vidal

(2021). Federated multi-task learning under a mixture of distributions. Advances in Neural Information Processing Systems, 34, 15434–15447.

26.

McMahan

Moore

Ramage

Hampson

Arcas

B. A.

(2017). Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics (pp. 1273–1282).

27.

Pei

J.-M.

Liu

W.-X.

J.-H.

Wang

L.-K.

Liu

(2024). A survey of federated learning methods in heterogeneous scenarios. IEEE Transactions on Consumer Electronics, 70(3), 5983–5999.

28.

Rieke

Hancox

Milletarì

Roth

H. R.

Albarqouni

Bakas

Galtier

M. N.

Landman

B. A.

Maier-Hein

(2020). The future of digital health with federated learning. NPJ Digital Medicine, 3, 119.

29.

Sadilek

Liu

Nguyen

Kamruzzaman

Serghiou

Rader

Ingerman

Mellem

Kairouz

Nsoesie

E. O.

(2021). Privacy-first health research with federated learning. NPJ Digital Medicine, 4(1), 132.

30.

Tan

Long

Liu

Zhou

Jiang

Zhang

(2022). Fedproto: Federated prototype learning across heterogeneous clients. In Proceedings of the AAAI conference on artificial intelligence (Vol. 36, No. 8, pp. 8432–8440).

31.

Tang

Guo

(2021). Personalized federated learning with contextualized generalization. arXiv preprint arXiv:2106.13044.

32.

Tao

Gao

Guo

(2022). Training vision transformers in federated learning with limited edge-device resources. Electronics, 11(17), 2638.

33.

Wang

Mathews

Kiddon

Eichner

Beaufays

Ramage

(2019). Federated evaluation of on-device personalization. arXiv preprint arXiv:1910.10252.

34.

Liu

Niu

Zhu

Tang

(2023). Bold but cautious: Unlocking the potential of personalized federated learning through cautiously aggressive collaboration. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 19375–19384).

35.

Yoon

Shin

Hwang

S. J.

Yang

(2021). Fedmix: Approximation of mixup under mean augmented federated learning. arXiv preprint arXiv:2107.00233.

36.

Zhang

Hua

Wang

Song

Xue

Guan

(2023). Fedala: Adaptive local aggregation for personalized federated learning. In Proceedings of the AAAI conference on artificial intelligence (Vol. 37, No. 9, pp. 11237–11244).

37.

Zhang

Luo

Bai

Duan

L.-Y.

(2021). Federated learning for non-iid data via unified feature learning and optimization objective alignment. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 4420–4428).

38.

Zhang

Sapra

Fidler

Yeung

Alvarez

J. M.

(2020). Personalized federated learning with first-order model optimization. arXiv preprint arXiv:2012.08565.

Dynamic Adaptive Collaboration for Personalized Federated Learning

Abstract

Keywords

Introduction

Related Work

Method

Overview of FedDAC

Problem Description

Dynamic Responsiveness Evaluation in FedDAC

Parameter Collaboration Strategy Aggregation Design

Important Parameter Aggregation Stage

Less Important Parameter Aggregation Stage: Adjustable Weighting Strategy

Experiments

Baseline Coverage and Result Interpretation

The Impact of Collaboration Strategy on the Experiment

The Impact of Dynamic Responsiveness Strategy

The Impact of the Adjustable Weighting Strategy

The Impact of Parameters on Experimental Results

The Impact of α

The Impact of τ

The Impact of k

Discussion

Summary of Findings and Contributions

Limitations and Future Work

Conclusions

Footnotes

ORCID iD

Funding

Declaration of Conflicting Interests

References

The Impact of $α$

The Impact of $τ$

The Impact of $k$