Curse or Redemption? How Data Heterogeneity Affects the Robustness of Federated Learning

The paper studies how client data heterogeneity in federated learning (FL) affects the robustness of models to backdoor attacks. While FL enables privacy-preserving collaborative training via local client updates and server-side aggregation, defending against poisoning attacks (especially targeted backdoors) is difficult because training data cannot be inspected. Existing FL backdoor studies largely assume IID or only lightly non-IID data, leaving a gap in understanding real-world heterogeneous settings. The authors aim to quantify and explain the impact of heterogeneity on backdoor effectiveness through extensive experiments on LEAF benchmarks (FEMNIST, Sent140) and CIFAR10, introducing a Heterogeneity Index (HI) to control and measure heterogeneity. They investigate how heterogeneity influences attack success rate (ASR), the role of malicious data distribution, the sensitivity to client-side attack timing, and the implications for defenses. Initial results suggest heterogeneity can reduce ASR and complicate attack design, but deeper analysis reveals new vulnerabilities ("curses") where proper timing and distribution choices can significantly boost attacks and undermine common defenses.

Related work covers three areas: (1) Data heterogeneity in FL: Prior studies show non-IID data affects convergence and model performance and can cause divergence of client updates (e.g., Li et al. 2019; Zhao et al. 2018; Sattler et al. 2019; Li et al. 2020). (2) Backdoor attacks: Classic data poisoning backdoors insert patterns to induce targeted misclassification (Chen et al. 2017). FL-specific strategies include model replacement (Bagdasaryan et al. 2018), boosting (Bhagoji et al. 2019), sybils (Fung et al. 2018), GAN-based (Zhang et al. 2019), and distributed attacks (Xie et al. 2019). (3) Defenses: Methods include filtering, fine-pruning, clustering, FoolsGold (cosine similarity), differential privacy and norm clipping (e.g., Steinhardt et al. 2017; Liu et al. 2018; Chen et al. 2018; Shen et al. 2016; Fung et al. 2018; Sun et al. 2019; Ma et al. 2019). However, few works analyze attack/defense effectiveness across varying degrees of heterogeneity, which this paper addresses.

Federated learning setup and datasets: Experiments use LEAF (FEMNIST and Sent140) and CIFAR10 (partitioned across 200 clients). Models: CNN for FEMNIST (handwritten characters, 62 classes), LSTM for Sent140 (tweet sentiment, 2 classes), and CNN for CIFAR10 (10 classes). Training configurations (Table 1): FEMNIST: 2 conv + 2 dense; 49,644/4,964 train/test; 179 total clients, 17 per round; learning rate 0.004, batch size 10; 1 local epoch; 2000 total rounds. Sent140: 100-cell LSTM + 2 dense; 6.553/655 train/test; 50 total clients, 10 per round; learning rate 0.0003, batch size 4; 1 local epoch; 10 total rounds. CIFAR10: 4 conv + 2 dense; 50,000/10,000 train/test; 200 total clients, 20 per round; learning rate 0.0005, batch size 32; 1 local epoch; 500 total rounds. Controlling and quantifying heterogeneity: The number of maximum classes per client is varied to control non-IID levels (fewer classes per client ⇒ more heterogeneity), following prior work. The Heterogeneity Index (HI) is defined as a normalized measure in [0,1] based on the max classes per client c and total classes Cmax; higher HI indicates greater heterogeneity. Additional experiments using Gaussian and Dirichlet partitions (Appendix) are consistent with HI-based results. Threat model: Attackers are malicious clients that can manipulate local training (data, hyperparameters, weights) and collude. Aggregation and benign client training are trusted. Multiple clients may be compromised. Backdoor objective and method: Targeted backdoor attacks insert a trigger (patterns for images, a token for text) in training samples to induce targeted misclassification while maintaining main-task accuracy and convergence. The study adopts the decentralized distributed backdoor attack (DBA) of Xie et al. (2019). A configured number of clients are selected as malicious; they inject triggers into a fraction of local data. Learning hyperparameters are the same for benign and malicious clients. Evaluation: At test time, the trigger is applied to 50% of test data (excluding samples originally from the target class), and ASR is the fraction classified as the target label. Relation to model poisoning: With a large scaling factor, backdooring approaches model replacement (model poisoning), as discussed in Bagdasaryan et al. (2018); analysis appears in the Appendix. Experimental factors: The study varies HI, malicious data distribution across classes, attack scale (fraction of compromised clients), total/local attack budget (fraction of backdoored data), and local/global attack timing (e.g., evenly across batches vs. early/middle/last batches). Figures summarize ASR trends under these variations.

Redemptions (apparent benefits of heterogeneity):

• Redemption 1: Higher data heterogeneity reduces backdoor attack effectiveness and increases variability. As HI increases, median ASR decreases and ASR becomes less stable (wider box-and-whiskers in Fig. 2), suggesting overfitting to skewed benign features can suppress backdoor features.
• Redemption 2: Malicious data distribution is a critical, overlooked factor. With fixed HI, ASR varies substantially across 25 different malicious data distributions (Fig. 3), demonstrating that the alignment of attacker data distribution with training distribution materially affects ASR.
• Redemption 3: Malicious data distribution dominates over attack scale and budget. Heatmaps varying attack scale (fraction of compromised clients) and total attack budget show no clear monotonic pattern (Fig. 4), contradicting common assumptions that larger scale/budget invariably increases ASR. Hence, designing effective attacks is more challenging under heterogeneous data. Curses (new vulnerabilities revealed under heterogeneity):
• Curse 1: Local attack timing is crucial; attacking the last batch(es) significantly boosts ASR with far smaller budgets. On FEMNIST, attacking only the last batch achieves high ASR comparable to attacking the last five batches but with about 80% less local attack budget. In the LATTER(last) strategy, total attack budget is only ~0.2% of total training data yet highly effective. Global attack timing has little impact relative to local timing (Fig. 5). As HI increases, last-batch attacks remain robust and increasingly outperform evenly distributed attacks (Fig. 6).
• Curse 2: Skewed-feature based defenses (e.g., cosine similarity) can fail. With higher HI, benign client updates naturally exhibit larger divergence (lower cosine similarity) from the global model, sometimes comparable to or exceeding malicious clients, making malicious updates harder to detect (Fig. 7).
• Curse 3: Attackers can leverage distribution similarity. ASR correlates strongly with how closely the malicious data distribution matches the global training distribution. Lower Chi-squared distance (ChiSq) between malicious and global distributions yields higher ASR and can outperform attacks with higher budgets but larger ChiSq (Fig. 8–9). Regression shows a strong negative correlation between ChiSq and ASR; heatmaps confirm low-ChiSq attacks generally achieve better ASR across budgets/scales. Defensive insights and quantitative effects:
• Active defense with a small IID dataset at the aggregator (≈10% of total data) to train client updates before aggregation markedly reduces ASR: CIFAR10 from 0.76 to 0.26; FEMNIST from 0.96 to 0.21; Sent140 from 1.00 to 0.36 (Table 2). Limitation: may conflict with secure aggregation.
• Client selection diversification (e.g., avoid selecting the same client in nearby rounds) reduces ASR across heterogeneity levels, with ASR not exceeding ~23% in reported experiments.
• Protecting or obfuscating global data distribution (e.g., misleading attackers about distribution or augmenting at the aggregator) lowers ASR on average from ~0.8 to ~0.46 when attackers build on a faked distribution with high ChiSq (~0.8).

The study directly addresses the central question of how data heterogeneity influences FL backdoor robustness. Empirically, heterogeneity both suppresses backdoor learning (by amplifying benign overfitting that can overshadow triggers) and complicates adversary planning (due to the dominant role of malicious data distribution), but simultaneously introduces exploitable vulnerabilities: precise local timing can induce overfitting to backdoor triggers with minimal budgets; and heterogeneity inflates benign update divergence, undermining skewed-feature defenses such as cosine similarity. These findings indicate that robustness assessments assuming IID data or ignoring malicious data distribution and timing are incomplete and can underestimate attacker capabilities in realistic heterogeneous settings. The defensive strategies proposed—active update generalization using a small IID set at the server, client selection diversification to limit accumulation of overfitted malicious updates, and concealing or misleading the global data distribution—demonstrate promising mitigation effects in experiments. Overall, the results emphasize the need for heterogeneity-aware defenses and evaluation protocols that consider distributional alignment and local training dynamics.

Extensive experiments on FEMNIST, Sent140 (LEAF), and CIFAR10 reveal that data heterogeneity has dual effects on backdoor robustness in federated learning. Initially appearing as a redemption—lowering ASR and complicating attack design via dependence on malicious data distribution—heterogeneity also brings curses: attackers can achieve high ASR with minimal budgets by targeting the last local batch, common skewed-feature defenses become less reliable as benign update divergence increases, and carefully matching malicious to global data distributions (low ChiSq) substantially strengthens attacks. The paper proposes practical, heterogeneity-aware defense directions: (i) server-side active defense using a small IID dataset to generalize client updates prior to aggregation; (ii) client selection policies to reduce repeated participation of the same clients in nearby rounds; and (iii) hiding or perturbing the apparent global data distribution. These insights guide the design of more robust FL systems and motivate future work on integrating distribution-aware detection (e.g., activation clustering, spectral signatures, gradient shaping) with scheduling and aggregation strategies under strong privacy constraints.

• The active defense relying on a small IID dataset at the aggregator may be incompatible with secure aggregation, making it difficult to train individual client updates on the IID data prior to aggregation.
• Empirical evaluations focus on specific datasets (FEMNIST, Sent140, CIFAR10), models, and attack mechanisms (e.g., DBA), with additional analyses referenced in the Appendix; generalization to other tasks and settings, while suggested by trends, is not exhaustively validated within the paper.