Tracing Back the Malicious Clients in Poisoning Attacks to Federated Learning

We sincerely appreciate your recognition of our work! Here are our responses to your questions:

Q1: Recovery from the attack: This is a very valuable suggestion. Indeed, if we save the checkpoint of each training round, including both the clients' updates and the global models, we can retain only the updates from benign clients, remove those identified as malicious by FLForensics, and directly perform aggregation, thus avoiding a complete re-run. On the other hand, if only partial intermediate states are saved, we may have to resume training from a specific intermediate round, which can still help mitigate the impact of backdoor attacks. Additionally, techniques such as unlearning [1] or recovery-based methods [2] can be used to further exclude the influence of malicious clients.

[1] Zeng, Yi, et al. "Adversarial unlearning of backdoors via implicit hypergradient." ICLR, 2022.

[2] Cao, Xiaoyu, et al. "Fedrecover: Recovering from poisoning attacks in federated learning using historical information." 2023 IEEE Symposium on Security and Privacy. IEEE, 2023.

Q2: Adapt the attacks: Since FLForensics only stores a subset of all intermediate states, malicious clients can evade the defense by reducing the frequency or probability of their attacks rather than launching them in every round. The top two subfigures in Figure 4 illustrate the results of such adaptive attacks. We observe that FLForensics is still able to accurately identify the malicious clients. However, when the attack becomes less effective, for example, with an attack frequency of 10 or an attack probability of 0.2, the influence scores of malicious clients become less suspicious, leading to a slight increase in the FPR and FNR of FLForensics. This highlights a trade-off between attack effectiveness and stealthiness.

Thank you for following up on my two suggestions/questions in the rebuttal. There are no concerns from my side regarding this paper, so retain my positive rating.

Thank you for the insightful comments. Here are our responses to your comments.

W1. Assumes a vanilla FL:

1). Following many prior works on federated learning defenses (e.g., “Learning to detect...”, FedCAM, and “Untargeted poisoning... ” mentioned by the reviewer), we adopt the commonly used assumption that the server is honest and directly receives model updates from clients.

2). If a trusted third party is available to assist with secure aggregation, it can also be leveraged to perform the forensic traceback procedure, which is orthogonal to our proposed method and can complement it.

3). We appreciate the reference to FedGT. As this line of work suggests, one possible way to ensure privacy while still enabling malicious client detection is to operate on aggregated group updates rather than individual ones. We believe our traceback approach could potentially be extended to such a group-based secure aggregation setting. In particular, instead of computing influence scores over individual client updates, we can compute group-level influence scores over aggregated updates, where each group consists of a small subset of clients. By designing overlapping groups and aggregating their updates, our traceback mechanism can still identify malicious clients using group-level influence patterns, similar in spirit to approaches like FedGT. We view this as a promising direction for future work.

4). Finally, we note that in many practical FL deployments (especially in cross-silo settings), secure aggregation is not yet widely adopted due to its added communication and system complexity. Thus, our work remains applicable to a broad range of real-world scenarios.

W2: Overstated and inaccurate claims: Our work is poison-forensics, meaning that after FL training is completed (even after the model has been deployed), if a user reports a misclassification event, FLForensics can trace back the malicious clients who participated in training. The papers mentioned by the reviewer have a different goal from ours. All four papers are FL aggregation rules, not post-training poison-forensics. Specifically:

“Learning to detect...” states: “After obtaining the spectral anomaly detection model, we apply it in every round of the FL model training to detect malicious client updates.”

“FedGT...” is also an aggregation rule: “a novel framework for identifying malicious clients in federated learning with secure aggregation”

“FedCAM” also states: “During each FL training round, the server calculates the geomed normalized AMs for each client-updated model using the trigger set”

"Untargeted poisoning..." states: “attestedFL removes the local models that are unreliable before computing the global model in each iteration of federated learning”

The above evidences show that all of these works perform defense during FL training, and can thus be considered robust aggregation rules. In contrast, our work is post-training poison-forensics. Therefore, we believe our claim, "the first poison-forensics method for FL to trace back malicious clients.", is correct.

W3: Attacks are not commonly used in the literature: Actually, the three attack papers we evaluate are all highly impactful, commonly used targeted attacks. Specifically, Scaling (published at AISTATS 2020) has around 2800 citations on Google Scholar, ALIE (NeurIPS 2019) has over 700 citations, and Edge-case (NeurIPS 2020) has over 800 citations.

W4: Writing quality: We would greatly appreciate it if the reviewer could provide specific suggestions (for example, where in the paper you observed inconsistencies or lapses), so that we can revise accordingly. However, we would like to clarify that our paper is not AI-generated.

W5: Results section is weak: We would appreciate it if the reviewer can point out which specific parts of our results are considered weak, so we will carefully consider and address the feedback.

Regarding FedGT, we appreciate the reviewer's clarification. Based on the reviewer's description, we believe FedGT is more similar to FLDetector [1], which also detects and removes malicious clients during the training process. In our paper, we discussed the limitations of FLDetector and its differences from our approach in the related work section (line 93 and 262 in the main paper).

Overall, FedGT, FLDetector, and secure aggregation rules are all training-phase defenses, whereas our paper focuses on post-training forensics. Therefore, FLForensics is fundamentally different from FedGT.

For realistic scenarios that preserve privacy, we add noise to all clients' model updates based on the widely adopted differential privacy mechanism [2]. Specifically, we use the following formula to add noise:

We experiment with different noise levels (i.e., different C and $\sigma$), and the results are as follows:

The results demonstrate that FLForensics can successfully trace back under varying levels of privacy.

[1]. Zhang, Zaixi, et al. "Fldetector: Defending federated learning against model poisoning attacks via detecting malicious clients." Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining. 2022.

[2]. Abadi, Martin, et al. "Deep learning with differential privacy." Proceedings of the 2016 ACM SIGSAC conference on computer and communications security. 2016.

Thank you for your feedback! We will carefully consider the theoretical analysis section and adjust the content of the paper accordingly.

Q1: Results is different from the original paper: Our experimental setup is largely aligned with that of FLTrust, with two main differences: the amount of data held by the server and the scaling factor used in the Scaling attack. In the FLTrust paper, the server holds 100 data samples, and the scaling factor is set to the number of clients (100 in their experiments). This means each malicious update is scaled by a factor of 100 to amplify its influence during aggregation. In contrast, our work uses only 50 server data samples and sets the scaling factor to 1. This design choice is motivated by the following considerations:

1). We found that when the server holds 100 data samples, the ASR of the Scaling attack drops to 0.01, which aligns with the results reported in the FLTrust paper. In such cases, using FLForensics to trace back malicious clients and retrain a global model becomes less meaningful. Therefore, we reduce the number of server data samples to make the attack more effective.

2). Setting the scaling factor to 1 avoids making the malicious updates significantly larger in magnitude than benign updates. If we used a scaling factor of 100, the malicious updates would become trivially detectable, for instance, by simply filtering out updates with abnormally large norms. To demonstrate that FLForensics can accurately trace back malicious clients even when malicious updates appear similar in scale to benign ones, we chose a scaling factor of 1.

Additionally, we also conducted experiments using the same settings as FLTrust. The results are as follows:

100 server data samples and a scaling factor of 1:

100 server data samples and a scaling factor of 100:

The results show that FLForensics can still accurately trace back malicious clients. Although the detection accuracy for certain attacks (e.g., Edge attack) is lower in these settings, this is largely because such attacks have negligible impact. This highlights a trade-off between attack effectiveness and traceback effectiveness.

Q2: Label-rich advantages: Our motivation for the "label-rich advantage" assumption stems from the empirical observation that a subset of benign clients consistently exhibits very high influence scores. Figure 2(a) illustrates this phenomenon: clients are color-coded as red (malicious), green (Category I benign), and blue (Category II benign). As shown, the influence scores of Category I benign clients are very close to those of malicious clients, while Category II benign clients display significantly different influence patterns. This also explains why FLForensics-A, which uses only the one-dimensional influence score, consistently exhibits some FPR. Since the 1D influence score cannot distinguish between malicious clients and Category I benign clients, it tends to misclassify Category I benign clients as malicious.

To further validate this assumption, we conducted the following experiment: we relabeled all test data samples to the target label and evaluated the cross-entropy loss on these inputs using each client’s final-round local model. Averaging the results, we found that malicious clients had an average loss of 4.82, while Category I and Category II benign clients had average losses of 0.18 and 12.04, respectively. This indicates that Category I clients are far more likely to predict arbitrary inputs as the target label, even more so than the malicious clients themselves. This clear separation in behavior provides strong empirical support for the "label-rich advantage" assumption.

Thank you for your response. We would like to ask whether our responses have fully addressed your concerns. If there are any remaining questions or suggestions, we would be happy to provide further clarification.

Thank you for your recognition of our work and your valuable comments. Below are our responses.

W1: Requirements seems not realistic: FLForensics traces back malicious clients based on a single reported misclassification event. We assume the server has received such an input from a client. To access client models, we store model checkpoints from selected rounds during training, eliminating the need to request any additional information post-training. In summary, the only requirement is a misclassification event reported by a client.

Furthermore, as discussed in Appendix G (“Detecting Misclassified Target Input”), our method can distinguish between misclassifications caused by poisoning attacks and those resulting from natural errors. Therefore, the misclassified input does not need to be a poisoning-induced target input.

W2: Relies on unsupervised manner: Please see the responses for Q1, Q2, and Q3.

Q1: Why scaling is necessary: The scaling operation enhances the robustness of our method. As illustrated in Figure 2(b), since $s_i$​ and $s’_i$ may be on different scales, clustering can become significantly biased, placing excessive weight on the dimension with larger magnitudes (typically $s’_i$​) while largely ignoring the other. As a result, clusters (i.e., points with different shapes) are formed primarily based on $s’_i$, leading to inaccurate separation. By scaling the two influence scores, clustering considers both dimensions equally, regardless of their original scales. In other words, the scaling operation ensures that FLForensics can robustly identify the malicious client cluster even when $s_i$ and $s’_i$ are on different scales. Therefore, the scaled case is consistently more stable and robust than the non-scaled case.

Q2: The strength of the attack: (1). As shown in Figure 4, the Adaptive Attack achieves very high ASR under some settings (e.g., attack frequency = 1, attack probability = 1, trigger size = 10, and trigger value is random or white), while the ASR is relatively low under others (e.g., attack frequency = 10, attack probability = 0.2, etc.). When the attack is highly effective, we achieve nearly 100% detection accuracy. Even when the attack is less effective, our method still maintains over 90% detection accuracy in most cases. The only exceptions arise when the trigger value is black or the trigger size is 2$\times$2, in which cases the detection accuracy drops. However, in these scenarios, the ASR is 0, indicating that the attacks are ineffective.

(2). According to Figure 4, there is generally a trade-off: higher ASR typically makes the attack easier to detect. However, even in many scenarios with very low ASR (e.g., attack frequency = 10, trigger location = UL, trigger value = grey, trigger size = 3$\times$3), our method still achieves nearly 100% detection accuracy.

(3). ASR is an important metric. However, due to space limitations, we had to place the ASR results in Table 6 of the Appendix. We mention this on line 263. We will add a citation to Table 6 at line 237.

Q3: Problem might depend on the practical settings: In common FL systems, clients exchange either models or model updates with the server. In our paper, we assume that model updates are exchanged. However, if full models are shared instead, the server can still compute the updates by subtracting the previous global model (e.g., $w^{(i)}_ {t+1} - w_ t$, where $w^{(i)}_ {t+1}$ is the shared model from client $i$.) allowing influence scores to be calculated for traceback. Additionally, the server can apply different aggregation rules when updating the global model. We demonstrate this in Figure 3(c), which shows results under various aggregation rules. These observations collectively indicate that FLForensics remains effective under different FL system settings.