Mitigating Backdoor Attacks in Federated Learning through Noise-Guided Aggregation

We would like to thank the reviewer for taking the time to review. We appreciate that you find that our method is novel, only requires noise, and improves benign accuracy by aligning with noise. In light of your insightful comments, we offer comprehensive responses below. We also add them into our main text or appendix in the revision. We hope that these responses and the revision have effectively resolved your concerns and enhanced the overall excellence of our work.

Q1: The reason for the effectiveness of noise alignment and filtering lacks clarity. It remains uncertain why attackers struggle to align their data with noise. It is advisable for the authors to conduct a toy experiment to illustrate this point.

Ans for Q1): Thanks for your valuable comments.

• We exploited t-SNE to visualize the feature distribution of the server and clients in Figures 4, 13, 14, and 15. It can be observed that the attacker’s feature distribution is more scattered and loose compared to the benign client, making it perform worse on the surrogate noise data.
• In response to your constructive comments, we have conducted toy experiments involving two clients. The benign data of both clients is the same while introducing additional poisoned samples to the first client. The accuracy of these two clients on noise samples is shown in Figure 8 Appendix E.11. We can see that the client introducing poisoned samples performs worse on noise samples, making it easier for the server to identify attackers.

We have added the above results and discussion to our revision.

Q2: The provided rationale for Nira's ability to defend against adaptive attacks is not clear too. It is suggested that the authors i) disable the scale-up operation, and ii) in the absence of the scale-up operation, substantiate the second reason with experimental results.

Ans for Q2): Thanks for your constructive comments.

• We apologize for the lack of detailed analysis and experimental support for Nira's defense against adaptive attacks. According to your valuable comments, we analyze the correlation between poisoned data, benign data, and surrogate data below. When attackers train local models, they align the conditional feature distributions between benign data and surrogate data to circumvent the server's detection. In this context, aligning the conditional feature distributions of poisoned data and surrogate data would also align the distributions of poisoned data and benign data, thereby reducing the damage of poisoned samples. If poisoned data and surrogate data are not aligned, the model's accuracy on surrogate samples will be low, making it easy for the server to detect. Therefore, the proposed method can effectively defend against malicious attacks whether poisoned data is aligned with surrogate data or not.
• To further validate the impact of scale-up on the performance of Nira, we conduct experiments without the scale-up operation, and the alignment loss between poisoned data and surrogate data is shown in Figure 9 Appendix E.11. We can see that the scale-up operation increases the alignment loss between poisoned data and surrogate data.

We have added the above results and discussion to our revision.

Q3: It might be beneficial for the authors to elaborate on the specific reasons why using malicious models for metric calculation could lead to biased results and ultimately undermine the defense. Additionally, it's worth addressing why Nira's use of malicious models in a similar context does not result in failure.

Ans for Q3): We appreciate your suggestion. We have added detailed explanations in the revised paper.

• In previous methods, malicious models are also involved in the calculation of these metrics, thus may yield tainted metrics and fail to achieve effective defense. For example, Krum [1] calculates the sum of the squared distances between each client model and the other client models as its score, and aggregates several models with the lowest scores. However, when the majority are attackers, the score of the malicious model may be relatively lower, leading the server to aggregate malicious models and resulting in defense failure.
• The proposed method aims to defend against backdoor attacks by designing a metric that will not be tainted by malicious models. To this end, we propose a metric that performs individual evaluation for each local model using surrogate data. This individual evaluation approach renders the metric impervious to variations in the attacker's ratio.
• The proposed metric (accuracy on surrogate data) can successfully filter out most malicious models, which is consistent with our experimental results reported in Table 5 Appendix E.4.

I am still concerning about your answer on Q2. Under the adaptive attack scenario, the attacker only aligns the benign data with surrogate data. But why it is claimed that "aligning the conditional feature distributions of poisoned data and surrogate data would also align the distributions of poisoned data and benign data, thereby reducing the damage of poisoned sample". Attacker is not going to "align the conditional feature distributions of poisoned data and surrogate data". The logic does not make sense here.

Q6: Clarification on the scale factor in replacement attacks is necessary. It would also be valuable to determine whether Nira remains effective when the scaling-up operation is disabled.

Ans for Q6): Thanks for your valuable comments. We have added the following clarification and experiments to our revision.

• The scale factor we used is $\frac{m}{n_a}$, where $m$ is the number of clients participating in aggregation each round and $n_a$ is the number of attackers, which is consistent with previous outstanding works [2,3].
• In response to your constructive comments, we have conducted experiments without the scale-up operation. The results are reported in Table 3-4 as below. We can see that the proposed method still outperforms baselines in the absence of the scale-up operation.

Table 3-4

Reference

[1] Blanchard, Peva, et al. "Machine learning with adversaries: Byzantine tolerant gradient descent." Advances in neural information processing systems 30 (2017).

[2] Bagdasaryan, Eugene, et al. "How to backdoor federated learning." International conference on artificial intelligence and statistics. PMLR, 2020.

[3] Wang, Hongyi, et al. "Attack of the tails: Yes, you really can backdoor federated learning." Advances in Neural Information Processing Systems 33 (2020): 16070-16084.

Q4: In Table 2, the impact of the poisoning ratio on benign accuracy appears significant, which is not a usual case for backdoor attacks. To better understand this phenomenon, it would be helpful to see the results for scenarios with only one attacker (attacker number = 1) and no attackers (attacker number = 0) under different poisoning ratios for both FedAvg and Nira.

Ans for Q4): Thanks for pointing out this potentially confusing problem.

• In Table 2, "Acc" refers to the prediction accuracy computed on the clean and poisoned test samples; "Main Acc" refers to the prediction accuracy computed on clean test samples. We can see that the poison ratio has a limited impact on the "Main Acc".
• In response to your constructive comments, we have conducted experiments with only one attacker (attacker number = 1) and no attackers (attacker number = 0) under different poison ratios for both FedAvg and Nira. The experimental results are shown in Table 3-1 as below. We can see that the proposed method achieves good performance under varying poison ratios.

Table 3-1

Following your valuable comments, we have added the above results and discussion in our revision.

Q5: It is suggested to provide experimental results when full client participation is employed, without client selection.

Ans for Q5): Thanks for your suggestion, which motivates the following experiments. We have added the following experiments without client selection to our revision.

• We have conducted experiments using a larger ratio, $30/50=60\%$. The results are reported in Table 3-2 as below. We can see that the proposed method achieves good performance even when attackers constitute the majority.

Table 3-2

• We have conducted experiments using larger ratios, $40\%, 60\%$, and $80\%$. The results are reported in Table 3-3 as below. We can see that attackers fail to enhance attack performance by increasing the poison ratio.

Table 3-3

In sum, experiments with larger ratios of malicious clients and poisons further underscore the robust effectiveness of the proposed method in diverse settings. Thanks again for your constructive comments.

We sincerely thank the reviewer for taking the time to review. We appreciate that you find that the backdoor attack is an important security issue in FL, the paper is clearly written, and the proposed method does not need the assumption that the majority of the clients are benign and does not need access to an auxiliary dataset which some previous methods require. According to your insightful and valuable comments, we provide detailed feedback below and add them into our main text or appendix in the revision. We hope these modifications and our revision can address your concerns and improve our work.

Q1: There are 50 clients in the experiment, and the number of attackers ranges from 0 to 12, which corresponds to 0% to 24% of malicious clients, not exceeding 50%.

Ans for Q1): We appreciate your valuable comments, which motivate the following experiments. We have added the following experiments to our revision.

In response to your constructive comments, we have conducted experiments using a larger ratio, $30/50=60\%$. The results are reported in Table 2-1. We can see that the proposed method achieves good performance even when attackers constitute the majority, which further underscores the robust effectiveness of the proposed method.

Table 2-1

Q2: In the appendix, it says that "train a few rounds on a small number of identified benign clients". If the thresholds are selected this way, knowing some identified benign clients is an important assumption of the method that should be mentioned in the main paper.

Ans for Q2): Thanks for pointing out the lack of information about the assumption. To efficiently select thresholds, we need to assume that the server can identify a small number of benign clients and simulate the training process. Following your valuable comments, we have highlighted this assumption in our revision.

Q3: This method seems to be limited to vision data. Federated learning can also be applied to other types of data, such as NLP, and graphs. How could the method generalize to other types of data?

Ans for Q3): Thanks for your constructive comments.

• Extending our proposed method to different data types, including NLP and graphs, is an interesting question.
• The promising direction may align with the work [1]. However, due to time limitations, we leave it as future work.

Reference

[1] Baradad Jurjo, Manel, et al. "Learning to see by looking at noise." Advances in Neural Information Processing Systems 34 (2021): 2556-2569.

Q7: In Table 1, the definitions of "Acc", "Atk Rate" and "Main Acc" are not explicitly explained and therefore confusing.

Ans for Q7): Thanks for pointing out this potentially confusing problem.

• "Acc": the prediction accuracy computed on the clean and poisoned test samples; "Atk Rate": the attack success rate measuring the proportion of poisoned test samples classified as target labels by the global model; "Main Acc": the prediction accuracy computed on clean test samples.
• We explained the above definitions in the first paragraph of Sec. 4 and the first paragraph in Sec. 4.2.

Following your valuable comments, we have highlighted these definitions in the table's caption to avoid confusion. Thanks again for your valuable comments.

Q8: The experiments are conducted on relatively limited settings, i.e., only one triggered-based backdoor attack is considered, and small datasets.

Ans for Q8): Thanks for your valuable comments.

• We set the experimental settings following previous outstanding works [1,2,3].
• We appreciate your suggestion. We would like to note that completing the experiments on the ImageNet dataset during the discussion period is challenging. Thus, we will add the mentioned large-scale datasets in our revision when we finish the experiments.

Q9: What is the ratio of the number of surrogate data and real data for each client? How will this ratio affect the performance?

Ans for Q9): Thanks for pointing out the lack of information about surrogate data. We have added the following information and experiments to our revision.

• The size of the surrogate dataset is $2000$, which is a small proportion of the utilized datasets, i.e., $3.33\%$ for CIFAR-10, $2.86\%$ for FMNIST, and $0.33\%$ for SVHN.
• Following your valuable question, we conduct ablations on the sensitivity of the size of the surrogate dataset and report results in Table 1-5 as below. We can see that the size of the surrogate dataset has a limited impact on the performance. Even when the surrogate dataset size is only 500, the proposed method still demonstrates excellent performance.

Table 1-5

Reference

[1] Xie, Chulin, et al. "Dba: Distributed backdoor attacks against federated learning." International conference on learning representations. 2019.

[2] Baruch, Gilad, Moran Baruch, and Yoav Goldberg. "A little is enough: Circumventing defenses for distributed learning." Advances in Neural Information Processing Systems 32 (2019).

[3] Zhang, Zhengming, et al. "Neurotoxin: Durable backdoors in federated learning." International Conference on Machine Learning. PMLR, 2022.

[4] Wang, Hongyi, et al. "Attack of the tails: Yes, you really can backdoor federated learning." Advances in Neural Information Processing Systems 33 (2020): 16070-16084.

Q4: Experimental results are limited to a small ratio of clients (12/50) and poisons (20%).

Ans for Q4): We appreciate your valuable comments. Accordingly, we have conducted the following experiments and added the following discussion to our revision.

• The ratio of malicious clients is set as $12/50=24\%$ larger than previous works [2,3,4]. In response to your constructive comments, we have conducted experiments using a larger ratio, $30/50=60\%$. The results are reported in Table 1-2 as below. We can see that the proposed method achieves good performance even when attackers constitute the majority.

Table 1-2

• The ratio of poisons is set as $20\%$ comparable with previous works [1,2]. In response to your constructive comments, we have conducted experiments using larger ratios, $40\%, 60\%$, and $80\%$. The results are reported in Table 1-3 as below. Our results show that our method still outperforms baseline methods and the attackers fail to enhance the attack rate by increasing the poison ratio.

Table 1-3

In sum, experiments with larger ratios of malicious clients and poisons further underscore the robust effectiveness of the proposed method in diverse settings. Thanks again for your constructive comments.

Q5: Why do the filtering thresholds vary with time in experiments? Are there any insights on how to adjust them systematically?

Ans for Q5): Thanks for pointing out this potentially confusing problem.

• In the federated learning scenario, the accuracy of client models on local data improves with increasing training rounds. Since the proposed method aligns the feature distributions of local data and surrogate data, the performance of client models on surrogate data also improves. Therefore, the accuracy threshold $\sigma_1$ increases and the distance threshold $\sigma_2$ decreases as the training rounds progress.
• We introduced an efficient method in Appendix E.5 for selecting thresholds: First, train a few rounds on a small number of identified benign clients, referring to the accuracy of the model on surrogate data and the similarity of features during the training process to assist in selecting $\sigma_1$ and $\sigma_2$. Then the selected $\sigma_1$ and $\sigma_2$ are used for the training of all clients.

Following your valuable comments, we have added the above discussion to our revision.

Q6: What does "features of the model" mean and how to determine the value of b?

Ans for Q6): We apologize for the confusing description.

• The "features of the model" refer to the output features of a specific layer in a neural network.

Following your valuable comments, we have fixed it in our revision.

• b stands for the number of noise samples used for evaluating local models. Considering that using all surrogate data will cause significant computation overhead, we set it to a fixed value of 128 for all experiments.
• Following your valuable question, we conduct ablations on the sensitivity of b and report results in Table 1-4 as below. We can see that b has a limited impact on the performance.

Table 1-4

We have added the above results and discussion to our revision.

We sincerely thank the reviewer for taking the time to review. We appreciate that you find that our method is technically novel and sound, our paper is well-written and easy to follow, and the experimental evaluations are comprehensive. According to your valuable comments, we provide detailed feedback below and also add them into our main text or appendix in the revision. We hope that these responses have addressed your concerns and improved our work.

Q1: The proposed approach is based on strong assumptions that the malicious attackers follow the proposed protocol, i.e. training both surrogate and local datasets with the proposed objectives exactly or adaptively.

Ans for Q1): Thanks for your insightful comments. We agree with the point that malicious attackers may reject to follow the proposed protocol. In this context, the model sent from malicious attackers will produce a poor performance on the constructed surrogate data. Accordingly, our method will detect these models as malicious models. Therefore, malicious attackers who reject to follow the proposed protocol fail to mount a backdoor attack.

We have highlighted this in-depth discussion in our revision. Thanks again for your constructive comments.

Q2: The introduction of surrogate data may easily prompt malicious attackers to inject backdoor triggers and objectives into the surrogate dataset as well. Have the authors considered these scenarios?

Ans for Q2): We apologize for the missing discussion. Following your valuable question, we have added the following experiments and discussion in our revision.

Malicious attacks may inject backdoor triggers into the surrogate data. However, if an attacker attempts to mount a backdoor attack on the surrogate data, the model sent from the attacker will achieve poor performance on the surrogate data shared among clients and server. As a result, the server will filter out these malicious models by evaluating their performance on the surrogate data. Therefore, injecting backdoor triggers on the surrogate data will also be detected by the proposed method. To verify the point, we inject backdoor triggers into the surrogate dataset on a malicious client while other settings are the same as our original setting. The results are reported in Table 1-1 as below. The results show that these malicious clients will not bypass the proposed method.

Table 1-1

Q3: The metric that the server used for filtering out malicious attackers appears to use a distance outliner (eq.6) and thresholds, which may be also tainted if the majority are attackers.

Ans for Q3): Thanks for pointing out the potentially confusing description. We have added detailed explanations in the revised paper.

• The proposed method aims to defend against backdoor attacks by designing a metric that will not be tainted by malicious models. To this end, we propose a metric that performs individual evaluation for each local model using surrogate data. This individual evaluation approach renders the metric impervious to variations in the attacker's ratio.
• The proposed metric (accuracy on surrogate data) can successfully filter out most malicious models, which is consistent with our experimental results reported in Table 5 Appendix E.4.
• However, it is challenging for a single metric to detect all malicious models, even if the metric is not tainted by malicious models. Thus, we introduce a supplementary metric to further promote the performance. The supplementary metric is inspired by existing methods, employing the distance among local models. Thus, the supplementary metric has the risk of being tainted.
• By integrating these two metrics, the proposed method achieves a more potent defense performance, particularly in scenarios with a significant proportion of attackers.

Dear reviewer #ig5G,

Thanks for your valuable time in reviewing and constructive comments, according to which we have tried our best to answer the questions and carefully revise the paper. Here is a summary of our response for your convenience:

• (1) The proposed approach is based on strong assumptions: We have explained that malicious attackers who reject to follow the proposed protocol fail to mount a backdoor attack.
• (2) Inject backdoor triggers and objectives into the surrogate dataset: Following your constructive comments, we have conducted experiments where we inject backdoor triggers into the surrogate dataset on a malicious client. Results are shown in the given Table 1-1, which illustrates that injecting backdoor triggers on the surrogate data will also be detected by the proposed method.
• (3) The metric may be tainted: We have explained that the proposed metric (accuracy on surrogate data) is impervious to variations in the attacker’s ratio, and the supplementary metric has the risk of being tainted. By integrating these two metrics, the proposed method achieves a more potent defense performance.
• (4) Small ratio of clients and poisons: Following your constructive comments, we conducted experiments using a larger attacker ratio, $30/50=60\%$, and larger poison ratios, $40\%, 60\%$, and $80\%$. Results are shown in the given Tables 1-2 and 1-3 in responses, which further underscore the robust effectiveness of the proposed method in diverse settings.
• (5) Filtering thresholds issues: We have discussed why thresholds vary with time in experiments and how to efficiently adjust them.
• (6) Determine the value of b: According to your insightful question, we explained the value of $b$ used in our paper. We also conducted ablations on the sensitivity of b and report results in Table 1-4, which shows b has a limited impact on the performance.
• (7) The ratio of the number of surrogate data: According to your insightful question, we explained the size of the surrogate dataset in our paper. We also conducted ablations on the sensitivity of the size of the surrogate dataset. The results shown in Table 1-5 in response illustrate that the proposed method still demonstrates excellent performance even when the surrogate dataset size is only 500.

We humbly hope our response has addressed your concerns. If you have any additional concerns or comments that we may have missed in our responses, we would be most grateful for any further feedback from you to help us further enhance our work.

Best regards

Authors of #1917

Based on your constructive comments, we have tried our best to answer the questions and carefully revise the paper. We would like to verify whether the answers meet your expectations. If there is anything that hasn't been done well, we stand prepared to continue supplementing in the upcoming time. Thank you again for your valuable insights, which enhanced our work a lot. Would you mind reconsidering the rating if our responses convince you?

Dear Reviewer ig5G,

We would like to express our sincere gratitude for your insightful review. Hope our responses have successfully addressed your concerns of this paper. As the final deadline for reviewer-author discussion is approaching, we still look forward to your valuable feedback if any concerns remain. We are ready to provide further elaboration and engage in a more in-depth discussion. If you are satisfied with our replies, please don't hesitate to update your score.

Best wishes, Authors