EA-PS: Estimated Attack Effectiveness based Poisoning Defense in Federated Learning under Parameter Constraint Strategy

Thank you for your thorough analysis and constructive feedback on our paper. We appreciate the opportunity to clarify the points raised and to provide additional insights into our research.

1. What is the definition of long-lasting attacks? Why $A_t - A_{t-1}$ can be interpreted as long-lasting attacks?

Response:

• We appreciate the opportunity to clarify the definition of long-lasting attacks. In our work, we follow the FL-WBC's observations on the long-lasting attack effect. The definition of "long-lasting" in the long-lasting attack is to describe the effects of an attack in the current round that can persist through multiple rounds of training. Therefore, there exists a slight misleading in our work. The correction is as follows: “long-lasting attack” to "attack with long-lasting effects ".

• $A_t-A_ {t-1}$ in our work is "the difference of different rounds in the coefficient of attack effects". $A_t$ is defined as "the coefficient of attack impact between two rounds". $A_t-A_ {t-1}$ can form a chain structure to better measure the accumulation of attack impact in different periods (i.e., long-lasting attack effects). For example, in our work, Theorem 4.1 shows that minimizing $A_t-A_ {t-1}$ yields a smaller optimization upper bound than a traditional method such as LeadFL; Theorem 5.2: Certified Radius analysis shows that the reduction of $A_t-A_ {t-1}$ improves the robustness of the model against long-term attack effects. The experimental results also support the above proof.

2. Figure 2 didn't illustrate the idea of parameter constraint strategy.

Response:

• We believe that Figure 2 illustrates the idea of the parameter constraint strategy. Based on the last paragraph of section 3, we will provide additional insights into our strategy. The goal of the parameter constraint strategy is to enhance the stability of poisoning attack defense by constraining the perturbation range of model parameters. The key components are 1)Optimized Manifold Space $A$; 2) Unit Space $I$; 3) Rank Constraint $λ$.

• Optimized Manifold Space $A$ represents the unconstrained parameter space of the model, which may involve high-dimensional or complex parameter distributions. In this space, malicious attacks (e.g., backdoor attacks) can create long-lasting effects through parameter perturbations. As illustrated in equations (formalized as equation (16): $I=B^{-1}AB$), the manifold space $A$ is mapped into a simpler, low-dimensional unit space (Unit Space $I$). By constraining parameter perturbations within a bounded region (Rank Constraint $λ$), the strategy suppresses the cumulative effects of adaptive or persistent attacks (formalized as equation (17): $AB=λB$). This ensures stable defense performance under long-lasting attack effects.

3. Why can we assume $\lambda$ is a linear set of $A$? What will be sacrifice with this assumption?

Response: For the reason why we can assume $\lambda$ is a linear set of $A$, based on the references and descriptions on page 5 of our work, we will provide additional insights into it. Firstly, we assume that $λ$ is a linear set of $A$ mainly because the linear decision rule can transform complex uncertainty descriptions into a more tractable linear form, thus yielding a computationally solvable robust optimization model. Specifically, the linear assumption simplifies the complex parameter constraints into a linear combination of historical information. Moreover, linear approximations facilitate the simplification of proofs for convergence and robustness guarantees. However, setting $λ$ as a linear combination of $A$ means ignoring possible nonlinear relationships, which may result in suboptimality according to the linear decision rule (Bertsimas et al., 2019), which has noted on page 5 of our work.

4. How is $H$ calculated?

Response: We appreciate the opportunity to add details about how $H$ is calculated. We will change the equation of $H_{t,e}^k$ in our work (section 4.2) to illustrate the details of how $H$ is calculated, as follows.

$H_{t,e}^k \overset{\bigtriangleup }{\underset{}{=}}\bigtriangledown ^2 F( \theta _{t,e}^k)= (θ _{t,e+1}^k-θ _{t,e}^k-∆θ _{t,e}^k)/ η_t.$

We hope this response adequately addresses your points and welcome the fruitful discussion. We are thankful for the contribution to the manuscript's refinement.

Thank you for your recognition of our work and for your insightful comments.

1. Table 1 caption is not quite clear benign accuracy / （attack success rate）？

Response: The metric used in Table 1 is backdoor accuracy, which is the attack success rate for backdoor attacks. We will change it to "backdoor accuracy" in the revision.

2. Can the author consider FEMINIST, which contains nature non-iid? I encourage the author to check the performance for untargeted attack (MPaf) and other targeted attack (DBA).

Response:

• We appreciate your suggestion to include the nature non-iid dataset (FEMINIST), MPaf, and DBA. In addition to the methods suggested above, we also added Spectrum (targeted) and Label-Flip (untargeted) to further enhance the experiment. The results are as follows.

• We also apply the suggested attack methods and our added methods on the CIFAR-10 dataset to further illustrate the performance. The results are as follows.

• It's important to note that none of the existing client-side defense methods focus on untargeted attacks. Through added experiments, we found that although our method struggles to defend against untargeted attacks, it still slightly outperforms the state - of - the - art client - side defense methods.

• For new target attacks, our method outperforms the state - of - the - art client - side defense methods with server-side defense methods.

The added code will still open-source to the original link in the manuscript.

We hope this response has addressed your concerns effectively. We are grateful for your valuable input.

References

[1] Wang, Tong et al. “An Invisible Black-Box Backdoor Attack Through Frequency Domain.” European Conference on Computer Vision (2022). (Spectrum)

[2]Zhang, Mengmei et al. “Adversarial Label-Flipping Attack and Defense for Graph Neural Networks.” 2020 IEEE International Conference on Data Mining (ICDM) (2020): 791-800. (Label-Flip)

Thank you for your thorough review and valuable feedback on our work.

1. The communication overhead should be compared to pure server-side defense.

Response: We'd like to address the concern regarding the communication overhead in our work. Since nothing but the parameter constraint strategy is used in this work, the communication overhead is the same as pure server-side defense per round.

2. I will suggest adding distributed trigger, adaptive backdoor, and CRFL as baselines.

Response: For distributed trigger and adaptive backdoor, we added DBA[2] and A3FL[1]. We also added CRFL to compare with Multi-Krum and Bulyan. The results are as follows.

3. How much additional time does the client-side defense require compared to standard FedAvg?

Response: As convergence analysis proved, our work is slightly inefficient compared with other methods. We appreciate your suggestion to add time overheads (seconds average round), and the experimental results on CIFAR-10 with IID distribution and FEMNIST with natural non-IID distribution are as follows.

4. Backdoor attacks is slightly inconsistent with Section 2.1.

Response: We appreciate the opportunity to change the "slightly inconsistent" expression that "one of the specific types of targeted attacks (known as backdoor attacks )" in section 2.1.

5. It would be beneficial to show model accuracy and backdoor accuracy to illustrate whether this defense slows down main task training.

Response: Details of MA (Main-task Accuracy) are in the Appendix. We will extract the MA to the experiment section.

6. The attack impact measures the differences between two rounds. I wonder why multiple rounds are not considered, as in a real-world FL system, malicious clients may not be selected every round.

Response: We appreciate the opportunity to clarify the "multiple rounds attack impact". Our work follows previous works setting as in Lead-FL and FL-WBC, where "In each adversarial round, malicious clients are randomly selected and participate in the training". We will highlight this in the experimental setting.

7. It is better to clarify the defender's knowledge and ability considering the difference between client-side defense and server-side defense. To make it more practical, how clients and/or sever exchange knowledge, information should be specified.

Response: 1) Only aggregation information is exchanged between clients and the server. 2) To clarify "the defender's knowledge and ability, considering the difference between client-side defense and server-side defense", we will add the comparisons in a table as follows.

The added code will still open-source to the original link in the manuscript. We are grateful for the chance to discuss our work's improvement, and wish to thank you again for your valuable input.

Reference:

[1]Zhang, Hangfan et al. “A3FL: Adversarially Adaptive Backdoor Attacks to Federated Learning.” Neural Information Processing Systems (2023).

[2] DBA: Xie, et al. "Distributed Backdoor Attacks against Federated Learning." International Conference on Learning Representations (2020)