Dual Defense: Enhancing Privacy and Mitigating Poisoning Attacks in Federated Learning

We appreciate the reviewer's comments and the raised concerns.

Resp to Q1: DDFed moves the comparison task to the client side because no FHE scheme efficiently supports both comparison operations and floating-point numerical computation over encrypted model updates. FHE schemes are generally classified into three categories: bit-wise approaches like FHEW and TFHE, word-wise approaches like BGV and BFV, and CKKS for efficient floating-point computations. There is currently no perfect FHE solution that efficiently handles all types of computational operations. Each category is more efficient for specific applications, leading to trade-offs when employing a particular FHE scheme.

In this paper, we use FHE to protect FL local models with millions of floating-point parameters, prioritizing computational efficiency. We adopt the CKKS scheme as our underlying FHE solution and delegate comparison tasks to the client side since CKKS does not efficiently support comparison operations.

We encountered issues with potential malicious clients in our initial DDFed design, as noted by the reviewer. This prompted us to propose a feedback-driven collaborative selection mechanism. Similar to blockchain consensus technology, this mechanism can tolerate up to 50% of clients being malicious. This aligns with our threat model assumption that DDFed can handle adversaries with the ratio lower than 0.5.

Resp to Q2: The introduction of FHE for privacy-preserving federated learning often raises computational concerns due to secure computation over encrypted models and communication overhead from the larger size of encrypted model updates, as noted by the reviewer.

From a computational perspective, the cost of FHE for privacy enhancement in FL depends on the size of the trained model, not the dataset. In our experiment, evaluating models on the MNIST and FMNIST datasets involved about 0.23 million parameters. Additionally, as shown in our released prototype implementation, we run each training round sequentially with each client rather than in parallel. We believe that if DDFed is applied to a real FL scenario where all local training and operations are conducted in parallel, the total training time will be lower than reported in Table 1.

The primary goal of this paper is to validate the effectiveness of the proposed dual defense approaches. As indicated in the released open-source code, we implemented a simulated FL framework rather than an actual FL system at this prototype stage. Building an FL system in a distributed real-world network environment slightly deviates from the core focus of this paper. Therefore, we did not evaluate communication overhead by measuring network latency caused by transferring encrypted model updates instead of plaintext model updates.

Formally, depending on the security parameter settings, the communication overhead is linearly related to the model size, rather than an exponential relationship. In our CKKS setting with a security parameter of 128, we use the above-mentioned model with 0.23 million parameters as an example. Our manual measurement shows that the original model size is approximately 0.9MB, while the encrypted model size is about 20.4MB. This indicates that the communication payload overhead is roughly 20 times greater compared to a non-FHE protected solution.

In summary, our work may introduce additional computational and communication costs similar to most existing FHE-based privacy-preserving federated learning solutions. However, this design provides a strong guarantee of privacy preservation. While applying DDFed in a cross-device FL scenario involving thousands of devices might raise practical concerns, we believe it remains feasible for cross-silo FL scenarios where organizations or companies with higher network bandwidth and servers conduct FL training. It is worth noting that the cross-silo FL scenario is an important type of application according to recent surveys on federated learning.

Again, we appreciate the reviewer's comments. We hope our response addresses these concerns and positively influences the final decision with higher rating scores.

We appreciate the reviewer's concerns and suggestions.

Resp to Q1: The primary purpose that we initiated the attack at round 50 is to demonstrate the effectiveness of defense mechanisms and clearly show the comparative effects of different defense methods before and after an attack. This setup can also illustrate how various defensive measures impact training convergence and model quality, even without attacks.

DDFed is resilient to poisoning attacks from the beginning of training. Our design is not constrained by the attack's initiation round. Supplementary experiments on the FMNIST dataset with 100 clients in a non-iid setting support this claim.

Resp to Q2: We use only the last layer for similarity computation because our main goal is to add privacy-preserving functionality to existing poisoning defense strategies, more than on optimizing existing defense mechanisms. Based on our exploration, similarity-based methods and their variants offer a comprehensive defense effectiveness that is robust to various threat assumption setting such as server reliance on validation data, types of model poisoning attacks, number of compromised clients. Therefore, we chose one typical similarity-based defense strategy (Cosine Defense) as a starting point to enhance privacy-preserving features. Our approach can easily extend to other similarity-based detection variants that use full layers for secure similarity computation. We conducted additional experiments considering full-layer secure similarity computation on a larger dataset (CIFAR10) under various attacks.

Resp to Q3: In the appendix 3, we provided an overview privacy analysis of differential privacy-enhanced FHE-based secure similarity computation but did not include a formal proof. Generally, we believe the reviewer is concerned about whether $[[x]]+ \Delta$ equals $x + \Delta$, where $x$ is under FHE protection. However, this depends on the precision of the employed FHE schemes. Proving such a statement theoretically may require delving into the specific construction algorithm of the FHE scheme, which is beyond the scope of machine learning-oriented venues.

This paper utilizes CKKS constructions, which natively support high-precision secure computation on floating-point numbers. As a result, adding DP noise to encrypted similarity results does not degrade performance. To validate this, we conducted supplementary experiments on CIFAR10 using a simulated DDFed setup where DP noise was added to non-encrypted parameters. The reported results support this claim.

Resp to Q4: As stated in response 2, the primary goal and contribution of this paper are to enhance existing model poisoning defense strategies with privacy preservation functionality, addressing both privacy and model poisoning issues in FL simultaneously. Therefore, we did not focus heavily on indicator selection.

Our exploration in this field shows that similarity-based model poisoning defense methods and their variants provide comprehensive defense effectiveness when considering threat model assumptions such as server reliance on validation data, types of model poisoning attacks, number of compromised clients, and attacking rounds. While it may not be optimal for a specific threat model assumption, it offers relatively good overall defense quality as demonstrated by related papers.

This paper focuses more on privacy-preserving functionality than on model poisoning defense strategies. Therefore, we did not conduct the latest attacks [1-2] for comparison, as suggested by the reviewer, because it is outside the focus of this study.

Supplementary response: We also conducted additional experiments to evaluate our work on the CIFAR10 using a ResNet model as shown in Resp to Q2 section. Regarding related work, we have discussed the referenced study [5], but omitted studies [3-4]. These will be included in the final version.

Finally, we hope our response, particularly regarding the last layer setting and DP noise addition, addresses these concerns and positively influences the final decision.

Resp to advance attacks: We thoroughly examined the advanced attack suggested by the reviewer, as referenced in [1] above. That paper only discusses attacks on Krum and its variant, Trimmed Mean, and Median—all of which are not conventional similarity-based approaches. Therefore, we adopted a similar attack strategy and implemented a dynamic attack: conducting n attack iterations after n non-attack iterations to collect "before-attack" global models. We then averaged the crafted models (using the most effective attack) with the "before-attack" global models for subsequent periods of attack iterations. The results on CIFAR10 show that DDFed remains effective under such dynamic attacks.

Resp to DP protection and evaluation: We would like to emphasize that the DP-based perturbation is used to prevent potential privacy leakage from similarity scores. Specifically, it prevents adversaries from inferring private information about benign clients by exploiting decrypted similarity scores and previous global models, rather than directly targeting a specific client's model update as in the previous DGL attack.

As shown in formula 4 of the paper, suppose a client can infer private information; they must (i) solve a multivariate linear equation problem with only one equation successfully—given $C$, $B$, and $C=<A,B>$, find all parameters of $A$—and (ii) break the DP-based perturbation to find $W^{t-1}_i$. To our knowledge, we do not believe an adversary can accomplish step (i) because each round's $A$ is not fixed. If the similarity computation does not use full layers, this challenge becomes even greater.

However, let's consider a harsh condition: during convergence phases when model parameters tend to stabilize and A is approximately fixed while using full-layer models for similarity computing. We then conducted a simple inversion attack (e.g., DLG attack) as suggested by the reviewer. We assessed the PSNR results of both DP-based perturbation and non-DP perturbation on the MNIST dataset. The results below demonstrate the effectiveness of the DP-based similarity score perturbation.

By the way, we didn't set up special access control for readers; we just used the default settings when inputting the last round's response. We also haven't provided a general response yet for that round. As suggested, we've revised the scope of previous responses and added a general response to address the current status of the rebuttal.

Finally, we appreciate the reviewer's concerns and suggestions again. We hope our response has addressed these issues and will continue to positively influence the final decision in this round of discussions.

We appreciate the reviewer's concerns and suggestions.

Resp to major concern 1: Collusion between the server and clients is beyond the scope of our threat model assumptions in Section 3.1. In PPFL, each solution includes a threat model that defines the adversary's capabilities and behavior, following the security research tradition of defining threat boundaries before proposing solutions since no solution can claim absolute security. Our threat model assumes an honest-but-curious server that does not collude with any clients, consistent with most related work's assumptions. Most current PPFL approaches cannot handle such collusion issues, which is a strong security assumption. We believe our work represents substantial progress in this field. Our solution eliminates the need for a non-colluding two-server assumption found in most recent related works, making private and robust FL solutions more practical.

Resp to major concern 2: Based on our exploration in model poisoning attacks and defenses, our findings indicate that similarity-based detection methods and their variants provide excellent and comprehensive defense outcomes regarding the server's reliance on validation data, types of model poisoning attacks, number of compromised clients, and attacking round. This is why we have chosen the (cosine) similarity-based poisoning model detection methodology. Additionally, our approach can easily extend to similarity-based poisoning model detection variants. However, it does not provide a unified support to all other defense methods such as Krum or those relying on the assumption that the server has partial validation datasets. The reviewer's question represents a promising future direction. Regarding comparisons with other defense approaches, we have demonstrated their effectiveness in existing experimental evaluations, such as in Figure 2. We will include discussions of these choices in the final version.

Besides, our solution does not rely on the assumption that the initial round of training must include only benign clients. In fact, our work supports starting attacks from the first training round, meaning compromised clients can be present initially. To substantiate this claim, we conducted additional experiment, comparing our approach with baselines by initiating attacks in the first training round on the FMNIST with 100 clients in a non-iid setting (see reported results below). The reviewer's misunderstanding likely arises from our experimental setup and results. We began the attack at round 50 to demonstrate the effectiveness of defense mechanisms and to clearly show the comparative effects of different defense methods before and after an attack. This helps illustrate how various defensive measures impact training convergence and model quality, even in the absence of attacks.

Resp to minor concern 1: Given the limited response time, we conducted an additional experiment to evaluate our approach against related baselines on the CIFAR10 using a ResNet model with 100 clients in a non-iid setting. As shown in the tables below, our proposed method remains effective under various attacks.

Resp to minor concern 2: Using HE for privacy enhancement does not add significant overhead in FL recently, contrary to what most non-cryptography researchers might expect. Despite the inherent challenges of developing HE technology, the minor computational overhead reported in Table 1 is due to the following factors:

1. The scale of HE used is relatively small, with total parameters of models being about 0.23 million.
2. Our design mechanism involves only simple computations over the encrypted model, with just one layer of multiplication depth. It is well-known that multiplication depth significantly impacts efficiency of HE. As suggested, we conducted additional experiments on the CIFAR10 dataset using a ResNet model (approximately 11.2 million parameters) and 100 clients. The results showed that it takes an extra 974 seconds (16 minutes) compared to the non-protected FedAvg baseline. This increase is reasonable and acceptable given that our proposed dual defense approach offers strong privacy and security guarantees.

We appreciate the reviewer's concerns and suggestions. We hope our response addresses these concerns and influences the final decision positively.

Thanks for the response and additional experiments. However, some of my concerns remains.

Collusion between clients and server: Firstly, assuming no collusion between clients and servers poses great vulnerabilities in real-life, particularly with expanding client pools. Servers can easily introduce fake clients to undermine this assumption. Recent studies [1][2] have successfully addressed privacy concerns and ensured robust aggregation, eliminating the need of this assumption. Secondly, the paper mentioned by the authors [3] relies on two servers to validate the gradient normalization. The authors remove the two-server setting at the cost of simply skipping the normalization step. How to ensure that the clients' local weights are normalized?

Choise of similarity-based detection methods: I understand that the solution does not rely on the assumption that the initial round of training must include only benign clients. However, the aggregation protocol the authors choose does not have formal convergence guarantee. As is mentioned by another reviewer, I expect the theoretical proof under the case with and without DP noise.

[1] Lycklama, H., Burkhalter, L., Viand, A., Küchler, N., & Hithnawi, A. (2023, May). Rofl: Robustness of secure federated learning. In 2023 IEEE Symposium on Security and Privacy (SP) (pp. 453-476). IEEE.

[2] So, J., Güler, B., & Avestimehr, A. S. (2020). Byzantine-resilient secure federated learning. IEEE Journal on Selected Areas in Communications, 39(7), 2168-2181.

[3] Ma, Z., Ma, J., Miao, Y., Li, Y., & Deng, R. H. (2022). ShieldFL: Mitigating model poisoning attacks in privacy-preserving federated learning. IEEE Transactions on Information Forensics and Security, 17, 1639-1654.

We appreciate the reviewer's positive decision and those raised concerns. The challenge of the paper lies in resolving the dilemma where detecting model poisoning requires plaintext model updates from each client, while privacy protection demands safeguarding these updates.

This is our first attempt to address this dilemma using a homomorphic encryption scheme as the privacy-enhancing technology. Unlike other methods such as secret sharing, pairwise masking, and functional encryption, the native design of the homomorphic encryption-based secure aggregation approach prevents the aggregation server from obtaining both anomaly detection results and aggregated models. Thus, such a design results in introducing an additional interaction step compared to the original federated learning paradigm and incurs extra time costs due to privacy protection. However, this is inevitable. Existing privacy-preserving federated learning solutions typically sacrifice either computational efficiency or communication efficiency—or both—to provide strong privacy and security guarantees.

In our paper, we note that feedback-driven collaborative selection requires an additional round of interaction due to the native security model of homomorphic encryption. In our next research plan, we aim to use advanced cryptographic computation technologies like functional encryption, which have different security models and can eliminate the need for extra interaction rounds.

From a computational cost perspective, the use of homomorphic encryption for privacy enhancement does not add significant overhead actually. This is contrary to what most non-cryptography researchers might expect, as shown by the initial time cost results in Table 1. Apart from the inherent challenges of developing homomorphic encryption technology, we believe the minor computational overhead reported in Table 1 is due to the following factors:

1. The scale of homomorphic encryption used is relatively small. The total parameters for evaluating models on the MNIST and FMNIST datasets are about 0.23 million. As requested by other reviewers, we will also include evaluations on larger datasets and models such as CIFAR10 in the final version.
2. Our design mechanism does not involves complex computations over the encrypted model, with only one layer of multiplication depth. It is well-known that multiplication depth significantly impacts efficiency in homomorphic encryption. Besides, although we provided a computational cost comparison with other approaches, we did not measure latency. As shown in the released open-source code, we implemented a simulated FL framework rather than an actual FL system at this prototype stage. Our goal is to validate the effectiveness of the proposed dual defense approaches. Building an FL system in a distributed real-world network environment deviates from the core focus of this paper and will be addressed in future research stages.

Here are specific responses to the raised questions:

1. Compared to native FL paradigms or existing non-private solutions, our work introduces homomorphic encryption to enhance privacy preservation and support poisoning model detection. Due to the introduction of homomorphic encryption, providing a precise formal time complexity analysis is challenging. We believe the reviewer would like a breakdown of the time costs involved. In addition to the original federated learning training time, extra time is required for homomorphic encryption processes such as encryption, decryption, and secure computation (i.e., inner-product and addition among encrypted models and fusion weights or perturbation noise). Each client is responsible only for encryption and decryption, while all secure computations are performed by the server.
2. Due to limited experimental conditions, we only report results for client sizes ranging from 10 to 100. Theoretically, our proposed framework supports a larger scale of clients. Since the input sizes for encryption, decryption, and secure computation are linear relative to the number of clients, the time cost is also linear based on current reported results. For example, in evaluating the MNIST dataset as shown in Table 1, compared to a non-private solution, our approach requires approximately an additional 2 seconds per training round with 100 clients participating in FL training. Therefore, scaling up to thousands of clients may take about 20 seconds per round without considering network latency.
3. As stated in Section 3.1 regarding threat assumptions (i.e., the threat model in security and privacy papers), our work can handle scenarios where less than half of the clients are compromised. We will further clarify these statements and analyses in the final version.

We hope we have addressed the reviewer's concerns. We will improve the final version of this paper and thank the reviewers for their valuable comments.

Still, thank you for the valuable comments given earlier.