Privacy-Preserving Federated Learning via Homomorphic Adversarial Networks

Thank you very much for your thorough review and valuable feedback on our work. Your comments have been instrumental in improving the quality of our manuscript, and we appreciate the opportunity to respond to your concerns. Below, we provide detailed responses to the issues you raised:

1. Regarding the discussion of Threshold Homomorphic Encryption (Threshold HE)
   We appreciate your suggestion regarding this domain. Threshold HE indeed has advantages in handling client dropout issues, making it an important research area. Thank you for recommending an expanded comparison. We understand that extending comparisons can help showcase the broad applicability of our method; however, due to space constraints, we have chosen to focus primarily on Multi-Key Homomorphic Encryption (MK-HE), which is more directly relevant to our work. The current experiments aim to validate the fundamental performance and key features of our proposed scheme, covering various commonly used datasets and attack models, which we believe is sufficient to support the objectives of this paper.

2. Regarding formal security proofs
   We acknowledge that the current security analysis does not meet strict formal standards. The discussion in Appendix G aims to evaluate system security through indirect methods, but it falls short of providing formal proof. However, due to the black-box nature of the current scheme, traditional formal security proof methods such as UC-Security are not applicable. For pseudo N-1 collusion attacks and their more complex variants, we will provide a more detailed analysis of potential collaborative attack scenarios in future work.

3. Regarding the novelty of the GAN-based approach
   We have made our contributions more explicit in the revised manuscript. Firstly, we have proposed a scheme that allows clients to encrypt data without depending on the original model, which has not been thoroughly discussed in existing work. This significantly reduces the risk of model exposure and enhances system privacy. Secondly, existing approaches do not address the challenge of key distribution, which is an important breakthrough in our method. Lastly, existing work primarily focuses on message encryption, while our design aims to achieve homomorphic encryption, which increases the difficulty level significantly. Homomorphic encryption requires a more complex trade-off between defense against attacks and maintaining accuracy. Unlike simple message encryption, homomorphic encryption adds complexity in defending against adversaries, which led us to employ a multi-step training mechanism to balance privacy protection and model performance.

4. On communication overhead
   The reviewer rightly pointed out the issue of communication bottlenecks. We agree that optimizing communication overhead is crucial for enhancing the practical usability of the proposed method. While we have provided a quantitative analysis of communication overhead in the paper, we also recognize that some ciphertext transmissions may be redundant in the current design. Future work will focus on optimizing both the encryption model and ciphertext design to reduce communication costs and improve system efficiency.

5. On improving the structure of the manuscript
   We have made modifications to Sections 1 and 3 to clarify the research problem more explicitly.

6. Regarding the experimental setup
   All experiments were conducted in a single-machine simulation environment, where we simulated multiple clients and extracted communication volumes to analyze the process.

7. On stronger attack models
   Privacy attacks, such as DLG attacks, tend to perform better on simpler models, making them critical for initial validation. Improvements in attack methods often aim to extend their applicability to more complex models. Intuitively, demonstrating security in simpler models implies that the scheme can also defend against attacks in more complex models. However, security in complex models does not necessarily guarantee security in simpler ones. Furthermore, given the diversity of complex models, comprehensive testing cannot cover all scenarios. Thus, we believe it is essential to validate the scheme using simpler models as a fundamental step. Future work will expand to more complex attack models to further validate the robustness of the scheme.

Thank you for your detailed review and valuable feedback on our manuscript. Below are our responses to your specific comments:

1. Regarding the assumption of at least two honest clients:
   We understand that the assumption of at least two honest clients may not always hold in certain scenarios. Traditional homomorphic encryption (HE) or multi-key homomorphic encryption (MK-HE) schemes also rely on similar assumptions, as with only one honest client, other clients can infer its data through simple computation. The primary objective of this study is to explore the potential of neural networks in emulating multi-key homomorphic encryption and validate their feasibility and initial performance. Thus, this assumption serves as a contextual condition for our work, which primarily focuses on comparisons with existing methods. This study lays the foundation for future efforts to explore more general security models, including potential modifications to relax this assumption.

2. On the formalization and rigor of security analysis:
   We acknowledge that the current security analysis may not meet the strict formalization standards of traditional cryptographic proofs, given the black-box nature of neural network encryption. In this work, we used experiments and attack simulations to validate the security of the proposed scheme. Future work will incorporate more theoretical analyses and mathematical modeling to systematically evaluate the security and applicability of this approach.

3. On comparative experiments, dataset diversity, and the adaptability of AHE:
   Thank you for your suggestions to expand the range of comparisons, increase dataset diversity, and validate the adaptability of the AHE method. The goal of this study is to propose a novel method and validate its fundamental performance and key characteristics. The current experiments, covering multiple commonly used datasets and attack models, are sufficient to support the research objectives of this paper. This work establishes a foundation for future research, including extending experiments to more datasets (e.g., text datasets), exploring more complex model architectures, and analyzing performance in diverse federated learning scenarios. We hope this study will inspire further interest in this direction and provide a basis for subsequent studies.

4. On communication overhead and its optimization:
   We agree that optimizing communication overhead is critical for enhancing the practical usability of the proposed method. While we have provided a quantitative analysis of the communication overhead in this paper, we also recognize that some ciphertext transmissions might be unnecessary in the current design. Future work will focus on optimizing the encryption model and ciphertext design to reduce communication costs and improve system efficiency.

Once again, thank you for your thorough review and constructive suggestions. We hope these responses address your concerns and we look forward to your further feedback.

We are sorry about this decision, but we would like to defend our contributions. 

Assumption of Two Honest Clients

The assumption of at least two honest clients is a commonly accepted practice in federated learning with homomorphic encryption, as demonstrated in prior works (e.g., [1-7]). This assumption ensures that the aggregation output does not trivially reveal the honest gradient. As we have mentioned, if only one honest client remains, an attacker can easily derive the values of the other clients and directly compute the honest client’s value, requiring no technical sophistication.

Experimental Validation

Similarly, the use of three datasets and models for validation is widely accepted in the field (e.g., [5-7]). Many related works primarily focus on small-scale datasets like MNIST to demonstrate the effectiveness of proposed methods. Based on these references and field practices, we believe our experimental setup sufficiently validates our method and ensures our core contributions are clear and focused.  

System Efficiency

Regarding system efficiency, we acknowledge the trade-offs between computation and communication. Compared to prior systems, HAN achieves a remarkable 6,075-fold improvement in aggregation speed, accompanied by a 29.2-fold increase in communication overhead. This trade-off is an inherent characteristic of secure aggregation methods, and different scenarios have varying requirements for computation and communication balance. Our method is designed to cater to cases where computational efficiency is a priority.  

Lastly, we sincerely thank the reviewer for their time and effort in providing detailed and constructive feedback. 



References

[1] Rathee, M., Shen, C., Wagh, S., & Popa, R. A. (2023, May). Elsa: Secure aggregation for federated learning with malicious actors. In 2023 IEEE Symposium on Security and Privacy (SP) (pp. 1961-1979). IEEE.

[2] Bell, J. H., Bonawitz, K. A., Gascon, A., Lepoint, T., & Raykova, M. (2020). Secure single-server aggregation with (poly)logarithmic overhead. In CCS, 2020.

[3] Bonawitz, K., Ivanov, V., Kreuter, B., Marcedone, A., McMahan, H. B., Patel, S., Ramage, D., Segal, A., & Seth, K. (2017). Practical secure aggregation for privacy-preserving machine learning. In CCS, 2017.

[4] Burkhalter, L., Lycklama, H., Viand, A., Kuchler, N., & Hithnawi, A. (2021). ROFL: Attestable robustness for secure federated learning. CoRR, 2021.

[5] Kadhe, S., Rajaraman, N., Koyluoglu, O. O., & Ramchandran, K. (2020). FastSecAgg: Scalable secure aggregation for privacy-preserving federated learning. CoRR, 2020. 

[6] Ma, J., Naas, S. A., Sigg, S., & Lyu, X. (2022). Privacy‐preserving federated learning based on multi‐key homomorphic encryption. International Journal of Intelligent Systems, 37(9), 5880-5901.

[7] Cai, Y., Ding, W., Xiao, Y., Yan, Z., Liu, X., & Wan, Z. (2023). SecFed: A secure and efficient federated learning based on multi-key homomorphic encryption. IEEE Transactions on Dependable and Secure Computing.

Thanks for the detailed response from the authors. After reviewing these responses, I have decided to maintain my original score for the following reasons:

1. The authors have not provided alternative security models or discussed the impact of an increased number of malicious clients, which was a key concern from my previous review.
2. The manuscript still lacks the rigorous security proofs or mathematical models that were requested. The current approach of relying on experiments and simulations is insufficient.
3. The authors have not expanded their comparisons to include a broader range of PPFL methods or demonstrated the adaptability of HANs across various datasets and model architectures.
4. The authors have acknowledged the issue but have not made substantial revisions to reduce communication costs or optimize the system's efficiency.

Thank you for your detailed review and valuable feedback on our manuscript. Your comments have been highly insightful and have significantly contributed to improving our work. Below are our responses to the issues you raised.

1. On the accuracy of the model and the precision of decrypted parameters

   We appreciate your observation regarding the potential inaccuracies in decrypted parameters due to the characteristics of the encryption-decryption model. As described in the manuscript, the design of HANs aims to balance computational accuracy and privacy preservation in federated learning by tolerating minor errors. Our experimental results demonstrate that this design achieves strong overall model performance while enhancing computational efficiency.

2. On the lack of formal security proof

   We understand the importance of formal security proofs and recognize the challenges posed by the black-box nature of neural network encryption. This paper introduces a pioneering framework that demonstrates the feasibility of neural network-based encryption for federated learning. However, we plan to explore additional methods for analyzing and validating the security of our approach in future research.

3. On the KeyGen mechanism and implementation details

   Regarding your concern about the KeyGen mechanism, our private keys are currently generated randomly. We rely on neural networks to learn this generation pattern, ensuring Perfect Key Uniformity. Experimental results confirm that the neural networks successfully learn this pattern, making a one-time-pad encryption scheme without key sharing feasible. This is consistent with the statement: “The learning does not require prescribing a particular set of cryptographic algorithms, nor indicating ways of applying these algorithms: it is based only on a secrecy specification represented by the training objectives.” . (Abadi & Andersen,2016)

   Additionally, chosen-plaintext attacks are infeasible in our scheme, as attackers cannot access the private model, analogous to not having access to a private key. As stated in our manuscript, chosen-plaintext attacks in our framework correspond to Known-Model Attacks (KMA). The effectiveness of our defenses against such attacks has been thoroughly discussed in the manuscript.

4. On the detailed description of model structures

   We acknowledge that the neural network-based multi-key homomorphic encryption method requires significant effort to further optimize model structure and training methods. The primary goal of this paper is to demonstrate the feasibility and effectiveness of such an approach. While the current model structure is not yet optimal, we believe that the training process and methodology described in the manuscript allow for the reproducibility of the experiments. If the paper is accepted, we will release the model weights to further assist in experiment replication.

Thank you again for your thoughtful feedback and constructive suggestions.

We sincerely thank you for your thoughtful and constructive comments. We have carefully addressed your suggestions and made revisions to enhance the manuscript. Below are our detailed responses:

1. Typographical Error and Proofreading:
   We appreciate your attention to detail. We have thoroughly reviewed the manuscript to address typographical, spelling, and formatting issues, ensuring improved accuracy and presentation throughout.

2. Clarity and Flow in the Introduction:
   We have revised the Introduction to clarify the core contributions earlier and to ensure a logical flow from problem context to solution. By introducing the use of neural networks as a substitute for MK-HE upfront, we aim to better communicate the novelty and significance of our method to readers.

3. Organization of the Method Section:
   We have integrated key content from the appendices into Section 3.6 and 3.7 to improve the clarity and accessibility of the methodology. However, as Section 3.1 introduces a new and complex concept, maintaining precision and completeness is critical. Further simplifications to this section would risk oversimplifying key ideas and reducing the comprehensibility of the framework. Consequently, we have preserved the necessary level of detail to ensure accuracy and reliability.
   Due to the numerous revisions made throughout the manuscript, we are unable to reproduce the updated content in its entirety within this response. We sincerely thank you again for your outstanding contributions, which have greatly helped us improve the quality and clarity of the paper.

We deeply appreciate your valuable feedback and thoughtful review, which have significantly contributed to refining our work. Thank you again for your time and effort.

Thank you for your detailed review and valuable feedback on our manuscript. Your comments have been highly insightful and have significantly contributed to improving our work. Below are our responses to the issues you raised.

We are grateful for your recognition of the formal definitions and the novelty of our proposed concept. These aspects form the foundation of our contributions.

1. On the detailed descriptions of the AHE scheme

   Thank you for your comment. To address this, we have made significant revisions to Section 3. Specifically, we have streamlined the definitions and statements to remove redundancy and focus on the essential components of the scheme. These changes aim to improve clarity and readability while maintaining academic rigor and ensuring reproducibility. We hope that these revisions make the design details more accessible and straightforward.

2. On the experimental analysis

   The experiments in the current work focus on validating the core aspects of the proposed scheme, including training accuracy, the impact of the PPU mechanism, and defenses against different types of attacks. These results demonstrate the feasibility and effectiveness of our approach as a foundation. While additional evaluations are needed for comprehensive validation, we believe the novelty of the proposed method makes it an important step forward. We aim to encourage community interest and collaborative exploration to further develop and refine this direction.

3. On the choice of simple models for security validation

   Privacy attacks like DLG are particularly effective on simpler models, making them critical for initial validation. By demonstrating that our scheme resists these attacks, we establish a fundamental level of security that supports its applicability to more complex scenarios. This stepwise approach enables future research to build upon a solid foundation while expanding to diverse models and datasets.

4. On the systematic nature of the security analysis

   We recognize the limitations of the current security analysis and the challenges posed by the black-box nature of neural network encryption. This work serves as a first step in proposing a novel approach. In addition to experimental validation, we aim to develop systematic methodologies for analyzing security in future studies. By introducing this framework, we hope to inspire further advancements in this area.

Once again, we deeply appreciate your constructive feedback, which helps us identify directions for improvement and inspires further research in this field. We hope our responses address your concerns.

Thank you for your detailed review and valuable suggestions. Your feedback provides important insights for improving our research. Below are our responses to your comments:

1. Understanding and Comparison of Related Work

   We sincerely appreciate the detailed feedback from the reviewer and acknowledge that our comparison with related work is indeed insufficient. In particular, we recognize the need to discuss and compare our approach with secret-sharing-based methods, such as ACORN and Bonawitz, which demonstrate strong resistance to collusion attacks through input validation mechanisms.

   To avoid overcomplicating the current paper and to ensure the clarity of our core contributions, we would like to clarify that our main contribution lies in leveraging neural networks to implement homomorphic encryption in privacy-preserving federated learning, addressing issues related to key distribution and collaborative decryption. We believe it is more appropriate to further explore these topics in follow-up research.

2. Communication Overhead

   The reviewer raises a valid concern regarding the communication overhead of HANs. While the current design prioritizes computational efficiency, we acknowledge that the increased communication cost may limit practical applications. In future work, we aim to optimize the ciphertext structure and model update mechanisms to minimize unnecessary transmissions, thereby improving the approach’s feasibility in real-world scenarios.

Once again, we deeply appreciate your constructive feedback, which helps us identify directions for improvement and inspires further research in this field. We hope our responses address your concerns.