Privacy-Preserving of Deep Learning Queries by Domain Shifting

We greatly appreciate the time and effort you dedicated to reviewing our work. Your valuable feedback has highlighted several important points that we will address in detail below:

1. Computational requirements in the model-agnostic setting

Thank you for raising the concern about computational overhead. We acknowledge that transforming input images through complex operations, such as training GANs, can be resource-intensive. In our experiments, we demonstrated that the training process could be completed within one week using a single GPU. While this may seem demanding, it is a one-time cost for the user, and the trained model can then be used repeatedly for inference. We believe this trade-off between computational effort and privacy preservation is practical for users with sufficient local computational resources. However, we acknowledge that this might not be feasible for all users, and we will address this limitation explicitly in the revised manuscript.

2. Access to labeled training data

You raise an important point regarding the need for labeled training data. Our work assumes that users have access to publicly available labeled datasets for training the encoder and decoder. While we recognize that using private datasets would be ideal, such datasets are often inaccessible due to commercial or legal restrictions. Hence, we designed and evaluated our approach using public datasets, which are more widely available to users. Regarding the suggestion of local inference, we agree that if users have sufficient data and computational resources, training a local inference model may indeed sidestep the need for privacy-preserving remote inference. However, in scenarios where users lack the resources or expertise to train complex models for local inference, our approach provides an alternative by focusing on privacy preservation in remote settings. We will clarify this distinction in the revised manuscript.

3. Privacy guarantees in the model-specific setting

Thank you for pointing out the importance of rigorous privacy guarantees and referencing noise-based mechanisms. While we focus on reducing the SSIM between the original and transformed images to demonstrate obfuscation effectiveness, we agree that SSIM alone does not constitute a formal privacy guarantee. We have considered model inversion attacks and input reconstruction attacks on the encoder-decoder mechanism. It is worth noting that these attacks are generally effective against shallow neural networks, while the encoder in our method uses a deep neural network, making successful reconstruction significantly more challenging. However, we acknowledge the need for stronger, more formal privacy evaluations. In response to your feedback, we will: 1 Include experiments evaluating membership inference attacks to assess privacy leakage. 2 Discuss the limitations of SSIM as a metric and explore additional metrics or methods to strengthen privacy guarantees. 3 Add evaluations on potential attacks and their results in the appendix.

We are grateful for your detailed review and constructive criticism. Your feedback has helped us identify areas for improvement, and we will incorporate these changes in the revised manuscript to strengthen the rigor and clarity of our work.

We greatly appreciate the time and effort you put into reviewing our paper and providing valuable suggestions and questions. Below, we address your comments in detail:

1. Risk of inference attack through repeated disguised data and output observation

Thank you for pointing out this potential risk. To mitigate the issue where an attacker could exploit repeated disguised data to infer output patterns and compromise the label replacement scheme, we propose introducing additional complexity to the obfuscation process. Specifically, users can randomly select one of several pre-defined permutation plans to obfuscate their data. This randomized approach significantly increases the difficulty for an attacker to identify consistent patterns, thereby enhancing privacy protection. We will discuss this scenario and our proposed countermeasure in the revised manuscript.

2. Clarification of the differences between encoder-decoder usage in [1] and [2]

We appreciate your suggestion to include a discussion of the referenced papers. Our approach, while leveraging encoder-decoder mechanisms, focuses primarily on input obfuscation for inference rather than federated learning. This distinction will be clarified in the revised manuscript. Thank you for pointing this out, and we will consider the relevance of these papers in future iterations of our work.

3. Exploration with more sensitive data

Thank you for this insightful suggestion. We agree that evaluating our method on more sensitive data, such as facial images or address data in NLP, would provide additional value. However, as most NLP-based privacy risks are associated with multi-query scenarios, we believe exploring obfuscation at a level higher than individual queries could be a promising direction. While this is beyond the current scope of our work, we are excited to explore these scenarios in future research and will mention this as a potential direction in the conclusion of our paper.

4. Ensuring consistent classification results across different models

Thank you for raising this important question. To ensure consistent classification results across models with different structures, we address this by training the models with the same labels during the training phase. By aligning the training data with consistent labels, we ensure that models are highly likely to produce consistent outputs for the pseudo-data, regardless of architectural differences. This training strategy significantly minimizes variability in classification outcomes and maintains the decoder’s accuracy. We will include this explanation in the revised manuscript to clarify our approach.

5. Formal privacy metrics for quantitative evaluation

Thank you for suggesting the inclusion of formal privacy metrics. While our current evaluation relies on SSIM to empirically measure the irrelevance between real and obfuscated inputs, we recognize the importance of leveraging formal metrics, such as membership inference attacks, to quantify privacy guarantees. We will conduct experiments using membership inference attacks in our revised evaluation and include these results to provide a more robust quantitative assessment of our method’s privacy-preserving capabilities.

We greatly appreciate the time and effort you put into reviewing our paper and providing valuable suggestions and questions. Thank you for your constructive feedback. Below, we address your comments in detail:

1. Comparison with existing approaches in the evaluation part

In our evaluation, we focus on three aspects for comparison with existing methods: • Model/Pipeline Performance • Privacy-Preserving Performance • Temporal Overhead Model/Pipeline Performance:In our experiments, we provide a detailed evaluation of performance degradation across various datasets and model architectures. Unlike homomorphic encryption and multi-party computation, which are designed to operate on basic computing operations and communication protocols, our method directly interacts with deep neural networks (DNNs). Since these existing methods do not operate on the DNN model itself, they inherently do not introduce performance degradation. However, this difference highlights that they are not directly comparable to our approach in this context. Privacy-Preserving Performance:We evaluate privacy-preserving performance by calculating the Structural Similarity Index Measure (SSIM) between the real input and obfuscated input. This demonstrates the irrelevance between the two, ensuring that an adversary cannot recover the real input from the obfuscated one. In contrast, existing approaches like homomorphic encryption rely on the inherent complexity of encryption algorithms during communication and inference. Temporal Overhead:We briefly discussed the temporal overhead of our method compared to existing approaches in the overhead section. In the revised version, we will include a more detailed comparison in the appendix to further substantiate our claims.

2. Clarification of the threat model

Regarding the threat model, we assume the service provider to be an honest-but-curious adversary. This means that while the service provider performs computations as intended, it may attempt to infer private information from the data it processes. On the adversary's side, they only have access to the obfuscated input and the obfuscated output. As demonstrated in the evaluation section, it is challenging for the adversary to recover the real input or output from these obfuscated forms. We will clarify this assumption in the threat model section and provide additional examples to strengthen its understanding.

We hope this addresses your concerns, and we will ensure these clarifications are incorporated into the revised manuscript. Thank you again for your thoughtful comments and for helping us improve our work.