OMS: One More Step Noise Searching to Enhance Membership Inference Attacks for Diffusion Models

Weakness: Analyzing why certain datasets/attack combinations benefit more from the approach. By incorporating such an angle, the contribution could be significantly broadened.

Thanks for your valuable suggestion. In the revised paper, we supplement discussion about this topic in experiments (Section 4.2). We show the performance boosts achieved by OMS are significantly influenced by the strength of the original attacker. Specifically, when the original attacker is relatively weak, the additional improvements provided by OMS are more pronounced. Our method can significantly augment the performance of weak attackers (Table 1, 20.77 in AUC for SecMI on LFW). This may be attributed to the fact that when the original attacker struggles to identify the samples, the additional information provided by OMS becomes more significant.

Weakness: Presentation Issues

We apologize for the presentation issues in our previous submission. We have taken steps to improve the presentation of our work. Specifically, we have revised Table 2 to include the percentage improvement in performance, which provides a clearer comparison and facilitates a more informed understanding of the effectiveness of our approach. We have corrected the citation commands from “\cite” to “\citep” and “\citet” and revised the abbreviation usage throughout the paper to improve clarity and readability. In addition, we have added a brief introduction for the fixed-point iteration process. To further support this introduction, we have provided additional preliminaries on fixed-point iteration in Appendix A. We believe that these improvements enhance the presentation of our work.

Weakness: What does it mean: "During the inference phase, due to the infeasibility of the training noise" $\rightarrow$ what is infeasible? Getting access to the original noise?

Yes. In the context of our work, we aim to emphasize the challenge of accessing the original noise used during the training stage.

Thank you for taking the time to review our paper, and for your helpful comments. Below we will address your concerns point by point:

Weakness: Overall, the contribution of just including an additional step into the MIAs seems comparably small. While a lot of effort it done to put the contribution into a theoretically sound presentation, it still relies on the observation made by previous MIAs that there is a noise difference between members and non-members.

We would like to very respectfully, but firmly, push back against the assertion that our method is only a minor variation on existing MIA. While we have no doubt that the reviewer understands these “simple” methods from previous works and our study, we believe there may be an oversight from the reviewer regarding the overall progress and current state of this research direction.

Loss-based methods continue to dominate the field of MIA due to their direct measure of the model’s familiarity with training data. Consequently, most initial work on MIAs against diffusion models has also employed loss, similar to MIAs on other deterministic models. However, these approaches fail to account for the unique randomness of diffusion models, which has not been analyzed in previous studies.

Our method is highly motivated by the stochastic nature of diffusion models. We reveal that the noise inconsistency results in unreliable loss estimation during the inference stage. We further ask why these unreliable losses can also achieve certain MIA performance. We answer this question by our proposed noise search framework. We reveal that the convergence speed, rather than the loss itself, is the key factor distinguishing members and nonmembers: members and nonmembers exhibit completely different convergence dynamics in the noise search process. We further analyze previous loss-based MIAs from the perspective of our proposed noise search framework and show that the “loss” (i.e., the noise difference) is actually a residual, indicating the convergence speed. Based on this analysis, we propose OMS.

We believe our method is distinct from existing MIA and our contribution is significant. Besides, we provide a deeper exploration of the convergence property in Appendix B. Specifically, we validate the convergence of the fixed-point process using the contraction mapping theorem, analyze the convergence speed through error analysis, and expand the validation of these convergence property to transformer-based diffusion models. In summary, we believe our contribution is significant and represents a meaningful advancement in the field of MIAs against diffusion models.

Weakness: Analysis of disparate effect over different members: Are certain members more vulnerable? Can that be detected better with the additional step?

Thanks for this advice! We supplement an analysis on the effects of the OMS on different samples. More specifically, we depict the score distribution before and after applying OMS to visualize the impact brought by OMS (Figure F.1(a)). Furthermore, we examine the label distribution of the 1795 members that are misclassified as nonmembers before OMS (Figure F.1(b)). The results indicate that these members are nearly uniformly distributed across various classes, suggesting that no significant subset of members benefits disproportionately from the additional step.

How is the threshold chosen that leads to the conclusion that 1795 nonmembers are now correctly classified as members?

We choose the threshold based on the implementation of previous studies (i.e., SecMI and PIA [8-9]). A detailed explanation of the threshold selection process is provided in the original submission (Appendix E). Specifically, “we randomly select a small fraction (10%) of samples to obtain an optimal threshold, which is then used to classify the remaining samples”. For the reported percentage, we test 50,000 samples on Cifar10 dataset, consisting of 25,000 member and 25,000 nonmember samples. This dataset and the experimental setup are described in detail in Appendix C of the original submission.

Why would one have interest in improving weak attacks?

We want to clarify that the term ‘weak’ in this context refers to inferior performance, rather than a fundamental weakness in the method itself. Attackers with strong performance exhibit limitations in other perspective. For instance, GSA [7] requires access to the model’s gradients, positioning it as a purely white-box method with strong performance. The behavior of PIA [2] to predict proximal points is abnormal and distinct from typical user behavior. SecMI [1] requires over ten queries to the target model, whereas NA [3] requires only one, hence SecMI's stronger performance.

In practical applications, the choice of attacker should be tailored to the specific scenario, taking into account the unique strengths and weaknesses of each MIA method. While performance is an important consideration, it should not be the sole criterion for selecting an attacker. Furthermore, as noted, our method can also enhance strong attackers. Therefore, we believe that it is not only important to further improve the performance of MIA with high accuracy but also to enhance weak attackers, as this holds practical significance in various contexts.

Reference

[1] Duan, Jinhao, et al. "Are diffusion models vulnerable to membership inference attacks?." ICML 2023.

[2] Kong, Fei, et al. "An Efficient Membership Inference Attack for the Diffusion Model by Proximal Initialization." ICLR 2024.

[3] Matsumoto et al. “Membership inference attacks against diffusion models.” Arxiv 2023.

[4] Berinde and Takens. “Iterative approximation of fixed points.” Springer 2007.

[5] Bertran et al. “Scalable membership inference attacks via quantile regression.” NeurIPS 2024.

[6] Tang et al. “Membership inference attacks on diffusion models via quantile regression.” Arxiv 2023.

[7] Pang et al. “White-box membership inference attacks against diffusion models.”

[8] https://github.com/jinhaoduan/SecMI

[9] https://github.com/kong13661/PIA

Dear Reviewer,

We hope to have resolved all your concerns. If you have any further comments, we will be happy to address them before the rebuttal period ends. If there are none, then we would appreciate it if you could reconsider your rating.

Regards,

Authors

Comparison of DDPM-Cifar10 and U-ViT-Cifar10

   Results on DDPM-Cifar10:

  Results on U-ViT-Cifar10:

To substantiate the efficacy of our approach on transformer-based diffusion models, we have extended our experimental analysis to encompass the U-ViT model, which was trained on the ImageNet dataset. We utilize the pre-trained U-ViT checkpoint (U-ViT-M/4) sourced from [1] directly. Specifically, we employ 10,000 samples from the ImageNet training set as members and 10,000 samples from the ImageNet validation set as nonmembers. The outcomes of these experiments are presented in the subsequent table.

Our findings indicate that our method can significantly enhance current MIAs on transformer-based diffusion models when applied to more complex datasets, thereby further validating the effectiveness of our approach.

[1] https://github.com/baofff/U-ViT

---

Thank you for your helpful comments again. We hope to have resolved all your concerns. If you have any further comments, we will be happy to address them before the rebuttal period ends.

We thank the reviewer for highlighting the importance of avoiding evaluation pitfalls, as discussed in [1]. We acknowledge the significance of this concern and are testing the performance of our method towards Stable Diffusion on the LAION-mi dataset to ensure rigorous evaluation. Given the impending end of the discussion period and the computational overload required for LAION-mi dataset (the deduplication and sanitization preprocess), we may not be able to provide these results before the deadline. However, we will endeavor to do so as soon as possible. If we are unable to meet this timeline, we will include the performance evaluation leveraging the LAION-mi dataset in the revised paper.

(Besides, the experimental results on U-ViT-ImageNet are available.)

Thank you again for your valuable feedback. If you have any further comments, we will be happy to address them before the rebuttal period ends.

[1] Dubiński, Jan, Antoni Kowalczuk, Stanisław Pawlak, Przemyslaw Rokita, Tomasz Trzciński, and Paweł Morawiecki. "Towards more realistic membership inference attacks on large diffusion models." In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 4860-4869. 2024.

Thank you for taking the time to review our paper, and for your helpful comments. Below we will address your concerns point by point:

Weakness: While the fixed-point iteration method is interesting, the paper could benefit from a deeper exploration of its convergence properties.

Thanks for this suggestion! We have enhanced our exploration of the convergence properties in the revised paper. In the original submission, we discussed the convergence of the generated sequence $\\{\\epsilon^n\\}$ and leveraged residuals as an indicator of convergence speed. To further deepen our analysis, we have now included an examination of the function $f$ which is composed by the diffusion models. Specifically, we have analyzed the contraction of the objective function and estimated the contraction constant $k$, observing that it remains continuously below 1, thereby reinforcing the convergence of the fixed-point iteration (More details can be found in Appendix B.1). Additionally, we have conducted an error analysis according to [1] to investigate the convergence speed, providing compelling evidence that members converge more rapidly than non-members (More details can be found in Appendix B.2).

Weakness & Question: More details on how OMS performs in terms of memory and computation time.

Thanks for this comment. In terms of memory and computation time, our proposed OMS method exhibits comparable performance to baseline attackers. Specifically, our method requires one additional query to the target diffusion model compared to the baseline methods. Consequently, in terms of memory consumption, our method utilizes the same amount of memory as the baseline attackers. Regarding computation efficacy, the increase in the number of queries translates to a minor increase in computation time. We provide the required queries for baseline attackers:

Question: Could the authors clarify how the convergence rate differs between member and non-member samples, and whether this could be generalized across various types of diffusion models? More insights into this mechanism would help to strengthen the theoretical foundation of the approach.

Thanks for the comments. We have conducted a more detailed analysis of the convergence rate differences between member and nonmember samples, and their generalization across various types of diffusion models. Specifically, we have included an error analysis in Appendix B.2, which provides an additional perspective on the convergence speed. Furthermore, we have conducted a convergence analysis of the transformer-based diffusion model (U-ViT), using both residual and error analysis, and presented the results in Figure B.2. These findings provide strong evidence that the assumption that member samples converge faster can be generalized across various types of diffusion models. This analysis helps to strengthen the theoretical foundation of our approach and provides deeper insights into the underlying mechanisms of convergence in fixed-point iteration.

[1] Berinde, V. "Iterative approximation of fixed points." (2007).

Dear Reviewer,

We hope to have resolved all your concerns. If you have any further comments, we will be happy to address them before the rebuttal period ends. If there are none, then we would appreciate it if you could reconsider your rating.

Regards,

Authors

Thank you for taking the time to review our paper, and for your helpful comments. Below we will address your concerns point by point:

Weakness: Some statements in the paper can lead to confusion. For example, in the introduction, it is stated: “To address these privacy concerns, Membership Inference Attacks (MIA) Shokri et al. (2017) have emerged as a potential solution.”

Thank you for pointing this out. In response to the potential confusion arising from the statement in the introduction, we have revised the corresponding part to enhance clarity and accuracy. Specifically, we have replaced “To address these privacy concerns” with “To audit these privacy risks”.

Weakness: While the methodology of this paper is technically sound, it would benefit from a clearer explanation of the real-world implications of privacy risks in diffusion models. To strengthen the paper, it is recommended that the authors include a specific section discussing the real-world impacts of privacy risks and their potential effects on individuals or organizations, along with 1-2 concrete examples to support this discussion.

Thanks for this advice! To give a clearer explanation of the privacy risks in diffusion models, we have expanded our discussion in Appendix H to provide a comprehensive analysis of the practical applications of MIA and the associated privacy risks. Specifically, we elaborate how MIA can serve as a powerful tool to detect unauthorized data abuse, which is crucial in the current digital landscape. This also raises awareness of data privacy concerns. Furthermore, we highlight the potential utility of MIA for organizations, as it can be leveraged as a privacy measurement and employed for auditing and regulatory purposes.

Weakness: Some terms and technical concepts are introduced without sufficient background or explanation.

Thanks for this advice. To enhance the clarity and accessibility of our paper, we have included a brief description of the fixed-point iteration process in the introduction (lines 70-72) to provide readers with a foundational understanding of this key concept. Additionally, we have appended a new section (Appendix A), introducing the preliminaries of the fixed-point iteration, which offers a more comprehensive overview of the technical details.

Question: Could the authors elaborate on how the noise search mechanism operates in practice? Specifically, what criteria are used to determine the optimal stopping point in the noise search process?

When to stop is a trade-off between effectiveness and computational costs. We have conducted experiments to analyze the impact of iteration times on the performance of the noise search mechanism. As demonstrated in Figure 3 of the main text, two iterations yield significant performance gains compared to one iteration (85.71 vs 78.28). However, the gains observed with three iterations compared to two iterations are relatively limited (87.06 vs 85.71). This experimental evidence suggests that while additional iterations may further enhance performance, stopping after two iterations represents a computationally economical and efficient choice. Furthermore, we provide additional ablations on iteration numbers (N) on Cifar10 dataset. The results are in the following table which show that further steps beyond the proposed OMS (N=2) do not result in significant performance gains, indicating a rapid performance saturation.

Therefore, based on these findings, stopping when N=2 is a practically viable choice for determining the optimal stopping point in the noise search process.

Question: How well does OMS perform across different types of diffusion models, beyond those tested in the experiments?

Our experiments in the original submission demonstrate that the proposed method exhibits strong generalization capabilities across different architectures (including conditional and unconditional, CNN-based and transformer-based diffusion models). Additionally, we have conducted further experiments on fine-tuned diffusion models to assess its performance in more specific contexts. Specifically, we evaluate our method on diffusion models finetuned on the Pokemon dataset following previous works [1-2]. The results of this experiment, as presented below, further validate the effectiveness of the OMS method.

Question: Are there specific classes of diffusion models where OMS may be less effective?

We have not identified any specific classes of diffusion models where the OMS may be less effective. However, we have observed that the performance gains achieved by OMS are influenced by the strength of the original attacker. Specifically, when the original attacker is relatively weak, the additional improvements provided by OMS are more pronounced. Our method can significantly augment the performance of weak attackers (Table 1, 20.77 in AUC for SecMI on LFW). Even for nearly saturated attackers, our method can achieve considerable improvements (Table 1, 94.89 to 97.26 in AUC for PIA on Cifar10). In summary, OMS demonstrates robust performance across a wide range of diffusion models, and is particularly effective for weak attackers.

[1] Duan, Jinhao, et al. "Are diffusion models vulnerable to membership inference attacks?." ICML 2023.

[2] Kong, Fei, et al. "An Efficient Membership Inference Attack for the Diffusion Model by Proximal Initialization." ICLR 2024.

Dear Reviewer,

We hope to have resolved all your concerns. If you have any further comments, we will be happy to address them before the rebuttal period ends. If there are none, then we would appreciate it if you could reconsider your rating.

Regards,

Authors