Towards Black-Box Membership Inference Attack for Diffusion Models

Q4: Could authors clarify the challenges that will be encountered while implementing MIA without accessing to the internal U-net, as well as how the author effectively addresses these challenges?

A4: We appreciate the reviewer’s insightful question. The primary challenge in implementing Membership Inference Attacks (MIA) without accessing the internal UNet lies in determining membership based solely on the model’s input and output, without direct access to the model’s parameters. This setting is reasonable because many companies are reluctant to release their code and weights of the model. To address this challenge, we reframe the problem as a series of individual detections, focusing on the model's ability to provide consistent predictions for training set images. Our method leverages the intuition that the model achieves more accurate and consistent outputs for samples it has seen during training. By repeatedly querying the model with perturbed versions of the target image, averaging the results, and comparing them to the original, we can infer membership by evaluating the consistency of the predictions. This enables us to perform effective image-to-image detection without accessing the model's internal parameters.

We thank the reviewer once again for the valuable and helpful suggestions. We will continue to provide clarifications if the reviewer has any further questions.

We sincerely thank the reviewer for the comments and suggestions. Below, we address the primary concern that has been raised.

Q1: A certain part of experimental results reveal that the proposed method can only achieve very limited improvements over the existing SOTA methods.

A1: We thank the reviewer for the comments. In the DDPM experiment setup, previous methods have already achieved high accuracy (AUC > 0.9), so the potential for improvement is inherently limited. However, in the Diffusion Transformer and Stable Diffusion setups, our method achieves a more significant AUC improvement, with an increase of about 10%. We would like to emphasize that our main contribution is removing the dependence on accessing the internal UNet structure, which allows our method to be applicable to a wider range of scenarios compared to previous approaches.

Additionally, we would like to clarify that the AUC metric is more stable than the True Positive Rate (TP) when the False Positive Rate is fixed at 1%. Because the classification threshold is determined by a small number of misclassified samples, leading to greater volatility. When we take three different random seeds for CIFAR-10, the results show significant variation. Below, we present the TPR @1% FPR for the baseline methods and our algorithm for the DDPM model with three random seeds on the CIFAR-10 dataset:

We also present the AUC for the baseline methods and our algorithm for the DDPM model with three random seeds on the CIFAR-10 dataset:

As seen, the TP metric exhibits large variance for all methods, whereas the more stable AUC metric clearly demonstrates that our method outperforms the others. We also want to mention that our model consistently outperforms baselines on Diffusion Transformer(Table 2) and Stable Diffusion(Table 3).

Q2: In the ablation experiments, the author attempts to test the impact of experimental parameters on the robustness of the algorithm; however, the evaluation metric is one-sided (i.e., only AUC).

A2: We thank the reviewer for the comments. In the main experimental results presented in Section 5, we report AUC, ASR, and TPR@1%FPR. In the ablation study section, we focus on AUC due to space limitations, but the ASR ablation is included in Appendix B with corresponding explanations in the main text. To provide more comprehensive details, we also present the TPR@1%FPR results for DDIM on CIFAR-10 from the ablation study below:

As observed from the experiments with random seeds in Q1, the TPR@1%FPR values exhibit inherent instability. However, the ablation study demonstrates that our algorithm remains effective, with performance not changing significantly across different hyperparameter choices.

Q3: The application experiments may have biases, as the author only conducts the evaluations on a single model (DALL-E) and relies on a small dataset (i.e., only 30 famous paintings and 30 generated paintings). Hence, the experiment results are therefore difficult to be convincing. Without comprehensive experiments, the effectiveness of the proposed methods cannot be validly verified.

A3: We appreciate the reviewer raising this question. Our main experimental results are provided in Section 5, where we evaluate the performance of our method in a more comprehensive setting. We conduct extensive experiments on DDIM, Diffusion Transformer and Stable Diffusion with many datasets including CIFAR-10/100, ImageNet, Laion5.

The purpose of Section 6 is to demonstrate the practical applicability of our algorithm to commercial models. Specifically, we use DALL-E 2’s variation API to show how our method can detect membership by leveraging the model’s outputs. Since DALL-E 2 does not provide a publicly accessible training set, the experiment in Section 6 primarily serves as demonstrative application scenario rather than an extensive evaluation with large-scale datasets. We agree that further evaluation on more diverse datasets and models would enhance the robustness of the findings, and we plan to include such experiments in future work.

Thanks again for your valuable feedback! Could you please let us know whether your concerns have been addressed? We are happy to make further updates if you have any other questions or suggestions.

We thank the reviewer for the comments and constructive suggestions. In the following, we address the main concern raised.

Q1: While the proposed attack method does not require model parameter access, it may not be feasible for API-only models.

A1: We appreciate the reviewer’s feedback. In Section 6 of the paper, we discuss an application scenario where membership inference attacks are performed using the variation API provided by OpenAI DALL-E[1] which is a popular API-only model. On the other hand, we believe that Stable Diffusion has not provided a variation API, likely because it has already released the model code and weights. We consider using only the text-to-image API for model attacks to be a valuable direction for future research. These results demonstrate that our approach can be applied to a real-world diffusion-model API.

Q2: It is reasonable and necessary to have comparison of computational resource requirements.

A2: We appreciate the reviewer’s question. From the perspective of computational complexity, the time required mainly depends on the average number $n$. The time for each detection is $n$ times that of the baseline algorithm. However, we conducted an ablation study in Section 5.5, and the experimental results show that our algorithm achieves an AUC greater than 0.9 even when $n=1$. Furthermore, with $n=10$, our algorithm inference 100000 CIFAR-10 images in approximately 2 minutes. We believe this time cost is within the reasonable range, and that the model’s access conditions and accuracy are more important factors. We will add these discussions to the limitation section.

Q3: The intuition behind the proposed algorithm in Section 4.2 is not clear enough. The statement, “If the noise prediction from the neural network exhibited high bias” is ambiguous. What specifically is meant by “bias” here, and why does this lead to the condition $\nabla_\theta L(\theta) = 0$ for a well-trained model?

A3: Thank you for the question. In this context, the "bias" refers to the expectation of the random variable $\epsilon - \epsilon_\theta \left( \sqrt{\bar{\alpha}_t} x_0 + \sqrt{1 - \bar{\alpha}_t} \epsilon, t \right)$.

The condition $\nabla_\theta L(\theta) = 0$ does not come directly from this statement but is instead a consequence of the assumption that our model is well-trained. Several papers [2][3][4] demonstrate that after sufficient training, neural networks converge to a stationary point where $\nabla_\theta L(\theta) = 0$.

On the other hand, the statement that “If the noise prediction from the neural network exhibited high bias, the network could adjust to fit the bias term, further reducing the training loss,” is an explanation for the equations in line 210.

Q4: Can the proposed method further generalize to fine-tuning of diffusion models?

A4: We greatly appreciate the reviewer for suggesting this new direction. We conduct relevant experiments by fine-tuning a DDIM model pre-trained on the STL-10 dataset. The fine-tuning dataset consists of 1000 randomly sampled images from CIFAR-10 or Tiny-ImageNet, while another 1000 images from the dataset are used as non-members. We finetune the model for 10000 iterations. The experimental results of our algorithm are as follows:

The results show that our algorithm also performs well in the fine-tuning setup, being able to determine whether an image is in the model's fine-tuning training set. We will add these results to the paper.

Q5: Although the DDIM model is widely used and much more effective than DDPM, can you still provide some results regarding the performance in DDPM?

A5: We appreciate the reviewer’s question. We have conducted experiments on the DDPM model across four different datasets with 30 diffusion steps. The results are as follows:

The experimental results show that our algorithm is also effective on the DDPM model. We will add these results to the paper.

We thank the reviewer once again for the valuable and helpful suggestions. We would be happy to provide further clarifications if the reviewer has any additional questions.

References

[1] The variation API of DALL-E. https://platform.openai.com/docs/guides/images/variations-dall-e-2-only

[2] Brutzkus A, Globerson A, Malach E, et al. SGD learns over-parameterized networks that provably generalize on linearly separable data[J]. arXiv preprint arXiv:1710.10174, 2017.

[3] Zhang C, Bengio S, Hardt M, et al. Understanding deep learning (still) requires rethinking generalization[J]. Communications of the ACM, 2021, 64(3): 107-115.

[4] Li H, Xu Z, Taylor G, et al. Visualizing the loss landscape of neural nets[J]. Advances in neural information processing systems, 2018, 31.

Thanks again for your valuable feedback! Could you please let us know whether your concerns have been addressed? We are happy to make further updates if you have any other questions or suggestions.

We express our gratitude to the reviewer for the insightful comments. Please find the details below.

Q1: Have relevant ideas been proposed in other types of membership inference attack methods?

A1: We did further survey upon the reviewer's suggestion. We found similar ideas in the context of Machine-Generated text detection, particularly with methods like DetectGPT[1]. DetectGPT uses a perturbation-based approach to determine whether a piece of text is generated by a language model. The core idea is to apply small perturbations to the input text multiple times and observe the model's response to these modifications. If the model’s probability distribution remains stable, it suggests the text is likely generated by the model. Our approach is analogous in that we treat the MIA as a multiple-access model problem. We add random noise to the image and examine whether the model’s denoised output remains stable, using this stability to infer whether the image is in the model’s training set. We will add this reference to the related work section in our paper.

Q2: The abstract is too brief. The paper could benefit from elaborating the abstract with more insights of the proposed method.

A2: We appreciate the reviewer’s suggestion. In response, we have revised the abstract to provide more insights of the proposed method. Please refer to the updated version of the manuscript.

Q3: Table 4 is vague, how to interpret the $L_1$ and $L_2$ distance as membership inference accuracy?

A3: In Algorithm 1, our approach relies on a distance function D. In Table 4, the $L_1$ and $L_2$ distances represent the $L_1$ and $L_2$ metrics in pixel space between two images, serving as a distance function. Similar to previous sections, we classify images with a distance smaller than a certain threshold as members, and those with a larger distance as non-members.

Q4: In general, is there a way to provide any confidence to the membership inference results?

A4: We thank the reviewer for suggesting this interesting direction. Based on the algorithm in this paper, one possible way to estimate confidence is by evaluating the size of the distance. For instance, if we set a threshold $\tau$, images with reconstruction errors close to $\tau$ would have lower confidence in their classification, while images further from $\tau$ would have higher confidence. We consider this a valuable direction for future work.

Finally, we thank the reviewer once again for the efforts in providing us with valuable and helpful suggestions. We will continue to provide clarifications if the reviewer has any further questions.

Reference

[1] Mitchell, Eric, et al. "Detectgpt: Zero-shot machine-generated text detection using probability curvature." International Conference on Machine Learning. PMLR, 2023.

We greatly appreciate the reviewer's comments and valuable suggestions. We address the reviewer's questions in more detail as follows:

Q1: The written of this paper can be further improved. The motivation discussed in Introduction is not solid enough in my opinion.

A1: We appreciate the reviewer's feedback on the need for a clearer explanation of the motivation. In response, we have expanded the discussion of the motivation in the introduction. The added content can be found in the highlighted sentences of the revised manuscript.

Q2: The TP value of REDIFFUSE on CIFAR-10 is much lower than others. Also, there is a lack of analysis on why longer diffusion steps, an important factor affecting performance, seriously degrade the results.

A2: We thank the reviewer for the comments.

First, while existing algorithms perform well on simpler tasks such as CIFAR-10, the primary contribution of our method lies in eliminating the need to access the UNet. Hence, we match previous performance while further avoiding the usage of the inner denoise models.

Second, we believe that the AUC is more reliable than the True Positive Rate (TPR) when the False Positive Rate is fixed at 1%. Because the classification threshold is determined by a small number of misclassified samples, leading to greater volatility. When we take three different random seeds for CIFAR-10, the results show significant variation. Below, we present the TPR @1% FPR for the baseline methods and our algorithm for the DDPM model with three random seeds on the CIFAR-10 dataset:

We also present the AUC for the baseline methods and our algorithm for the DDPM model with three random seeds on the CIFAR-10 dataset:

As seen, the TP metric exhibits large variance for all methods, whereas the more stable AUC metric clearly demonstrates that our method outperforms the others. We also want to mention that our model consistently outperforms baselines on Diffusion Transformer(Table 2) and Stable Diffusion(Table 3).

Third, regarding longer diffusion steps, we observe that the reconstruction error has higher variance. Although the average for members is an unbiased estimator, the smaller sample size results in lower accuracy. To reduce this variance, we can increase the number of averages, which improves accuracy. The AUC results for CIFAR-10 with diffusion steps=400 under different average numbers are shown in the table below:

Q3: More cases should be provided to analyze the differences between member and non-member samples. But there is only one case in Sec.6.

A3: We greatly appreciate the reviewer’s suggestion. In the revised version of our manuscript, we have added additional examples in Appendix D, demonstrating the changes after applying the variation API to both member and non-member inputs. The results show that DALL-E 2’s variation API induces fewer changes to images in the member set compared to the non-member set.

Once again, we sincerely thank the reviewer for the constructive comments, and we are eager to engage in further discussions to clarify any concerns.

References

[1] Matsumoto, Tomoya, Takayuki Miura, and Naoto Yanai. "Membership inference attacks against diffusion models." 2023 IEEE Security and Privacy Workshops (SPW). IEEE, 2023.

[2] Duan, Jinhao, et al. "Are diffusion models vulnerable to membership inference attacks?." International Conference on Machine Learning. PMLR, 2023.

[3] Kong, Fei, et al. "An efficient membership inference attack for the diffusion model by proximal initialization." arXiv preprint arXiv:2305.18355 (2023).

Thanks again for your valuable feedback! Could you please let us know whether your concerns have been addressed? We are happy to make further updates if you have any other questions or suggestions.