Pseudo-Private Data Guided Model Inversion Attacks

Sincerely thank you for your constructive comments and generous supports! Please see our detailed responses to your comments and suggestions below.

W1, W2: Missing evaluations on PLG-MI and state-of-the-art model inversion defenses.

Regarding these additional evaluations, please refer to the general response. These results will be included in the main results section (i.e., Section 4.2) of the final version of our paper.

W3. The evaluation focuses on metrics computed on an evaluation model trained on the same data as the target model ... further support the paper's claims.

We appreciate the reviewer's insightful comments regarding the potential limitations of using an evaluation model trained on the same data as the target model. We have indeed considered this concern in the paper. For the PPA-based experiments, to mitigate the risk of misleading results, particularly with regard to adversarial features, we calculated the KNN Dist metric using the penultimate layer of a pre-trained FaceNet model, as you suggested. Details of this setup are provided in Appendix C.5 of the paper.

Additionally, in response to concerns about Fig. 5, we have computed the FaceNet distance as an additional evaluation metric, with results as follows:

These results are consistent with the Attack Accuracy metric, further supporting the effectiveness of our method.

W4. Giving more intuition on why MMD and CT are used ... FaceNet distance?

• We chose MMD and CT over KL divergence primarily due to the inherent limitations of KL divergence, which requires the two probability distributions to have the same support and is often inapplicable when one or both distributions are implicit with unknown probability density functions (PDFs) [r1, r2]. In contrast, MMD and CT do not have these limitations. They are also amenable to mini-batch based optimization and are straightforward to implement in practice.
• Regarding the use of LPIPS as a similarity metric, we initially experimented with FaceNet distance, using the penultimate layer features of a pre-trained FaceNet model to measure similarity. However, we found that the results were not satisfactory, possibly because using features from a single layer did not provide sufficient discriminative semantic information. Therefore, we adopted the approach outlined in the original StyleGAN2 paper [r3], using LPIPS, which measures similarity based on a concatenation of multiple hidden layer representations from a VGG feature extractor. This method captures more comprehensive semantic information, making LPIPS more discriminative and, consequently, more effective than FaceNet distance for our purposes.

W5. (Minor remarks):

W5.1. The term "MMD" used in the caption of Fig. 1 should be introduced ... lead to confusion of the reader.

Thank you for your helpful suggestions. We have updated the caption of Fig. 1 to include the full term for the abbreviation "MMD," which stands for maximum mean discrepancy. Additionally, we have clarified the term "MI" on line 39 by introducing its full name, "model inversion (MI)," to avoid any confusion for the readers.

W5.2. Section 2.2: There should be some motivation for why MMD and CT are introduced here ... understanding these concepts.

Thank you for your valuable suggestions. We have revised Section 2.2 to include additional context explaining the motivation for introducing MMD and CT at this point in the paper. Specifically, we added 1-2 introductory sentences to clarify the relevance of these measures to our method. Additionally, we included a brief explanation of what MMD and CT measure and how they differ, providing readers with a clearer understanding of these concepts. The specific content added is as follows:

"To effectively align distributions in our subsequent methods, it is essential to introduce metrics that can accurately quantify the differences between them. Two commonly used measures for this purpose are maximum mean discrepancy (MMD) and conditional transport (CT). MMD focuses on mean differences using kernel methods, while CT incorporates cost-based transport distances, offering complementary perspectives on distributional discrepancies."

W5.3. L132: From my understanding, the "linear interpolation" between both distributions means that the sample share $\alpha$ is sampled from X_prior and $(1-\alpha)$ is sampled from X_private. This should be clarified in the writing.

Thank you for your feedback. We have made the necessary clarifications in the manuscript. The specific content added is as follows:

"To evaluate the impact of this distribution discrepancy on MI performance, we create a series of proxy prior distributions through linear interpolation, where a mixing coefficient $\alpha\in [0,1]$ determines the proportion of samples drawn from each distribution. Specifically, a fraction $\alpha$ of samples is drawn from $\mathrm{P}(\mathcal{X}{\text{prior}})$, and the remaining $(1-\alpha)$ is drawn from $\mathrm{P}(\mathcal{X}{\text{private}})$."

W5.4. Fig. 3: The font size in the legend ... Currently, those data points are hard to see in (b). W5.6. L296: The figure reference is broken, it currently states "Fig. 4.3" instead of "Fig. 5".

Thank you for your detailed observations. We have made the necessary adjustments based on your suggestions. Specifically, we increased the font size in the legend and enlarged the data points in Fig. 3 to enhance their visibility and make them easier to interpret. Additionally, we corrected the figure reference on line 296, updating it from "Fig. 4.3" to "Fig. 5" as appropriate.

W5.5. To make the approach more clear, think about adding "Step-4: Repeating the attack with the updated generator" ... be repeated after this step.

Thank you for your suggestion. As clarified in Section 3.2, PPDG-MI consists of three iterative steps, so adding a separate "Step-4: Repeating the attack with the updated generator" may be redundant. However, to enhance clarity, we will add a note at the end of Step-3 indicating "return to Step-1 and repeat the attack with the updated generator." This addition should make the iterative nature of the process more explicit to all readers.

Q1. There is the risk of a mode collapse in fine-tuning the generator. Does the approach require regularization to avoid this failure case, or is the method already robust enough to avoid it?

• For MIAs focusing on low-resolution tasks, we adopt a principled tuning strategy, fine-tuning $\mathrm{G}$ and $\mathrm{D}$ using the original GAN training objective on $\mathcal{D}\_{\text{public}} \cup \mathcal{D}\_{\text{private}}^{\text{s}}$. This approach mitigates the risk of mode collapse.
• However, for MIAs targeting high-resolution tasks, such as PPA [r4], we are unable to apply a principled tuning strategy due to the lack of access to the GAN training specifics. Instead, we employ an empirical strategy to fine-tune $\mathrm{G}$. While this approach may affect the quality of the generated images, it does not lead to mode collapse, as we only make slight alterations to the generator $\mathrm{G}$. Currently, we do not employ regularization to avoid this failure case but instead manage it by controlling the fine-tuning strength (i.e., through hyperparameter adjustment). We believe that incorporating regularization could further enhance the robustness of this process.

Q2. Regarding the Ratio metric.

Thank you for your insightful question. To ensure a fair comparison, we maintained an equal number of queries to the target model $\mathbf{M}$ during the inversion process for both the baseline PPA and PPDG-MI. For example, in the experiment where $\mathcal{D}_{\text{public}}$ = FFHQ and $\mathrm{M}$ = ResNet-18, the baseline attack's optimization iterations were set to 70, while PPDG-MI was configured with 35 optimization steps per round. Due to space constraints, we have detailed these attack parameters in Appendix C.4.

Additionally, after considering your perspective, we agree that maintaining the same number of optimization iterations per round as the baseline is a more reasonable approach to validate the effectiveness of PPDG-MI. This setup would only further enhance our experimental results. We appreciate your feedback and will consider this for futher experiments.

Limitations: The limitation section could include some failure cases and additional limitations of the method.

We appreciate the reviewer's suggestion. We will further examine the experimental results and include a discussion of failure cases and additional limitations of the method in the final version of our paper.

---

References:

[r1] Tran et al. "Hierarchical Implicit Models and Likelihood-Free Variational Inference." In NeurIPS, 2017.

[r2] Yin et al. "Semi-Implicit Variational Inference." In ICML, 2018.

[r3] Karras et al. "Analyzing and Improving the Image Quality of StyleGAN." In CVPR, 2020.

[r4] Struppek et al. "Plug & Play Attacks: Towards Robust and Flexible Model Inversion Attacks." In ICML, 2022.

Thank you for your time in reviewing our work and for your constructive comments. Please see our detailed responses to your comments and suggestions below.

W1. So while the idea is straightforward and makes a lot of sense ... in the context of MIs.

First, we would like to clarify that the problem investigated in our paper focuses on model inversion attacks, where the adversary seeks to recover private training data by exploiting access to only a well-trained target model $\mathrm{M}$. In this context, the adversary is limited to querying $\mathrm{M}$ and possesses knowledge of the target data domain but lacks specific details about $\mathcal{D}_{\text{private}}$. This is fundamentally different from the two papers you cited, which investigate gradient inversion attacks. In gradient inversion attacks, the adversary has access to data gradients; however, this information is not available in our case.

Regarding the novelty of our work, we do not claim that reliance on a generative prior is our primary contribution. Rather, this approach was first proposed by GMI [r1]. However, our work is the first to introduce the use of a dynamic prior, whereas previous studies [r1, r2, r3] in generative MIAs have relied on a fixed generative prior.

W2.1. With respect to point b): if the generative model is conditioned on the pseudo-generations ... 'actual' training data.

We would like to emphasize that generative model inversion attacks have demonstrated the ability to recover samples that closely resemble actual training data [r2, r3], by leveraging advanced model inversion optimization techniques and strong image priors.

The ability to recover such samples stems from the fact that the well-trained target model has learned discriminative features specific to the class. The effectiveness of these attacks is a key reason why this area has garnered increasing attention, as it underscores the privacy leakage risks inherent in machine learning models.

W2.2. ...but lets assume the images generated in this step are valid samples ... high similarity to the pseudo-private ones.

We would like to clarify a few points regarding your concerns. First, the distinction between "easier" and "more difficult" or informative samples is not as clear-cut in the context of model inversion attacks, especially when using balanced datasets, which are commonly employed in this domain. For instance, private training datasets like FaceScrub consist of 106,863 face images from 530 celebrities, with about 200 images per person, and the CelebA subset contains 30,000 face images of 1,000 identities, with about 30 images per person.

In generative model inversion attacks, the underlying assumption is that the model has already learned discriminative features between classes; otherwise, the task would be impossible to accomplish. As long as the well-trained target model captures these discriminative features and the discrepancy between the prior distribution and the private data distribution is small, representative images reflecting these features can be recovered with generative model inversion techniques. These representative images are sufficient to reveal privacy-sensitive information related to the training data.

Under this assumption, the primary challenge is to minimize the discrepancy between the prior distribution and the private data distribution, thereby increasing the likelihood of sampling actual training data (cf. right panel of Fig. 2). The dynamic prior method we propose specifically aims to reduce this discrepancy. Furthermore, both qualitative and quantitative experimental results have demonstrated the effectiveness of our method, i.e., it can recover samples that more closely resemble actual training data.

W3. One step which is mentioned as 'optional' is selection of samples that are meaningful. What does this imply? ...

We appreciate the reviewer's insightful question. Step-2 is labeled as "optional" to reflect the differences between low-resolution MIAs and high-resolution MIAs at this stage.

Specifically, for low-resolution MIAs, we adopt a principled tuning strategy. In this case, we fine-tune $\mathrm{G}$ and $\mathrm{D}$ using the original GAN training objective on $\mathcal{D}\_{\text{public}}$ (e.g., 30,000 samples) and $\mathcal{D}\_{\text{private}}^{\text{s}}$ (e.g., 1,000 samples per identity). We utilize all pseudo-private data because some MIA algorithms involve highly time-consuming optimization processes (e.g., black-box MIAs), and effective density enhancement requires a sufficient amount of pseudo-private data. Additionally, by incorporating $\mathcal{D}\_{\text{public}}$ during fine-tuning, we mitigate the risk of mode collapse.

For high-resolution MIAs, we propose a tuning strategy that leverages only the high-quality pseudo-private dataset $\mathcal{D}\_{\text{private}}^{\text{s}'}$, due to the high memory consumption during optimization, which necessitates selecting a subset of high-quality samples (e.g., 10 out of 100 samples). While this approach may affect the quality of the generated images, it does not lead to mode collapse. We mitigate this risk by carefully adjusting the hyperparameters, resulting in only slight alterations to the generator $\mathrm{G}$.

To address any confusion, we will remove the "optional" label and clarify the differences in pseudo-private data selection during Step-2 in the attack parameters section (Appendix C.4).

W4. While the focus of the work is clearly computer vision ... which the paper positions itself to be.

Model inversion attacks have garnered increasing attention in the trustworthy machine learning area due to their potential to reveal privacy risks in machine learning models. While current generative MIAs primarily focus on either the initial training process of GANs or the optimization techniques used in the attacks, our paper takes a different direction. We introduce a novel method by fine-tuning the GAN's generator based on the attack results from previous runs. This approach introduces a dynamic and iterative dimension to model inversion attacks, expanding the current understanding and application of generative MIAs. Although our work is primarily focused on computer vision, the underlying principles and methodologies could potentially be adapted to other domains, making this more than just an incremental improvement.

W5. Minor: the convention is typically MI for model inversion, rather than MIA (which is often used for membership inference), making it a bit more difficult to follow.

Indeed, both "MI attacks" and "MIAs" have been used to refer to model inversion attacks in the literature. For example, "MI attacks" is used in works such as GMI [r1] and LOMMA [r3], while "MIAs" is the terminology adopted in PPA [r2] and LS [r4].

Q1. Reason for making a connection to adversarial samples in the background.

Thank you for highlighting this point. Traditional MIAs [r5] on DNNs trained with image data adopt direct optimization in the input space, which can lead to the generation of adversarial samples. This limitation led Zhang et al. [r1] to propose generative MIAs as a more effective alternative. We have outlined this progression in the introduction section, specifically in lines 26-41. To eliminate any confusion, we will make the necessary clarifications in the problem setup section.

Q2. The difference between the low/high dimensional reconstructions sounds rather artificial (it seems this is just pre-trained vs FT GAN), is it really necessary to separate these? Are there are other differences making this separation clearer?

We appreciate the reviewer's constructive comments. The distinction between low-resolution and high-resolution settings is based on the typical approach in each scenario: the low-resolution setting usually involves training GANs from scratch using low-resolution public auxiliary data, while the high-resolution setting involves using pre-trained StyleGAN models trained on high-resolution data. This separation was originally introduced by the state-of-the-art model inversion defense LS [r4], and we followed this established setup. We acknowledge your point and will consider your suggestion to categorize the approaches based on whether the GAN is trained from scratch or a pre-trained StyleGAN is used.

---

References:

[r1] Zhang et al. "The Secret Revealer: Generative Model-inversion Attacks Against Deep Neural Networks." In CVPR, 2020.

[r2] Struppek et al. "Plug & Play Attacks: Towards Robust and Flexible Model Inversion Attacks." In ICML, 2022.

[r3] Nguyen et al. "Re-thinking Model Inversion Attacks Against Deep Neural Networks ." In CVPR, 2023.

[r4] Struppek et al. "Be Careful What You Smooth For: Label Smoothing Can Be a Privacy Shield but Also a Catalyst for Model Inversion Attacks." In ICLR, 2024.

[r5] Fredrikson et al. "Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures." In CCS, 2015.

We're glad to have addressed some of your concerns, and we would like to take this opportunity to further address the remaining points.

Re: The dynamic prior.

We appreciate your feedback and understand that the term 'dynamic prior' in our work may have led to some misunderstanding. To address this, we will revise the manuscript to use a more precise description of our approach. Specifically, we will clarify that we fine-tune the generator $\mathrm{G}$, which represents the prior, during the model inversion process.

The key difference we emphasize is that, in our approach, the generator is continuously fine-tuned throughout the model inversion process. In contrast, previous works such as [r1-r3, r6-r9] select a prior based on a heuristic but do not fine-tune the generator during the model inversion process. This continuous fine-tuning process distinguishes our method. To further clarify the distinction between our approach and previous generative model inversion attacks [r1-r3, r6-r9], we provide the following step-by-step explanation.

In previous generative model inversion attacks, the generative model $\mathrm{G}$ remains frozen throughout the model inversion process:

• Initialize latent codes: $\mathbf{Z}=\\{\mathbf{z\_i} \mid \mathbf{z}\_i \in \mathcal{Z}, i = 1,\ldots, N\\}$;
• Obtain optimized latent codes: $\hat{\mathbf{Z}}=\\{\hat{\mathbf{z}} = \text{argmin}~\mathcal{L}\_{\text{id}}(\mathbf{z};y,\mathrm{M}, \mathrm{G}) + \lambda \mathcal{L}\_{\text{prior}}(\mathbf{z};\mathrm{G},\mathrm{D}) \mid \mathbf{z} \in \mathbf{Z}\\}$;
• Generate recovered samples: $\mathcal{D}_{\text{private}}^{\text{s}} = \\{\hat{\mathbf{x}} = \mathrm{G}(\hat{\mathbf{z}}) \mid \hat{\mathbf{z}} \in \hat{\mathbf{Z}} \\}$.

In contrast, our pseudo-private data guided model inversion attacks (refer to Section 3.2) involve a more dynamic process that includes the following three iterative steps:

• Generate pseudo-private dataset with generative model inversion attacks (frozen $\mathrm{G}$): $\mathcal{D}_{\text{private}}^{\text{s}}$;
• Select high-quality pseudo-private dataset from $\mathcal{D}\_{\text{private}}^{\text{s}}$ (frozen $\mathrm{G}$): $\mathcal{D}\_{\text{private}}^{\text{s}'}$;
• Density enhancement around high-quality pseudo-private data $\mathcal{D}\_{\text{private}}^{\text{s}'}$ (unfrozen $\mathrm{G}$): $\mathrm{G}, \mathrm{D} \leftarrow `Fine-tune`(\mathrm{G}, \mathrm{D}, \mathcal{D}\_{\text{private}}^{\text{s}'})$.

As observed in our approach, the generative model $\mathrm{G}$ is not static but is fine-tuned iteratively based on the selected pseudo-private data. This differs from the static nature of the generator $\mathrm{G}$ in the previous generative model inversion attacks.

Thank you for your constructive comments and generous supports! Please see our detailed responses to your comments and suggestions below, where we use references in our manuscript due to the token limit.

W1. There are a number of typographic errors (e.g. “vallina”)

Thank you for your detailed review of our paper and for identifying the typographic errors. We have made the necessary corrections, including changing "vallina" to "vanilla." Your attention to detail is greatly appreciated, and we have carefully reviewed the manuscript to address all identified issues.

W2. A brief related work section which situates their contribution in the main body would have been appreciated.

Thanks for this suggestion! We have revised the paper to include a brief introduction of related work. Due to space constraints, we integrated this section into the problem setup (i.e., Section 2.1), while the detailed related work is provided in Appendix A.1. The specific content added is as follows:

"Current generative MIAs primarily concentrate on either the initial training process of GANs [Chen et al., 2021, Yuan et al., 2023, Nguyen et al., 2024] or the optimization techniques used in the attacks [Zhang et al., 2020, Wang et al., 2021a, Struppek et al., 2022, Kahla et al., 2022, Nguyen et al., 2023]. In this paper, we take another direction and introduce a novel approach by fine-tuning the GAN’s generator based on the attack results from previous runs. This approach introduces a dynamic and iterative dimension to model inversion attacks, expanding the current understanding and application of generative MIAs."

W3. There is verbatim repetition in the problem setup.

Thank you for highlighting the verbatim repetition in the problem setup, particularly in the phrases "the goal is to find a sample $\mathbf{x}$ that maximizes the model $\mathrm{M}$'s prediction score for class $y$" and "aimed to optimize for an optimal synthetic sample $\mathbf{x}^*=\mathrm{G}(\mathbf{z}^*)$ to maximize the target model's prediction probability in the target class $y$" in the original main text. We have made the necessary revisions to eliminate the repetition.

W4. They claim that "all state-of-the-art generative MIAs" are limited due to the utilization of a fixed prior during inversion. This bold claim is not backed up.

Thank you for pointing out the need for further clarification regarding our claim that "all state-of-the-art generative MIAs" are limited due to the utilization of a fixed prior during inversion. We recognize that this statement may seem bold without sufficient supporting evidence.

However, our intention is not to overgeneralize but to emphasize a prevalent trend observed in multiple state-of-the-art generative MIAs, which we have validated through rationale-driven analysis and main experiments. To address this concern, we have revised the manuscript by changing "all state-of-the-art generative MIAs" to "state-of-the-art generative MIAs" and have cited specific examples validated in our experiments to demonstrate the common use of fixed priors and their associated limitations.

We appreciate your feedback and will ensure that our claims are accurately represented and well-supported in the revised manuscript.

W5. Some of the symbols are not properly defined (e.g. \lambda is undefined)

We have carefully reviewed the entire manuscript and added definitions for all previously undefined symbols, including $\lambda$ and $\alpha$. Thank you for bringing this to our attention.

Q1. What are the real-world settings that this threat model could be observed with this work?

In real-world scenarios, the threat model described in our work can manifest in various situations where sensitive data is vulnerable to exposure through model inversion attacks. A prominent example is in security and surveillance. Models deployed in these contexts, such as facial recognition systems, are particularly susceptible to model inversion attacks that could reveal personal identities [r1, r2]. Another critical example is in healthcare systems, where machine learning models are used to analyze sensitive medical data, such as diagnostic images or patient records. In these settings, an adversary could exploit model inversion techniques to reconstruct confidential information, like detailed diagnostic images (e.g., CT scans), thereby compromising patient privacy [r3].

Q2. How might this approach extend to diffusion-based models?

To the best of our knowledge, diffusion models have not yet been applied to optimization-based generative model inversion attacks (i.e., the approach outlined in Eq. (1)). We hypothesize that this is primarily due to the technical challenges posed by the multi-step sampling process in diffusion models.

One potential challenge arises in the optimization of the latent code $\mathbf{z}$ during model inversion process, as the denoising Markov chain involves multiple steps. This process requires storing the generator's gradients at each step for backpropagation, leading to substantial memory consumption. Additionally, errors can accumulate throughout the optimization (i.e., sampling) process, potentially resulting in a suboptimal or inaccurate latent code.

Therefore, while we do not have a definitive answer on how to extend this approach to diffusion models, we believe that diffusion model-based MIAs represent a highly interesting and promising area of research, given the superior generative performance of diffusion models compared to GANs.

---

References:

[r1] Zhang et al. "The Secret Revealer: Generative Model-inversion Attacks Against Deep Neural Networks." In CVPR, 2020.

[r2] Struppek et al. "Plug & Play Attacks: Towards Robust and Flexible Model Inversion Attacks." In ICML, 2022.

[r3] Wang et al. "Variational Model Inversion Attacks." In NeurIPS, 2021.

Dear Reviewer CEeM,

Thank you once again for taking the time to review our paper and for your ongoing support. Your thoughtful feedback has been very valuable in improving our work.

Sincerely,

Authors of Submission #7116

Thank you for your time and careful review of our work. Please see our detailed responses to your comments and suggestions below.

W1, W2: Missing evaluations on PLG-MI and state-of-the-art model inversion defenses.

Regarding these additional evaluations, please refer to the general response. These results will be included in the main results section (i.e., Section 4.2) of the final version of our paper.

W3. The middle panel in Fig 5 seems wrong. The attack accuracy of random samples is higher than the one of high-quality samples.
W4. In Section 4.3, the citation for Fig 5 appears to be Fig 4.3.

Thank you for your careful review of our paper and for pointing out these issues. We have made the necessary corrections. The middle panel in Fig. 5 has been revised to represent the data accurately, and the citation error in Section 4.3 has been corrected to refer to Fig. 5 as intended.

Q1, Q2: The removal of the final results selection stage and the change of parameters in the pre-attack latent code selection stage.

Our proposed approach, PPDG-MI, requires identity-wise fine-tuning of the generator $\mathrm{G}$ during the model inversion process. Consequently, all three stages in the attack pipeline—latent code sampling, optimization, and final result selection—should be designed to function in an identity-wise manner.

Taking the experiment, where $\mathcal{D}_{\text{public}}$ = FFHQ and $\mathrm{M}$ = ResNet-18, as an example, the original PPA setting involved selecting 200 candidates from 5000 latent codes, running 70 optimization iterations per latent code, and then choosing the 50 recovered samples with the highest average prediction scores as the final results. The time cost we measured for the latent code sampling stage was approximately 52s/identity, the optimization stage took around 565s/identity, and the final result selection stage took about 5s/identity (these measurements were conducted on a single A100 GPU). Therefore, the time overhead is substantial, particularly since our method necessitates at least two rounds of attacks.

Given that latent code sampling in PPA is performed only once, whereas our approach requires identity-wise sampling, we modified the experimental setting by selecting 100 candidates from 500 latent codes to save on experiment time. This also significantly reduced the time required for the optimization stage. In addition, to maximize the amount of test data in our evaluation, we removed the final result selection stage, retaining all 100 optimized latent codes. It is important to note that these setting changes were applied equally to both the baseline PPA and our PPDG-MI, ensuring a fair comparison. Therefore, these adjustments do not affect the validity of our method's effectiveness.

We'd appreciate it if you could consider the above responses when making the final evaluation of our work. Please let us know if you have any outstanding questions.

Dear Reviewer 4Zsf,

Thanks very much for your time and constructive comments.

Would you mind checking our responses and confirming whether you have any further questions? Any comments and discussions are welcome!

Thanks for your attention and best regards.

Authors of Submission #7116