Generative Model Inversion Through the Lens of the Manifold Hypothesis

Thank you for your great comments and efforts in reviewing our paper. Please see our detailed responses to your comments and suggestions below.

Q1. What is the difference between this and model inversion (such as psp, e4e, etc.)? Where is the specific manifestation of the attack?

Thank you for your question. To clarify the distinction between generative model inversion Attacks (MIA) and model inversion methods such as PSP and e4e, we can identify several key differences in both their approaches and objectives.

MIA is a category of privacy attack aimed at reconstructing representative samples by exploiting access to only a well-trained model. Early work [r1] framed MIAs as an input-space optimization problem, where the objective is to use gradient descent to find inputs that maximize the prediction score for a given target class.

Generative MIA [r2] extends this framework by integrating generative models such as GANs to learn an image prior. This prior is used to constrain the inversion process to the latent space of the generator, significantly improving the visual quality and semantic relevance of the reconstructed samples, which enhances the effectiveness of model inversion attacks.

In contrast, model inversion methods like PSP and E4E focus on inverting a pre-trained generative model (e.g., StyleGAN) to generate latent representations or embeddings for a given target image. This enables tasks such as image manipulation and editing, where the goal is not necessarily to infer sensitive information but to alter or reconstruct the image from the latent representation.

We hope this clarifies the conceptual differences between these two approaches. Please let us know if further clarification is needed!

Q2. The datasets targeted by the author are all facial datasets ... in this manuscript.

Thank you for your question. We followed the common evaluation protocol established in prior MIA literature [r2-4], which predominantly uses facial datasets due to their relevance to real-world privacy concerns. These datasets reflect scenarios where the primary risk lies in revealing sensitive attributes such as facial features. As we clarified earlier, the goal of MIA is to reconstruct representative samples that reveal class-sensitive information. Consequently, the background area does not carry significant class-sensitive information, and therefore is less critical to the effectiveness of the inversion in our context.

Q3. The manuscript did not report experimental error bars or confidence intervals.

Thank you for pointing this out. To address this concern, we have conducted three independent runs (with different random seeds) of our key experiments in Table 2 to assess the consistency and statistical significance of our results. The averaged results and standard deviations are reported in the following table:

These results confirm that our method consistently outperforms the baselines across runs, demonstrating statistical reliability. We will include the updated results in the main results section (Section 6.3) of the final version of our paper to enhance the reliability of the proposed method.

Q4. Only a few SOTA defenses such as BiDO and NegLS were compared ... the robustness of the method.

Thank you for your comment. We chose to evaluate with BiDO, NegLS, and TL-DMI because they represent state-of-the-art defense mechanisms specifically designed to counter model inversion attacks. However, as we understand it, adversarial training and gradient masking are primarily techniques designed to defend against adversarial attacks (please correct us if our understanding is incorrect), rather than directly addressing the unique vulnerabilities posed by MIAs. As a result, these methods fall outside the scope of our evaluation.

L1. Is the method of this manuscript applicable to all generative models?

Thank you for your question. The method proposed in this manuscript is applicable to single-step generative models such as generative flow, VAE, and GAN, where the generation process can be expressed as $\mathbf{x}=G(\mathbf{z})$ through a single forward pass of the generator. This structure enables direct computation of the Jacobian, allowing us to apply projection-based methods for analysis.

However, for diffusion models, the sampling/generation process is iterative and involves denoising at each step, which makes it more complex and fundamentally different from single-step models. As a result, our projection-based method is not directly applicable to diffusion models.

L2. Re: Extending AlignMI to black-box settings.

Thank you for your comment. As stated in our problem setup (Section 2), our study focuses on the white-box setting, since AlignMI relies on inversion loss gradients to improve gradient–manifold alignment—this naturally requires access to the model's internal gradients.

We acknowledge that model inversion attacks have also been studied in black-box settings, where several works have proposed search-based methods in the latent space to approximate gradients using surrogate signals [r5-6]. While our method is not directly applicable in such scenarios, we believe the core idea of promoting alignment between surrogate gradients and the data manifold holds potential for generalization to the black-box setting. We view this as a promising direction for future research.

L3. Generative MIA highly relies on the quality of pre-trained GANs (such as StyleGAN) ... the inversion effect will be significantly reduced.

Thank you for your comment. The goal of MIA is to reconstruct representative samples that reveal class-sensitive information, and therefore specific poses and expressions may not be critical to our context. We acknowledge that the distributional gap between the generator and the private data can affect inversion quality [r7]. However, recent studies have shown that characteristic features of target classes can still be revealed even under large distributional shifts [r3]. Moreover, while this limitation exists, it primarily arises from the choice and quality of the generator itself. Addressing this issue falls outside the scope of our work, which focuses on enhancing inversion through gradient-manifold alignment.

L4. The projection operation in aligning ... it may still limit its application in large-scale tasks.

Thank you for your comment. Compared to the standard classification objective, the primary computational overhead of the gradient-manifold alignment training objective comes from the additional single matrix-vector multiplication required for the projection operation. However, this increase in computational cost is marginal relative to the overall time spent on training.

To further investigate this, we tested the gradient-manifold alignment training objective against the standard classification objective across three models. We conducted experiments over 30 epochs, with each experiment repeated three times. The results are as follows:

These findings suggest that while there is a slight increase in computational cost, it has negligible impact on overall training efficiency.

We hope our responses have clarified the confusion and addressed your concerns. We would greatly appreciate it if you could take them into consideration during your final evaluation of our work. Please let us know if you have any outstanding questions.

---

References:

[r1] Fredrikson et al. Model inversion attacks that exploit confidence information and basic countermeasures. In CCS, 2015.

[r2] Zhang et al. The secret revealer: Generative model-inversion attacks against deep neural networks. In CVPR, 2020.

[r3] Struppek et al. Plug & play attacks: Towards robust and flexible model inversion attacks. In ICML, 2022.

[r4] Peng et al. Pseudo-private data guided model inversion attacks. In NeurIPS, 2024.

[r5] An et al. Mirror: Model inversion for deep learning network with high fidelity. In NDSS, 2022.

[r6] Kahla et al. Label-only model inversion attacks via boundary repulsion. In CVPR, 2022.

We are glad to have resolved most of your concerns and would now like to address the remaining points with further clarification.

Q1. Why are some of the error bars in the response to Q3 (ResNet-18 + TAA, ResNeSt-50 + TAA) so large?

Thank you for your question. Due to time constraints during the rebuttal period, we initially conducted three independent runs per setting. This limited number of trials may have contributed to slightly higher standard deviations for certain configurations (i.e., ResNet-18 + TAA, ResNeSt-50 + TAA), as you point out:

While these values may appear relatively large compared to other entries, we would like to emphasize that the absolute magnitudes of the deviations remain modest. More importantly, even within these variance bounds, the TAA configurations consistently outperform the baseline (PPA) across all evaluated setups.

To further address this concern, we have added 5 more runs for both cases. The updated results are as follows:

These additional experiments show that the initially observed variance diminishes with increased repetitions, confirming the robustness and consistency of our proposed method.

Q2. Why is the KNN Dist value in Table 1 of the original text so high, while the values in other tables are relatively low?

Thank you for your observation. The discrepancy in KNN distance values between Table 1 and Table 2 in the manuscript primarily arises from differences in the feature extractors used across resolution settings. In Table 1 (hypothesis validation), we adopt a low-dimensional setting (64×64). as computing and storing the decoder Jacobian at high resolution (224×224) is prohibitively expensive. Since the goal here is to validate the main hypothesis rather than implement a practical attack, experiments in low-dimensional settings are sufficient and appropriate for empirical analysis. For the main experiments in Table 2, we evaluate the effectiveness of AlignMI under high-resolution settings (224×224), which reflect more realistic model inversion attack scenarios.

As for the KNN distance metric: It measures the $\ell_2$ distance between the reconstructed image and its nearest neighbor in the training set, computed in an embedding space. Following established MIA literature [r1–r2], we use different feature extractors for different resolutions. In high-resolution settings, we employ the penultimate layer of InceptionResnetV1 (224×224), which outputs normalized feature vectors, resulting in smaller KNN distance values. In low-resolution settings, we use the penultimate layer of FaceNet (64×64), which does not normalize its feature embeddings, thereby yielding larger raw distances. Thus, the observed difference in KNN distance values reflects inherent differences in feature extractor behavior rather than any inconsistency in our evaluation methodology.

Q3. The author needs to explain the meanings of the two indicators, $L_\text{CE}$ and $L_\text{Align}$ in L4.

Thank you for your question. We are happy to clarify the meanings of the two indicators $L_\text{CE}$ and $L_\text{Align}$ mentioned in L4. (1) $L_\text{CE}$ refers to the standard cross-entropy (CE) loss, which is the conventional objective used for training classification models. (2) $L_\text{Align}$ refers to the total loss used in alignment-aware training, as defined in Equation (7) of our paper. This objective augments the cross-entropy loss with an alignment promotion term, which encourages the model’s input gradients to align with the estimated tangent space of the data manifold.

In the timing experiments (L4), these two terms are used to compare the training efficiency between standard training and alignment-aware training.

---

References:

[r1] Zhang et al. The secret revealer: Generative model-inversion attacks against deep neural networks. In CVPR, 2020.

[r2] Struppek et al. Plug & play attacks: Towards robust and flexible model inversion attacks. In ICML, 2022.

Thank you for your constructive comments and generous support! Please see our detailed responses to your comments and suggestions below.

W1. Lack of causal analysis between alignment and leakage: The central claim that "better alignment between gradients and the data manifold leads to higher vulnerability to MIAs" is only empirically correlated. No controlled experiments or ablation studies are provided to demonstrate a causal link.

Thank you for your comment. We acknowledge the importance of establishing a causal relationship between gradient-manifold alignment and MIA vulnerability. In Section 6.2, we provide empirical validation of our hypothesis. The interpretation of our hypothesis validation requires a nuanced understanding, and we would like to clarify our reasoning through a step-by-step analysis:

1. Increase in $AS_{tr}$ leads to decrease in test accuracy (Figure 4(a)):

Ideally, to rigorously test the hypothesis, we would compare models with varying training-time alignment scores ($AS_{tr}$) while keeping test accuracy constant. However, in practice, alignment-aware training inevitably reduces test accuracy. We hypothesize that this trade-off could stem from inherent limitations of modern deep neural network architectures or the inductive biases introduced by SGD optimization. A parallel example is adversarial robustness training [r1], which improves robustness but consistently compromises natural accuracy.

2. Lower test accuracy leads to reduced model inversion vulnerability:

Previous work [r2] has shown that models with higher test accuracy tend to be more susceptible to generative MIAs, whereas those with lower test accuracy are typically more resistant. This is intuitively understandable—lower test accuracy implies weaker dependencies between input features and predictions, thus making it harder to infer input features from model outputs.

3. Interpreting Figure 5 (the inverted‑U‑shaped curve):

If gradient–manifold alignment were entirely unrelated to model inversion vulnerability, one would expect a monotonic decline in attack accuracy with increasing $AS_{tr}$, due to the corresponding drop in test accuracy (as discussed in Point 1). However, Figure 5 reveals a non-monotonic trend—attack accuracy initially increases before eventually declining. This is because improvements in gradient–manifold alignment create a new attack surface, leading to increased model inversion vulnerability. At early stages, the benefits of improved alignment outweigh the negative impact of reduced test accuracy, hence attack accuracy rises. Beyond a certain point, however, adverse effects of declining test accuracy become dominant, and attack accuracy begins to decline. This trend holds consistently across both architectures we studied. Therefore, we can infer that, for models with comparable test accuracy, those with better gradient–manifold alignment are more vulnerable to MIAs, thus supporting our core hypothesis.

Additionally, the effectiveness of AlignMI compared to the baseline further supports the hypothesis, as it explicitly promotes better alignment and improves inversion performance.

W2. Unclear motivation for explicit projection in Section 5.

Thank you for your comment. To clarify, we do not introduce an explicit projection operation in Section 5. Instead, we propose AlignMI, a method that constructs an alignment-enhanced loss gradient:

$$\widetilde{\nabla}\mathcal{L}(\mathbf{x}) = \mathbb{E}_{\mathbf{x}'\sim p(\cdot\mid \mathbf{x})}\bigl[\nabla \mathcal{L}(\mathbf{x}')\bigr]$$

This method is designed to better align with the generator manifold compared to the original loss gradient $\nabla_{\mathbf{x}} \mathcal{L}$. Unlike implicit projection, which naturally arises from the generator's structure during optimization, AlignMI explicitly modifies the gradient to enhance this alignment and leverages it to boost the effectiveness of generative model inversion attacks. This design offers a complementary benefit to the implicit mechanism described in Section 3 by directly reinforcing alignment, rather than relying solely on the generator's inductive bias.

We support the AlignMI method with both quantitative and qualitative evidence: (1) Figure 12 (Appendix E.2) shows that AlignMI achieves higher alignment scores than the baseline. (2) Figures 13, 14, and 15 (Appendix E.4) visualize loss gradients and demonstrate that AlignMI produces gradients with more semantically meaningful features, indicating stronger alignment with the generator manifold.

W3. No discussion of Jacobian degeneracy in practice: The generator Jacobian is assumed to be full-rank, but in complex generators like StyleGAN, the Jacobian may degrade or collapse in certain regions of the latent space.

Thank you for raising this important point. We acknowledge that the analysis in Section 3 assumes the generator Jacobian $J_G$ is full-rank, which forms the basis for interpreting $Range(J_G)$ as the tangent space of the generator manifold. This assumption enables a clean and tractable theoretical framework for analyzing the alignment between gradients and the generator’s manifold.

While Jacobian degeneracy is theoretically possible, the likelihood of such degeneracy is generally low in practice. This is primarily due to the dimensional disparity between the image space (e.g., 64×64×3 or 224×224×3) and the latent space (e.g., dimension 512), which reduces the risk of rank deficiency in the Jacobian.

To further investigate, we conducted a statistical analysis of the Jacobian’s rank during the inversion optimization process. Specifically, we employed singular value decomposition (SVD) to evaluate the smallest singular values of $J_G$, as a non-zero minimum singular value indicates full rank. We sampled 10,000 samples during the optimization and report the distribution of the smallest singular values in the table below.

The results indicate that the generator Jacobian remains consistently full-rank throughout the inversion process, as evidenced by the strictly positive smallest singular values observed across all sampled instances.

We hope the clarifications provided have addressed your concerns and reinforced your confidence in our work. Please feel free to reach out if you have any further questions or require additional details.

---

References:

[r1] Fredrikson et al. Model inversion attacks that exploit confidence information and basic countermeasures. In CCS, 2015.

[r2] Zhang et al. The secret revealer: Generative model-inversion attacks against deep neural networks. In CVPR, 2020.

Dear Reviewer WVQ7,

We appreciate your valuable feedback and generous support, as well as the time you dedicated to reviewing our work.

Sincerely,

The Anonymous Authors

We sincerely thank you for your constructive comments and generous support! Please see our detailed responses to your comments and suggestions below.

W1. The method is limited to relatively low-dimensional setting due to its computational cost.

Thank you for the observation. The low-dimensional setting is used only for hypothesis validation, as computing and storing the decoder Jacobian at high resolution (e.g., 224×224×3) would be prohibitively expensive. Since the goal here is to validate the main hypothesis rather than implement a practical attack, experiments in low-dimensional settings are sufficient and appropriate for empirical analysis. Importantly, when evaluating the AlignMI attack algorithm, we conduct experiments in both low- and high-resolution settings to demonstrate its effectiveness in realistic scenarios.

W2.1. In my opinion, the experiment section doesn’t completely validate the author’s hypothesis formulated in section 4 ... I’m therefore not sure that this approach can be used to rigorously validate the authors hypothesis.

Q1. How do you explain the behavior of the VGG16 model in Figure 5?

Thank you for raising this important point. We agree that the interpretation of our hypothesis validation requires a nuanced understanding, and we would like to clarify our reasoning through a step-by-step analysis:

1. Increase in $AS_{tr}$ leads to decrease in test accuracy (Figure 4(a)):

Ideally, to rigorously test the hypothesis, we would compare models with varying training-time alignment scores ($AS_{tr}$) while keeping test accuracy constant. However, in practice, alignment-aware training inevitably reduces test accuracy. We hypothesize that this trade-off could stem from inherent limitations of modern deep neural network architectures or the inductive biases introduced by SGD optimization. A parallel example is adversarial robustness training [r1], which improves robustness but consistently compromises natural accuracy.

2. Lower test accuracy leads to reduced model inversion vulnerability:

Previous work [r2] has shown that models with higher test accuracy tend to be more susceptible to generative MIAs, whereas those with lower test accuracy are typically more resistant. This is intuitively understandable—lower test accuracy implies weaker dependencies between input features and predictions, thus making it harder to infer input features from model outputs.

3. Interpreting Figure 5 (the inverted-U-shaped curve):

If gradient–manifold alignment were entirely unrelated to model inversion vulnerability, one would expect a monotonic decline in attack accuracy with increasing $AS_{tr}$, due to the corresponding drop in test accuracy (as discussed in Point 1). However, Figure 5 reveals a non-monotonic trend—attack accuracy initially increases before eventually declining. This is because improvements in gradient–manifold alignment create a new attack surface, leading to increased model inversion vulnerability. At early stages, the benefits of improved alignment outweigh the negative impact of reduced test accuracy, hence attack accuracy rises. Beyond a certain point, however, adverse effects of declining test accuracy become dominant, and attack accuracy begins to decline. This trend holds consistently across both architectures we studied. Therefore, we can infer that, for models with comparable test accuracy, those with better gradient–manifold alignment are more vulnerable to MIAs, thus supporting our core hypothesis.

W2.2. At the opposite, the training-free approach seems to me to better validate the author’s hypothesis ... since it's mostly based on an intuition.

Thank you for this insightful observation. As you noted, the training-free approach is primarily grounded in geometric intuition, whereas the alignment-aware training provides a more principled way to validate the hypothesis, as discussed earlier.

To address your request for more concrete evidence, we note that our initial submission already included quantitative and qualitative evidence demonstrating that the training-free AlignMI approach indeed improves gradient–manifold alignment. Specifically, Figure 12 in Appendix E.2 presents a comparison of the $AS_{tr}$ distributions between the baseline and both the PAA and TAA variants of AlignMI. The results clearly show that the AlignMI approach increases the alignment scores relative to the baseline.

Furthermore, Figures 13, 14, and 15 in Appendix E.4 visualize the loss gradients for both the baseline and AlignMI. These visualizations reveal that the gradients produced by AlignMI exhibit more semantically meaningful features, offering qualitative evidence that our approach enhances alignment between the loss gradients and the generator manifold.

Q2. Could a similar technique, based on your analysis, be developed to reduce the MIA vulnerability of models?

We appreciate your insightful question and your intuition is indeed correct. We investigated how reducing $AS_{tr}$ (i.e., gradient-manifold alignment) affects a model’s vulnerability to generative MIAs. As shown in the following table, decreasing this alignment consistently leads to lower attack success rates, indicating a reduction in vulnerability.

These findings support the promising direction that techniques explicitly designed to reduce gradient–manifold alignment could be developed as effective defenses against generative MIAs. We view this as a compelling area for future work and appreciate your suggestion in helping shape that research direction.

Re: Discussion of Limitations.

Thank you for pointing this out. Due to space constraints in the initial submission, we included the discussion on the limitations of our method in the appendix. We agree that this discussion is important and will move it to the main text in the final version of the paper to ensure greater clarity and completeness.

We hope the clarifications provided have addressed your concerns and reinforced your confidence in our work. Please feel free to reach out if you have any further questions or require additional details.

---

References:

[r1] Fredrikson et al. Model inversion attacks that exploit confidence information and basic countermeasures. In CCS, 2015.

[r2] Zhang et al. The secret revealer: Generative model-inversion attacks against deep neural networks. In CVPR, 2020.

Thank you for your thoughtful comments and generous support! We appreciate your recognition of our contributions and the strengths of the paper. Please see our detailed responses to your comments and suggestions below.

W1, Q1. Lack of multiple experimental runs to ensure statistical significance.

Thank you for pointing this out. To address this concern, we have conducted three independent runs (with different random seeds) of our key experiments in Table 2 to assess the consistency and statistical significance of our results. We report the mean and standard deviation of the results in the table below:

These results confirm that our method consistently outperforms the baselines across runs, demonstrating the statistical robustness of our approach. We will include the updated results in the main results section (Section 6.3) of the final version of our paper to enhance the reliability of the proposed method.

W2, Q2. The intuition is not fully convincing ... having gradient along data manifold can be helpful.

Thank you for the thoughtful question. The motivation behind our work is grounded in the evolution of model inversion attacks (MIAs). The seminal work [r1] formulated MIAs as an input-space optimization problem. However, this traditional approach often produces noisy features, especially when the target DNNs are trained on high-dimensional data.

To address this, generative model inversion (GMI) approaches [r2] introduced image priors via pretrained generators (e.g., GANs), constraining the optimization to the generator’s latent space. This significantly improved visual quality and semantic relevance of reconstructed samples. These advancements naturally led us to ask: Why are GMI approaches so effective?

This question drove the analysis presented in Section 3. Our derivation shows that GMI methods implicitly denoise gradients by projecting them onto the generator’s manifold, thereby preserving only the gradient components aligned with the tangent space of the generator manifold and filtering out directions that deviate from it (Figure 2). Surprisingly, despite the effectiveness of GMI approaches, we found the degree of alignment between the gradients and the generator manifold to be remarkably low (Figure 3).

This observation led us to our core idea: can we explicitly promote gradient–manifold alignment to further improve inversion performance? To the best of our knowledge, this question has not been explored in prior work, and it forms one of the central contributions of our paper. Before proposing a new attack method, we rigorously validated our hypothesis that improving gradient–manifold alignment would boost inversion performance. This hypothesis is formalized in Section 4 and empirically validated in Section 6.2.

W3, Q3. The use of VAE may be unfair ... understanding of the data before implementing the attack.

We would like to clarify that the use of VAE is not intended to facilitate the attack itself, but rather to estimate the data manifold for validating our hypothesis. In this context, model inversion attacks are used as an evaluation tool to assess how different levels of gradient–manifold alignment impact a model’s inversion robustness. Therefore, the use of a pretrained VAE to estimate the data manifold is not meant to simulate a realistic attack scenario, but instead to enable a principled evaluation of the core hypothesis being examined.

We hope the clarifications provided have addressed your concerns and reinforced your confidence in our work. Please feel free to reach out if you have any further questions or require additional details.

---

References:

[r1] Fredrikson et al. Model inversion attacks that exploit confidence information and basic countermeasures. In CCS, 2015.

[r2] Zhang et al. The secret revealer: Generative model-inversion attacks against deep neural networks. In CVPR, 2020.