Trap-MID: Trapdoor-based Defense against Model Inversion Attacks

Thank you for your insightful review and valuable feedback. We address the specific weaknesses and questions raised in your comments below:

W1: The defense should be tested against black-box/label-only attacks

The experiments show that BREP-MI, a label-only attack, is ineffective on Trap-MID, requiring over 820,000 iterations to initialize latents for all identities and getting 0% attack accuracy in untargeted attacks recovering 300 targets, which indicates Trap-MID's efficacy. Detailed results are available in Table 1 of the attached file in Author Rebuttal. We will add this analysis to our paper.

W2: It is important to investigate a high-resolution setting

We add the experiments against PPA for a high-resolution setting. Due to time constraints, we modified the attack to optimize 20 samples and select 5 in the final stage (PPA originally optimized 200 samples and selected 50).

The following table shows the defense performance on DenseNet-169. Although Trap-MID doesn't fully mitigate PPA, it preserves privacy to a certain degree without an accuracy drop. Besides, increasing trapdoor loss weight or combining Trap-MID with NegLS can improve the defense further. We discuss more about the defense combination in Q1.

Results including other models can be found in Table 2 of the attached file. We regret that we cannot conduct hyper-parameter tuning on other models due to time constraints. We will add this analysis to our paper.

W3: The evaluation metrics are limited

We acknowledge that each metric has limitations, yet attack accuracy/KNN distance and FID can complement each other. The formers estimate the extracted attributes, while the latter measures the naturalness and styling similarity. However, we agree that a universal metric would be valuable for a straightforward comparison.

Our PPA experiments at W2 include additional metrics such as FaceNet's feature distance used by PPA and improved precision, recall, density, and coverage used by VMI. We also use them to evaluate PLG-MI's results for a more comprehensive analysis. The results are available in Tables 2 and 3 of the attached file. We will add this analysis to our paper.

W4: The analysis should use a sample-wise measurement to provide a reliable measurement of trigger visibility

Thank you for highlighting this concern. The "trapdoor visibility" in Definition 2 might be misleading. We use KL divergence to ensure that triggered data is natural enough to be generated in attacks. Making them similar to the original images is one method to achieve this objective. If triggered data is similar enough to its counterpart, we have

$\text{If } \forall x \in X,\\; \log{p(x)} - \log{p(\Pi(x))} \le \epsilon,\text{ then } D_{KL}(p(X)||p(\Pi(X))) \le \epsilon,$

which fulfills Definition 2.

We believe this theoretical analysis suggests the potential of various trigger designs. For instance, if anyone in a green shirt is classified as identity 1, the attacks could be misled into manipulating shirt colors.

W5: Missing related work

Thank you for mentioning these works. We will add them to Related Work.

W6: The definition of objective might not be true for all attacks

This definition doesn't mean that attacks aim to recover the entire posterior distribution. Instead, the attack relies on such distribution. Most MI attacks guide optimization by identity and prior loss, maximizing $p(y|X)$ and $p(X)$. According to Bayes' theorem, given target label $y$, the attack process can then be viewed as maximizing the posterior probability $p(X|y)$.

We will refine the description here to make it clearer.

W7: The link to NegLS's implementation

Since NegLS hadn't released their code when we conducted experiments, we adapted KED-MI's code to implement NegLS by ourselves. We will include the link to our source code after releasing it.

W8: Missing evaluation model's architecture

Following GMI, the evaluation classifier is Face.evoLVe with an input size of 224x224. We will add this information to our paper.

Q1: Can inference-time mitigation strategies from the backdoor/shortcut literature eliminate the model's shortcut behavior?

If such mitigations are differentiable, one method is to apply them before feeding synthetic data into the victim model. Our work is a first step in trapdoor-based defense. Future work could explore the impact of recent backdoor/shortcut techniques on both attacks and defenses.

Besides, we found that combining Trap-MID and NegLS further improves defense, suggesting that Trap-MID can complement existing methods. Intuitively, NegLS makes it harder to extract private data and therefore makes trapdoors more attractive. A future direction explores hybrid defense to counter specific adaptive attacks.

Results against other attacks can be found in Table 4 of the attached file. We will add this analysis to our paper.

Q2: Why does the attack performance increase for other attacks besides PLG-MI when using trapdoor loss and discriminator loss?

This may come from the capacities of GAN in attacks.

Trapdoor loss search for easier triggers for the target model to retain utility. However, these crafted triggers may be harder to generate, leading to a lower defense performance against weaker generators.

Discriminator loss encourages invisible triggers, which is essential to deceive stronger discriminators like PLG-MI. However, generating invisible triggers requires fine-grain adjustment, making the weaker generators less likely to be misled.

Thank you for your thoughtful questions and feedback. We address them as follows:

Q1: Why not use something like FaceNet also for the 64x64 experiments?

The FaceNet evaluation of PLG-MI's 64x64 experiments can be found in Table 3 of our attached file at Author Rebuttal, which includes the following additional metrics:

1. $\delta_{face}$: The FaceNet feature distance between each recovered image and the nearest private data. A higher value indicates less similarity to the private data.
2. Improved Precision: Measures whether each recovered image lies within the estimated manifold of private data in the InceptionV3 feature space. A lower value indicates less similarity to the private data.
3. Improved Recall: Evaluates whether each private image lies within the manifold of recovered data in the InceptionV3 feature space. A lower value suggests that private data is less likely to be reproduced by the generator.
4. Density: Quantifies how many private-sample neighborhood spheres contain each recovered image in the InceptionV3 feature space. A lower value indicates less similarity to the private data.
5. Coverage: Assesses how many private samples have a neighborhood sphere containing at least one recovered image in the InceptionV3 feature space. A lower value suggests that private data is less likely to be reproduced by the generator.

As shown in the results below, Trap-MID outperforms existing defenses in FaceNet distance and ranks second to NegLS in most other metrics.

We notice that Trap-MID's ability to classify arbitrary images with injected triggers as corresponding classes results in more diverse recovered images, leading to a broader manifold and higher recall. Besides, all metrics except FaceNet distance utilize the same InceptionV3 model as FID, making NegLS excel in these metrics with its less natural recovered images.

Q2: It seems like the authors implemented the defense by themselves, which should also be fine

Yes, the authors of NegLS released their code after we implemented it. We have verified that our implementation adheres to the original training algorithm, and the detailed configurations can be found in Appendix C.6 in our paper.

Q3: The impact of Trap-MID in defending in the high-resolution setting against PPA seems limited

It is worth noting that with proper configuration, Trap-MID can still perform effectively against PPA. For example, as shown in Table 2 of our attached file, Trap-MID reduces PPA's attack accuracy to 11.78% on DenseNet-169 when $\beta=0.5$. However, we acknowledge that tuning hyper-parameters is necessary for optimal defense across different datasets and architectures.

Besides, we noticed that the success of hybrid defense like Trap-MID + NegLS is not merely owed to either of them but lies in their orthogonal strategies that (1) make it harder to extract private data and (2) provide misleading shortcuts. For instance, as shown in Table 2 of our paper and Table 4 of the attached file, this combination reduces LOMMA (KED-MI)'s attack accuracy to 42.47%, whereas either method alone achieves only 61.25% or 77.67%.

Q4: While the runtime is still somewhat reasonable, it limits the approach to some extent

We agree that computational efficiency is an important consideration, despite the effectiveness of Trap-MID. We will add this discussion to our Future Work section.

Q5: Regarding the evaluation model, is the input size of the Face.evolve model really 224x224?

We apologize for the mistake. The input size of the evaluation model is actually 112x112. The 64x64 images are indeed resized to fit the model's input size.

Thank you for your thoughtful review and helpful comments. We address the specific weaknesses and questions raised in your review below:

W1: “MI attacks leverage a discriminator to approximate generic prior and ensure natural outcomes” This statement might not generalize to MI attacks that don't utilise GAN discriminator, such as PPA

We appreciate your feedback. This statement refers to the challenge of misleading MI attacks using trapdoors. Since MI attacks often rely on GAN to produce natural outcomes without explicit constraints like $l_2$ or $l_\infty$ distance in adversarial attacks, it becomes harder to design effective triggers.

PPA uses a pre-trained StyleGAN2 generator and optimizes latents during attacks. Although the discriminator isn't involved in the process, it guides the generator to produce realistic images during GAN training, which indirectly ensures natural outcomes. We will refine this discussion to make it clearer.

W2: The evaluation is based on low-resolution (64x64) MI attacks, which are not very practical for modern models. The main results are on VGG-16, which is quite outdated.

To address this, we have added experiments against PPA to demonstrate a high-resolution (224x224) scenario with modern models. Due to time and computational constraints, we modified the attack to optimize 20 samples and select 5 in the final stage (PPA originally optimized 200 samples and selected 50, which would take about 10 days on our machine).

The following table shows the defense performance on DenseNet-169. Although Trap-MID doesn't fully mitigate PPA with default settings, it preserves privacy to a certain degree without an accuracy drop. Besides, increasing trapdoor loss weight further improves defense performance, and combining Trap-MID with NegLS (keeping $\beta$=0.2) even reduces attack accuracy to 2%. This result demonstrates Trap-MID's effectiveness in privacy protection under different scenarios. We discuss more about the defense combination in Q3 of Author Rebuttal.

Detailed results, including other architectures, can be found in Table 2 of the attached file in Author Rebuttal We regret that we cannot conduct hyper-parameter tuning on other models due to time constraints. We will add this analysis to our paper.

W3: The visualization results are not very convincing to me. I suggest the authors provide more visualization for reference and conduct a comprehensive study to further confirm the effectiveness of Trap-MI.

We acknowledge that visualization can be subjective due to the ambiguous definition of "class representative." However, in Figure 2 of our paper, we found that recovered images from Trap-MID show more distinct attributes from private data. For example, Identity 1 images have different skin tones, Identity 2 and 5 images have different hairstyles, and Identity 4 images show different genders. We also provide more recovered samples in Figure 1 of the attached file in Author Rebuttal, in which Identity 4 images from Trap-MID have different hair colors from private data compared to other defenses.

For the quantitative analysis, our PPA experiments in W2 include additional evaluation metrics, such as FaceNet's feature distance used in [1], as well as improved precision, recall, density, and coverage used in [2]. We also conduct the same evaluation on our PLG-MI results. These results, found in Tables 2 and 3 of the attached file Auther Rebuttal, provide a more comprehensive analysis of Trap-MID. We will add this analysis to our paper.

Q1: How sensitive is the Trap-MID to the hyper-parameters?

We discussed the impact of blend ratio in Appendix D.3 and trapdoor loss weight in Appendix D.4. According to Tables 7 and 8 in our paper, Trap-MID is not very sensitive to these hyper-parameters regarding test accuracy, maintaining about 81-84% accuracy under different configurations.

For defense performance, Table 7 in our paper shows that a lower blend ratio generally provides better defense. There is an abrupt drop in attack accuracy against KED-MI and PLG-MI (e.g., PLG-MI's attack accuracy drops from 93.86% to 1.92% when the blend ratio decreases from 0.05 to 0.03), indicating that an invisible enough trigger is essential to mislead certain attacks.

Additionally, Table 8 in our paper shows that increasing trapdoor loss weight from 0.02 to 0.2 reduces PLG-MI's attack accuracy from 23.84% to 6.23%.

[1] Struppek et al., Plug & Play Attacks: Towards Robust and Flexible Model Inversion Attacks. ICML 2022 [2] Wang et al., Variational model inversion attacks. NeurIPS 2021

Thank you for your valuable comments. We address the specific weaknesses raised in your review below:

W1: The proposed method involves multiple optimization processes, making it computationally expensive and potentially impractical for large-scale or resource-constrained applications.

We appreciate your concern. Here are the training times of different defense methods in our experiments:

While Trap-MID does take the longest time due to three gradient updates per epoch (discriminator, triggers, and target model), it is worth noting that it also significantly surpasses other defenses against recent MI attacks. Moreover, Trap-MID still requires less data and computational cost than the existing trap-based defense, NetGuard, which demands an additional dataset, training an extra classifier, and conducting shadow attacks.

We believe that it would be a valuable future direction to develop a more efficient trigger generation to make Trap-MID more practical for large-scale applications (e.g., pre-computing triggers with fewer steps to reduce overhead during model training).

W2: The paper does not analyze the discrepancy between the two posterior probability distributions, which is crucial for understanding the practical implications of the defense mechanism.

Theorem 1 establishes a lower bound for the posterior probability of poisoned data compared to benign data:

$\mathbb{E}\_{Y \sim p(Y)} \mathbb{E}\_{X \sim p(X)}[\log p_f(\Pi_y(X)|Y)] \ge \mathbb{E}\_{(X, Y) \sim p(X, Y)}[\log p_f(X|Y)] + (\delta - \epsilon)$

For example, since the unprotected model isn't injected with a trapdoor, it would have a negative trapdoor effectiveness $\delta$, leading to a lower expected posterior probability of poisoned data $p_f(\Pi_y(X)|y)$ than benign data $p_f(X|y)$ and making MI attacks more likely to extract private data.

In contrast, a trapdoored model with a stronger predictive power on invisibly triggered data (especially when $\delta > \epsilon$) would result in a higher expected posterior probability for poisoned data compared to benign data, misleading MI attacks to recover triggered data instead.

W3: The paper's organization and logical flow can be further improved for better readability and comprehension.

Thank you for highlighting this issue. We will refine our explanations and improve the organization for better readability. We would greatly appreciate any specific feedback on the sections that need improvement to help us enhance our presentation and address any confusion.

Thank you for your detailed and insightful review. We address the specific weaknesses and questions raised in your comments below:

W1: Trap-MID assumes trust between data providers and model owners, which isn't always feasible.

This limitation is discussed in Appendix A.2. We follow a common scenario that relies on model owners to protect private data. Since backdoor attacks can be conducted by poisoning the dataset, we believe Trap-MID's success can inspire future work enabling data owners to preserve privacy.

W2: It would be beneficial to explore the effectiveness of Trap-MID against black-box and label-only attacks.

To address this, we conduct experiments against BREP-MI, a label-only attack. Our result shows that BREP-MI is ineffective on Trap-MID, requiring over 820,000 iterations to initialize latents for all identities and getting 0% attack accuracy in untargeted attacks (recovering 300 identities), demonstrating Trap-MID's efficacy and generalizability:

Results against other defenses can be found in Table 1 of the attached file in Author Rebuttal. We will add this analysis to our paper.

W3 & Q1: Could the authors provide more insights into KD's impact on Trap-MID and how to make it more robust against KD-based attacks?

KD reduces Trap-MID's efficacy as student models typically do not learn trapdoor behaviors. Conducting shadow KD during trapdoor injection could be one of the solutions [1]. Besides, we found that combining Trap-MID with NegLS also improves defense, even against KD-based attacks:

Intuitively, NegLS makes it harder to extract private data and therefore makes trapdoor more attractive. This suggests Trap-MID to be an orthogonal work to existing defenses, and a hybrid approach may enhance robustness against specific adaptive attacks.

Results against other attacks are available in Table 4 of the attached file. We will add this analysis to our paper.

W4: It would be valuable to investigate Trap-MID's robustness against other types of adaptive attacks.

Thank you for the suggestions. Due to time constraints, we couldn't explore this further in this work. We recognize the importance of this future direction to help improve the trapdoor design.

W5: The randomly initialized triggers introduce variability in defense performance.

We acknowledged this limitation in Appendix A.2. However, the worst-case performance of Trap-MID still surpasses the best-case performance of existing defense against most attacks, indicating its effectiveness:

W6: Exploring more sophisticated trigger designs could improve the defense further.

This future direction is discussed in Appendix A.2. Our work serves as the first step in trapdoor-based defense. Besides, Appendix D.1 compares Trap-MID with patch-based triggers, showing the importance of trigger design. Future research could explore advanced triggers to improve defense further.

W7 & Q5: How effective is Trap-MID against other reconstruction-based attacks?

We discuss this future work in Appendix A.2. As these reconstruction attacks also optimize inputs to satisfy the adversary's objectives, we believe that Trap-MID can be extended to mitigate them with proper "shortcuts." However, since this work focuses on MI defense, the extension is beyond the scope, and we leave it to future research.

Q2: How sensitive is Trap-MID to the specific trigger design? Are there any guidelines for choosing a stable trigger design?

Appendix D.1 compares Trap-MID with patch-based triggers, showing the importance of trigger design. In this work, we use trapdoor loss and discriminator loss to optimize the triggers, which already surpassed existing defenses. We leave further customization to future work (e.g., improving trigger initialization and optimization, or using recent backdoor techniques).

Q3: Could the authors elaborate on the accuracy-privacy trade-off from trapdoor loss weight?

We discussed this trade-off in Appendix D.4. A larger weight encourages the model to learn trapdoor behavior instead of the main task, enhancing defense at the cost of accuracy. Table 8 shows that increasing this weight from 0.02 to 0.2 reduces PLG-MI's attack accuracy from 24% to 6%, while the test accuracy decreases from 84% to 81%.

Q4: Could the authors provide a more detailed analysis of how distributional shifts impact Trap-MID's defense performance and how it compares to existing defenses in such scenarios?

Section 4.2 and Appendix E.2 analyze this impact. Distributional shifts in auxiliary data degrade attack performance, especially against Trap-MID. In particular, distributional shifts make extracting private data harder, which therefore makes trapdoors more attractive to the attacks. Tables 4 and 11 show that PLG-MI's attack accuracy drops from 6% to nearly 0% with distributional shifts, highlighting its effectiveness in this case.

[1] Ge et al., Anti-Distillation Backdoor Attacks: Backdoors Can Really Survive in Knowledge Distillation, ACM MM 2021