Dear Reviewer EAeB,
thank you very much for reviewing our paper. We are glad you find our analyses novel and important, the writing excellent and the experiments comprehensive. We try to address all your remarks and questions sufficiently.

1.) Lack of theoretical analysis framework: Providing a grounded theoretical framework for our work already requires a universally accepted theoretical framework for model inversion attacks. Unfortunately, to date, there is no such framework available, and since sensitive features and privacy leakage themselves are hard to clearly define, this is no trivial task. We also stress that previous attack and defense methods all also not provide a theoretical framework. However, we believe that our extensive experiments and analyses already provide proof for our statements.

2.) Privacy protection mechanism against other types of attacks: We kindly emphasize that our focus lies only on model inversion attacks. We already provided an experimental evaluation on the impact of label smoothing on other model inversion algorithms besides PPA in Appx. C.5.. Moreover, as requested by other reviewers, we also added empirical results for adversarial attacks (see Appx. C.8 or our answer to Reviewers uLpV and wJ7G). For other privacy attacks besides model inversion attacks, we cannot make any guarantees, e.g., differential privacy. For such a privacy guarantee, the community is still missing a clear definition of privacy leakage in the feature space. There exists no clear definition of when we can call an inversion attack successful, or at which point a person's identity is leaked from a reconstructed sample. That is why existing attacks try to approximate such statements with metrics like the attack accuracy, the FaceNet distance, or our novel knowledge extraction score.

3.) Can an analysis be provided comparing the performance of using negative label smoothing to the performance under positive label smoothing? We are not sure if we understand your point here correctly. We state for all models their classification performance as the prediction accuracy on a hold-out test set, for example, in Table 1 and Table 2. In addition, we state each model's calibration error (ECE) and a total of 7 metrics for quantifying the attack performance of model inversion attacks. If we misunderstand your definition of performance, please let us know how we can provide you with the desired information.

4.) Are there any known vulnerabilities introduced by using negative label smoothing? Not directly a vulnerability but a side effect of using negative label smoothing is the loss of model calibration, i.e., prediction scores matching the true probability of a predictions correctness. Moreover, we added several experiments to investigate the impact of label smoothing on adversarial examples (see Appx. C.8). In case of adversarial robustness, our results show that negative label smoothing can even increase a models robustness, making it less susceptible to image manipulations.

5.) How generalizable are the findings of this paper? Please note that we already conducted experiments on different datasets (FaceScrub and CelebA), as well as on different common model architectures (ResNet, ResNeXt, DenseNet, VGG16). So we can already conclude that the effect is not limited to a single dataset and model architecture. Since label smoothing is applicable to all kinds of classification tasks and not dependent on a specific domain, we expect our results to be also valid for other domains and model architectures.

3.) Include a discussion on the potential harm to models' robustness when protecting training data privacy: We added additional analyses on the impact of label smoothing on a model's robustness to adversarial examples (Appx. C.8). Results on backdoor attacks will be added as soon as they become available.

For adversarial attacks, we investigated four attack algorithms (FGSM, PGD, BIM, One-Pixel Attack) in two modes (targeted + untargeted) to measure the impact of label smoothing on a model's robustness. Please refer to Appx. C.8 for the detailed hyperparameters and attack results. Overall, we found that training with positive label smoothing makes the models slightly more robust to adversarial examples. Interestingly, training with negative label smoothing markedly increased the models' robustness to adversarial attacks. So, as a first result in this research direction, we can conclude that negative label smoothing not only helps mitigate privacy leakage but can also make a model more robust to adversarial examples. We will further investigate this phenomenon in future work.

Columns 3-6 state the attack success rate for untargeted adversarial attacks, columns 7-10 the success rate for targeted attacks.

FaceScrub

CelebA

Dear Reviewer wJ7G,
we just added the results for the backdoor experiments. Due to the relatively small number of samples per class in FaceScrub and CelebA, we were not able to stably train the models - independently of the smoothing factor used. Therefore, we trained ResNet-152 models on poisoned ImageNette datasets, which is a subset of ten ImageNet classes. Specifically, we investigated the common BadNets and Blended attacks.

Overall, positive label smoothing seems not to make a model more vulnerable to backdoor attacks; training with standard cross-entropy loss already builds a strong connection with the attacks' trigger. Negative label smoothing, however, reacts sensitively to the poisoned datasets, leading to decreased clean accuracy and, in the case of Blended backdoors, also a substantially lower attack success rate. So negative label smoothing can act as some kind of indicator of the dataset quality since it lowers the clean model accuracy, which might be a sign that the dataset is somehow manipulated.

For more details and hyperparameters, please take a look at Appx. C.9.

2.) Observations are partly known in literature: The stated previous literature investigated the impact of positive label smoothing on knowledge distillation and showed that this can result in better class-wise separation of penultimate layer representations. In this work, however, we look at the information leakage by model inversion attacks, a completely different setting since the attack goal is not to train a student model but to reconstruct sensitive features of individual classes Also, compared to previous work, we also incorporate an analysis of negative label smoothing. However, we added references to previous research in Section 4.3 to place our findings better within the existing literature.

3.) Training and Test Accuracy of Classifiers in Fig. 4? As stated in the paper ("In the following, we focus our analyses on the FaceScrub models..."), all analyses are based on the FaceScrub ResNet-152 models, which are stated in Table 1 and, with more details, in Table 2 in Appendix C.1. However, we understand that this information might easily be overlooked, so we made it more clear in the paper. Regarding the training and test accuracy, the classifiers achieved the following results: Consequently, the model used for Figure 4c) is no poorly trained classifier but the ResNet-152 FaceScrub model achieves generally good performance on the test data.

4.) Why use a warm-up period when training with negative LS? We follow here the insights of Wei et al. [2]. The authors motivate this decision to improve the model performance. We also experimented with training models directly with negative LS and no warm-up period but found the training to become rather unstable. In some training iterations, the optimization gets stuck in a poor local minimum, substantially degrading the model's prediction accuracy. Adding the warm-up period successfully solves this problem. However, we think by selecting better warm-up parameters, the performance of models trained with negative LS can even further be improved. Due to resource restrictions, we leave more comprehensive parameter optimization for future work.

(1) Zhang et al., The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks
(2) Wei et al., To Smooth or Not? When Label Smoothing Meets Noisy Labels

We are currently working on adding more results for other model inversion attacks. We will add the results in the next couple of days. Please excuse the delay - getting the different code bases running and conducting the experiments takes some time. We also plan to add a user study.

As requested, we conducted a qualitative user study to compare the effects of different defense methods on the visual similarity between attack results and the true identity. We compared the attack results on the ResNet-152 FaceScrub and CelebA models trained with negative LS to the attack results against models trained with MID and BiDO, respectively. The study was conducted using the manual labeling service of https://thehive.ai/. More specifically, we randomly sampled $150$ target classes from the FaceScrub and CelebA datasets, respectively. We then took for each target model and target class the attack results that achieved the highest robustness on the target model using PPA's final selection algorithm. For each class, the annotators were presented with $5$ randomly selected images of the true identity and the $3$ attack results from the attack against negative LS, MID, and BiDO models. The annotators were then asked to "select the person (options 1-3) that looks the least similar to the person depicted in the example images". Noticeably, the annotators were not provided any information on the different defense mechanisms or even the attack setting at all to avoid undesired side effects during the labeling process. Each sample was labeled by three different annotators and the final decision was made on a majority vote. In total, 37 annotators took part in the labeling process. To filter out unreliable annotators, some honeypot tasks were added to ensure the annotation quality.

The following table states the user study results as the relative preference for the individual defense mechanisms. For the FaceScrub model, the annotators found about 48% of the attack results from attacks against the negative LS model to look the least similar to the target class. For the CelebA experiment, even 71.5% of the results were preferred. Overall, this means that the defensive effect of negative LS was qualitatively higher valued than MID and BiDO as defense mechanisms, particularly on the CelebA models. At the same time, negative LS also maintained the model's utility in terms of test accuracy better than the other defense mechanisms.

Fore more details, please take a look at Appx. C.10 in the paper.

Dear dNRe,
we just added the MID results to the table above. We hope with this we answered all your questions and added all requested experimental results. If you have any questions left, we are happy to answer them before the end of the discussion phase.

Best,
the authors

Dear Reviewer dNRe,

thank you very much for acknowledging our rebuttal and increasing your score.

We are still eager to address your concerns and convince you of our work. Due to the close end of the rebuttal phase, please excuse that our answers and additional results are limited.

1.) User study: we thank you for suggesting improvements for the user study. If you desire, we can add a second study to the final version of the paper, if it gets accepted.

2.) Fig. 4 We did not use the weight-based projection as suggested by [A, B] but rather directly projected the penultimate activations into 2D space by using t-SNE. We will look into the visualization of Müller et al. but will probably not finish it until the rebuttal deadline.

3.) Lack of Theoretical/Analytical Support: Providing a grounded theoretical framework for our work already requires a universally accepted theoretical framework for model inversion attacks. Unfortunately, to date, there is no such framework available, and since sensitive features and privacy leakage themselves are hard to clearly define, this is no trivial task. We also stress that previous attack and defense methods all also not provide a theoretical framework. However, we believe that our extensive experiments and analyses already provide proof for our statements.

4.) Why does positive label smoothing provide better defense compared to no label smoothing for LOMMA (GMI), despite having higher test accuracy? Please note that positive LS smoothing increases the attack success/privacy leakage in 7 out of 8 attack settings: PPA (high resolution), GMI, KED, KED + LOMMA, PLG-MI, RLB-MI, and BREP-MI. So there is a clear pattern. For GMI + LOMMA we think there are two reasons why positive LS does not outperform the hard-label case. First, positive label smoothing might hinder the knowledge distillation process by training the augmentation models, as previously discussed, e.g., by Müller et al. [A]. So including the augmentation models in this attack setting might actually limit the attack's success. KED+LOMMA uses the same augmentation models but also includes the target-specific GAN, which provides a substantial attack performance boost against the positive LS model, as demonstrated by the attack results for KED without LOMMA. Second, the target objective of LOMMA (logit score + regularization) might interfere with the fact that label smoothing encourages representations in the penultimate layer to group in tight equally distant clusters, see also [A]. So we believe that the overall reason why GMI+LOMMA does not perform better on the positive LS models probably is due to the attack design that does not work well with the model's feature representations.

5.) Why does negative label smoothing provide worse defense compared to no label smoothing for PLG-MI, despite a significantly lower test accuracy? Indeed, the attack accuracy indicates misleading attack results. Whereas the attack accuracy is quite similar for both models, the other attack metrics (feature distance and FID score) are substantially worse for the negative LS model. Since the attack accuracy is susceptible to adversarial examples and does not make a statement about the sample quality, it is important to also consider the other metrics.

To provide additional qualitative proof of the defensive effect of negative LS, we just plotted attack results for the first ten classes in Fig. 11 in Appx. C.5. Importantly, the samples were not cherry-picked but we took the first generated sample for each class. When taking a look at the image it becomes clear that PLG-MI indeed produces qualitative worse results for the negative LS model. So negative LS in this case limits the privacy leakage by limitating the constructed samples' quality. Also, the variance of the different samples of one class is very low (not depicted in the image) whereas the attack against the other models constructs visually different samples for each single class.

So we stress that the results for LOMMA (GMI) and PLG-MI are not contradictory to the general message of the paper ("Label Smoothing Can Be a Privacy Shield but Also a Catalyst for Model Inversion Attacks"). For LOMMA (GMI), the attack design simply does not match the characteristics of positive label smoothing training, and for PLG-MI the qualitative samples and other metrics clearly show that negative LS indeed mitigates the privacy leakage of the model.

Again, thank you very much for your insightful review. We are happy to make any suggested changes and additional results to the final paper.

Best,
the authors

2.) Can an adjusted attack algorithm achieve different results? We conducted additional experiments by changing or replacing specific aspects of PPA. Specifically, we increased the number of initial sample vectors by factor 5, we experimented with different learning rates, we added an additional prior loss based on the GANs discriminator, and we tested two different loss functions from recent literature [1, 2]. While some of the changes could indeed improve the attack's success, the overall attack performance is still substantially below those for models trained without negative LS. For example, the FaceNet distance, which states the visual similarity between reconstructed samples and training samples from the target class, is still about 0.3-0.7 higher than for attacks against the model trained without any LS (avg. FaceNet distance 0.71). We, therefore, conclude that by selecting good attack parameters and making adjustments to the optimization goal, one might improve the attack success but compared to models trained without LS or with positive LS, the attack still falls short by a large margin.

Our results including a more detailed experimental setting and more elaborated discussion are also added in Appendix C.7.

(1) Zhang et al., The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks
(2) Nguyen et al., Re-Thinking Model Inversion Attacks Against Deep Neural Networks

3.) Why is the gradient similarity for positive LS lower than for training with hard labels? During the attack optimization process, the attack explores the GAN's latent space by guidance through the target model. We hypothesize that the optimization landscape defined by the target model trained with LS - compared to the model trained with hard labels - leads to more pronounced changes in direction since the model reveals more fine-grained details on the features. Recovering such fine-grained features might need more adjustments to the optimization direction during the optimization since the next gradient step is based on the current position in the latent space. Keeping the same direction might probably is not ideal for recovering certain features. So we suppose that the model trained with hard labels provides not enough guidance signal but rather keeps the overall optimization direction, which leads to generally good results but some fine-grained details might not be recovered. The model trained with positive LS, however, provides a stronger guidance signal, which leads to locally more pronounced changes in the optimization direction. The model trained with negative LS often fails to exploit local information sufficiently and provide a stable guidance signal.