TARP-VP: Towards Evaluation of Transferred Adversarial Robustness and Privacy on Label Mapping Visual Prompting Models

W1. Generally, in existing work, the MIA success rate is typically larger than 50%. A value close to 50% indicates that the attack is invalid, as this is like a random guess. Thus, a successful defense against MIA would result in an MIA success rate close to 50%, as seen in Table 4 of [1], Table 3 of [2], and Tables 2, and 3 of [3]. In these studies, some defenses reduce MIA from 60%-80% to around 50%, but not below 50%.

W2. We appreciate for pointing out these errors in writing, we will correct them in our revised manuscript.

Q1. The MIA we use is based on Yeom et. al [4], and actually, Song et al. [2] also evaluate MIA and use the MIA based on Yeom [4]. We grouped the two references in Ln102, which may cause ambiguity, and we will clarify this in our manuscript.

L1. Actually, we use epsilon size 8/255 as it is a commonly used setting when using ResNet18 or WideResNet as threat models on CIFAR-10 after [5]. As per the reviewer’s suggestion, we select two pre-trained models in LM-VP models to conduct experiments under epsilon size 4/255, although the results are slightly different with 8/255, we see similar trends, and we will complete the whole ablation experiments in our manuscript.

[1] Milad Nasr, Reza Shokri, and Amir Houmansadr. Machine learning with membership privacy using adversarial regularization. In Proceedings of the 2018 ACM SIGSAC conference on computer and communications security, pages 634–646, 2018.

[2] Shokri R, Stronati M, Song C, et al. Membership inference attacks against machine learning models[C]//2017 IEEE symposium on security and privacy (SP). IEEE, 2017: 3-18.

[3] Liwei Song, Reza Shokri, and Prateek Mittal. Privacy risks of securing machine learning models against adversarial examples. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pages 241–257, 2019.

[4] Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy risk in machine learning: Analyzing the connection to overfitting. In 2018 IEEE 31st computer security foundations symposium (CSF), pages 268–282. IEEE, 2018.

[5] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan.Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning, pages 7472–7482. PMLR, 2019

We appreciate the reviewers' recognition of our work. Our main contribution is that we first introduce a method that simultaneously enhances transfer adversarial robustness and privacy. As a new research prospect, we are happy to discuss any questions you may have. Additionally, we will also fully release our code to enhance the contribution of this work to the AI security community.

W1. As per the reviewer’s suggestion, we conducted experiments on Tiny-ImageNet, which has a resolution of 64x64 and contains 200 classes. We select different pre-trained models and the results indicate that: The LM-VP model with transfer AT improves transfer adversarial robustness by 3%-24% and mitigates the MIA success rate by 3%-12% compared to the LM-VP model with standard training, on both CIFAR-10 and Tiny-ImageNet, transfer AT on the LM-VP model demonstrates a better robustness-privacy trade-off, which shows a good generalization performance.

W2. We thank the reviewer for pointing out this concern. Regarding this, we guess the reviewer might think that the transfer adversarial attacks generated by ResNet-18 are weak. However, from Table 1 in our manuscript, for pre-trained models other than ConvNext and EVA, the transfer adversarial attacks pose a significant threat to them, evidenced by a substantial gap in natural accuracy and adversarial robustness. The choice of pre-trained models is important in the LM-VP models, ConvNext and EVA have 197.96M and 304.14M parameters, respectively, and are fully pre-trained on ImageNet. We have provided the results of adversarial robustness against ResNet-18 transfer attacks on the LM-VP models using ConvNext and EVA (Table 1 and Table 2 in our manuscript which indicate the attack success rate (1-adversarial robustness), high adversarial robustness shows their good synergy with LM-VP models.

We speculate the reviewer wants to see the ResNet-18 transfer adversarial attacks on ConvNext and EVA when transferred to CIFAR-10 after traditional finetune. After a full finetune, the standard accuracy reaches nearly 100% and the attack success rate is 12.50% for ConvNext and 18.75% for EVA, with MIA success rates of 62.1% and 76.80% respectively, finetune results are similar to LM-VP results, indicating that ConvNext and EVA are indeed robust to such transfer attacks on CIFAR-10 after transfer learning, but they are vulnerable to MIA.

Lastly, we want to emphasize that our work does not solely consider the transfer adversarial robustness as the only metric. If we only consider this, it is evidence that it is highly dependent on the performance of the pre-trained model (Table 1 in our manuscript ), e.g., large-size pre-trained models achieve better transfer adversarial robustness. However, we also address privacy concerns associated with the LM-VP model, ConvNext and EVA achieve high adversarial robustness, but they still have much room for improvement in terms of privacy (Table 3 in our manuscript: Standard trained LM-VP models). Applying transfer AT to the LM-VP models not only further enhances adversarial robustness but also mitigates privacy issues, which is the primary contribution we claim.

W3. We thank the reviewer for raising this question and rethinking the white-box attack on LM-VP models. According to our new attempt, white-box adversarial attacks LM-VP models can be indeed applied to LM-VP models. However, as shown in the table below, the adversarial robustness of the LM-VP model varies greatly when selecting different pre-trained models, making it difficult to obtain a consistent conclusion. Different from a general model, the pre-trained model plays an important role in the LM-VP model but the parameters are fixed and do not participate in training, VP plays limit role in defending against the white-box adversarial attack, thus its adversarial robustness may largely reflect the pre-trained models’ inherent adversarial robustness when transferred to the target dataset. Furthermore, we study the standard form of AT in the LM-VP model which is different from the transfer AT in our manuscript.

Unlike transfer AT, which can improve transfer adversarial robustness, standard AT can be regarded as invalid in the LM-VP model. It has really poor performance in both natural accuracy and standard adversarial robustness. The reasons may lie in:

(1) The adversarial examples generated by LM-VP models rely heavily on pre-trained models. The information of the source dataset may lead to unsatisfactory results of training with those adversarial examples on the target dataset.

(2) The adversarial perturbation directly affects VP, which further damages VP performance during AT.

We choose different pre-trained models to illustrate the above situation regarding the white-box adversarial robustness. However, our work focuses on studying the transfer adversarial robustness and privacy of the LM-VP model and proves the effectiveness of transfer AT. The same transfer attack model (ResNet18) is used to ensure the consistency of the experimental environment. Considering the white-box adversarial attack on the LM-VP model, its essence may be to attack fixed pre-trained models. How to study or improve its adversarial robustness is an interesting topic. We will revise the corresponding part in our manuscript and add the analysis and experiments of the LM-VP model under white-box adversarial attacks.

W4. We thank the reviewer for pointing out this mistake, LM as an FC layer, also has parameters, like in Section 3.1.3 we mentioned the parameters w2 of the fc layer, this is a writing error in Section 3.3, and we will correct it.

I still think that the so-called transferred adversarial robustness is far away from the adversarial robustness.

1. In the paper, "adversarial robustness" is used multiple times, Line 156, Line 254, Line 272. However, it is "transferred adversarial robustness" according to the author's response.
2. In the second table of your rebuttal, ResNet50 standard test accuracy is 80.52% and in Table 1 of your submission, ResNet50 standard test accuracy is 86.3%. I am confused about your setup.
3. In the second table of your rebuttal, ResNet152 standard PGD accuracy is 57.09% and is even higher than 35.99% in your submission, as transfer attack PGD20 accuracy. I am confused why the white-box attacks are even worse.
4. I need authors to double check the adversarial attack setting in both the original submission and the rebuttal.

We agree that transferred adversarial robustness is not exactly the same as adversarial robustness; to clarify this, we will use the term "transferred adversarial robustness" to make our work as precise as possible in the updated version, we appreciate the reviewer pointing this out. We also agree that the white-box standard adversarial robustness of LM-VP models is a topic worth studying. However, we want to provide some concerns about why evaluating the standard adversarial robustness of LM-VP models can be challenging. In contrast, the black-box transfer adversarial robustness of LM-VP models does not suffer from the same issues described below.

• The standard adversarial robustness of LM-VP models may largely reflect more about the inherent properties of a fixed pretrained model rather than an effective evaluation of the LM and VP components. In Table 2 of our rebuttal, there are significant differences in the best adversarial robustness of LM-VP models with different pretrained models. Despite using the same VP and LM components, their best adversarial robustness varies greatly and lacks a clear pattern, e.g., larger size pretrained models may not have better (or worse) adversarial robustness. This indicates the standard adversarial robustness may significantly be influenced by inherent features of the fixed pretrained model rather than the whole LM-VP model, whereas our goal is to evaluate the whole LM-VP model rather than just the pretrained component.
• Whether the standard adversarial robustness makes sense for LM-VP models or not: Adversarial examples generated by the LM-VP model are significantly influenced by the fixed pretrained model trained on the source dataset. The validity of using these samples to assess the LM-VP model's adversarial robustness on the target dataset requires further consideration.

Q1. Since our manuscript focuses on the transferred adversarial robustness of LM-VP models, we frequently use the terms “transfer” or “transferred” throughout the paper, e.g., we mention we enhanced the transferred adversarial robustness in the contribution in Ln 57. The original manuscript does not consider the standard adversarial robustness, thus in some parts, we use adversarial robustness as transferred adversarial robustness. During rebuttal, since we rethink the standard adversarial robustness in LM-VP, we will check and correct the relevant statements in the revised manuscript and add the analysis of the standard adversarial robustness.

Q2. The two columns on the left of Table 2 in the rebuttal are the results after standard training on LM-VP models. “Best performance” refers to the performance under the epoch of the best (standard or transferred) adversarial robustness, i.e., in Table 2 of rebuttal, when the standard PGD-20 adversarial robustness is best at 8.33%, the corresponding natural accuracy is 80.52% under that epoch; In contrast, in the manuscript, when the ResNet18 transfer PGD-20 adversarial robustness is best at 35.61%, it has a natural accuracy of 86.3%. We will add the description of the term “best performance” in the revised manuscript to clarify this, as it can indeed be easily misunderstood.

Q3. Table 2 of the rebuttal reflects the standard adversarial robustness of LM-VP models. Based on these two values alone (57.09% and 35.99%), we can only infer that, at a specific stage of LM-VP model training, the adversarial examples generated by ResNet18 are more aggressive than those generated by the LM-VP model based on pretrained ResNet152. For the general trained model, we generally believe that white-box attacks are stronger than black-box transfer attacks. However, the LM-VP model is influenced by pretrained models, which are trained on source datasets. It is conceivable that the adversarial examples under some pretrained models are strong and some are weak. For example, from Table 2 of the rebuttal, pretrained Swin shows an extremely strong white-box adversarial attack while pretrained ResNet152 is relatively weak. However, the strength of ResNet18 transfer adversarial attack remains consistent.

We would also like to mention that 57.09% of the standard adversarial robustness occurs only in the first epoch of training. This early-stage standard adversarial robustness likely reflects the inherent properties of the pretrained ResNet152, but its standard adversarial robustness decreases along with training, e.g., only 19.81% remains at epoch 9. Conversely, the ResNet18 transfer adversarial robustness maintains a relatively stable level, fluctuating between 32% and 36% for pretrained ResNet152.

Q4. Regarding the ResNet18, WRN-34-10 transfer adversarial attack, and the standard adversarial attack, our settings are consistent: PGD with epsilon=8/255, num_steps=10, step_size=2/255 for training, and only change the num_steps=20 for testing, the above settings are the same in the manuscript and rebuttal.

W1 and W2. The main novelty of this work does not lie in the theoretical aspect. The main contribution of our work is actually to introduce a novel method to jointly improve the transfer adversarial robustness and privacy of LM-VP models. This issue has not been fully explored before and we are the first to study the robustness and privacy trade-off of the LM-VP model, in this paper, we introduce the transfer AT for LM-VP models and conduct comprehensive experiments varying different pre-trained models to valid the effectiveness of transfer AT compared with standard training on LM-VP models, i.e., jointly improve the transfer adversarial robustness and training data privacy.

On the other hand, we admit that a theoretical understanding of the interaction between adversarial robustness and privacy for LM-VP models is a significant research problem. Yet, it is still an open challenge in the community. Song et.al [1] tried to analyze this interaction on general machine learning models. Through extensive experiments, they concluded that larger generalization errors and larger training data sensitivity make the model more susceptible to MIA but did not provide any principled theoretical analysis. From Table 1 and Table 3 in our manuscript, the generalization error on the train and test accuracy may not be a key factor that influences the training data privacy for LM-VP models, as each pre-trained model does not show a significant generalization error, but their MIA values vary.

Q1 and L1. We think some possible insights can be explored to tackle the theoretical challenges, for example: (1) During transfer AT, the original training examples are perturbed before feeding into the model, which means these data are not exposed to the trained model, this may be one factor that transfer AT helps mitigate the MIA issue because LM-VP models do not suffer from large generalization error and increased training data sensitivity (Table1, 2 and Figure3 in our manuscript) and transfer AT do not train the original training examples.

(2) Differential privacy (DP) is a method to guarantee privacy, both adversarial examples and VP introduce noise to original images, which may potentially resemble some operation of DP, thus mitigating the MIA issue. Other aspects, such as dataset complexity, training data size, pre-trained model architecture, and different adversarial training strategies will also be important considerations for future work to build the theory.

[1] Liwei Song, Reza Shokri, and Prateek Mittal. Privacy risks of securing machine learning models against adversarial examples. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pages 241–257, 2019.

We sincerely thank the reviewer for the valuable feedback.

The idea of this paper, i.e., studying the relation of the adversarial robustness and privacy within the LM-VP model stems from similar concerns observed in general trained models. In the context of general trained models, a key consideration is the boundary relationship between standard adversarial robustness and privacy. However, existing research [1] highlights a conflict between these boundaries when employing white-box adversarial attacks. As a result, in this paper, we use the transferred adversarial attacks to study the robustness instead of using white-box attacks, which from our point of view, is more sensible; the specific reasons are listed below:

1. A crucial distinction between the LM-VP model and a general model lies in the presence of a pre-trained model that does not participate in training [2]. Evaluating the LM-VP model using white-box adversarial robustness metrics can be heavily influenced by the choice of pre-trained model, as shown in Table 2 of the PDF. There is no clear pattern in their best adversarial robustness, potentially leading to different boundary relationships between adversarial robustness and privacy. Moreover, since we train the LM-VP model on the target dataset, but the generation of adversarial examples relies on the fixed pre-trained model from the source dataset domain, as a result, different pre-trained models may result in varying boundary performances, i.e., in this sense, using white-box adversarial attacks for the evaluation would make it difficult to draw a consistent conclusion.

2. In contrast, when considering the transferred adversarial robustness of LM-VP models, the intensity of the transfer attack remains constant once the attack model is selected. This consistency holds regardless of the chosen pre-trained model, ensuring that the transfer adversarial training process consistently optimizes in the same direction. This inherent consistency thus is more helpful for exploring and establishing a sensible boundary relationship between transferred adversarial robustness and privacy within LM-VP models. Therefore, utilizing transferred adversarial attacks serves as a more reliable and insightful evaluation method in this context.

Modeling Approach: Within the framework of transfer AT, the LM-VP model, comprising VP (Visual Prompt), a pre-trained model, and LM (label mapping), is treated as a unified black-box system. A fixed-parameter attack model, excluded from the training process, is employed to conduct transferred adversarial attacks. Under this setup, this paper systematically evaluates both standard training and transfer AT across LM-VP models that are based on various pre-trained models, with a focus on their impact on the trade-off between transfer adversarial robustness and privacy.

Quantification and Analysis: To quantify the boundary relationship between robustness and privacy, we leverage numerical metrics of transfer adversarial robustness and the success rate of membership inference attacks. By systematically comparing these metrics under both standard training and transfer AT across a range of pre-trained models, we aim to arrive at a consistent and generalizable conclusion. This analysis seeks to demonstrate that transfer AT effectively achieves a superior balance between transfer adversarial robustness and privacy boundaries, ultimately establishing it as a more secure training approach for LM-VP models.

[1] Liwei Song, Reza Shokri, and Prateek Mittal. Privacy risks of securing machine learning models against adversarial examples. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pages 241–257, 2019.

[2] Bahng H, Jahanian A, Sankaranarayanan S, et al. Exploring visual prompts for adapting large-scale models[J]. arXiv preprint arXiv:2203.17274, 2022.

Dear Reviewers,

The authors have responded to your valuable comments. Please take a look at them! Thanks!

Best, AC