Endowing Visual Reprogramming with Adversarial Robustness

Thank you for your expertise and attention to detail that has been instrumental in improving the quality of my work. We are delighted that you consider our research to be effective, and interesting. Our point-by-point responses to the reviewer's mentioned questions (Q) and weaknesses (W) are provided as follows.

W1: I am concerned about the application scenarios of the proposed method. Visual reprogramming aims to leverage less data and computational resource overhead to stimulate the base model's capabilities on target tasks, which means that timeliness and performance are more important than adversarial robustness. The methods proposed in the article hinder these two aspects to some extent.

We greatly appreciate your concerns regarding the application scenarios of Visual Reprogramming (VR). Here, we aim to clarify the design motivations behind our work and the importance of adversarial robustness. In certain critical application scenarios, such as medical diagnosis, autonomous driving, and financial transactions, the security of the target task is paramount. In these cases, the adversarial robustness of the model is directly tied to the reliability and safety of the overall system. Therefore, adversarial robustness is an indispensable requirement in such applications, as important as performance such as accuracy.

While existing VR research has achieved promising results, it generally lacks an in-depth exploration of adversarial robustness. Our research seeks to address this gap by providing baseline results in this area, offering theoretical insights and experimental evidence to inspire future research.

Currently, adversarial training is the most effective and widely adopted method to enhance the adversarial robustness of models. Therefore, to effectively improve the adversarial robustness of VR, we conduct experiments involving adversarial training. However, this does not diminish the inherent advantages of VR. For example, since VR does not require modifications to the pre-trained model and involves only a small number of trainable parameters, employing adversarial training in VR can still reduce computational resource requirements compared to training a model from scratch. This makes it possible to obtain a larger robust model within the same computational budget.

Regarding the impact on timeliness and performance, these are well-recognized challenges in adversarial training research and are also present in VR. In our experiments, we adopt PGD-AT and TRADES as they are two classical and effective adversarial training methods. Our primary goal is to demonstrate that adversarial training is an effective approach to improving the robustness of VR. We believe that designing better adversarial training algorithms tailored for VR could further enhance adversarial robustness and performance while reducing computational resource consumption.

Moreover, our experimental results demonstrate that even without adversarial training, utilizing adversarially pre-trained models can provide certain robustness. For example, as shown in the table below:

Table 1: The performance comparison of VR with XCIT-S12 (A.T.) on different datasets, and PGD-20 is used for adversarial testing. Using PGD-10 ($\epsilon=4/255$) as adversarial training. The results are presented in the format of robustness (clean accuracy). The best robustness is highlighted in bold.

Table 2: The performance of VR with XCIT-S12 (A.T.) under natural training (N.T.) on different datasets, and PGD-20 is used for adversarial testing. The results are presented in the format of robustness (clean accuracy). The best robustness is highlighted in bold.

Continue by the next post.

When applying an adversarially pre-trained ViT on ImageNet to the OxfordPets target dataset, adversarial training achieves 49.41% robustness, while non-adversarial training only decreases by 6.33% robustness while maintaining almost the same accuracy. These findings highlight the potential of leveraging adversarial pre-training as a complementary approach to improve robustness in visual reprogramming tasks, offering a practical solution for safety scenarios where training costs need to be reduced. We believe this provides valuable insights and a strong foundation for future research aimed at enhancing both robustness and efficiency in VR applications.

W2: The paper primarily uses the projected gradient descent (PGD) method to generate adversarial samples. However, different generation methods may have different impacts on the model's robustness, and there is a lack of discussion on more adversarial sample generation methods.

Regarding adversarial training methods, we use PGD-AT and TRADES as the primary methods for generating adversarial samples during adversarial training. This is because these two methods are well-established and effective adversarial training approaches commonly employed in the literature, and our primary goal is to demonstrate that adversarial training is an effective approach to improving the robustness of VR. We believe that employing more advanced adversarial sample generation methods could potentially lead to improved results. For example, in VR, methods such as AVmixup [1] or DAJAT [2] could be employed to generate additional adversarial examples under various augmentation strategies, potentially further enhancing the robust generalization ability of VR. Additionally, approaches like MART [3] and SCORE [4], which refine the objective of adversarial training, could be utilized to better balance the accuracy and robustness of VR.

[1] Saehyung Lee, Hyungyu Lee, and Sungroh Yoon. Adversarial vertex mixup: Toward better adversarially robust generalization, 2020b.

[2] Sravanti Addepalli, Samyak Jain, and R. Venkatesh Babu. Efficient and effective augmentation strategy for adversarial training, 2022.

[3] Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In ICLR, 2020.

[4] Tianyu Pang, Min Lin, Xiao Yang, Jun Zhu, and Shuicheng Yan. Robustness and accuracy could be reconcilable by (proper) definition, 2022.

W3: This work focuses on classification accuracy on adversarial samples, but adversarial robustness is a multidimensional concept that requires more comprehensive evaluation metrics. Adversarial training has some limitations, such as high computational costs and the risk of overfitting, which require more discussion.

We agree that adversarial robustness is a multidimensional concept. According to your suggestion, we include the following additional adversarial robustness evaluation methods to further evaluate the adversarial robustness of VR: CW, Square, and AutoAttack. We conduct additional experiments using these methods to evaluate the robustness of VR, and the results are presented in the table below:

Table 3: The performance comparison of VR (Full / FullyLM) with ViT-S+ConvStem (A.T.) on different datasets under different adversarial attacks. Using PGD-10 ($\epsilon=4/255$) as adversarial training. The results are presented in the format of robustness (clean accuracy).

From the results, it can be observed that our strategy is robust under various types of adversarial attacks.

Continue by the next post.

Q1: The paper focuses on image classification tasks. VR can be applied to a wider range of tasks, such as object detection and semantic segmentation. Can the methods in this article be extended to other downstream tasks?

Our method can indeed be extended to other downstream tasks beyond image classification. While the primary focus of this work is on image classification, as outlined in IBM/model-reprogramming, the proposed approach can be adapted to different tasks by designing appropriate input transformations and output label mappings, depending on the specific source and target domains.

For example, [1] has demonstrated the vulnerability of semantic segmentation models to adversarial attacks. Building on this, one can leverage an adversarially pretrained vision model to achieve a robust semantic segmentation model by VR. Alternatively, by utilizing existing robust semantic segmentation models, as shown in [2], our method can be applied to adapt these models to new robust semantic segmentation tasks.

[1] Volker Fischer, Mummadi Chaithanya Kumar, Jan Hendrik Metzen, and Thomas Brox. Adversarial examples for semantic image segmentation, 2017

[2] Jindong Gu, Hengshuang Zhao, Volker Tresp, and Philip Torr. Segpgd: An effective and efficient adversarial attack for evaluating and boosting segmentation robustness, 2023.

We hope that our responses can satisfactorily address your concerns. Thank you again for your time to provide us with valuable feedback.

We genuinely appreciate your thorough review and constructive comments on our manuscript. Your feedback has provided us with valuable insights to help us further improve our work. We hope that our rebuttal has addressed your concerns. If you have further questions and suggestions, we would be happy to continue the discussion with you! Your participation in our work is greatly appreciated.

Thank you so much for your valuable comments! Our point-by-point responses to the reviewer's mentioned questions (Q) and weaknesses (W) are provided as follows.

W1: Limited novelty, the main component of the method is using adversarially pre-trained backbones, and using adversarial training during visual re-programming, which is a direct application of prior work.

We would like to emphasize that our primary goal is to explore the performance of visual reprogramming (VR) methods in adversarial scenarios, providing an important theoretical framework and experimental validation, rather than proposing a new vanilla VR method or a adversarial training technique. As VR methods leverage the potential of pre-trained model capabilities, an increasing number of VR methods have been proposed. However, these methods lack investigations focused on security, which severely hinders their application in security-sensitive domains. Adversarial robustness is a critical metric in security-critical fields, and adversarial training is a commonly used and effective approach for enhancing adversarial robustness. Therefore, in our research, we incorporate widely used adversarial training methods (i.e., PGD and TRADES) into our discussions. Furthermore, to explore the combination of different VR methods in adversarial scenarios, we include commonly used VR methods (i.e., Pad, FULL, RandLM, IterLM, and FreqLM) in our discussions.

As a fundamental research, our work theoretically and empirically demonstrates that the robustness of VR is influenced by various factors, such as the robustness of pre-trained models, the use of adversarial training, and different input transformations and output label mappings. These findings can serve as baseline results and help provide reliable insights for future research.

In our humble opinion, proposing entirely new methods is not the sole measure of contribution; our work is the first to investigate adversarial robustness in VR, providing theoretical analysis and experimental validation, which we believe represents a foundational and meaningful step in this area. There are many such foundational works, such as: AutoVP [1], which designs a framework to automate visual prompting for downstream image classification tasks; ARD [2], which combines knowledge distillation with adversarial training, motivates subsequent research.

[1] Hsi-Ai Tsao, Lei Hsiung, Pin-Yu Chen, Sijia Liu, and Tsung-Yi Ho. Autovp: An automated visual prompting framework and benchmark. In ICLR, 2024.

[2] Goldblum, Micah and Fowl, Liam and Feizi, Soheil and Goldstein, Tom. Adversarially Robust Distillation. In AAAI, 2020.

W2: The theoretical portion of the paper tries to tackle an interesting question, however the relevance to the method is limited ? The method itself is very empirical and intuitive, and the theory seems like an afterthought.

Our method is derived from our theoretical framework, and our experimental results validate our theory. They are intrinsically related, and the theory was not introduced subsequently. In Section 5.2, we have added more detailed discussions based on both experimental and theoretical results, including our reasons for using adversarially pre-trained models as backbones and incorporating adversarial examples from the target domain into the reprogramming, highlighted in blue.

Specifically, the first term of the upper bound in Theorem 1 indicates that a pre-trained model with robustness in the source domain can enhance the robustness performance after reprogramming. Therefore, we employed adversarially pre-trained models as the backbone in Strategies 3 and 4, which performed better than Strategy 1, thereby validating the rationale of our theory. Furthermore, the second term of the upper bound in Theorem 1 suggests that the robustness of visual reprogramming is influenced by $f_{\mathrm{in}}$, $f_{\mathrm{out}}$ and and the discrepancy between the source and target domains. Consequently, we utilized various $f_{\mathrm{in}}$ and $f_{\mathrm{out}}$ methods in our experiments. In most cases, SMM outperforms other $f_{\mathrm{in}}$ methods, while FullyLM surpasses other $f_{\mathrm{out}}$ methods. This is because these methods possess more complex function spaces, making it easier to bridge the gap between the source and target domains. These findings are consistent with our theoretical analysis. Additionally, the second term is related to adversarial margin disparity (Equation 12), which implies that incorporating adversarial training into the reprogramming process can mitigate risks, as adversarial margin disparity represents the distance in the adversarial space. Therefore, in Strategies 2 and 4, we employed commonly used adversarial training methods. Our experimental results confirm this, demonstrating that adversarial training consistently enhances the robustness of visual reprogramming.

Continue by the next post.

W3: Though the authors demonstrate results on various input transform/label mapping techniques, in terms of backbones only ResNet18 is used, and only on CIFAR-10/100 and GTSRB datasets, which consist of small images, which is qualitatively different from practical image recognition tasks.

Following your suggestions, we have added some experiments involving larger images and more general scenarios (Flowers102, OxfordPets, SUN397, and Food101). Additionally, we have included experiments with a variety of different model architectures (ViT-S+ConvStem and XCIT-S12). These experiments further validate our findings and support our theoretical framework. We will provide more comprehensive experiments in the subsequent version of the manuscript.

Table 1: The performance comparison of VR with XCIT-S12 (A.T.) on different datasets, and PGD-20 is used for adversarial testing. Using PGD-10 ($\epsilon=4/255$) as adversarial training. The results are presented in the format of robustness (clean accuracy). The best robustness is highlighted in bold.

Table 2: The performance of VR with XCIT-S12 (A.T.) under natural training (N.T.) on different datasets, and PGD-20 is used for adversarial testing. The results are presented in the format of robustness (clean accuracy). The best robustness is highlighted in bold.

Table 3: The performance comparison of VR with XCIT-S12 (N.T.) on different datasets, and PGD-20 is used for adversarial testing. Using PGD-10 ($\epsilon=4/255$) as adversarial training. The results are presented in the format of robustness (clean accuracy). The best robustness is highlighted in bold.

Table 4: The performance comparison of VR with ViT-S+ConvStem (A.T.) on different datasets, and PGD-20 is used for adversarial testing. Using PGD-10 ($\epsilon=4/255$) as adversarial training. The results are presented in the format of robustness (clean accuracy). The best robustness is highlighted in bold.

Table 5: The performance of VR with ViT-S+ConvStem (A.T.) under natural training (N.T.) on different datasets, and PGD-20 is used for adversarial testing. The results are presented in the format of robustness (clean accuracy). The best robustness is highlighted in bold.

Continue by the next post.

Q1: The authors could clarify more on how their theoretical work on guaranteed adversarial robustness risk upper bound affects their method and potentially what are the implications of their proof for this task.

Specifically, the first term of the upper bound in Theorem 1 suggests that a source domain pre-trained model with inherent robustness can improve robustness performance post-reprogramming. Consequently, we utilized adversarially pre-trained models as the foundation in Strategies 3 and 4. These strategies outperformed Strategy 1, thereby supporting the theoretical basis of our approach. Additionally, the second term of the upper bound in Theorem 1 implies that the robustness of visual reprogramming is affected by $f_{\mathrm{in}}$, $f_{\mathrm{out}}$ and the disparity between the source and target domains. As a result, we implemented various $f_{\mathrm{in}}$ and $f_{\mathrm{out}}$ methods in our experiments. In most instances, SMM outperformed other $f_{\mathrm{in}}$ methods, while FullyLM exceeded other $f_{\mathrm{out}}$ methods. This is attributed to the more complex function spaces of these methods, which facilitate bridging the gap between source and target domains. These results align with our theoretical analysis. Furthermore, the second term relates to adversarial margin disparity, indicating that integrating adversarial training into the reprogramming process can reduce risks, as adversarial margin disparity reflects the distance in the adversarial space. Therefore, in Strategies 2 and 4, we employed widely-used adversarial training techniques. Our experimental findings corroborate this, showing that adversarial training consistently enhances the robustness of visual reprogramming.

Our theory demonstrates that the robustness of VR is influenced by various factors (i.e., the robustness of pre-trained models, the use of adversarial training, and different input transformations and output label mappings). These insights can inspire the design of pre-trained models specifically tailored for adversarially robust VR (the first term in theorem), methods to evaluate and optimize the robustness distance between two domains (the second term in theorem), techniques for $f_{\mathrm{in}}$ and $f_{\mathrm{out}}$ specifically designed for adversarially robust VR (the second term in theorem), adversarial training methods specifically designed for adversarially robust VR (the second term in theorem) and optimization strategies targeting the third term of our theoretical framework.

Q2: Would these results generalize to more practical datasets such as ImageNet and recent models such as VisionTransformers, which have very different robustness characteristics than CNNs ?

We greatly appreciate your suggestions. Based on your suggestions, we have included some experiments involving more practical datasets and updated model architectures as shown in the reply of W3. We will provide more comprehensive experimental results in future versions of our manuscript. These experimental results also demonstrate the effectiveness of our strategy and validate our theory.

We hope that our responses can satisfactorily address your concerns. Thank you again for your time to provide us with valuable feedback.

We greatly appreciate your involvement in our work. Your insightful feedback has helped us enhance our work. We have addressed your concerns and suggestions in the rebuttal, and we would be much appreciated if you could kindly check our rebuttal. Please let us know if you have any other questions, and we would be delighted to continue the discussion with you. We are grateful for your efforts in improving our work.

We sincerely thank you for your thorough review and constructive comments on our manuscript. We appreciate your recognition of the clarity and proposed method. Your feedback has provided us with valuable insights that will help us further improve our work. Next, we will address your questions (Q) and weaknesses (W) in detail below.

W1: The technical novelty of this paper is limited. The modules for visual programming, such as input transformation and label mapping, are similar to (Chen, 2022; Cai et al. 2024). And those modules for adversarial robustness also have been proposed by previous literature (Madry et al. 2018; Zhang et al., 2019a)

We would like to emphasize that the primary goal of our work is to investigate the performance of Visual Reprogramming (VR) in adversarial scenarios, providing both a solid theoretical framework and experimental validation. While many VR methods have been proposed in the existing literature (e.g., [1]), there is a lack of research targeting safety-critical tasks, where adversarial robustness is particularly important. In these contexts, adversarial training (e.g., [2], [3]) is the most effective and widely adopted strategies to enhance model robustness.

Our work serves as a foundational study, theoretically and experimentally demonstrating that the robustness of VR is influenced by multiple factors: the robustness of the pretrained model, the use of adversarial training, and different input transformations and output label mappings. These findings can act as baseline results, helping to reduce redundancy in future research while providing reliable insights for subsequent innovations.

In our humble opinion, proposing entirely new methods is not the sole measure of contribution; our work is the first to investigate adversarial robustness in VR, providing theoretical analysis and experimental validation, which we believe represents a foundational and meaningful step in this area. There are many such foundational works, such as: AutoVP [4], which designs a framework to automate visual prompting for downstream image classification tasks; ARD [5], which combines knowledge distillation with adversarial training, motivates subsequent research.

Additionally, our work differs significantly from [6]. Their study can be understood as focusing on scenarios where the source and target domains are the same. For example, the experiments in [6] only utilize non-robust pretrained models on CIFAR-10 and leverage VP to add prompts to the inputs, achieving adversarial robustness on CIFAR-10. In contrast, our work focuses on the adversarial robustness of VR, where the source and target domains may differ. For instance, we use pretrained models on ImageNet and apply them to target tasks such as CIFAR-10 or GTSRB. As a result, we not only adopt different input transformations but also employ distinct output label mappings to adapt to the target domain. Furthermore, we evaluate the impact of adversarially pretrained models and non-adversarially pretrained models, which is different from the setting in [6].

[1] Chengyi Cai, Zesheng Ye, Lei Feng, Jianzhong Qi, and Feng Liu. Sample-specific masks for visual reprogramming-based prompting. In ICML, pp. 5383–5408, 2024.

[2] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR, 2018.T

[3] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In ICML, pp. 7472–7482, 2019a.

[4] Hsi-Ai Tsao, Lei Hsiung, Pin-Yu Chen, Sijia Liu, and Tsung-Yi Ho. Autovp: An automated visual prompting framework and benchmark. In ICLR, 2024.

[5] Goldblum, Micah and Fowl, Liam and Feizi, Soheil and Goldstein, Tom. Adversarially Robust Distillation. In AAAI, 2020.

[6] Aochuan Chen, Peter Lorenz, Yuguang Yao, Pin-Yu Chen, and Sijia Liu. Visual prompting for adversarial robustness. In ICASSP 2023, pp. 1–5, 2023a.

Continue by the next post.

W2: I'm a little confused about the task setting, especially in the availability of adversarially pre-trained models. For the source domain, it may not be practical to have both a pre-trained model and an adversarially pre-trained model. Obviously, we can’t temporarily train a source model via adversarial training. if we can, why don't we train a target model directly with clean and adversarial data, because both datasets are also available?

The purpose of VR is to apply well-trained pretrained models to other tasks without the need to retrain or modify the parameters of the pretrained models from scratch. There are many publicly available adversarially pretrained models with strong adversarial robustness, which can be obtained from platforms like RobustBench.

In practical applications, it is not necessary to simultaneously acquire both non-adversarially pretrained models and adversarially pretrained models. In our experiments, we set up this comparison to explore the impact of these two types of pretrained models on the robustness of VR and to validate the first term in the upper bound of Theorem 1: the influence of the adversarial risk of pretrained models on the source domain on VR robustness. Our results demonstrate that using adversarially pretrained models tends to provide better robustness for VR, which can inspire future research to better leverage the capabilities of adversarially pretrained models.

Moreover, it might be possible to directly train a robust model on the target domain, but this would be computationally inefficient to train a model from scratch. In contrast, as stated above, the purpose of VR is to better utilize existing pretrained models, with only a few learnable parameters. Therefore, VR can significantly reduce the training cost, making it possible to obtain larger-scale models with limited training resources.

W3: Experiments in Sec. 5 are more like ablation studies, to discuss the effectiveness of different designs for the proposed framework. However, it lacks comparisons with kinds of baselines or SOTA methods. Moreover, it would be more significant to evaluate the proposed method with more complex models on more large datasets, because that is the point of visual reprogramming to reduce the training and storage costs for target domains.

Our experiments include four strategies in terms of whether to use adversarially pre-trained models and whether to apply adversarial training. Among these, the strategy S1 employing non-adversarially pretrained models and non-adversarial training corresponds to the standard approach adopted in existing VR works, which can be considered as the baseline method. Moreover, because adversarial robustness remains an underexplored aspect in VR, no existing methods are explicitly designed to address this problem. Our work demonstrates that current VR methods are vulnerable to adversarial attacks and that leveraging adversarially pretrained models and adversarial training can enhance the adversarial robustness of VR. We believe this contribution provides both theoretical and empirical groundwork for future research in this area.

In order to further verify the effectiveness of our strategy on more complex models and larger datasets, we conducte additional experiments with ViT and XCiT on larger datasets, including Flowers102, OxfordPets, SUN397, and Food101 according to your suggestions:

Table 1: The performance comparison of VR with XCIT-S12 (A.T.) on different datasets, and PGD-20 is used for adversarial testing. Using PGD-10 ($\epsilon=4/255$) as adversarial training. The results are presented in the format of robustness (clean accuracy). The best robustness is highlighted in bold.

Table 2: The performance of VR with XCIT-S12 (A.T.) under natural training (N.T.) on different datasets, and PGD-20 is used for adversarial testing. The results are presented in the format of robustness (clean accuracy). The best robustness is highlighted in bold.

Continue by the next post.

Table 3: The performance comparison of VR with XCIT-S12 (N.T.) on different datasets, and PGD-20 is used for adversarial testing. Using PGD-10 ($\epsilon=4/255$) as adversarial training. The results are presented in the format of robustness (clean accuracy). The best robustness is highlighted in bold.

Table 4: The performance comparison of VR with ViT-S+ConvStem (A.T.) on different datasets, and PGD-20 is used for adversarial testing. Using PGD-10 ($\epsilon=4/255$) as adversarial training. The results are presented in the format of robustness (clean accuracy). The best robustness is highlighted in bold.

Table 5: The performance of VR with ViT-S+ConvStem (A.T.) under natural training (N.T.) on different datasets, and PGD-20 is used for adversarial testing. The results are presented in the format of robustness (clean accuracy). The best robustness is highlighted in bold.

These experimental results have been added to Appendix C, and We will provide more comprehensive experiments in the subsequent version of the manuscript. These experimental results demonstrate the effectiveness of our strategy on more complex models as well as on larger datasets.

W4: In Eq. 10, $\Phi_\rho$ has two input variables (i.e., $x$ and $y$), but in the definition of Eq. 11, it only has one.

Thank you for your suggestion. There is some ambiguity in the expression of Eq. 10. To provide a clearer expression, we have rewritten Eq. 10 as follows:

We hope that our responses can satisfactorily address your concerns. Thank you again for your time to provide us with valuable feedback.

We sincerely appreciate you taking the time to provide constructive feedback on our manuscript. In our rebuttal, we have addressed your concerns and questions, particularly regarding the technical novelty and the task setting. If you have any further concerns or suggestions, we would be happy to continue the discussion with you! We once again appreciate your efforts in enhancing our work.

We sincerely appreciate your time and valuable feedback on our manuscript. Your recognition of the effectiveness and importance of our theoretically guaranteed, simple yet effective research is truly inspiring. The minor issues you pointed out will be carefully addressed in the revised version of the manuscript. Our point-by-point responses to the reviewer's mentioned questions (Q) and weaknesses (W) are provided as follows.

W1: The two proposed strategies are mature techniques in the field. Technically, it's simply an extension to visual reprogramming and there are no surprising results. Meanwhile, as the number of learnable parameters in visual reprogramming is limited, adversarial training may not be an ideal solution to improve the robustness.

We agree with the reviewer that the proposed strategies are based on well-established techniques. However, we would like to emphasize that our main contribution is to provide a pioneer study of adversarial robustness for visual reprogramming (VR). The significance of VR lies in applying existing pre-trained models to new tasks while reducing the computational costs. However, the current VR field lacks research on scenarios where the target tasks are security-critical. Therefore, to address this gap, our work serves as a foundational study, theoretically and experimentally demonstrating that the robustness of VR is influenced by multiple factors: the robustness of the pretrained model, the use of adversarial training, and different input transformations and output label mappings. We believe that this comprehensive framework can guide future research to improve the robustness of VR by combining more robust pre-trained models with stronger adversarial training specifically tailored for VR.

In our humble opinion, proposing entirely new methods is not the sole measure of contribution; our work is the first to investigate adversarial robustness in VR, providing theoretical analysis and experimental validation, which we believe represents a foundational and meaningful step in this area. There are many such foundational works, such as: AutoVP [1], which designs a framework to automate visual prompting for downstream image classification tasks; ARD [2], which combines knowledge distillation with adversarial training, motivates subsequent research.

Additionally, we agree that the number of learnable parameters in visual reprogramming is limited; however, we would like to emphasize that adversarial training remains a crucial technique for enhancing model robustness in adversarial machine learning. To defend against adversarial attacks, as listed in Section IV of the review [3], most adversarial defense approaches are based on adversarial training or propose improved methods for adversarial training. Moreover, our experimental results demonstrate that adversarial training can indeed effectively enhance adversarial robustness in visual reprogramming. Therefore, we believe that adversarial training remains an important method to improve the robustness of models in adversarial settings, even when the number of learnable parameters is constrained. Furthermore, we also hope that our proposed comprehensive theoretical framework will inspire further research to develop methods for enhancing the adversarial robustness of visual reprogramming with a limited number of learnable parameters.

[1] Hsi-Ai Tsao, Lei Hsiung, Pin-Yu Chen, Sijia Liu, and Tsung-Yi Ho. Autovp: An automated visual prompting framework and benchmark. In ICLR, 2024.

[2] Goldblum, Micah and Fowl, Liam and Feizi, Soheil and Goldstein, Tom. Adversarially Robust Distillation. In AAAI, 2020.

[3] Baoyuan Wu and Shaokui Wei and Mingli Zhu and Meixi Zheng and Zihao Zhu and Mingda Zhang and Hongrui Chen and Danni Yuan and Li Liu and Qingshan Liu. Defenses in Adversarial Machine Learning: A Survey. arXiv, 2023.

Continue by the next post.

W2: Although the theoretical analysis correponds to the proposed strategies, the AMDD-VRA part, which takes up the most context, is not reflected or applied in practice. This term seems to be the most important factor in VR, which is neither taken into consideration in methods nor discussed based on experimental results. For instance, the performance-robustness trade-off on GT-SRB is more noticeable than that on CIFAR. Does it has anything to do with the gap between the source domain and the target domain?

We have added a more detailed discussion based on experimental and theoretical results in Section 5.2, highlighted in blue. Our experimental results indicate that visual reprogramming with adversarially pre-trained models yields greater robustness compared to using naturally pre-trained models. This aligns with the first term of the upper bound in Theorem 1, which suggests that pre-trained models with differing levels of robustness exhibit distinct adversarial risks in the source domain. Consequently, employing a more robust pre-trained model may lead to a more robust visual reprogramming model.

In addition, the second term of the upper bound in Theorem 1 suggests that the robustness of visual reprogramming is influenced by $f_\text{in}$, $f_\text{out}$, and the disparity between the source and target domains. For instance, in most cases, SMM outperforms other $f_\text{in}$ methods, and FullyLM outperforms other $f_\text{out}$ methods. These findings are consistent with our theoretical analysis, indicating that designing better $f_\text{in}$ and $f_\text{out}$ can lead to improved robustness. Furthermore, the second term of the upper bound in Theorem 1 is related to adversarial margin disparity (Eq. 12). This implies that incorporating adversarial training during reprogramming can reduce the risk. Our experimental results confirm this, as adversarial training consistently enhances the robustness of visual reprogramming.

The second term of the upper bound in Theorem 1 also indicates that the robustness of visual reprogramming is affected by the discrepancy between the source and target domains. As the reviewer noted, the performance-robustness trade-off on GTSRB is more noticeable than that on CIFAR. This discrepancy can likely be attributed to the fact that the models used in our experiments were pretrained on ImageNet. ImageNet and CIFAR both involve natural object classification tasks, whereas GTSRB focuses on traffic signs, which are more specific and differ significantly in style and context from the complex natural backgrounds in ImageNet. Since the purpose of $f_\text{in}$ is to serve as an input transformer, designing more powerful $f_\text{in}$ (e.g., SMM) may further reduce the gap between the source and target domains. For example, as shown in Figure 3, the performance gap between Full/FullyLM and SMM/FullyLM is relatively small on the CIFAR-10 dataset, whereas on GTSRB, the robustness of SMM/FullyLM is significantly higher than that of Full/FullyLM.

Continue by the next post.

W3: There can be more discussions on the recent advancements in Visual Reprogramming and Visual Prompting. For instance, [1] has studied VP with adversarial robustness, what's the difference between your work and theirs. Also, [2] has extended VP to visual foundation models, is your work applicable to more significant scenarios like MLLMs?

The task settings of Visual Reprogramming (VR) and Visual Prompting (VP) indeed differ significantly. In VR, the source domain and target domain can belong to entirely different domains, as demonstrated in IBM/model-reprogramming, where the source and target domains may even span different modalities (e.g., images, text, or audio). This allows for the flexible selection of various source domains for VR operations. Furthermore, Section 2.2 of [3] highlights that existing VP methods differ in terms of the placement and functionality of prompts, whereas VR offers a model-agnostic prompting technique. This involves adding trainable noise to input images before forward propagation, which neither alters their visual essence nor relies on specific model architectures.

The differences between our work and [1] align with the above description. In [1], the pretrained model and the dataset used to generate prompts belong to the same dataset (i.e., CIFAR-10). In contrast, in our work, the source domain dataset for the pretrained model and the target dataset for VR are distinct. For instance, we use various models pretrained on ImageNet while employing CIFAR-10 as the target dataset. Additionally, [1] only considers non-robust pretrained models, whereas we analyze the impact of robust and non-robust pretrained models from both theoretical and experimental perspectives.

Due to the differences between the source and target domains in our work, we also utilize distinct visual input transformations and introduce output label mapping, which is not employed in [1], to perform experiments with different combinations. As a foundational study, our work explores the potential of endowing VR with adversarial robustness from both theoretical and experimental angles.

Finally, our research holds significant potential for practical applications, particularly in security-critical MLLMs. Previous studies, such as those cited in IBM/model-reprogramming, have already demonstrated the powerful potential of reprogramming techniques across different tasks and modalities. For example, when applying general-purpose MLLMs to financial transaction scenarios, the lack of task-specific training may result in elevated security risks. Our work can provide a viable direction to address this challenge.

[1] Aochuan Chen, Peter Lorenz, Yuguang Yao, Pin-Yu Chen, and Sijia Liu. Visual prompting for adversarial robustness. In ICASSP 2023, pp. 1–5, 2023a.

[2] Zhang, Yichi, et al. "Exploring the Transferability of Visual Prompting for Multimodal Large Language Models." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.

[3] Chengyi Cai, Zesheng Ye, Lei Feng, Jianzhong Qi, and Feng Liu. Sample-specific masks for visual reprogramming-based prompting. In ICML, pp. 5383–5408, 2024.

W4: Minor: There are some typos in submission, e.g., 'iss' in line 191.

Thank you for pointing out these issues, which will be very helpful for us to improve our paper. We have rechecked our manuscript and corrected the typos and clarity issues in the revised manuscript.

Q1: Why $f_\text{out}$ is concatenated to $f_\mathcal{S}$ in the last term of the equation in Definition 1? Is there any intuitive explanation on why the source domain $\mathcal{S}$ should also be mapped to the $\mathbb{R}^{\mathcal{T}}$?

AMDD-VRA (Definition 1) measures the adversarial margin disparity between the source and target domains within hypothesis spaces $\mathcal{F} _{\mathcal{S}}$ . Since the label spaces $\mathcal{Y}$ of the source and target domains differ, we add the label mapping method to map the source domain label space $\mathcal{Y} _{\mathcal{S}}$ to the target domain $\mathcal{Y} _{\mathcal{T}}$. These methods typically consider only certain labels from the source domain (e.g., FullyLM uses a one-to-one mapping based on frequency), so the labels not considered do not impact VR. In the last term of the equation in Definition 1, $f _\text{out}$ focuses only on the mapped labels from the source domain, as only these labels relate to the performance of VR.

We hope that our responses can satisfactorily address your concerns. Thank you again for your time to provide us with valuable feedback.

Thank you for your response. Our responses to your new questions (Q) are provided as follows.

Q1: Is there any unique or new challenge in the adversarial robustness of visual reprogramming compared to image classification? This can help highlight the significance of your work.

Compared to image classification, there are unique or new challenges in the adversarial robustness of visual reprogramming as follows:

• Compatibility Between Adversarial Training Methods and VR Frameworks: In VR, the trainable components are limited to input transformation $f_\text{in}$ and output label mapping $f_\text{out}$. This restriction presents a unique challenge in designing adversarial training algorithms. For example, when using adversarially pre-trained models, it is a new challenge to simultaneously account for such constraints and better leverage the robustness of pre-trained models to develop a more compatible and efficient adversarial training method.

• Design of $f_\text{in}$ and $f_\text{out}$ Methods: Our experiments demonstrate that the choice and design of $f_\text{in}$ and $f_\text{out}$ methods significantly impact the adversarial robustness of the model. Designing $f_\text{in}$ and $f_\text{out}$ methods with inherent consideration for adversarial robustness is an unique challenge in the adversarial robustness of VR.

In our work, we theoretically and experimentally demonstrate that the robustness of VR is influenced by multiple factors: the robustness of the pretrained model, the use of adversarial training, and different input transformations and output label mappings. This lays the foundation for future research to address the challenges mentioned above.

Q2: What is the role of $f_\text{out}\circ f_\mathcal{S}$ on the domain $\mathcal{S}$ ? To me, this term does not have a exact meaning in the context of visual reprogramming.

Thanks so much for pointing out the issue of $f_\text{out}\circ f_\mathcal{S}$, which helps us identify an oversight in our theorem. We have carefully reviewed Definition 1 and Theorem 1 and found that $f_\text{out}$ was mistakenly added to the last term. The revised Definition 1 is given as follows:

Specifically, the last term is the adversarial margin disparity (AMD) on the source domain, which quantifies the maximum margin disparity between models $f^{\prime} _\mathcal{S}$ and $f _\mathcal{S}$ within the $\epsilon$-perturbation range around the instance $\boldsymbol{x}$, meaning the expectation of inconsistent classification between $f^{\prime} _\mathcal{S}$ and $f _\mathcal{S}$ under $\epsilon$-perturbation in the domain $\mathcal{S}$. The first term is similarly defined on the target domain. The difference between these two terms in Adversarial Margin Disparity Discrepancy of Visual Reprogramming with Adversarial Robustness (AMDD-VRA) is used to measure the adversarial distribution distances between the source domain $\mathcal{S}$ and the target domain $\mathcal{T}$. It can be observed that $f _\text{in}$ and $f _\text{out}$ influence the value of AMDD-VRA. For instance, when the source domain and target domain are completely identical, if both $f _\text{in}$ and $f _\text{out}$ are identity mappings, the AMDD-VRA value is 0. However, in the context of VR settings, where the source and target domains are not identical, $f _\text{in}$ and $f _\text{out}$ can play a role in either narrowing or widening the adversarial discrepancy between the two different domains. Furthermore, based on the definition of AMD, it can be inferred that whether adversarial training is employed for $f _\text{in}$ and $f _\text{out}$ may impact the first term of AMDD-VRA, thereby affecting the overall value of AMDD-VRA.

Fortunately, we have thoroughly examined the paper and confirmed that this oversight does not affect our conclusions. We have revised Definition 1 and the proof of Theorem 1 in Appendix A, highlighting the changes in blue, and have uploaded the revised version. Furthermore, the issue ($f_\mathrm{out} \circ f_\mathcal{S}$) addressed in Q2 will no longer be present in the revised manuscript.

Thanks again for the reviewer's carefulness!