Bag of Tricks to Boost Adversarial Transferability

Responses to Weakness&Questions 2-3

A2: As you suggested, we provide analysis and evaluation as follows,
[Analysis] In our appendix (Fig. C4), we categorize the adversarial attack process into four distinct stages: initialization, input transformation, ensemble, and adversarial perturbation updating. Our techniques are designed to integrate seamlessly into these stages. For instance, we apply the random global momentum initialization and dual example in the initialization stage, high-frequency perturbation and spectral-based augmentations in the input transformation stage, gradient alignment, model shuffle, and asynchronous input transformation in the ensemble stage, and the scheduled step size in the adversarial perturbation updating stage. This structured approach ensures that each technique operates independently, contributing uniquely to the overall performance enhancement.
[Experimental results] To empirically validate the orthogonality of our techniques, we conducted experiments combining them in pairs. The studied tricks include random global momentum initialization (RGI), dual example with scheduled step size (DS), high-frequency spectral transformation (HT), gradient alignment (GA), asynchronous input transformation (AIT), and model shuffle (MS). We use the MI-FGSM as the optimization method for attack under the ensemble settings in our paper. We report the average attack success rates against the defense models for study, including the AT, FAT, HGD, RS, and NRP. The results are presented as follows,

Tab. 1 Average attack success rates (%) of the combination of our combined tricks on the defense methods.

Tricks	RGI	DS	HT	AIT	MS	GA
RGI	48.2	51.5	53.4	49.3	48.5	48.7
DS	--	49.3	55.9	51.8	50.6	53.9
HT	--	--	50.2	57.0	53.4	51.8
AIT	--	--	--	48.5	49.3	49.0
MS	--	--	--	--	47.6	48.5
GA	--	--	--	--	--	48.1

The results demonstrate that the combination of different techniques leads to a significant improvement in performance. For example, combining random global momentum initialization (RGI) with dual example and scheduled step size (DS) resulted in a higher success rate (51.5%) compared to using either technique alone (RGI: 48.2%, DS: 49.3%). On the one hand, the RGI guides the optimization towards the global optima. On the other hand, more near-sample gradients are utilized by the scheduled step size to improve the adversarial transferability. This evidence strongly supports our claim of orthogonality and synergistic effectiveness.

A3: Our proposed tricks can be integrated into various attacks. As you suggeted, we update the results of our revised paper with the use of these methods, including MIG [a], TGR [b], NAA [c], and RAP[d], with or without our proposed tricks to generate adversarial examples under the ensemble setting, and attack the advanced defense models and Google's vision API. We generate $1,000$ adversarial examples to attack defense models and $100$ adversarial examples to attack Google's vision API. The results are depicted as follows,

Tab.1 Attack success rates (%) of our combined tricks and various attacks on the defense methods and Google’s vision API.

Method	Tricks	AT	FAT	HFD	RS	NRP	Google
MIG [a]	$\times$	72.1	38.6	69.5	24.6	65.3	66.0
MIG [a]	$\checkmark$	91.3	52.1	88.5	42.4	81.6	79.0
TGR [b]	$\times$	75.1	35.3	82.4	29.5	73.0	70.0
TGR [b]	$\checkmark$	97.3	61.5	95.4	57.6	91.3	100.0
NAA [c]	$\times$	69.5	34.3	80.2	25.5	64.6	59.0
NAA [c]	$\checkmark$	80.2	61.3	85.7	35.9	73.1	71.0
RAP [d]	$\times$	59.2	33.5	75.3	20.6	58.1	52.0
RAP [d]	$\checkmark$	65.3	36.9	79.2	24.8	65.7	62.0

The results clearly illustrate that applying our techniques significantly improves the adversarial transferability of these recent methods. This updated analysis and the corresponding results have been added to Section 4 of our revised manuscript.

Responses to Weakness&Questions 1

A1: [Analysis on the hyper-parameter study] As you suggested, we have revised our paper to present a more in-depth analysis of our hyper-parameter study, which can be summarized as follows,

• [Number of iterations T] (1) The results of I-FGSM in Fig. 2a by varying the number of iterations suggest an up-to-down trend in attack success rates, indicating the importance of gradients near the input image in crafting transferable adversarial examples. (2) GIMI-FGSM performs best at around T = 5, leveraging global momentum computed using the first several iterations to bypass nearby local optima effectively. However, its attack success rate diminishes with larger T values, indicating that the effectiveness of this momentum diminishes against distant local optima.

• [The scale factor for step size] We observed a notable performance boost with an optimal step size range (Fig. 2b). An adequately large step size can be utilized to skip over the local optima and boost the adversarial transferability, which can be verified by I-, MI-, NI, etc. The excessively large step size can overlook the optima and cause inefficient optimization under $l_∞$ constraint, thus harming attack performance, which can be observed in PI- and EMI-FGSM.

• [The decay factor for momentum] By varying the momentum decay factor (Fig. 2c), we can see a significant performance change in different attacks. The decay factor balances the historical momentum and novel gradient, adjusting which to an appropriate ratio can stabilize the update directions, thus better searching for the direction towards optima to boost the adversarial transferability.

[Takeaways] The results of the hyper-parameter study suggest that adversarial transferability is much more sensitive to some basic settings than we thought. We appeal to more concerns on the overlooked confounders when benchmarking attacks.

[Inspiration] We are motivated by the insights deriving from the study on the number of iterations and scale factor for step size to design the scheduled step size and dual example strategy, thus better utilizing the near-sample gradients for better adversarial transferability. We are motivated by the analysis of GI-MIFGSM and the study of the momentum decay factor to design the random global momentum initialization technique to find a better initialization value for momentum, thus stabilizing the optimization direction towards the optima. However, our proposed tricks are just simple utilization of these insights, which remain more in-depth theoretical analysis and practice exploration for future work.

Dear Reviewer A7nY,
We have submitted our response to your questions and revised our paper as you suggested. We sincerely appreciate your valuable feedback on improving the quality of our paper.
Are there any additional questions or concerns we can answer? Thanks for your reply!

Sincerely,
Authors

Responses to Weakness & Questions

A1: Thank you for your insightful question. The theoretical analysis in understanding adversarial transferability remains less explored but is fundamentally important [1]. To the best of our knowledge, theoretical work on the effectiveness of attack methods in adversarial transferability is quite limited and challenging [2-5].

Our work is an empirical study, which starts from the hyper-parameter study, deriving lots of intuitive insights from these results and correspondingly proposing a bag of tricks to boost the adversarial transferability. In particular, the numeric results in the hyper-parameter study suggest that adversarial transferability is much more sensitive to some basic settings than we thought. We hope our empirical findings with extensive practical experiments can motivate theoretical insights and analysis on this sensitivity to understand adversarial transferability better. We add more in-depth analysis in the hyper-parameter study of Section 3 and provide a detailed discussion, including the potential theoretical direction to utilize the insights, in Appendix G of our revision.

[1] Bose, Joey, et al. "Adversarial example games." NeurIPS 2020.
[2] Wang, Yilin, and Farzan Farnia. "On the role of generalization in transferability of adversarial examples." UAI 2023.
[3] Yang, Zhuolin, et al. "Trs: Transferability reduced ensemble via promoting gradient diversity and model smoothness." NeurIPS 2021.
[4] Zhang, Yechao, et al. "Why Does Little Robustness Help? A Further Step Towards Understanding Adversarial Transferability." S&P 2024.
[5] Zhao, Anqi, et al. "Minimizing Maximum Model Discrepancy for Transferable Black-box Targeted Attacks." CVPR 2023.

A2: The computation overheads in adversarial attacks mainly derive from the forward and backward propagations. Random global momentum initialization (RGI) with additional iterations definitely increases the computation overhead and training time (around $1.5 \times$). However, it is noteworthy that this increase in computation facilitates a substantial enhancement in adversarial transferability, with improvements of up to 9.2% in MI-FGSM. Given the significant enhancement in attack capability, the increased computational time is justifiable, especially from the perspective of an attacker prioritizing attack efficiency.

Besides, most of our tricks do not take additional forward and backward propagations, which do not decay the computational efficiency. For instance, in our proposed tricks under the non-ensemble setting, the scheduled step size does not incur any additional computational cost. Additionally, the dual example and high-frequency transformation introduce a minor computational overhead with minimal impact on the overall computational duration.

To support our argument, we conducted an experiment using the MI-FGSM method with VGG-16 to craft 10,000 adversarial images, integrating several tricks, including RGI. The results are as follows:

Method	MI	MI+RGI	MI+Scheduled step size	MI+Dual Example	MI+High-Frequency Transformation
Time (s)	332	495	332	332	339

It can be seen that the introduction of RGI results in around a 50% increase in computation time compared to the standard MI-FGSM, while others merely influence the efficiency. Considering the efficiency of adversarial example generation and the attack performance improvement, we think this additional time is acceptable for better performance in practical applications.

A3: Due to the strong capacity of certified defense, most existing adversarial attacks diminish the performance, while the integration of our tricks can boost the attack performance. As shown in Tab. 5 of our revised paper, while RS effectively defends the various advanced attack methods, the integration of our tricks to these methods consistently brings a significant improvement in attack success rates (17.9% on average). It indicates that our tricks can be employed to attack more invariant robust features, thus improving the attack performance.

Dear Reviewer Pj5y,
We have submitted our response to your questions and revised our paper as you suggested. We sincerely appreciate your valuable feedback on improving the quality of our paper.
Are there any additional questions or concerns we can answer? Thanks for your reply!

Sincerely,
Authors

Responses to Questions

A1: In Fig. 1, we combine 5 tricks into different attacks, namely the random global momentum initialization, scheduled step size, dual example, high-frequency transformation, and spectral input transformations.

A2: For all experiments in our paper, we use the ImageNet-1K dataset. We randomly select $10$ images from each class in ImageNet-1K for evaluation. All of these images are classified correctly by $8$ studied models in our paper.

A3: MI-FGSM is generally adopted as the backbone attack method for various attack methods in many previous studies, such as DIM, TIM, Admix, etc. Thus, we also use the MI-FGSM as the base method under the ensemble setting to generate adversarial examples.

We have provided the experimental results of applying our tricks to advanced ensemble-based attacks (SVRE and SSA) under the ensemble setting in Tab. D.3 of Appendix D.3.

Here, to further verify the effectiveness of our proposed tricks, we replace MI-FGSM with two other gradient-based attacks, namely PI- and VMI-FGSM, under the same ensemble setting in our paper for evaluation. The results are depicted as follows,

Strategy	Trick	ASR (PI/VMI)
Loss	--	75.3/77.2
Loss	GA	76.5/79.3
Loss	AIT	75.8/78.4
Logit	--	86.3/90.1
Logit	AIT	88.2/93.5
Logit	GA	88.1/91.7
Prediction	--	75.6/78.1
Prediction	GA	76.9/79.3
Prediction	AIT	78.5/81.6
Longitude	--	87.2/92.3
Longitude	MS	90.0/94.1

From the results, we can see the integration of our tricks (GA, AIT, and MS) enhances the ensemble attack performance of PI- and VMI-FGSM, consistent with the results on MI-FGSM in our paper. It validates the effectiveness of ensemble strategies.

Responses to Weaknesses

A1: Thanks for your careful review! We fixed these typo issues in our revision and polished the whole paper carefully.

A2: Thanks for your suggestion! we give a more in-depth analysis in the hyper-parameter study of our revision.

A2.1: Different from other gradient-based methods, GIMI-FGSM initializes the momentum with a pre-computed global momentum. We infer that it is the global momentum that helps GIMI-FGSM skip the local optima near benign samples faster than others, but it is still easily influenced by multiple local optima away from the benign samples. Thus, GIMI-FGSM has a peak performance at around $T=5$ in our experiment, which is much earlier than others.

A2.2: There are two reasons for the performance degradation with excessively large step sizes. First, it is easy to skip over the optima using the excessively large step size for optimization, leading to poor adversarial transferability. Second, following the general setup of previous work, the crafted adversarial perturbation is limited within $\frac{16}{255}$. With the excessively large step size, the generated perturbation easily exceeds the limit, leading to being clipped to satisfy the constraint. This will fail the effectiveness of optimization in the current iteration, leading to performance degradation.

A2.3: When the momentum decay factor is less than one, it indicates a decreasing impact of historical gradients in the computation of momentum, instead of the decreasement of momentum. Following previous studies [1,2,3], we apply the decay factor to the historical accumulated gradients, i.e.,

$m_{t+1}=\gamma \cdot m_{t} + g_{t+1},$

where we denote $m_{t}$ and $m_{t+1}$ as the momentum, $g_{t+1}$ as the novel gradient, and $\gamma$ as the decay factor. The same presentation is also consistently adopted in our algorithms in the appendix, such as Alg.1 (random global momentum initialization) in Appendix B.1 and Alg. 2 (dual example with ensemble strategy) in Appendix B.2. With decreasing $\gamma$, it is not good for stabilizing the optimization, leading the optimization of adversarial examples easily stuck in local optima for performance degradation.

Thanks for your point. We revised our paper to make it more clear.
[1] Wang et al. Enhancing the transferability of adversarial attacks through variance tuning. CVPR 2021.
[2] Gao et al. Patch-wise attack for fooling deep neural network. ECCV 2020.
[3] Lin et al. Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attacks. ICLR 2020.

A3: Following previous work [1], we conduct an experiment on attacking audio-visual models in the event classification task. For the surrogate model, we use the model that uses ResNet-18 for both audio and visual encoders and sum operation as the fusion module for features of audio and visual modalities. The targeted model involves two models, which respectively use VGG-16 and ResNet-34 as the visual and audio encoders. We use the audio-visual model with the ResNet-18 as the backbone to craft 200 adversarial examples and attack the other two models. The 200 audio-visual samples are randomly selected from the MIT-MUSIC dataset, which are classified correctly by the studied models. We use MI-FGSM to craft the adversarial examples and respectively apply the proposed random global initialization (RGI), scheduled step size, and dual example strategies to MI-FGSM. The results of attack success rates are reported as follows,

Audio-Visual Models	ResNet-18	VGG-16	ResNet-34
MI	100.0	39.5	40.6
MI+RGI	100.0	43.7	42.8
MI+Scheduled step size	100.0	42.1	41.9
MI+Dual example	100.0	45.2	46.9

From the results, we can see a clear improvement (3.8% on average) by integrating the tricks into MI-FGSM. It shows the scalability of our tricks in audio-visual applications. We add these results in Appendix H of our revision.

[1] Tian, Yapeng, and Chenliang Xu. "Can audio-visual integration strengthen robustness under multimodal attacks?." CVPR 2021.

A4: In the long run, future studies are expected to design novel training methods to improve the model robustness, e.g., adversarial training and certified defense by random smoothing. For real-world applications, it is valuable to explore input transformation-based methods to purify the adversarial perturbation, thus achieving a good balance between efficiency and robustness.

Responses to Weaknesses and Questions

A.4: To verify the scalability of our proposed tricks, we respectively integrate the random global momentum initialization (RGI), dual example (DE), gradient alignment (GA), asynchronous input transformation (AIT), and model shuffle (MS) into the momentum- and ensemble-based method (MI-CWA) [2]. Following the same setting in our paper, we evaluate the attack performance against the defense methods. The results are depicted in the following table.

Method	AT	FAT	HGD	RS	NRP
MI-CWA	82.7	45.2	89.1	39.2	83.5
MI-CWA+RGI	83.1	45.7	90.5	43.0	84.6
MI-CWA+DE	86.9	48.5	91.6	44.7	85.2
MI-CWA+GA	82.8	45.2	89.0	39.4	83.7
MI-CWA+AIT	92.8	55.4	97.2	51.5	87.2
MI-CWA+MS	83.2	45.5	89.1	40.0	83.9
MI-CWA+combination	99.1	62.4	100.0	59.1	92.8

From the results, we can see the introduction of different tricks brings different improvements to the attack performance against various models. The MI-CWA integrated with combinational tricks achieved the best performance, surpassing the best result achieved by VMI+comnination in our paper with an improvement of 2.33% on average. Although MI-CWA achieves the SOTA performance among the ensemble-based methods, our tricks can still boost the adversarial transferability, which further validates the generality and superiority of the proposed tricks.
For a better demonstration of the effectiveness and scalability of our tricks to advanced attacks, we added it to our discussion and the corresponding results in Tab.5 of our revision.

[1] Chen, H., Zhang, Y., Dong, Y., & Zhu, J. (2023). Rethinking Model Ensemble in Transfer-based Adversarial Attacks. arXiv preprint arXiv:2303.09105

A.5: TIM [2] adopts Gaussian smooth on the gradient to approximate the average gradient of a set of translated images to update the adversary, which is a classical input transformation-based attack method. In our paper, however, we mainly study the effect of number of copies and diversity on the performance of input transformation-based attack methods. Thus, we do not include the comparison with TIM in our main paper. We have the comparison with TIM in our supplementary, shown in Tab. F9. We discussed it in Section 2 of the revision as you suggested. Thanks.

[2] Dong, Yinpeng, et al. "Evading defenses to transferable adversarial examples by translation-invariant attacks." CVPR 2019.

Responses to Weaknesses and Questions

A.1 && A.2: We thank the reviewer for reading our paper carefully and giving the above positive comments！

A.3: In the hyper-parameters study, we select 7 FGSM-based approaches, including I-FGSM, MI-FGSM, NI-FGSM, PI-FGSM, EMI-FGSM, VMI-FGSM, GIMI-FGSM, for a comprehensive comparison.

In Section 3.2, we study the influence of momentum initialization and the proposed trick, random global momentum initialization (RGI). We did not include the performance comparison with I-FGSM because I-FGSM does not use the momentum for crafting adversarial perturbation. We already include the comparison with GIMI-FGSM in Tab. 1. We denote the global momentum initialization technique as GI in Tab. 1 and apply it to compared methods, including MI-, NI-, PI-, EMI-, and VMI-FGSM. The integration of GI- into MI-FGSM is the GIMI-FGSM. Thus, we actually compare 6 methods in this section. We will clarify it in our revision.

In Sections 3.3 and 3.4, we study the performance of proposed tricks, namely the scheduled step size and dual example. The proposed tricks aim sample better gradients around the input samples for crafting transferable adversarial examples. In the current version, we do not include the NI-FGSM and EMI-FGSM because they adopt look-ahead (calculating the gradient on a sample around the input sample) for gradient computation to skip the local optima, while the other five methods focus on using gradients better to update directly the adversarial example, which are more matched with our proposed tricks. As you suggested, Following the same setting in our paper, we also integrate our tricks into NI-FGSM and EMI-FGSM, as presented in the following tables.

Tab. 1: Average attack success rates (%) of gradient-based attacks using various scheduled step sizes, including NI-FGSM and EMI-FGSM. We report the performance under the default setting (T = 10)/optimal setting. We denote ”Ori.” as the identity step size. We use the VGG-16 as the surrogate model and evaluate the adversarial transferability on other 7 models.

Step	NI-FGSM	EMI-FGSM
Ori.	57.4/58.6	65.0/67.1
log	58.6/59.1	65.9/67.4
linear	60.4/60.7	66.7/68.3
exp	57.9/57.9	63.8/64.5
pvalue	58.3/59.0	64.2/64.9

Tab. 2 : Average attack success rates (%) when applying dual example w/o or w ensemble strategy using various sequences as step size to NI-FGSM and EMI-FGSM. We denote ”N/A” as vanilla adversarial attacks, and others as dual example with scheduled step size. We use the VGG-16 as the surrogate model and evaluate the adversarial transferability on the other 7 models.

Step	NI-FGSM	EMI-FGSM
N.A.	57.4	65.0
Ori.	59.3/64.0	66.3/68.9
log	62.5/66.9	63.8/69.4
linear	61.7/67.3	67.1/69.4
exp	61.3/65.8	65.9/67.3
pvalue	62.0/64.2	66.2/68.7

In Tab.1, we present the performance of the application of scheduled step size. We can see a clear attack success rate improvement by simply adopting a suitable scheduled step size sequence. With the linear step size, the NI- and EMI-FGSM have an improvement of $2.6\%$ and $1.5\%$ on average, respectively.
In Tab.2, we present the performance of the application of dual example with scheduled step size. We can see a significant improvement from the numeric results. Specifically, by using ensemble strategy of the dual example, NI- and EMI-FGSM have a large improvement of up to $9.9\%$ and $4.4\%$ in average attack success rate.
On the one hand, we can observe that the proposed tricks are also applicable to these attacks, which further validate their superiority and generality. On the other hand, consistent with our paper, different attack methods have different suitable scheduled step sizes for the best performance. It remains an open problem about how to decide the best scheduled or dynamic step size to boost the adversarial transferability. We hope our results can motivate future work researching this interesting problem.
We added the results to Tab.2 and Tab.3 of our revision as you suggested. Thanks.

Dear Reviewer uKVA,
We have submitted our response to your questions and revised our paper as you suggested. We sincerely appreciate your valuable feedback on improving the quality of our paper.
Are there any additional questions or concerns we can answer? Thanks for your reply!

Sincerely,
Authors