Rethinking Audio-Visual Adversarial Vulnerability from Temporal and Modality Perspectives

Q: [1] What is the meaning of (a)synchronous in context of Line 138?
A: In Figs. 1 and 2, we randomly mask frames at a ratio of $\rho$ to evaluate the corruption robustness of audio-visual models. "Synchronous" indicates that we first randomly select frames at the ratio $\rho$ and perturb both the video and audio data together on the selected frames. In contrast, "asynchronous" means perturbing the video and audio on separate segments of the video and audio streams.

[2] Is "contact" in line 411 and 415 is "concat"?
A: Yes. Thanks for your help! We have fixed these typos in our revision.

[3] Line 366: Is the masking done synchronously or asynchronously?
A: The masking is done synchronously. The model is more fragile when the perturbation is synchronous, where the information from different modalities can be complementary under the asynchronous perturbation as shown in Fig. 1 and Fig.2.

[4] Figure 3. What does it mean minimizing prediction $p_{av}$,$p_a * p_v$?
A: Thank you for your question; we understand it might cause some confusion.

• $p_{av}$ represents the model's prediction based on the fusion of audio-visual information. Minimizing $p_{av}$ corresponds to maximizing the classification loss.
• $p_a \cdot p_v$ measures the alignment between the predictions based solely on audio and visual inputs. Minimizing $p_a \cdot p_v$ aims to generate adversarial perturbations that disrupt the alignment between audio and visual modalities, thereby enhancing the attack's effectiveness.

To make this clearer, we have revised the figure in our paper.

[5] Based on Eq. (4), loss in Eq. (2) will be minimized. However, if we want to exploit the temporal consistency (line 52), why not maximizing this Eq. (2) loss? I think, maximizing variation of the extracted features between time step would make the temporal more inconsistent, which can make the attack more efficient.
A: The motivation behind minimizing Eq. (2) is to target more robust features along the temporal dimension, which helps improve adversarial transferability.

Maximizing the variation of extracted features between frames is an inspiring idea, as it could diversify the temporal features. However, this approach tends to distract the attack from focusing on robust features, making it easier to fool the surrogate (white-box) model but less effective in attacking victim (black-box) models.

To validate this, we conducted an experiment. Using the same experimental setup, we employed AcA (AlexNet-concat-AlexNet) as the surrogate model to attack RsR (ResNet-sum-ResNet). In a more challenging scenario, we generated adversarial examples using only 5 steps, where our method achieved a white-box attack success rate of 82.3% and a black-box attack success rate of 54.9% against RsR. When we instead maximized Eq. (2) to diversify temporal features, the white-box attack success rate improved to 86.7%, but the black-box attack success rate dropped to 50.5%.

This experiment demonstrates that attacking temporal consistent features is crucial for enhancing audio-visual adversarial transferability.

We hope our response addresses your concerns, and we kindly request your consideration in improving the score. Thank you for your valuable review and assistance in improving our revision!

Q: [1] In experiment comparison, some figures use TMA to represent the proposed method, while some of them use Ours. It is suggested to use the same.

A: We use "TMA" to represent our proposed attack method and "Ours" to denote our proposed defense method. To minimize confusion, we have clarified this distinction in the figure captions.

[2] For Table 2, does the ablation study use the proposed method TMA? If yes, it is suggested to add the information in the caption.
A: Yes. We have added this information to the caption.

[3] According to Table 2, the successful defense rate continuously increases with the addition of sampling ratio. Where is the change point? The experiment of the ablation study needs to show the sampling ratio value from where the result performance will drop down.

A: With increasing the sampling ratio, the successful defense rate will continuously increase until it converges. We propose this strategy based on the temporal redundancy of video and audio modalities, where there are many frames redundant in similar information. While generating the adversarial perturbation for all frames for adversarial training is time-consuming, we only generate the perturbation for a part of them and share the perturbation with neighbor frames to improve the efficiency of adversarial training as well as improve the model robustness. To validate this, we further increase the sampling ratio from 25% to 40%. While the computation overhead increases from 21.3 to 34.2 hours, there is only a 0.4 improvement in the adversarial robustness.

We hope our response addresses your concerns, and we kindly request your consideration in improving the score. Thank you for your valuable review and assistance in improving our revision!

W: (1) The experiments related to the proposed method are not sufficiently comprehensive. In Section 5.3, Adversarial Curriculum Training is introduced, which includes two strategies: the Data-level strategy and the Model-level strategy. The core of Adversarial Curriculum Training involves the iterative adjustment of the data masking ratio and the model dropout ratio. However, the paper does not present corresponding experimental results to substantiate the significance of this iterative design. Therefore, it is recommended to add experimental results to support the validity of this design.

A: Thanks for your suggestion! We add this ablation study in Appendix B to verify the significance of our proposed training strategy. Specifically, we evaluate the impacts of different schedulers on audio-visual robustness, including constant and synchronous ratios, constant yet asynchronous asynchronized ratios, a linear scheduler, and a cosine scheduler. Evaluation results demonstrate that by utilizing data-level and model-level strategies and dynamically adjusting the ratios, we effectively boost the robustness of audio-visual models. The reviewer may refer to the appendix B for more details.

(2) The paper evaluates unimodal adversarial attack methods (such as FGSM, I-FGSM, and MI-FGSM in the image domain). However, studies such as [1,2] have proposed multimodal attacks. It remains unclear whether the proposed defense method is effective against multimodal adversarial attacks similar to the Audio-Visual attacks discussed in [1]. I suggest adding experiments to evaluate the method's performance under multimodal attack scenarios.

A: Thank you for your insightful point! There are certain similarities between audio-visual attack/defense and vision-language attack/defense methods, as both require consideration of alignment and consistency between the two modalities. However, there are also notable differences between them. (1) Task difference: vision-language attacks aim at attacking content retrieval-related problems while audio-visual attacks focus on classification problems. (2) Operation difference: vision-language attacks perturb input data by optimizing latent embeddings while our method perturbs input by adjusting output logits. (3) Modality difference: vision-language attacks focus on static images while our approach considers the temporal redundancy of dynamic videos. This redundancy motivates our design of curriculum training to exploit sparsity, enhancing adversarial robustness while improving training efficiency.

Considering these reasons, we do not apply [1, 2] to the audio-visual domain. However, we believe these works are inspiring and we think exploring audio-visual attacks from the perspectives of model pretraining, vision-language alignment, and content retrieval is a valuable future direction. We have included this discussion in Appendix C. Thank you for the suggestion!

[1] Zhang et al. Towards adversarial attack on vision-language pre-training models[C]/ACM MM 2022.
[2] Lu et al. Set-level Guidance Attack: Boosting Adversarial Transferability of Vision-Language Pre-training Models[J]. arXiv preprint arXiv:2307.14061, 2023.

Q: Please refer to the questions raised in the Weakness section. Additionally, the reviewer is interested in the following questions: (1) What's the selection criterion when choosing datasets for the experiment? (2) Could the proposed method be incorporated into the pre-trained multi-modal models, such as CLIP or BLIP?

A: (1) We choose datasets based on their generality and popularity. Both Kinetics-Sounds and MIT-Music datasets are widely used in the audio-visual area [1, 2, 3]. To further verify the scalability of our proposed method, we additionally conduct experiments on the MIT-Music dataset, with results provided in Appendix A.
[1] Tian et al. "Can audio-visual integration strengthen robustness under multimodal attacks?." CVPR 2021.
[2] Li et al. "On adversarial robustness of large-scale audio visual learning." ICASSP 2022.
[3] Yang et al. "Quantifying and enhancing multi-modal robustness with modality preference." ICLR 2024.

(2) Both the proposed attack and defense methods primarily leverage temporal sparsity and redundancy to enhance performance, which is more pronounced in audio-visual data than in vision-language data. Exploring the presence of redundancy in vision-language modalities and developing adversarial training strategies for pre-trained multi-modal models with limited data is an inspiring research direction. While our current work focuses on audio-visual data, we have included a discussion of this perspective in the appendix, with the aim of motivating further research in this area.

We hope our response addresses your concerns, and we kindly request your consideration in improving the score. Thank you for your valuable review and assistance in improving our revision!

W: The proposed defending method seems not designed exactly towards the proposed attacks. In particular, the modality-misalignment-based attack is not specifically addressed.

A: During the adversarial training process, we utilize our proposed Temporal and Modality-based Attack (TMA), which includes modality-misalignment-based attack, to craft adversarial examples and train the model on these generated examples. As a result, the trained model can effectively defend against modality-misalignment-based attacks.

The primary challenge in audio-visual adversarial training lies in efficiency. Therefore, our focus has been on proposing a new strategy to enhance efficiency by leveraging temporal redundancy. We understand this may have caused some confusion, and we have revised the paper to clarify this point.

Q: Maybe it is better to show the effectiveness of the method by some real audio-visual samples, i.e., applying the proposed attacks can indeed attack existing methods while the proposed defending method can handle/alleviate it

A: Thank you for your suggestion. To demonstrate the scalability of our method in real-world applications, we utilized our proposed attack method, TMA, to generate 100 audio-visual adversarial examples to deceive the multi-modal large language model, VideoLLaMA2 [1], achieving an attack success rate of 74%. Detailed experimental results are provided in Appendix D.

In our work, we designed an efficient adversarial training method that leverages audio-visual redundancy to enhance adversarial robustness while improving training efficiency. However, due to limited computational resources and restricted access to the pre-training datasets of current MLLMs, it is challenging to apply adversarial training directly to foundation models like VideoLLaMA2. We hope this research inspires future studies to explore more efficient approaches for improving adversarial robustness from both temporal and modality perspectives.

We hope our response addresses your concerns, and we kindly request your consideration in improving the score. Thank you for your valuable review and assistance in improving our revision!

[1] Cheng, Zesen, et al. "VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs." arXiv preprint arXiv:2406.07476 (2024).