Few-shot Text Adversarial Attack for Black-box Multi-task Learning

We thank the reviewer for the valuable comments on our paper. Below is a detailed response to the weaknesses and questions the reviewer mentioned.

Weaknesses 1.1
We include a discussion of these works in Section 2.1 of the paper. Furthermore, we provide an overview of related work on transfer attacks in Section A.4 of the appendix. Please review the updated version of the paper.

Weaknesses 1.2
In Section K of the Appendix, we investigate the impact of varying query numbers and the amount of available training data. Specifically, we set the query numbers and available training data to 10, 50, 100, 1000, and 2000, respectively. Notably, 100 queries correspond to the value used by CEMA in the original paper. Given that a query number of 10,000 is excessively large, we limit the maximum query number to 2000. The results of these experiments are presented in Table 12 of Section J. Our findings indicate that the attack effectiveness increases with the number of queries. Even with only 10 queries, CEMA achieves an attack success rate exceeding 30%.

Additionally, in Section 5.7, we discuss the attack scenario involving zero-shot training data. In this scenario, the attacker cannot access the training data but is aware of its relevant attributes. For example, both the SST5 and Emotion datasets are related to sentiment analysis, but their label spaces and distributions differ significantly. To test this, we used $100$ unlabeled texts from the Emotion validation set as auxiliary data for the SST5 attack, and vice versa.
The experimental results, presented in Table \ref{data-data}, show that even with limited auxiliary data and significant differences between the auxiliary data and victim texts, CEMA achieves an attack success rate of $66.40\%$ and a BLEU score of $0.27$.
The findings indicate that an attacker requires only partial knowledge of the training dataset and gathers the relevant data from the Internet. Utilizing CEMA, they can execute a substantial attack on the multi-task system.

Weaknesses 2
We implement a defense method in Section Q of the appendix by utilizing shuffle between two models during both querying and attack phases. CEMA employs three attack methods: DWB, FD, and Textbugger. However, due to the complexity of the code for FD and Textbugger attacks and the requirement to shuffle between two substitute models, it is challenging to complete the necessary code for these attacks within the rebuttal timeframe. As a result, we select DWB and TextFooler as the attack methods for CEMA and enable both methods to shuffle between two models during the querying and attack phases.

The experimental results, presented in Table 19 of Section Q in the appendix, show that while Random Shuffle reduces CEMA's attack effectiveness, CEMA still achieves reasonable performance under this defense.

Additionally, we present two other defense methods—Preceding Language Modifier and Adversarial Training—in Section P.

Weaknesses 3
We introduce three victim models in Sections I, J, and P.2 of the appendix. The victim models in Section I comprise six tasks, including four text classification tasks and two text translation tasks. Section J presents three tasks: text translation, summarization, and text-to-image generation. Section P.2 features two text classification tasks. The attack results of CEMA on these models are provided in Sections I, J, and P.2, demonstrating that CEMA achieves significant attack performance across all three victim models.

We appreciate your invaluable time and insightful comments. We have provided more details to your questions and concerns and added experiments. Can you kindly check them and let us know if they address your concerns? If you have further comments, we are happy to have a discussion.

Thank you very much!

We thank the reviewer for the valuable comments on our paper. Below is a detailed response to the weaknesses and questions the reviewer mentioned.

Weaknesses 1 and Questions 1

We train five substitute models on a server equipped with a 24GB NVIDIA 3090 GPU. Each model is trained over two epochs using a dataset containing 100 samples. The training time for a single model is approximately 4 minutes, and the size of each trained model is 418 MB. We added a section on the computation overhead of substitute model training in Section E of the Appendix.

Weaknesses 2 and Questions 2

We expand the number of tasks to six downstream tasks, comprising four classification tasks and two translation tasks. The corresponding experimental results are presented in Tables 10 and 11, located in Sections H and I of the appendix. We observe that CEMA achieves an ASR of over 60% on both the SST5 and Emotion datasets, with all BLEU scores below 0.3. These results suggest that the CEMA method can be effectively extended to multi-task learning systems with a broader range of tasks.

Furthermore, we expand the task variety to include three additional types: translation, summarization, and text-to-image generation. CEMA demonstrates consistently significant attack performance, highlighting the adaptability of our method to a broader range and larger number of tasks. Detailed results for the six downstream tasks are presented in Section I of the Appendix, while the outcomes for the additional tasks are included in Section J.

Weaknesses 3 and Questions 3

We initiate an extensive exploration of defensive strategies to counter CEMA. In practical systems, we thoroughly investigate various defense mechanisms, including train-free adjustments (Preceding Language Modifier) and adversarial training. The victim models used in our study are after-trained models sourced from the Huggingface website, Ali Translator, and Baidu Translator. Since the training details of these pre-trained models are not publicly available, re-training them using adversarial training is infeasible. Consequently, we adopt training-free defense methods. Specifically, we implement the same approach proposed by [1] and apply prompt learning techniques to large language models (LLMs) to mitigate adversarial text inputs. For this, we provide LLMs with the following prompt:

"Please revise the following text for grammatical errors, improve the spelling, grammar, clarity, concision, and overall readability."

The results are presented in Table 17. The attack effectiveness of CEMA decreases significantly after employing this defense method. However, the reduced attack performance remains noteworthy, even with an ASR still reaching 40%.

In addition, we train four classification models as victim models and conduct adversarial training to evaluate the impact of adversarial training on CEMA's attack effectiveness. All four models are based on the BERT architecture and are labeled Bert1, Bert2, Bert3, and Bert4. Specifically, Bert1 and Bert3 are trained on the SST5 dataset, while Bert2 and Bert4 are trained on the Emotion dataset. The results of adversarial training are provided in Table 18, Section O of the Appendix. Our findings indicate that although adversarial training reduces CEMA's attack effectiveness, it still demonstrates considerable performance.

Weaknesses 3: More Baselines

We compare our method with approaches that generate adversarial examples using substitute models, with the results presented in Table 16 of Section O in the appendix. CEMA outperforms adversarial examples generated by various substitute models. Specifically, CEMA achieves an attack success rate (ASR) that is 30% higher than the second-best ASR and a BLEU score that is 0.15 higher than the second-best BLEU score.

Weaknesses 4

For each dataset, 100 queries are performed per task, with the SST5 dataset containing 2,210 texts and the Emotion dataset containing 2,000 texts, resulting in an average of 0.045 and 0.05 queries per task, respectively.

To provide further clarification, we have computed the total number of queries for each attack method, as shown in Table 13 and Table 14 of Section M in the appendix. If you prefer to use the total number of queries, we can replace the data in these tables accordingly.

We thank the reviewer for the valuable comments on our paper. Below is a detailed response to the weaknesses and questions the reviewer mentioned.

Weaknesses 1

The substitute model functions as a white-box model, which the attacker can access indefinitely, enabling them to obtain any information necessary for the attack. This unrestricted access allows the attacker to gather the required details and generate adversarial examples capable of successfully deceiving the substitute model. Given that adversarial examples exhibit transferability, those that successfully attack the substitute model are then used to target the victim model.

In this context, transferability refers to the ability of adversarial examples generated for one model to produce similar attack effects on different models or systems. Specifically, it means that an adversarial example created for one model can cause an unseen target model to make incorrect predictions or classifications. Even if the target model differs in architecture, training data, or optimization methods from the source model, the adversarial examples can still "transfer" and have an effect on the target model.

The process of crafting adversarial examples using the substitute model is known as a transfer attack. We provide a detailed explanation of this process in Appendix L. Additionally, we summarize the concepts of transfer attacks and transferability in Section 2.1.

Weaknesses 2.1 Main Contribution

We broaden the scope of text multi-task adversarial attacks through the introduction of CEMA, which extends the application of such attacks in several key ways.

1. Query Count: CEMA achieves state-of-the-art (SOTA) attack performance with just 100 queries. Notably, even when the number of queries is reduced to 10, the attack success rate (ASR) remains above 30% (detailed experimental results are provided in Section K of the appendix). To the best of our knowledge, this is the first instance in the adversarial attack community where adversarial examples are successfully generated with such a low query count.

2. Black-box Setting: CEMA operates in a black-box setting, where the attacker only has access to the output texts of the multi-task model. In contrast, most previous methods for text multi-task adversarial attacks operate in white-box settings. Black-box settings, however, are more realistic and practical for real-world scenarios.

3. Multi-task Learning Structures: CEMA explores multi-task learning structures involving multiple distinct tasks. In our original paper, we evaluate CEMA’s performance on text classification and translation tasks. Additionally, Section J of the appendix presents further experiments on summarization and text-to-image generation tasks. Previous research on text multi-task adversarial attacks primarily focuses on attacking multi-task models that handle multiple text classification tasks, whereas our work expands the scope to a wider range of tasks.

4. Multi-model Multi-task Learning Systems: We introduce an attack scenario for Multi-model Multi-task Learning systems, a setting that has not been explored in previous adversarial attack research. Past studies primarily focus on parameter-sharing multi-task learning systems, leaving Multi-model Multi-task Learning systems largely unaddressed in the context of adversarial attacks.

Overall, our work proposes four more practical attack scenarios and aims to bridge the gap in existing research on text multi-task adversarial attacks, thereby broadening the scope and application of adversarial attacks in multi-task learning systems.

To address these challenges, we propose CEMA, a plug-and-play framework that simplifies the previously mentioned complex attack scenarios into straightforward text classification attacks. Moreover, CEMA adapts to any text classification adversarial method. We also present our attack scenarios as four key research questions in lines 55-60 of the paper’s introduction, with our contributions further summarized in lines 88-101 of the introduction.

Weaknesses 2.2: Compare CEMA with the Adversarial Examples Generated on Various Substitute Models

We compare our method with approaches that generate adversarial examples using substitute models, and the results are presented in Table 16 of Section O in the appendix. CEMA outperforms adversarial examples generated on various substitute models. Specifically, CEMA achieves an attack success rate (ASR) that is 30% higher than the second-best ASR, and its BLEU score is 0.15 higher than the second-best BLEU score.

Weaknesses 1

We define the attack constraint as the semantic similarity between the adversarial and original texts. This constraint is motivated by two main considerations. First, in text classification tasks, sentence-level adversarial attacks generate adversarial examples by modifying sentence structures and text styles, often resulting in substantial word changes. As a result, these methods typically use the semantic similarity between the adversarial and original texts as a constraint. Second, in text translation adversarial attacks, the semantic similarity between the adversarial and original texts is also commonly used as the constraint. The similarity of adversarial examples on the SST5 and Emotion datasets is presented in Section L of the appendix, with values of 0.934 and 0.931, respectively. Additionally, Section R of the appendix provides the definitions of adversarial examples for text classification and text translation. Additionally, we assume that the attack methods are effective, and therefore, in the paper, we assume that the success probability of each attack method is greater than 0.

Weaknesses 2.1

Since the exact success probability for each attack method is unavailable, we estimate the attack success probability using the frequency of successful attacks for each method. In the table, we report the frequency $P(AB)$ of both methods successfully attacking, as well as the frequencies $P(A)$ and $P(B)$, which correspond to the success rates of Method A and Method B, respectively. Our analysis shows that $P(AB)$ is approximately equal to $P(A) \times P(B)$, with the average value of $P(A) \times P(B) - P(AB)$ being only 0.17%. The detailed experimental results are presented in Table 20 (Section S) of the supplementary materials. We remove the sections on the union bound and the transferability of adversarial examples in the appendix.

Event independence is defined as the assumption that the occurrence of event A does not affect the occurrence of event B. We assume that the success of adversarial examples generated by Method A does not influence the success of adversarial examples generated by Method B, implying that A and B are independent.

However, this assumption of independence is based solely on experimental validation, which may lack sufficient rigor. Consequently, Sections 4.4 and T provide further discussion on the case of non-independence. We find that, even in non-independent scenarios, using more methods to generate adversarial examples increases the likelihood of successfully attacking the victim model.

Weaknesses 2.2 and Weaknesses 2.3 The maximum entropy distribution is primarily because, during the clustering process, we selected the clustering result that closely approximates a uniform distribution based on the maximum entropy principle. We have provided additional details on the clustering process in the original paper. Furthermore, we have simplified the proof in Section B of the appendix.

Weaknesses 3.1：Multi-model Multi-task Learning Multi-model Multi-task Learning (MMMTL) is a machine learning approach that combines Multi-task Learning (MTL) and Multi-model Learning. MTL involves training multiple related tasks using a shared model, allowing the model to optimize multiple objectives by sharing representations and parameters, as seen in tasks like sentiment analysis and text classification. In contrast, Multi-model Learning employs multiple models to solve a problem, leveraging different models' strengths, such as neural networks, decision trees, and support vector machines. MMMTL merges these two concepts, using multiple models to tackle several related tasks while sharing information between models and tasks, thus improving performance across tasks.