Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets

Thank you for your review. But it seems that you may have mistakenly pasted review comments intended for another paper here. So, we will wait for your correction.

Thank you for taking the time to carefully review our work and provide constructive feedback.

Claims&Wekaness1. The claim seems to broad.

A1. Thanks a lot for the valuable suggestion. We will narrow down the scope to the text and graph domains to ensure rigor.

We'd like to kindly clarify that we have mentioned rationalization is mainly used for NLP (the right part of L93–94). Although our experiments are conducted on text and graph data, the theoretical analysis in Sec4 is based on abstract variables and is not restricted to a specific scenario. Could you kindly clarify your concern in more detail regarding why you believe the analysis may not generalize to other scenarios? This would help us address your concern more effectively.

We believe such correlations may also arise in the image domain. For example, consider a cat-vs-dog classification dataset in which every original image contains a green pixel. If the generator selects a subset of pixels as the rationale, it may (during this selection process) consistently include the green pixel as part of the rationale for all images labeled 1, while excluding it from images labeled 0. This kind of sampling bias would introduce a spurious correlation.

However, our focus in this paper is solely on explaining why such situations can arise (i.e., identifying a potential risk or vulnerability). Assessing how likely this is to occur in real-world settings is somewhat beyond the scope of this paper. We appreciate your suggestion and will clarify and narrow down the scope in our revision.

Weakness2. Complexity and running time.

A2. Thanks for the valuable suggestion. Below is a comparison between the complexity of vanilla RNP and RNP+A2I (referred to as A2I for short). In general, A2I introduces an additional attacker module, and as a result, A2I has approximately 1.5 times the number of parameters of vanilla RNP. However, we would like to note that introducing extra modules is common practice in the field of rationalization. For example, our A2I model has a comparable number of parameters to Inter-RAT, a previously proposed variant of RNP.

Below are the practical time and memory consumption of RNP and A2I on the Beer-Appearance dataset (we add the results of Inter-RAT as a reference).

Per epoch, RNP consumes only about 60% of the training time compared to RNP+A2I. However, A2I converges significantly faster than vanilla RNP and typically requires fewer training epochs (about 50% on most datasets). Since memory usage is influenced by batch size, we further assign A2I a batch size of 256 (the same as Inter-RAT). Under this setting, the memory usage is 3664 MB, which is comparable to that of Inter-RAT.

Regarding the second question, "How does the proposed method scale to large datasets?", we are not entirely sure about the specific concern. Are you referring to datasets with more samples, or datasets with longer text inputs? If you mean the former, we believe the table in the Appendix addresses this concern: the largest dataset we use (Hotel-Cleanliness) contains 150k text samples. If you mean the latter, we follow one of our baseline Inter-RAT to test our A2I on the Movie dataset, which has much longer texts (the average length is about 775, and we set the max length to be 1024). The results are as follows (the results of other baselines are copied from Inter-RAT):

Our A2I still significantly outperforms RNP in terms of F1 score on the long-text dataset.

Weakness3. It does not thoroughly explore conditions under which the adversarial intervention might fail or succeed universally.

A3. We are sorry, but we are not very sure about your specific concern. Could you please specify it with more details?

We would like to clarify that our analysis already covers both sides of cooperative rationalization and how our attack succeeds in each case. Specifically, in the first part of Sec 4.3 (L241–316), we discuss how our attacker can correct the predictor and generator when they select the wrong rationales. Then, from L318 to the end of Sec4.3, we discuss how, when the predictor and generator select the correct rationales, our attacker does not introduce any negative impact. We think these two cases together cover both sides of the issue.

Typos. Thank you very much for the careful reading, it refers to Fig.3. We will fix it in our revision._

Thank you deeply for taking the time to thoroughly review our paper.

If we understand correctly, the only weakness you mentioned is about the lack of ablation study.

Q1. There is no ablation study on how each of the component of the proposed algorithm contribute to the success the effectiveness of the method seems to rely on the attacker’s ability to identify trivial patterns.

A1. We think this is a misunderstanding. Compared to the standard rationalization framework RNP, our A2I introduces only one extra component (the attacker). So, the results of RNP can directly serve as the ablation study of our RNP+A2I. Also, the results of FR can serve as the ablation study of our FR+A2I.

And we have designed a special experiment to verify the effectiveness of the attacker. The attack success rate of RNP in Figure 6 implies that our attacker can successfully identify the trivial patterns learned by the predictor. And the low attack success rate of our RNP+A2I implies that our attacker-based instruction can effectively deter the predictor from adopting trivial patterns.

We sincerely thank you for dedicating your time and expertise to review our paper. Your insightful comments and suggestions are highly valued and appreciated.

Q1. Experiment of Figure 3: I am not fully convinced that this experiment provides the evidence that the generator-predictor interaction creates spurious correlation. Here is why: Since the generator is not trained with the ground truth label information, for a given trained g, Z and Y are independent. The independence of g and Z on Y means that (in the notation of equation (5)), P[Y=1|Z=t+, g] = P[Y=1]. Therefore, the orange curve should not include the spurious correlation. This contradicts with the statement in the paper.

A1. We are sorry to cause such a misunderstanding. We agree that $g$ is not trained with $Y$, but we can not say $P[Y=1|Z=t+, g] = P[Y=1]$. Here is a simple intuitive toy example.

Suppose there is a box containing approximately 200 cat photos and 200 dog photos. The cat images are labeled as 1, and the dog images are labeled as 0. For both cats and dogs, half of the images have a red background and the other half have a green background. In this dataset, background color is considered a trivial pattern and is independent of the label $Y$.

Now, suppose a "hand" randomly samples 100 cat images and 100 dog images from the box to form a new dataset. Although the hand is a random hand and never uses the labels during sampling, biases may still occur in the specific sample. For example, the sampled cat images might include 50 with red backgrounds, while the sampled dog images might only include 30 with red backgrounds. As a result, under this particular sampling by the hand, background color and label $Y$ are no longer independent.

Returning to rationalization. We consider that in the original dataset, all text samples end with a period. However, one possible scenario is that in the random sample drawn by $g$, 51% of the positive samples include the period as part of $Z$, while only 49% of the negative samples include the period in $Z$. As a result, "whether $Z$ contains a period" is no longer independent of the label $Y$. This spurious correlation is caused by the sampling bias of $g$. This is because once the generator, originally a random variable, is instantiated into a specific value $g$, it inevitably contains bias.

In the example above, the spurious correlation caused by bias may not seem very severe, as we only considered a single trivial pattern. However, given that text should be regarded as a high-dimensional vector, chances are that the accumulation of biases from multiple different trivial patterns can become quite significant.

Q2. While they claim their algorithm’s performance is comparative to LLama3.1-8b-instruct, the LLM is relatively small, and it is better to compare it with larger models or other models. I cannot judge whether their algorithm is state of the art.

A2. We are sorry to say that our RTX3090 and RTX 4090 GPUs cannot afford LoRA fine-tuning for models larger than 8B.

Our methods is SOTA compared to previous rationalization methods. But we acknowledge that our method can not outperform more powerful LLMs like GPT4. However, the training of those LLMs usually involves extensive human alignment, which can be very expensive. Our model is small. Besides, we do not use human-annotated rationales for training. So, our model is much cheaper and can work in places with limited resources.

Despite the issues of SOTA results, our two major contributions still make sense. First, we find a new kind of spurious correlation, which may open up new avenues for future research. Second, we propose a direction to mitigate this issue with attack, which may inspire others to develop better methods.

Q3. In Equation (11), The meaning of [0.5,0.5] is unclear. Does this represent a random variable?

A3. Yes, it represents a random noise. Thanks for the reminder, and we will clarify this in the revision.

Q4. Suggestions about $g$ and Eq.8.

A4. Thank you for your suggestion. We will do it in our revision.