Attacking for Inspection and Instruction: Debiasing Self-explaining Text Classification

Q2. If the authors manually label some data from broader topics and more diverse targets and conduct experiments on them, there would be evidence that the selecting bias is common and inherent exists in the RNP framework, and the proposed method could well alleviate it.

A2 Thank you for your valuable suggestion. We have now added an experiment conducted with GNNs. We use a very widely used graph classification dataset in the field of explainable GNNs: BA2Motifs. There are labels for the gold rationales: the house motif for class 0 and the cycle motif for class 1. We report the overlap (F1 score) of the selected nodes and gold rationales. The base model for each player is a 2-layer GCN. We report the overlap (F1 score) of the selected nodes and gold rationales. The results are as follows:

Notes: The numbers in "()" are the standard deviations.

Q3. By providing more high-quality rationales, the predictions should be more accurate.

A3. In general, high-quality rationales do improve classification accuracy. However, the prediction accuracy is also influenced by various other factors, leading to some minor fluctuations.

Also, due to the sampling bias we analyzed in this study, poor rationales do not necessarily lead to poor prediction (empirically supported by the experiments in Fig.5). Some experimental results in recent papers also show that better rationale quality does not lead to better prediction. In the paper of CR [A], the best rationale quality is achieved by CR, but the best prediction performance is achieved by FR. In the paper of A2R [B], A2R achieves the best rationale quality, but not the best classification performance.

We also find an interesting paper [C] that argues that spurious features can sometimes improve the accuracy.

References
[A] Towards trustworthy explanation: On causal rationalization. ICML 2023.
[B] Understanding Interlocking Dynamics of Cooperative Rationalization. NeurIPS 2021.
[C] Spuriosity Didn't Kill the Classifier: Using Invariant Predictions to Harness Spurious Features. arXiv:2307.

Thank you very much for your valuable comments and suggestions!

Q1. The potential of this work is significantly suffered from the fact that the RNP framework cannot be practically aligned with large pre-trained models.

A1. We agree with you. We have put this issue in the section of limitations, and we leave formally investigating the obstacles in utilizing powerful pre-trained language models under the rationalization framework as the future work.

We have now re-run RNP and FR with the Bert encoder on the three aspects of BeerAdvocate. Although fine-tuning Bert is still very difficult, we find that using a smaller learning rate for the Bert encoder than for the linear layer works somewhat. So we use a learning rate of $1e-5$ for the Bert encoder and $1e-4$ for the linear layer. The maximum sequence length is 256 (this is enough for the Beer dataset). The batch size is 24.

From the second table, we see that Bert_RNP performs much worse than GRU_RNP, so there must be some unknown issues prevent RNP to work well with Bert. The sampling bias problem studied in our paper may not be the only obstacle that prevents Bert_RNP to work well, so in most cases, A2I does not improve Bert_RNP a lot.

But from the first table, we see that Bert_FR improves Bert_RNP a lot. So the obstacles introduced by the Bert encoder may have somewhat been overcomed by FR. Under this case, our A2I improves Bert_FR a lot (more than $5\%$ in 6 of 9 settings). We believe that future researchers will further explore harnessing the power of Bert within the RNP framework. At that point, we can also build upon subsequent methods to incorporate our A2I.

Notes: We highlight the results only when FR+A2I gives an improvement over $5\%$ compared to FR

We are really grateful for your constructive feedback and guidance!

Q1. One of the concerns regarding this work is its contribution; the proposed method seems to be merely an add-on to a specific type of problem.

A1. We are sorry for the confusion, but we think this is a misunderstanding. Aside from the proposed method, the identification of the specific sampling bias problem is also one of our contributions. We are the first one to identify this problem, and this finding is supported by theoretical analysis (Section 4.2) and empirical evidence (Section 5, Fig.5, Fig.6, Fig.7-9).

Although the proposed method is an add-on to RNP and its variants, but on the other hand, our method is flexible enough to benefit a number of approaches in this field.

Q2. The conclusions drawn from binary classification may not easily generalize to a multi-class setting. In binary classification, selecting the reversed label represents a clear worst-case scenario. However, it is not clear how this approach could extend to multi-class cases. In the appendix, the authors provide formulations for multi-class scenarios in equations (17) and (18). I encourage the authors to deliberate on the specific method for "choosing the $Y'\neq Y$ —should this $Y$ be sampled from a uniform distribution, for instance?

A2. Thank you for your valuable suggestion. We would like to first clarify a misunderstanding, the $Y$ in Equation 17 is the true label of $X$. $Y'\neq Y$ means the prediction does not match the true label. The $\min_{Y'}$ operator means we want to find a $Y'$ that makes the attack easiest.

Here is the detailed clarification on Equation 17:
The general idea is the same as for binary classification: Consider an n-class classification task. We denote $X^j$ as the text of the $j$-th class. For the $i$-th class, the goal of the attacker is to find $Z_A$ from those $X^j$ ($j$ can be any value not equal to $i$, i.e., $X^j$ doesn't belong to the $i$-th class) and get the predictor to predict $Z_A$ as the $i$-th class. Equation 17 is a practical implementation of this idea: for an aribitrary data point $X^Y$, the attacker finds the attack rationale $Z_A=f_a(X^Y)$, and the predictor's output is $f_p(Z_A)$. The significance of the $\min_{\theta_a}$ operator in Equation 17 is such that $f_p(Z_A)$ is categorized as different from $Y$. And the $\min_{Y'}$ operator is to see which class $f_p(Z_A)$ is closest to, making the attack easier. If $f_p(Z_A)$ is successfully classified as class $Y'$, then we think the predictor uses some trivial patterns that do not belong exclusively to the $Y'$-th class to make predictions about $Y'$.
You can understand it in a simpler way: $Y,Y'$ in Equation 17 correspond to the above $j$ adn $i$, respectively. There is a simple way to bridge binary and multi-class classification: In binary classification, $[i,j]=[1,0]$ or $[0,1]$. In n-class calssification, $i,j$ can be any values as long as $i\neq j, 1\leq i,j\leq n$.
And Equation 18 (the instruction term) is no different from the second term of Equation 5.

Also, our analysis about the problem is not limited to binary classification tasks. The most important analysis about the root of the problem is Equation 7. Here we consider an n-class classification task.
Equation 7 can be extended as follows:
The setup: $T\in \{t_1,t_2,...,t_m\}$ ($m$ can be an arbitrary integer). $\forall i \in [1,m], \ P(Y)=P(Y|t_i)$, which means $T$ is a trivial pattern independent with $Y$.
The first row of Equation 7 becomes: $\exist 1\leq j\leq n,\ 1\leq i\leq m,\ P(Y=j|Z=t_i,g)>P(Y=j),$ which means that $t_i$ becomes an indicator for the $j$-th class under the condition of the generator.

Q3. Moreover, the authors' method assumes the availability of a balanced dataset. How would the algorithm be modified in the presence of imbalanced labels?

A3. I guess your question stems from our assumption $P(Y=1)=P(Y=0)=0.5$ in Equation 7. This assumption is made for the sake of theoretical analysis simplicity. For example, if $P(Y=1)>P(Y=0)$ in the original dataset, then a predictor does not need any input to get a accuracy higher than $50\%$. For unbalanced datasets, we can employ existing methods designed to handle unbalanced labels (e.g., random under-sampling). In fact, the BeerAdvocate dataset we used is a very unbalanced dataset, and the method we used to handle it is random under-sampling. Here are the statistics for the training data of BeerAdvocate:

We really appreciate your valuable comments and suggestions!

Q1. No code to verify the results

A1. We have now updated the code and the instructions at the general response. They are only visible to reviewers.

Q2. While the paper discusses the theoretical aspects of sampling bias and introduces a solution for self-explanation, it falls short in discussing the real-world implications of this bias in AI applications. Providing concrete examples or case studies of how sampling bias can impact decision-making systems would enhance the paper's practical relevance.

A2. Thank you for your suggestion. We have now added two case studies in Fig.10 of Appendix A.7.

Q3. The datasets BeerAdvocate and HotelReview seem small and basic. I am curious to see how this method performs on larger scale datasets like CIFAR10.

A3. While BeerAdvocate is relatively small (about 30,000 texts for each aspect), the HotelReview is much larger. Hotel-Service has more than 100,000 texts and Hotel-Cleanliness has more than 150,000 texts. We'd like to conduct experiments on CIFAR10, but the problem is that it does not have human-annotated rationales for evaluation. As we have mentioned in Section 5.2.1, the sampling bias problem makes the prediction performance not a good metric.

As an alternative, we now add experiments on a graph neural network dataset to validate the generalization capabilities of the method. We use a wiedly used dataset in the field of explainable GNNs: BA2Motifs. This is a graph classification dataset and has labeled gold rationales for evaluation. The base model for each player is a 2-layer GCN. We report the overlap (F1 score) of the selected nodes and gold rationales. The results are as follows:

Notes: The numbers in "()" are the standard deviations.

Q4. No reporting of the mean and standard deviation of multiple experiments to see if the results are significant

A4. We now report the standard deviation. In our original experiments, we use a fixed random seed of 12252018 (inherited from the code provided by our baseline FR), because we think that experiments with 12 different settings (beer: 3 aspects $*$ 3 sparsity, hotel: 3 aspects) under the same random seed are somewhat sufficient to verify the stability of the models. Here we rerun RNP and RNP+A2I with 4 additional seeds and report the standard deviation over the five random seeds. Notes: The format of the numbers is "avg(std)".

Due to the large amount of GPU resources consumed by Bert related experiments, the random seed experiments for FR are still in preparation.

Q4. The use of GRUs and GloVe embeddings, while understandable for comparison purposes, may not reflect the current state-of-the-art in NLP, where transformer-based models like BERT are prevalent. It would be interesting to see how the proposed method performs with such models.

A4. Thanks for your suggestion. We have now re-run RNP and FR with the Bert encoder on the three aspects of BeerAdvocate. Although fine-tuning Bert is still very difficult, we find that using a smaller learning rate for the Bert encoder than for the linear layer works somewhat. So we use a learning rate of $1e-5$ for the Bert encoder and $1e-4$ for the linear layer. The maximum sequence length is 256 (this is enough for the Beer dataset). The batch size is 24.

From the second table, we see that Bert_RNP performs much worse than GRU_RNP, so there must be some unknown issues prevent RNP to work well with Bert. The sampling bias problem studied in our paper may not be the only obstacle that prevents Bert_RNP to work well, so in most cases, A2I does not improve Bert_RNP a lot.

But from the first table, we see that Bert_FR improves Bert_RNP a lot. So the obstacles introduced by the Bert encoder may have somewhat been overcomed by FR. Under this case, our A2I improves Bert_FR a lot (more than $5\%$ in 6 of 9 settings). We believe that future researchers will further explore harnessing the power of Bert within the RNP framework. At that point, we can also build upon subsequent methods to incorporate our A2I.

Notes: We highlight the results only when FR+A2I gives an improvement over $5\%$ compared to FR.

We greatly appreciate your detailed review and constructive feedback on our paper!

Q1.* The paper primarily focuses on binary classification tasks, and it is not clear how well the proposed method would generalize to multi-class classification or other types of machine learning tasks.

A1. Although we use binary classification datasets in our experiments (since there are no proper multi-class classification datasets that have human-annotated rationales), we have also provided formulations for multi-class scenarios in Appendix A.2. The general idea is the same as for binary classification: Consider an n-class classification task. We denote $X^j$ as the text of the $j$-th class. For the $i$-th class, the goal of the attacker is to find $Z_A$ from those $X^j$ ($j$ can be any value not equal to $i$, i.e., $X^j$ doesn't belong to the $i$-th class) and get the predictor to predict $Z_A$ as the $i$-th class. Equation 17 is a practical implementation of this idea: for an aribitrary data point $X^Y$, the attacker finds the attack rationale $Z_A=f_a(X^Y)$, and the predictor's output is $f_p(Z_A)$. The significance of the $\min_{\theta_a}$ operator in Equation 17 is such that $f_p(Z_A)$ is categorized as different from $Y$. And the $\min_{Y'}$ operator is to see which class $f_p(Z_A)$ is closest to, making the attack easier. If $f_p(Z_A)$ is successfully classified as class $Y'$, then we think the predictor uses some trivial patterns that do not belong exclusively to the $Y'$-th class to make predictions about $Y'$.
You can understand it in a simpler way: $Y,Y'$ in Equation 17 correspond to the above $j$ adn $i$, respectively. There is a simple way to bridge binary and multi-class classification: In binary classification, $[i,j]=[1,0]$ or $[0,1]$. In n-class calssification, $i,j$ can be any values as long as $i\neq j, 1\leq i,j\leq n$.
And Equation 18 (the instruction term) is no different from the second term of Equation 5.

Also, our analysis about the problem is not limited to binary classification tasks. The most important analysis about the root of the problem is Equation 7. Here we consider an n-class classification task.
Equation 7 can be extended as follows:
The setup: $T\in \{t_1,t_2,...,t_m\}$ ($m$ can be an arbitrary integer). $\forall i \in [1,m], \ P(Y)=P(Y|t_i)$, which means $T$ is a trivial pattern independent with $Y$.
The first row of Equation 7 becomes: $\exist 1\leq j\leq n,\ 1\leq i\leq m,\ P(Y=j|Z=t_i,g)>P(Y=j),$ which means that $t_i$ becomes an indicator for the $j$-th class under the condition of the generator.

Experiments on GNNs: We have now generalized our method to other machine learning tasks. In particular, we extend it to explainable graph neural networks. We conduct experiments on BA2Motifs, which is a widely used dataset in the field of explainable GNNs. It's a graph classification dataset. There are labels for the gold rationales: the house motif for class 0 and the cycle motif for class 1. The base model for each player is a 2-layer GCN. We report the overlap (F1 score) of the selected nodes and gold rationales.

Notes: The numbers in "()" are the standard deviations.

Q2. While the authors mention that the proposed method is model-agnostic, the experiments are limited to the RNP framework and its variants. It would be beneficial to see the method applied to other types of models to assess its generalizability.

A2. We are grateful for the suggestion, but we think there is a misunderstanding. RNP is not a niche field, as mentioned in the related work section, with new research in this domain emerging every year. This is attributed to the fact that the cooperative game proposed by RNP is a mainstream approach in NLP for constructing self-explanatory models. When we refer to "model-agnostic," we are referring specifically to RNP and its variants. Now, we have added additional experiments to show that the method can be generalized to GNNs, which is shown in A1 above.

Q3. The paper could benefit from a more detailed discussion on the limitations of the proposed method, including potential scenarios where the adversarial game might fail to detect certain types of biases or where the instruction phase might not effectively debias the predictor.

A3. Thanks a lot for your suggestions. Since we are focusing on the bias introduced by the generator's selection, one type of the bias that we cannot deal with is the bias already exist in the dataset. We will add it to the present limitations.

Q2. The proposed method makes sense, but there are several much simpler solutions that the authors should try and compare with. First, it seems to me that the root cause of the problem is that the generator is overpowered -- it's able to internally detect the label, and then feed special tokens correlated with the label to the predictor which do not have semantic meaning. As such, some simple solutions would be to reduce the capacity of the generator; add regularization to the generator; or to train the generator and predictor in an alternating fashion, with more steps for the predictor. The final suggestion is similar to how GANs are trained. In addition, the problem examined is very similar to mode collapse in GANs, and some of the solutions there (e.g. a diversity regularizer [1]) could work as well.

A2. Thanks you for your insightful suggestions. Our initial focus was on training a good predictor to guide the generator in producing appropriate rationales. This is because we believe that the generator must first identify the sentiment and then select the rationales that represent the sentiment. So, our focus is on avoiding the second part of your suggestion (feed special tokens correlated with the label to the predictor which do not have semantic meaning) . Also, experiments in the reviewer-author discussion of FR (one of our baselines) showed that adding more layers to the encoder of the RNP's generator does not necessarily decrease its performance (more powerful generator does not lead to worse results, so reducing the capacity may not be the first choice for us). Finally, we want to point out that the root cause is not simply that the generator is overpowered. We think the root is $Y\perp Z\nRightarrow Y\perp Z|g$, meaning that any random bias in the generator's selection can make a trivial pattern (independent with $Y$) be correlated with $Y$. Note that the punctuation example is only an extreme toy example. Experiments in Fig.5(a) show empirical evidence for this claim. The orange line corresponds to a randomly initialized generator (not trained to recognize any semantics), but the predictor can still make correct predictions with the random selected rationales.
For the above reasons, reducing the capacity of the generator is not the primary focus of our efforts. Nevertheless, we have also ever considered some of the above methods.
A2.1 (about reducing the capacity of the generator).
We do not deny that some methods can work. But it is non-trivial to design a good method that works well. Since we need to maintain the generotor's power for identifing the sentiment rationales, we need to be very careful when reducing the capacity of the generator to get a trade-off. Practically, we have ever considered to add spectral normalization (SN) to the generator's linear layer to restrict its Lipschitz continuity and produce a more flatter model. But we found that this trick actually hurts the model. When the SN is added, the model cannot be trained at all. A2.2 (about adding regularization to the generator).
In fact, a previous method 3PLAYER [A] has tried to do this. It takes the unselected part into consideration and the regularization term for the generator is to make the unselected part to be unable be classified correctly. But it has been empirically verified by several later papers [B,C] that it does not improve the rationale quality significantly as compared to the vanilla RNP. A2.3 (about training more steps for the predictor).
In the paper of FR, it has been empirically shown (Fig.1 in FR) that training the predictor with a higher learning rate than the generator (which is somewhat similar to training the predictor for more steps) unexpectedly results in a decrease in the quality of rationales. Instead, training the generator with a higher learning rate improves the rationale quality. A2.4 (mode collapse).
Although we call the generator a generator, but actually in fact a selector which outputs binary masks (e.g., "0,1,1,0,0,1") instead of real text. How to measure the diversity is not easy. Also, the problem is not completely the same as mode collapse. A well-known method to address mode collapse in GANs is using a smaller learning rate for the generator [D]. But according to A2.3, this method does not work for the cooperative game of rationalization.

We are grateful for your valuable suggestions, and we do not claim that this method is the sole or optimal solution to this problem. However, exploring more advanced methods is somewhat beyond the scope of this paper. We would like to respectfully clarify that our contributions in this paper are not limited to the proposed method. Another important contribution is the identification of this special sampling bias problem (Q1 and A1 above).

We sincerely appreciate your efforts to help improve the quality of this study. Here are some more detailed clarifications that may address your concerns.

Q1. The authors do not sufficiently show that this degeneration is an issue empirically in my opinion. To start, the authors should show a few real examples where vanilla RNP gives a nonsense justification while the predictor still outputs the correct label; and show that RNP + A2I fixes these cases. In addition, the authors could consider plotting a histogram of the length of the rationale (for RNP and RNP+A2I), and showing that samples with short justifications correspond to degenerate cases (e.g. the punctuation example). Overall, the sparsity of the A2I augmented models (in Table 1) do not seem significantly different from the sparsity of the base models, and so I am not convinced that A2I solves the issue presented.

A1.1. (The authors do not sufficiently show that this degeneration is an issue empirically in my opinion.)
Sorry for the confusion, we indeed did not show it in the introduction. But we have shown it with practical experiments in Section 5.

We first show the risk of this bias with experiments designed from three perspectives in Section 5.1 (i.e., Fig.5, Fig.8, and Fig.9). We present three types of prediction accuracies for the BeerAdvocate dataset: (1) A predictor trained with the full input text. (2) A predictor trained with randomly selected patterns. For the generator, we remove the other objectives and only train it with the sparsity constraints. In other words, the generator is trained to randomly select 10% of the input text, and the predictor is then trained to classify using these randomly selected texts. (3) We use the randomly selected texts from (2) to feed the predictor trained in (1).
The result for the Aroma aspect is shown in Figure 5. From Figure 5(a), we observe that even with the randomly selected patterns (i.e., patterns unlikely to contain real rationales), the predictor can still achieve a very high prediction accuracy (represented by the orange line, approximately 95%). This accuracy is close to that of the classifier trained with the full texts. A followed question is: Does this result suggest that the 10% randomly selected patterns already contain enough sentiment inclination for classification? The answer is no. Consider the green line, which represents the outcome when we feed the randomly selected texts to the predictor denoted by the blue line. We observe that the green line indicates a significantly lower accuracy (about 58%), implying that the randomly selected patterns contain only minimal sentiment information. Thus, the orange predictor incorrectly treats certain randomly selected trivial patterns as indicative features.

Then, we show the existence of the bias in the normally trained RNP framework with Attack Success Rate (ASR) in Section 5.2 (i.e., Fig.6 and Fig.7). In Fig.6(a), the attack success rate is very high (about $85\%$), which indicates that the predictor indeed utilizes some category-agnostic features for classification, and the attacker has successfully identified these features.

A1.2 (the authors should show a few real examples where vanilla RNP gives a nonsense justification while the predictor still outputs the correct label; and show that RNP + A2I fixes these cases.)
Thank you for your suggestions. We have now added two examples of the sampling bias in Figure 10 of Appendix A.7. The sparsity is about $10\%$ (note that $10\%$ is the average sparsity across the dataset, and it is manually determined by $s$ in Equation 4 rather than the model's power. So it is possible that some texts may have low sparsity and others have high sparsity.).

A1.3 (In addition, the authors could consider plotting a histogram of the length of the rationale (for RNP and RNP+A2I), and showing that samples with short justifications correspond to degenerate cases (e.g. the punctuation example). Overall, the sparsity of the A2I augmented models (in Table 1) do not seem significantly different from the sparsity of the base models, and so I am not convinced that A2I solves the issue presented.).
We are sorry, but we think this is a misunderstanding. The sparsity in Table 1 is the average sparsity over the entir dataset, and it is manually determined by $s$ in Equation 4 rather than the model's power. The punctuation example is just an imaginary toy example to better understand this problem. You can also refer to the example in our Fig.2, where the sentence "I went to a hotel yesterday" is a trivial pattern and plays the same role as "." in the punctuation example. In fact, short justifications do not correspond to degenerate cases and vice versa.

Q7. The authors do not show any confidence intervals for their results, so it is unclear whether performance gains are statistically significant. They also only evaluate on two datasets, though these seem to be standard datasets in the RNP community.

A7.1. (standard deviation) We now report the standard deviation. In our original experiments, we use a fixed random seed of 12252018 (inherited from the code provided by our baseline FR), because we think that experiments with 12 different settings (beer: 3 aspects $*$ 3 sparsity, hotel: 3 aspects) under the same random seed are somewhat sufficient to verify the stability of the models. Here we rerun RNP and RNP+A2I with 4 additional seeds and report the standard deviation over the five random seeds. Notes: The format of the numbers is "avg(std)".

Due to the large amount of GPU resources consumed by Bert related experiments, the random seed experiments for FR are still in preparation.

A7.2. (datasets) We fully understand your concern. However, datasets containing manually annotated rationales are precious, and it is common to utilize only two datasets in the field of rationalization. Our baseline method FR, and two other relevant papers [C,E] all use two datasets. Although we use only two datasets, each of these datasets contains three independently annotated aspects and each aspect is trained independently. Thus, to some extent, they can be considered as six datasets to some extent.

References
[A] Rethinking Cooperative Rationalization: Introspective Extraction and Complement Control. EMNLP 2019.
[B] Invariant Rationalization. ICML 2020.
[C] Understanding Interlocking Dynamics of Cooperative Rationalization. NeurIPS 2021.
[D] GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium. NeurIPS 2017.
[E] D-Separation for Causal Self-Explanation. NeurIPS 2023.
[F] A Survey on Bias and Fairness in Machine Learning. ACM computing surveys 2021.

Q6. Concerns about applying A2I to BERT and GNNs A6. Thanks for your suggestion. We have now re-run RNP and FR with the Bert encoder on the three aspects of BeerAdvocate. Although fine-tuning Bert is still very difficult, we find that using a smaller learning rate for the Bert encoder than for the linear layer works somewhat. So we use a learning rate of $1e-5$ for the Bert encoder and $1e-4$ for the linear layer. The maximum sequence length is 256 (this is enough for the Beer dataset). The batch size is 24.

From the second table, we see that Bert_RNP performs much worse than GRU_RNP, so there must be some unknown issues prevent RNP to work well with Bert. The sampling bias problem studied in our paper may not be the only obstacle that prevents Bert_RNP to work well, so in most cases, A2I does not improve Bert_RNP a lot.

But from the first table, we see that Bert_FR improves Bert_RNP a lot. So the obstacles introduced by the Bert encoder may have somewhat been overcomed by FR. Under this case, our A2I improves Bert_FR a lot (more than $5\%$ in 6 of 9 settings). We believe that future researchers will further explore harnessing the power of Bert within the RNP framework. At that point, we can also build upon subsequent methods to incorporate our A2I.

Notes: Due to limited GPU resources and time, we run the experiments with a fixed random seed 12252018 (inherited from the code provided by our baseline FR). Although not very solid, having the same seed for the nine different settings (3 different aspects $*$ 3 different sparsities) may somewhat reflect the stability of the effectiveness of A2I. We highlight the results only when FR+A2I gives an improvement over $5\%$ compared to FR.

Following your valuable suggestion, we have now also further applied RNP and our AI2 to explainable GNNs. We use a very widely used graph classification dataset in the field of explainable GNNs: BA2Motifs. There are labels for the gold rationales: the house motif for class 0 and the cycle motif for class 1. We report the overlap (F1 score) of the selected nodes and gold rationales. The base model is a 2-layer GCN. The results are as follows:

Notes: The numbers in "()" are the standard deviations.

Q3. There are a few edge cases which I am not convinced that A2I will be able to fix. Primarily, these deal with the circumstance where, in the toy example, the do not appear in the negative samples, and vice versa -- so is a token that appears almost exclusively in positive examples, and is a token that appears almost exclusively in negative examples. Such spurious correlations have been found in natural language tasks [3-4]. In these cases, it seems like the attacker would not be able to choose the corresponding token, and would thus still output random noise.

A3. If we understand correctly, you are discussing the spurious correlations that already exist in the original dataset. It involves another line of research that is orthogonal to this paper. Existing research on causality has primarily focused on spurious correlations inherent in the dataset. However, our research investigates a novel question: if the dataset itself is clean and lacks spurious correlations, could the selection process of the generator introduce additional spurious correlations? This poses a unique research question in our study.

Our research problem in this study is the spurious correlations introduced by the generator's selection (as implied in Section 4.2, trivial patterns in this paper refer to those independent with $Y$). We acknowledge that we cannot deal with those spurious correlations that exist in the original dataset. We see a recent study MCD [E] in the field of rationalization has already noticed this problem and addressed it well. As we have implied in our submission, our A2I is model-agnostic, we will consider building our A2I on top of MCD in the future, just like FR+A2I in our paper.

Q4. Another concern deals with the singular sentiment assumption. This seems like a strong assumption that is very dataset and task specific, and the authors already discuss its failure modes in the appendices. The presence of negation seems to be another case where the assumption would be violated. As such, I am not convinced in the generalizability of the method to other datasets and tasks. Regardless, the authors should formulate this assumption mathematically in the text.

A4. $S\in \{s_+,s_-\}$ constitutes indicative features for the category (e.g., sentiment orientation), i.e., $Y$ is primarily determined by $S$ within $X$. Additionally, there may exist descriptions $K\in \{k_+,k_-\}$ in $X$ related to sentiment but not decisive for the label (e.g., softened tones).
Formally, the singular sentiment assumption is $P(\phi(x,s_-)|Y=1)=0, P(\phi(x,s_+)|Y=0)=0$.
The reason we make this assumption is that we think that $s_+$ and $s_-$ will not appear in one $x$ simultaneously, otherwise we would not be able to determine to which category this $x$ belongs. And if $s_+$ and $k_-$ appear in one $x$, that $x$ still belongs to $Y=1$. So, $K$ is still a trivial pattern, and this is what we discussed in the appendix.

If you still think this assumption is too strong, we can make it a relaxed one: $P(\phi(x,s_-)|Y=1)=\epsilon=P(\phi(x,s_+)|Y=0), \ s.t., \epsilon \ll P(\phi(x,s_-))$, which means that it is very hard for the attacker to find $s_-$ from $X$ labeled $1$. Based on it, our analysis in the last paragraph of Section 4.2 still makes sense.

Q5. Overall, the clarity of the paper could be improved. Some of the formulation sections are hard to parse. For example, the authors formulate the problem as one of sampling bias, which makes sense intuitively. However, the mathematical formulation and causal graphs for this section don't follow the prior work in sampling bias [2].

A5. Thank you for your suggestion. But could you please provide more details about the unclarity? Sampling bias can take various forms [F], and the issue we discuss may differ from that in reference [2]. The sampling bias we discuss in this study arises from approximating $P(Y|Z,g)$ as $P(Y|Z)$. And Equation (7) shows why such an approximation can introduce a bias. The format of a causal graph depends on the process it describes. In rationalization, there are two different data-generating processes (DGP): one is the normal DGP of the original dataset, and another is the DGP of the rationale selection. In our study, the causal graph (Fig.4) describes a small local of the generator's updating process (i.e., the second DGP mentioned above) rather than the DGP of the original dataset, as most causality research does (since our research problem is the spurious correlations introduced by the generator, rather than the spurious correlations in the original dataset). Thus, it is not surprising that our causal graph differs from that of [2].

A1. The experiments are designed for two purposes:

1. To verify the claims proposed in this paper. We compare with the vanilla RNP to verify the existence of the sampling bias problem (this part is also verified by Section 5.1) and the effectiveness of our AI2 in addressing it. And we think this is the most important point of this study.
2. To show the competitiveness of the proposed method. So we compare with two recent methods FR and Inter_RAT to support this point.

All three methods you mentioned are somewhat old in the current literature. 3PLAYER was published in 2019. DMR and A2R were published in 2021. Besides, the performance of them is not as good as the more recent methods we have compared with. Experiments in A2R show that 3PLAYER sometimes does not even beat the vanilla RNP (note that A2R and 3PLAYER are published by the same team of authors). Also, FR has been empirically shown to outperform DMR and A2R by a large margin (up to $10.3\%$ in terms of F1 score). Note that the datasets we use in this study are the same as those in FR.

So, comparing with these old methods can neither support the first nor the second goal we mentioned above, and thus can bring few new insights to the readers. Moreover, these methods involve extra modules and hyperparameters. If you have ever reimplemented these methods yourself, you will know that it is not easy to find good parameters for them, which may lead to unfair comparisons.

A2. There are three reasons that may address your question.

1. Till our submission, Inter_RAT has not been formally published. There are no instructions in its released code. And the implementation details are not very clear. So, building our A2I on top of it is hard.

2. The goal of the comparison with advanced methods is to show the competitiveness of the proposed method. From the results, it seems that FR is much better than Inter_RAT. So, comparing FR and FR+A2I can better support this goal.

3. RNP+A2I has already outperformed Inter_RAT by a large margin, showing that A2I is a better method than Inter_RAT in this scenario.

A3. Thank you for providing the new references. They are relevant papers, but they are too recent to compare.
KDD 2023: August 6-10, 2023.
ACL 2023: July 9-14, 2023.
Submission ddl for ICLR 2024: September 28, 2023.

According to the ICLR policy (https://iclr.cc/Conferences/2024/ReviewerGuide), authors are not required to compare with papers published within the last four months.

Nevertheless, we are grateful for your suggestions. We find that the core idea of the DR paper [1] provides a very simple and flexible trick. It is orthogonal to our research and can be easily combined with our A2I. We are preparing the experiments for DR+A2I.

We will release the code and instructions (they are now visible only to reviewers) in case of acceptance.

We welcome any further questions! Just add your comments below.

[1] Decoupled Rationalization with Asymmetric Learning Rates: A Flexible Lipschitz Restraint. KDD 2023

update:
We have done a quick test for DR on part of the Beer dataset. The core idea of DR is to assign $\lambda=\frac{\eta_g}{\eta_p}>1$ for RNP, where $\eta_g, \eta_p$ are the learning rates for the generator and the predictor, respectively. In the original paper, the authors provided a heuristic method to relate $\lambda$ and sparsity. For simplicity in reimplementation, we simply set $\lambda$ to 10, as there is some empirical evidence from the original paper suggesting that 10 is a reasonable choice. The results are as follows:

Finally, we reiterate that the research questions addressed in these new works are orthogonal to ours, and there is potential to integrate different methods to address distinct issues.