Towards Faithful Explanations: Boosting Rationalization with Shortcuts Discovery

Thank you for your time and insightful suggestions! According to your comments, we conduct additional experiments and provide the responses as follows:

Comment1:I would be curious whether transferring the knowledge from one task to another could be possible to some extent (not necessary from movie-dataset-1 to movie-dataset-2).

This is an interesting question! We conduct experiments to illustrate this. As shown in the table, we first train ${SSR}_{unif}$ and AT-BMC (the state-of-the-art (SOTA) supervised rationalization approach) on the BoolQ dataset, and then evaluate them on Movie and MultiRC.

From the experimental results, we can see that both AT-BMC and ${SSR}_{unif}$ cannot achieve good results.

The reason may be the data distributions are too different.

Comment2:The relationship between the number of augmented data vs task/rational performance is currently unclear. I would appreciate having a graph showing how the performance evolve according the number of added data

Here, we compare ${SSR}_{unif}$ with different percentages (0-25%) of semantic DA on the Evidence Inference dataset.

From the experimental results, we can observe that both task and rationale performance are improving as the number of added data increases. Among them, the rationale performance improvement is more obvious and has a significant improvement when the number of added data reaches 15% of the total data.

We appreciate your comments! To address your concerns, below we prudently justify the details of our proposed method and conduct more experiments.

Comment1: The definition and suggested algorithm for selection is too simplistic.

Thank you for your comments, below we present the details of the definition and suggested algorithm for selection. Definition: Given the text input $x={x_{1}, x_{2}, \ldots, x_{n}}$, the goal of selection is to first generate a mask variable $m=\{m_{1}, m_{2}, \ldots, m_{n}\}$, where $m_{j} \in\{0,1\}$ indicates whether the $i$-th token is a part of the rationale. Finally, the selection function selects the rationales as $z = m \odot x= \{m_{1}\cdot x_{1}, m_{2} \cdot x_{2}, \ldots, m_{n} \cdot x_{n} \}$. Briefly, the goal of the selection is to extract a subsequence from the original text to be utilized as rationales to support the prediction results.

Algorithm: In order to select whether each token is rationale or not, we reduce the rationale extraction algorithm to a token-level binary classification task, i.e., predicting the probability that a token is a rationale token. Specifically, we map each token $x_{i}$ to the probability of being selected as part of rationale: $p_{\theta}(m_{i}|x_{i}) = softmax(W_{s_{un}}f_{s_{un}}(x_{i}))$. Among them, $f_{s_{un}}(\cdot)$ represents an encoder encoding $x_{i}$ into a $d$-dimensional vector, and $W_{s_{un}} \in {R}^{2 \times d}$.

Then, in order to train the model to be able to extract rationales, there are different training approaches for unsupervised and supervised rationalization:

For unsupervised rationalization, due to the lack of labeled rationales, the selection function must be cascaded with the predictor, where the rationale learning signal for unsupervised rationalization must rely on comparing the prediction of the predictor with the ground-truth task label.

For supervised rationalization, since the labeled rationale is known, we can use the token-level binary cross-entropy (BCE) cost function to train supervised rationalization.

Comment2: The authors use only one example ('received a lukewarm approach' in the film review) throughout the article. Doesn't such a solution reduce the proposal to a supervised approach? Please use another example of a shortcut.

First, the shortcut does not reduce the proposal to a supervised approach. Specifically, the example we give in the paper is a real example from the Movie dataset, where the shortcut 'received a lukewarm approach' is obtained by our shortcut discovery strategy, not by a priori human annotation. Besides, it is noted that our shortcut discovery strategy relies on the rationale of the true annotation. However, since the rationale annotation is difficult to obtain for most tasks, shortcuts can not reduce the proposal to a supervised approach in most cases. Therefore, in our SSR, we use a semi-supervised approach. Finally, we also visualize some of the shortcuts obtained by our shortcut discovery strategy in Appendix C.5, which we further illustrate one of them here:

We visualize another example on Movies (the label is positive), where the italic tokens represent the real rationales, and the bolded ones are the predicted rationales.:

From the results, we can find although both Vanilla Un-RAT and ${SSR}\_{unif}$ predict the label as positive correctly, Vanilla Un-RAT still extracts shortcuts as rationales. Specifically, "Mozart is a famous musician" is the shortcuts. Although Mozart was a great musician, it has no relevance to how good his biographical film is. ${SSR}\_{unif}$ avoids these shortcuts, but Vanilla Un-RAT extracts these as rationales.

Comment3: Could you clearly state your hypothesis in the introduction?

Thank you very much for your suggestion, putting our assumptions in the introduction is really beneficial for readers to read. Therefore, we have revised the introduction (the blue text) in the revised version and uploaded it to ICLR. Please review it, and if there are any problems, please let us know.

Comment4: Is the whole of section 2 your proposition, or the definition of possible solutions used now? It is not clear.

Thank you again for your suggestions. Based on your suggestions, we have organized section 2 and updated it in the revised version.

Thank you for your insightful suggestions! According to your comments, below we clarify the misunderstandings, and conduct more experiments.

Comment 1: It's unclear whether shortcuts always exist in the text. If they aren’t, what would the model predict?

We argue that shortcuts should always exist in texts such as entity bias [1,2] and statistical bias [3]. Previous studies [4,5,6,7] that study shortcuts in texts also support our viewpoint. Moreover, from the causal graph we constructed in Appendix D, we observe that since there always exists a backdoor path S↔Z→Y which makes shortcut Z and the label Y spuriously correlated, we conclude shortcuts always exist in the text. Finally, we assume that some datasets have been processed without significant shortcuts, enabling the model to learn the true causal relationships within the data and function as a debiased model. However, this assumption is overly idealistic when applied to real-world datasets.

Comment2: The author asserts that unsupervised methods frequently identify shortcuts as inefficient rationales and provides examples. Yet, the reader would be curious about how often that happens.

As in the answer to comment 1, shortcuts commonly occur in text. Meanwhile, [5] has shown that the vanilla rationalization criterion in unsupervised methods is prone to highlighting spurious correlations (shortcuts) between the input features and the output as valid explanations. Therefore, we argue that unsupervised methods will frequently identify shortcuts as inefficient rationales.

Comment3: If the model already yields good results, why do we have to give up shortcuts? I suggest a stronger argument for why finding a good rationale is significant, such as it can be helpful for other reasoning tasks.

This is a valuable question! If the shortcut helps us get better results, do we still need to remove the shortcut? Our answer is yes. Specifically, when faced with test data that is distributed with the same distribution as the training data, the shortcut facilitates us to obtain accurate results. However, since shortcuts do not reflect the real causal relationship between the data and labels, when the test data is distributed differently from the training data (i.e., the shortcut does not exist in the test data), a model that relies on shortcuts will have poor prediction results. For example, in the following table (Table 3 in the original manuscript), Vanilla Un-RAT achieves promising results when we test it on an identically distributed dataset IMDB, but it achieves poor predictive results on out-of-distribution data SST-2. Thus, we can conclude that when the model relies on the shortcut in the data for training and prediction, the model fails in the face of new data distributions, further limiting the application of the model.

Furthermore, finding a good rationale is significant:

1. Extracting a good rationale can help us avoid extracting shortcuts as rationales, which can help the model focus on the real causal relationship between the data and labels during training and prediction. It can increase the accuracy and robustness of the model when facing the test data with different distributions.
2. Extracting a good rationale can improve the interpretability of the prediction results, which can be applied in some high-risk domains, such as law [4].
3. Rationalization can help other reasoning tasks. For example, rationalization is used as a knowledge extractor in [8] to extract the most responsible features for the predictions. After that, [8] expands the extractive rationales using commonsense resources to generate natural language explanations and give the final prediction.

References

[1] Yongchun Zhu, Qiang Sheng, Juan Cao, Shuokai Li, Danding Wang, Fuzhen Zhuang. Generalizing to the Future: Mitigating Entity Bias in Fake News Detection. In SIGIR2022.

[2] Fei Wang, Wenjie Mo, Yiwei Wang, Wenxuan Zhou, and Muhao Chen. A Causal View of Entity Bias in (Large) Language Models. In Arixv2023.

[3] Xiaobao Wu, Chunping Li, Yishu Miao. Discovering Topics in Long-tailed Corpora with Causal Intervention. In ACL2021.

[4] Linan Yue, Qi Liu, Li Wang, Yanqing An, Yichao Du, Zhenya Huang. Interventional Rationalization. In EMNLP2023.

[5] Shiyu Chang, Yang Zhang, Mo Yu and Tommi Jaakkola. Invariant Rationalization In ICML2020.

[6] Wei Liu and Jun Wang, Haozhao Wang, Ruixuan Li, Zhiying Deng, YuanKai Zhang, and Yang Qiu. D-Separation for Causal Self-Explanation. In NeurIPS2023.

[7] Du M, Manjunatha V, Jain R, et al. Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU models. In NAACL2021.

[8] Bodhisattwa Prasad Majumder, Oana-Maria Camburu, Thomas Lukasiewicz, Julian McAuley. Knowledge-Grounded Self-Rationalization via Extractive and Natural Language Explanations. In ICML2022.

Thanks for your reply. I now know that your implemented Vanilla Semi-RAT also uses this setup, so I think my question has been answered. However, I would like to take this opportunity to point out the special contributions of our work "FR".

While I acknowledge that ST-RAT also utilizes a shared encoder, it should be noted that it does not fall under the category of rationalization. In the realm of rationalization, the classifier's input should be a rationale, which ST-RAT does not adhere to. In fact, ST-RAT cannot even be classified in the XAI domain. Its classification and rationale extraction are two entirely independent tasks. One could interpret ST-RAT as performing two separate tasks—text classification (although NLU in the original paper) and NER—simultaneously through the concept of multi-task learning, leveraging two sets of annotated data for training (classification and NER). The performance improvement resulting from multi-task learning in this case is a straightforward consequence of simultaneously utilizing labeled data for both tasks.

In (unsupervised) selective rationalization, the extraction of rationales and the prediction of the classifier form a pipeline task. The only available labels are for classification, and there has been no prior analysis on how sharing encoders in a pipeline (beyond the scope of the rationalization pipeline) enhances performance or the mechanisms by which performance improvement occurs. Clearly, the pipeline task is distinct from multi-task learning.

We will be grateful if you ccould discuss its effectiveness when you employ this trick. For instance, you might consider phrasing such as, "Liu et al. (2022) developed a unified encoder to induce a better predictor by accessing valuable information blocked by the selector." as seen in https://arxiv.org/abs/2306.14115.

Thanks for your public comment. The shared encoder trick for the selector and predictor is a classical operation, and a similar empirical trick was used in ST-RAT [2] before FR [1]. Since our method is a semi-supervised method and ST-RAT is our important semi-supervised baseline, we also use the shared encoder in SSR. Similarly, the same setup is used for Vanilla Semi-RAT and WSEE in the replication of them.

Finally, we argue that the shared encoder is a common trick, especially after you theoretically prove that it works. However, this is not the main central point of this paper. Our goal in this paper is to address the problem that models may exploit shortcuts in the data to make predictions and compose rationales. To solve this problem, in this paper, we propose a shortcuts-fused method by incorporating shortcuts explicitly with a shortcut discovery approach.

References

[1] Wei Liu, Haozhao Wang, Jun Wang, Ruixuan Li, Chao Yue, YuanKai Zhang. FR: Folded rationalization with a unified encoder. In NeurIPS2022.

[2] Meghana Moorthy Bhat, Alessandro Sordoni, and Subhabrata Mukherjee. Self-training with few-shot rationalization: Teacher explanations aid student in few-shot nlu. In EMNLP2021.

Hi,

I've been studying about rationalization lately as well. But I don't agree with your comments on INVRAT. From the original INVRAT, I find that the environment in the beer dataset is achieved through the linear prediction error. This suggests that the environments can be inferred from the data in some way. I can share my method which can achieve INVRAT with you:

1.Train an RNP and obtain the non-rationale representation of all samples (non-rationale can be used for environment partitioning [1,2]).

2.Cluster all the non-rationale representations using k-means clustering (refer to [2] for the use of clustering for environmental inference).

3.Assume each cluster of samples belongs to a single environment.

Of course, there are many methods for environmental inference, and the above method I used is relatively simple. You can also try other methods that use variational inference to infer environments, such as [3,4].

Finally, I am also currently submitting to ICLR. I notice that you first commented on 06 Dec 2023, which is a blocking period for replies for authors. According to ICLR rules, from 05 Dec 2023, authors can't upload any comments to openreivew. As far as my experience is concerned, authors can reply to your comments on openreivew only after the decision result of the paper is out.

References

[1] Provably invariant learning without domain information. Xiaoyu Tan, LIN Yong, Shengyu Zhu, Chao Qu, Xihe Qiu, Xu Yinghui, Peng Cui, Yuan Qi. ICML2023.

[2] Learning Invariant Graph Representations for Out-of-Distribution Generalization. Haoyang Li, Ziwei Zhang, Xin Wang, Wenwu Zhu. NeurIPS 2022.

[3] Learning Substructure Invariance for Out-of-Distribution Molecular Representations. Nianzu Yang, Kaipeng Zeng, Qitian Wu, Xiaosong Jia and Junchi Yan. NeurIPS 2022.

[4] Towards Out-of-Distribution Sequential Event Prediction: A Causal Treatment. Chenxiao Yang, Qitian Wu, Qingsong Wen, Zhiqiang Zhou, Liang Sun and Junchi Yan. NeurIPS 2022.

Dear readers,

It's clear that this commenter has been making no sense. If you are interested, you can see from our conversation at https://github.com/yuelinan/Codes-of-Inter-RAT/issues/1 that I have responded to all of his questions and have open sourced the code.

Furthermore, in response to his query, I clearly stated "I would like to state that it is my conclusion (since the publication of neurips2023) to use clustering methods to classify environments and to implement invrat based on this. Until then, I use my empirical clustering methods to classify environments. So when you ask a question in December 2023, am I going to use my 2022 experience to answer you? I think it is more responsible to answer you using current conclusions I have already reached."

Besides, during the iclr review, this commenter intentionally commented at a stage when the authors were unable to respond to public comments in an attempt to influence the ac's decision on the paper. I consider this to be malicious harassment by a peer.

I'll keep publicizing the discussion on GitHub.