Is the MMI Criterion Necessary for Interpretability? Degenerating Non-causal Features to Plain Noise for Self-Rationalization

Thank you deeply for taking the time to thoroughly review our paper. We are truly grateful for the insights and recommendations you've provided.

Weakness1 (The reasoning presented in Sections 4.1 and 4.2 is insufficient) For Section 4.1, we guess that you think $I(Y;S)$ can be as high as $I(Y;C)$ because we use the expression

$\mathcal{L_{MMI}}(C)\leq\mathcal{L_{MMI}}(S)<\mathcal{L_{MMI}}(N)$ (L209)

Here, we use "≤" only for mathematical rigor, but in practice, equality is hard to achieve. Using "$<$" directly might be more concise and clear. Below, we will analyze why equality is hard to achieve.

The overall idea is the Data Processing Inequality. For the three variables $Y,C,S$, we have that
$P(Y,C,S)=P(Y)P(C|Y)P(S|C,Y)$.
And from Figure 2(a), we know that $S \perp Y|C$. So, we further have $P(Y,C,S)=P(Y)P(C|Y)P(S|C)$, which means that they form a Markov chain $Y\rightarrow C \rightarrow S$ (note that the arrows here do not mean causality). Thus, with the data processing inequality, we have that $I(Y;C)\geq I(Y;S)$, where the equality holds if and only if $Y \perp S|C$.

$C$ is the direct cause of $Y$, while $S$ is correlated to $C$ through some intermediate variables, and then to $Y$. Therefore, this conditional independence hardly holds in practical applications. As a result, we can hardly have $I(Y;S)=I(Y;C)$. Therefore, we can say $I(Y;C)>I(Y;S)$. And in our analysis of the damage caused by spurious correlations in Figure 3, we use this setting, i.e., $\mathcal{L_{MMI}}(C)< \mathcal{L_{MMI}}(S)<\mathcal{L_{MMI}}(N)$. Even in this setting, spurious correlations can still pose obstacles to model training.

Regarding the question in Section 4.2, if $D_{KL}(P(Y|X_{-C})||P(Y|X))=0$, we will have $P(Y|X_{-C})=P(Y|X)$, which means that $Y \perp S|C$. Going back to the analysis above, this case can hardly happen. That is why we have Equation 7 (L251).

Weakness2 (datasets). Thank you for your careful observation. The Beer dataset has indeed been withdrawn from the official website, so it is necessary to contact the dataset's authors to obtain authorization for academic use. We had already received permission by email before our research.

Our datasets are the same as our main baseline MCD (NeurIPS 2023).

Here are some other papers that use the Hotel dataset: DMAHR [1], DMR [2], FR [3], CR [4], GR [5]. Among them, CR and MCD are already included in our baselines. The most recently published GR is publicly available at the end of March 2024, so we do not take it as a baseline.

Besides Beer and Hotel, there is another benchmark for rationalization, namely ERASER [6]. Compared to Beer and Hotel, the main advantage of ERASER is that it contains manually annotated rationales in the training set, which can be used for supervised rationale extraction. However, the datasets in it are not designed to verify spurious correlations. Considering this drawback, we did not use ERASER. We will add this discussion in our revision.

Here, we use the graph classification dataset to verify the generalizability of our method in other fields, as this domain has a dataset, GOODMotif (GOOD means graph out of distribution, it is from a public graph OOD benchmark), which contains both ground-truth rationales and spurious correlations.

We compare our MRD to the standard RNP and the strongest baseline MCD (other baselines such as Inter_RAT, NIR, and CR are designed for text data only and cannot be applied to graph data). The encoder is a three-layer GIN. We select a set of nodes from a graph to form the rationale. The sparsity is set to about $30\%$ (close to the sparsity of ground-truth rationales). The results are as follows:

[1] Deriving machine attention from human rationales. EMNLP 2018.
[2] Distribution matching for rationalization. AAAI 2021.
[3] FR: Folded Rationalization with a Unified Encoder. NeurIPS 2022.
[4] Towards trustworthy explanation: On causal rationalization. ICML 2023.
[5] Learning Robust Rationales for Model Explainability: A Guidance-based Approach. AAAI 2024.
[6] ERASER: A Benchmark to Evaluate Rationalized NLP Models. ACL 2020.

Thanks to the authors for their detailed response. While I think the theoretical exposition needs further discussion, the additional empirical results further support the merit of proposed approach. In lights of these results I have updated my rating from 5 to 6.

We are grateful for your detailed review and the thoughtful suggestions you provided.

Weakness 1. Now results Tables only F1 numbers are bolded. Please bold best and underline 2nd best for the other columns also.

A. Thank you for your suggestion, we will do it in our revision.

Question 1. I wonder why some of the results in Tables 1 & 2 are not consistent. Such as Table 2, NIR F1 goes up when spurious rate is increased and then goes back down. Can this be explained somehow? Some random chance?

A. We think this is a misunderstanding. The term $S$ represents the average sparsity of the selected rationales, that is, the average percentage of selected tokens in relation to the full text.

The reason why "NIR F1 goes up when the spurious rate is increased and then goes back down" could be due to two factors. First, the sparsity of the ground-truth rationale is approximately between 10% and 20%. When the specified sparsity exceeds 20%, the model is forced to select additional tokens, leading to a decrease in F1. When the sparsity is very low, the model might randomly select either spurious correlations or the real rationales, resulting in a relatively low F1 as well.

Limitations. Does not directly work with LLMs.

A. We now add the experiments conducted with the llama-3.1-8b-instruct model. We perform both 2-shot prompting and supervised finetuning. The results are in the General Response (at the top of this page). We find that our method can sometimes outperform the finetuned llama-3.1-8b-instruct.

We do not compare with GPT-4 because GPT-4's training involved extensive human alignment, which is very expensive. GPT-4 is overly powerful and not representative, as many studies on LLMs even use GPT-4 as a judge to evaluate different methods. Additionally, GPT-4 cannot be privately deployed.

We sincerely thank you for dedicating your time and expertise to review our paper. Your insightful comments and suggestions are highly valued and appreciated.

1. Weakness1 (It would be beneficial to demonstrate the generalization of the criterion through more experiments on different types of data).

A. Thank you for your insightful suggestion. We now add a graph classification task.

Although the framework of RNP can be applied to other fields, most of the improvement methods are designed for text data. When applying it to other domains, it is not easy to find enough proper baselines. Additionally, to evaluate the quality of rationales extracted by different methods, we need manually annotated rationales as a test set, which are rarely available in the image and speech domains. Here, we use the graph classification task to verify the generalizability of our method in other fields, as this domain has a dataset, GOODMotif (GOOD means graph out of distribution, it is from a public graph OOD benchmark), which contains both ground-truth rationales and spurious correlations.

We compare our MRD to the standard RNP and the strongest baseline MCD (other baselines such as Inter_RAT, NIR, and CR are designed for text data only and cannot be applied to graph data). The encoder is a three-layer GIN. We select a set of nodes from a graph to form the rationale. The sparsity is set to about $30\%$ (close to the sparsity of ground-truth rationales). The results are as follows:

2. Weakness2 (it would be beneficial to include a LLM as one of the baselines)

A. Thank you for your valuable suggestion. We now add the experiments conducted with the llama-3.1-8b-instruct model. We perform both 2-shot prompting and supervised finetuning. The results are in the General Response (at the top of this page). We find that our method can sometimes outperform the finetuned llama-3.1-8b-instruct.

We do not compare with GPT-4 because GPT-4's training involved extensive human alignment, which is very expensive. GPT-4 is overly powerful and not representative, as many studies on LLMs even use GPT-4 as a judge to evaluate different methods. Additionally, GPT-4 cannot be privately deployed.

3. Question1 (using the BERT encoder is no better than using the GRU encoder)

A. This phenomenon is indeed counterintuitive but has been validated by a set of previous papers. One of our baselines, MCD, also summarizes this phenomenon. There are roughly two possible reasons for this: first, when implementing RNP using BERT, it is highly sensitive to hyperparameters; second, it is prone to overfitting.

4. Question2 (bolded and underlined numbers)

A. Yes, they represent the best and second-best results. We will add this explanation to our revision.

We are thankful for your detailed review and valuable suggestions. Your feedback has greatly aided in refining our paper. Best wishes to you and yours!

Thank you for taking the time to carefully review our work and provide constructive feedback.

Weakness1. The datasets seem to be quite similar in terms of the task.

A. Thank you for your valuable suggestion. We now add a graph classification task. Different from the previous text classification datasets, the graph classification dataset is a more challenging 3-class classification dataset.

Our research topic imposes special requirements on the dataset. Firstly, the primary challenge in the dataset needs to be spurious correlations. Classification tasks are relatively easier to construct datasets with spurious correlations. As far as we know, the vast majority of studies on spurious correlations use classification tasks for experiments. Additionally, to compare the quality of rationales extracted by different methods, we need manually annotated ground-truth rationales as a test set, which further constrains the dataset. Here, we use a graph classification task to verify the generalizability of our method in other fields, as this domain has a dataset, GOODMotif (GOOD means graph out of distribution, it is from a public graph OOD benchmark), which contains both ground-truth rationales and spurious correlations.

We compare our MRD to the standard RNP and the strongest baseline MCD (other baselines such as Inter_RAT, NIR, and CR are designed for text data only and cannot be applied to graph data). The encoder is a three-layer GIN. We select a set of nodes from a graph to form the rationale. The sparsity is set to about $30$% (close to the sparsity of ground-truth rationales). The results are as follows:

Weakness2. I would have liked to see the actual binary task performances.

A. Some experimental results are copied from baseline papers. We followed the Inter_RAT settings and initially did not report classification accuracy. Now we report the classification accuracy of the standard RNP, the strongest baseline MCD, and our MRD. Additionally, we trained a regular classifier not used for interpretability, which we refer to as Classifier. (Due to time constraints, we are report results from a single random seed on the three beer-related datasets.) And we will consider a t-test in future work.

The results with $S\approx$10% are as follows:

The results with $S\approx$20% are as follows:

The results with $S\approx$30% are as follows:

Question1. Indicate which metric the 10.4% performance gains is.

A. It represent the rationale quality, which is measured by F1 score (the overlap between human-annotated rationales and model-selected tokens). On the Beer-Palated dataset with $S\approx 10$%, the F1 score of the second best baseline MCR is $53.1$%, while our MRD gets $63.5$%. The improvement is $63.5$%-$53.1$%=$10.4$% (L316). We will include this discussion in our revision.

Limitations. If the given example of wolves being commonly depicted in snow is in actuality the only (or clearly over-represented) occurrence case for both categories ("snow" and "wolf") within a dataset, the correlation will be seen as a causal one within the model training.

A. We greatly appreciate your reminder and will include the following discussion in our revision.

We agree that our method cannot handle this situation, but it is a too extreme case. If a wolf always appears with snow in an image, and snow never appears without a wolf, then snow and wolf will have identical statistical properties in this dataset. In fact, we believe that even humans would not be able to handle this situation. Suppose a human is asked to classify these images without being told the basis for classification. In that case, they wouldn't know whether to classify based on the wolf or the snow, as either could be used to complete the classification task.

Thank you for the thorough rebuttal. I am satisfied with the additional experiments and additions.

Regarding the point in limitations: I understand that the case is extreme, but still a very realistic one, especially when dealing with rare categories and finite datasets. I do not necessarily agree that humans would always fail the task in a case where the categories in question would indicate strong semantic links to other categories based on which one could form an analogy and perform inference. However, this speculation is obviously out of the scope of the current study.

I am increasing my rating for the overall paper from 6 to 7.