Measuring Information in Text Explanations

Thank you for reading this paper and providing comments. To the points you mentioned:

• What is the training data and procedure for InfoNCE? We have train - validation - test splits in the dataset (Appendix A.1 elaborates more on the details). The InfoNCE, as well as other estimators, are trained on the train set.
• About the CLUB method, and why preferring InfoNCE over CLUB? Appendix A.2 shows the procedure of finding one estimator among the collections. On correlated Gaussian, the mean squared errors of any estimators except DoE were acceptable. However, on e-SNLI, other estimators have much larger variance on the validation set than InfoNCE. This motivates us to choose InfoNCE rather than CLUB on the experiments (which uses e-SNLI problems).
• Why use V-info vs MI estimation method? V-info has different applicability from the MI estimation problems. V-info is suitable for “vector-to-scalar” data, whereas MI estimators (CLUB, DoE, InfoNCE, MINE, NWJ, SMILE, etc.) are for “vector-to-vector” data. Sections 3.4 and 3.5 contains more formal elaborations.
• Does this method depend on the accuracy of MI estimator? Yes. This method also depends on how well the embeddings encode the related information.
• Proposed information-theoretic scores vs other metrics? Each metric provides a unique perspective. Our scores take root in communication channels and information theory, and reveal mechanisms about the NLEs that we consider to be quite interesting. One benefit of our score is the flexibility and the potential to generalize across multiple modalities (given suitable embedding).
• About the innovation and usefulness. We consider the innovation and usefulness mainly in two aspects: (1) A step towards unifying the evaluations of two different types of text explanations. (2) Some insights about the mechanisms of the explanations, e.g., by the informativeness-relevance trade-off.

Thanks for the recommendation. Here is an attempt following this line of thought. (Note that we do not have as strong background to carry out the theory analysis to the same level as the authors of the CLUB paper and many other papers that propose information estimator algorithms.)

Many other estimators have larger variances than InfoNCE, echoing the results shown in e.g., figure 1 of the CLUB paper. McAllester and Stratos (2020) showed that lower bound MI estimation from N samples can’t be larger than O(log N), and InfoNCE is one of the lower bounds. We used batch sizes of 64 (except for NLE-roberta where we used 32), so the upper bound on the estimated amount is around a constant times 4 nats. This explains why even the most informative explanations have I(X;E) of around 4 nats (in Figures 4 and 5). Also, selecting the estimator by the least variance doesn't seem to be the optimal criterion (while it's the most convenient criterion in the absence of ground truth). Comparing the behavior of CLUB to InfoNCE, and other estimators on the explanation data has a lot of research opportunities left in the future works. We will add these discussions to the papers.

Thank you for commending on the novelty, feasibility, and the quality of the paper! To the points in weaknesses and questions:

• Improve the structure of experiments and results. Thanks — we updated the pdf. The current version lists the sections in a more organized way.
• Figure 4 name confusion: Yes this confusion is avoided in this version. We changed the “informativeness” in the silver labels into “informational support”.

Thank you for considering this paper interesting, and for commending on the empirical solidity of this paper! To the points mentioned in weaknesses and questions:

• The evaluation plan and goals. We added the experiment protocol and goals to the first paragraphs in 5.1 to 5.3. We also updated the pdf to make it more organized.
• Only automatic and no human evaluations? Yes — human eval of explanation is a huge topic. Each format of explanation format (NLE or rationale), and each topic of the explanation contents can have a unique impact to the audience [1]. Also the situation of the audience matters [2]. The human evaluations in a careful enough manner would lead to the scales of experiments that significantly go beyond the scope of this paper, so we can only focus on the automatic evaluation perspective.
• New insights beyond the correlations to some scores? There are at least three insights beyond correlations: (1) With ANOVA, we quantify how much each aspect (lexical-semantic, reasoning, clarity, relevance) can explain the variation in the measured information scores. (2) We show a tradeoff between relevance-informativeness phenomenon. (3) We illustrate that the openai and cohere embeddings preserve the distributions, whereas the roberta embeddings focus more on the decision boundary.
• Why choose embeddings to compute the entropy? Embedding is the crucial component that allows multiple types of explanations (NLE, rationale, and potentially other types) to be comparable on the same ground. The pipelines involving conditional probabilities usually need another module that computes the one-dimensional probability value from the embeddings.
• Comments about the formats:
  • Reference for “Inter-score correlations are low”: Indeed. They are from Figure 7. We updated the references in the pdf to make this clearer.
  • References for the ANOVA results: We updated the references to the ANOVA details to make this clearer.
  • Figures 4-6, and moving some statistics to the main content. We moved around some parts. Figures 5-6 takes more space than the prompt templates (which we also consider quite important, since LLM’s behavior is heavily affected by the actual wording of the prompts) so we are keeping them at the current version.
  • Naming confusion for “informativeness” (GPTScore vs I(Y;E)). Good catch. We changed to informational support (info_support in the tables to save space) to avoid confusion.
  • Table 1 in Figure 3 x-axis: The “informativeness” (now info_support) is added now. The past version didn’t contain this due to a confusion with I(Y;E).
  • Changing the order of x-axis in Figure 3 to match the previous table: Done.

References:

[1] For example: Chen, Chacha, et al. "Machine explanations and human understanding." arXiv preprint arXiv:2202.04092 (2022). The “How are explanations useful to humans?” section in this reading list compiles some papers on this topic: https://ziningzhu.notion.site/Explanations-reading-list-56cd203b1d1c4fd79e8fcf319b1560a8?pvs=4

[2] Zhu, Zining, et al. "Situated Natural Language Explanations." arXiv preprint arXiv:2308.14115 (2023).

Dear Reviewer 7iHa,

Following the previous response, we would like to further discuss the meaning of the two information-theoretic scores and put them into context.

The two scores take root in the communication channels: one unit of information (bit) quantifies the amount of information that helps us remove the uncertainty of a fair coin toss. (Machine learning uses ln instead of log, which we follow. It's just differing by a constant.) Intuitively, there should be a score that describes how much information is "transmitted" through each channel in the communication. We apply information estimation tools and showed it's possible to do so.

We call the I(X;E) score relevance, since this score describes more of how the explanan relates to the input. The experiments verify that the I(X;E) is correlated to many lexical-semantic features, especially the cosine similarity. Additionally, the features in the lexical-semantic category can explain a high portion of the variance in I(X;E), illustrating that I(X;E) describes more about the relatedness of the input.

We call the I(Y;E) score predictive informativeness, because (1) the computation approach uses predictive V-info, and (2) this score describes more of how the explanan predicts the output label. To predict the label (i.e., do the task specified by the dataset), a wide range of mechanisms might happen under the hood, including reasoning, inferring, pattern matching based on the embedding spaces, etc. These are illustrated from our correlation experiments and the analysis of variance experiments. Overall, it's hard to claim that this score is about e.g., reasoning, so we do not draw such a conclusion.

We hope the above elaboration clarifies a bit. If there are further questions, please do not hesitate to follow up on the comments.

Thank you for commending on the contribution! We believe that model evaluation can benefit from information-theoretic approaches. While the area of information estimator is not as popular as e.g., prompting LLMs, or LLM agents, there are still some meaningful progress in the past a few years, and we still believe there will be improvement potential.

To your question about the experiment procedures: we updated the pdf. Each subsection there now starts with a paragraph describing the experiment setup. The procedure is now clearer and reflects how each experiment is set up.