CELL your Model: Contrastive Explanations for Large Language Models

Main weaknesses:

1. The authors propose contrastive explanations for LLMs based on prompt perturbations. Whereas the authors nicely demonstrate use cases of their method, there are important factors that have not been systematically explored to evaluate the robustness and generalizability of this approach. This point is set aside the fact that the scoring function is use case-dependent and thus its formalization changes for every case. In particular, I am curious whether the authors find different results based on the delta parameter value described in equation 1:

Do the authors find distinct qualitative results depending on that value? If not, does this also hold across use cases?

Answering these questions would greatly benefit the reader in having more information in terms of applicability of this method.

2. I feel like there is a step between finding prompt perturbations that lead to some desirable properties in the output and explaining why these perturbations generate what they generate. I understand this is a tricky issue, but in my opinion a linguistic interpretation of the perturbations has more value than simply providing the perturbation itself. Did the authors try to analyse the linguistic outputs of the perturbed prompt to gain more insights in terms of explainability. It seems that currently, the value of the proposed method rests in applications of making “better” LLM outputs rather than explaining why the models output what they do. I would definitely clarify the contributions of this work in terms of explainability, in order to situate it better within that field.

Secondary weaknesses:

1. I suggest making the connection between the g function described in equation 1 and the distinct scoring methods described in section 3.1 more explicit. This will increase the clarity of the paper.

2. On line 277, it is confusing to call this a “first iteration”, as in fact there are n iterations at this level. I would call it: “first n iterations”.

• My main concern is with the contrastive prompts found by CELL. While the authors report a 0.99 similarity score between the original and contrastive prompts, many examples in the paper seem semantically different. Using BERT embeddings to assess similarity may be problematic, as they primarily capture word overlap, and changing a word can lead to entirely different meanings. This raises questions about the use cases the author provided:

  1. Use case of automated red teaming: A successful adversarial example (contrastive prompts in this context) should have the same meaning for humans while eliciting unusual behavior from LLMs. The responses of LLMs in Figure 4 appear reasonable to me, as the contrastive prompts introduce ambiguity. For instance, "is a competitor" is crucial for LLMs to answer "No," while altering it to "is not part" changes the relationship between the two companies, rendering them non-competitive. I believe responding "Yes" in this scenario is not inappropriate.

  2. Explaining conversational degradation: I think neither of the two cases in Figure 5 demonstrates degradation. In the first example, the LLM provided less detail because the CELL altered the assistant's previous response, which was, "That's really funny. Are you going to call this a lesson?" This response had a humorous tone, suggesting that the user's request was overly simple. As a user, I would expect a straightforward answer listing the basic steps. In the second example, the term "harming" was directly changed to "talking," which completely alters the context. I also don't think not calling the emergency services has any problem. If the authors' intention is merely to explain why the LLM responded in that way—although I don't think these explanations significantly enhance our understanding of LLM behavior—then it may be acceptable. However, labeling it as "conversational degradation" suggests that it's a flaw in LLMs that we should aim to fix.

• As the examples provided seem invalid, it raises further concerns about the effectiveness of the evaluation metric. Since the goal is to explain the behavior of LLMs, the most straightforward approach would be to conduct a human study to demonstrate that contrastive explanations align with human expectations. Specifically, do the prompts identified by CELL reveal unintended behaviors from LLMs while maintaining a similar meaning to the original prompts?

• While the algorithms appear valid and the experiments are comprehensive, I'm still unclear about the specific insights that CELL offers and how these explanations can be utilized to fix LLMs. From what I've observed, CELL primarily indicates that LLMs alter their responses based on changes in the prompt. I may be overlooking something, but I don't grasp the advantages of contrastive explanations in this context.

1. The framework for generating contrastive explanations lacks clear definitional boundaries. For instance, in Figure 2's example ("My car is making a weird noise when I accelerate. Can you help me diagnose the problem?"), there's insufficient clarity regarding the relative importance hierarchy among phrases like "car," "weird noise," "accelerate," etc.
2. A potential typographical error appears in Line 190, where "y1 contradicts y1" likely should read "y1 contradicts y2."
3. The examples presented in Figure 4 appear to misalign with standard red teaming definitions and practices. A more rigorous adherence to red teaming principles would strengthen the paper. More appropriate examples should demonstrate how minimal prompt changes could expose concerning model behaviors, such as safety issues, biases, etc.
4. The choice of traditional PLMs (BERT, BART, T5) as infiller functions (I(.)) raises concerns about semantic coherence in generated texts, as these models often struggle with maintaining semantic correctness in their outputs.

1. One major weakness of this paper is that the setup and takeaways for the Experiments section is not very clear. The major question I have is how to define the success of finding contrastive explanations. In Section 5.1, the results use the scoring function preferences, which characterize the pairwise preference between the given explanation and the baseline explanation generated by LLM through direct prompting. However, given the pairwise comparison model (i.e., stanfordnlp/SteamSHP-flan-t5-xl) is not specifically trained to provide preference for such explanation, I am wondering whether the scores output by the model have high correlation with human judgment. As a follow-up question, what's the criterion to evaluate the contrastive explanation if we ask humans to evaluate it. I think defining what is a good explanation is essential and non-trivial. With this concern, even though Section 5.2 analyze the properties of contrastive explanation algorithms (i.e., flip rate, edit rate, and content preservation), I don't exactly know how to interpret the results or what are the actionable takeaways since the definition of contrastive explanations for LLMs is not exactly the same with the contrastive explanations for classifiers.
2. It's commendable that in Section 6, the authors provide concrete use cases for the proposed contrastive explanations in the context of LLMs. However, especially for the first use case, automated red teaming, I don't know what's the advantage for using the contrastive explanation compared to using other red teaming. Providing results on comparing using contrastive explanation and other red teaming approaches could strengthen the argument. For the second use case, the paper mentions "beyond explanations, the generated contrastive examples produced by our method provide useful data for improving the model’s conversational ability", I am wondering if you can elaborate on this or is there any proof-of-concept result.

• The novelty and contribution are not prominent. The authors claim that “this paper offers the first contrastive explanation methods for LLMs.” However, [1] has already proposed counterfactual explanations for LLMs, which are conceptually similar to contrastive explanations. This prior work is notably absent from the related literature review. While the authors emphasize that many previous studies focus on using LLMs to generate contrastive explanations for classification tasks, I do not see significant technical differences between classification and generation tasks in this context.
• The clarity of the paper could be improved. For instance, in the upper part of Figure 1, the contrastive response does not contradict the original response. However, in the lower part, there seems to be a typo in the prompt, as “kill and save” presents a contradiction. Additionally, the original and contrastive responses are generated from different input prompts, making it unclear why it is reasonable to compare the preferences for the two responses.
• The evaluation is limited to Llama 2 (13b, 70b) and bart-large-xsum. It would be beneficial to include evaluations of more recent LLMs, such as Llama 3, GPT-4, Gemini, and Claude.
• The results are not compelling. For instance, CELL and CELL-budget produce similar outcomes across different values of split_k, and CELL-budget is not consistently more efficient.
• There is a lack of evaluation metrics for the explanations, such as faithfulness and plausibility [2, 3].
• The rationale for considering four scoring functions is unclear. Do they operate similarly, or do they provide different insights in various settings?
• The analysis and discussion of the two use cases lack depth and insight. It may be reasonable to reduce the size of Figures 4 and 5 to save space, allowing for a more detailed discussion about the use cases.

[1] Andreas Madsen, Sarath Chandar, and Siva Reddy. 2024. Are self-explanations from Large Language Models faithful?. In Findings of the Association for Computational Linguistics: ACL 2024, pages 295–337, Bangkok, Thailand. Association for Computational Linguistics.

[2] Lyu, Qing, Marianna Apidianaki, and Chris Callison-Burch. "Towards faithful model explanation in nlp: A survey." Computational Linguistics (2024): 1-67.

[3] Zhao, Haiyan, et al. "Explainability for large language models: A survey." ACM Transactions on Intelligent Systems and Technology 15.2 (2024): 1-38.