ContextCite: Attributing Model Generation to Context

We thank the reviewer for their feedback. We address each question individually below.

Effect of number of context sources (and sparsity) on sample complexity.

We agree with the reviewer that computational cost is a significant limitation of ContextCite. However, we would like to point out that the number of sample ablations needed often does not actually grow quickly as the context scales. As we discuss in Appendix C.1.1, the number of ablations required to learn an accurate surrogate model only grows linearly in the number of relevant sources (rather than growing linearly in the total number of sources). In Figure 3(a), we observe empirically that the number of relevant sentences often remains small ($<10$) even when the context consists of hundreds of sources. This suggests that a small number of context ablations may suffice even for very long contexts (in practice, we find that using just $32$ context ablations often works well).

We would also like to point out that while ContextCite is expensive to run, it can be applied in a post-hoc manner. A user can interact with an LLM as usual and can only choose to apply ContextCite and incur this cost in cases when they are suspicious of a model’s generated response and would like to see attributions.

Effect of linearity in context attribution.

We agree with the reviewer that a linear model may not handle complex dependencies between sources, such as those that might arise when reasoning over many sources. We do find it promising that a linear model works well for Hotpot QA, in which every question explicitly requires combining information from two or more documents. That said, proper attribution for more sophisticated reasoning may require more complex surrogate models (or interaction terms for the linear model), which is an interesting direction for future work.

Why does selecting only relevant sources help?

The reviewer makes a good point that if the model uses the wrong sources to generate a response, then selecting just these sources is unlikely to help. To understand why relevant sources can actually help, we consider two failure modes associated with long contexts:

1. The model identifies the wrong sources. Selecting these sources will not help.
2. The model identifies the correct sources, but misinterprets information because it is distracted by other irrelevant information. Selecting these sources can help because a shorter context including these sources could help.

We intend to include this discussion of why selecting relevant sources helps in the next revision.

We thank the reviewer for their feedback. We address each question individually below.

Token-level context attribution.

Our task actually does consider attributing arbitrary selections from the response (including individual tokens). We discuss this in Section 2.3. When we evaluate ContextCite in Section 4, we evaluate the attributions of individual sentences in the response: “given an example, we (1) split the response into sentences using an off-the-shelf tokenizer and (2) compute attribution scores for each sentence”; see lines 202-203. We have also included a figure in the global response illustrating an example of a ContextCite attribution on Llama-3-70B for a particular generated statement.

Effect of ablations on context coherence and model output.

The reviewer makes a good point that a context source might not directly contribute to a response, but ablating it might influence the response indirectly by making the context incoherent. For example, consider the sentences: “John lives in Boston. Charlie lives in New York. He sometimes visits San Francisco.” In this case, “He” refers to Charlie. However, if we ablate just the sentence about Charlie, “He” will now refer to “John.” So, if we attribute the generated statement “Charlie sometimes visits San Francisco,” then we’ll get two sources: (1) “Charlie lives in New York” and (2) “He sometimes visits San Francisco.” While, by some definition, (2) is the actual source, having both (1) and (2) as sources is also reasonable behavior in our view. So, we do not view this issue as a significant limitation. Extending ContextCite to account for such dependencies (e.g., via a non-linear surrogate model) is an interesting avenue for future work.

Experiments on larger models and other NLP tasks.

• Larger models: Our experiments in the paper evaluate ContextCite on models ranging from 4B to 8B parameters. To showcase the scalability of our method, we have included an additional figure in the global response, which illustrates a ContextCite attribution for Llama-3-70B. Furthermore, in Appendix B.3, we show that our experiment from Section 5.2 (selecting query-relevant context sources) improves results on LLama-3-70B as well.
• Other NLP tasks: The tasks we use to evaluate ContextCite are reasonably diverse, including summarization (CNN DailyMail), knowledge extraction (TyDi QA), and reasoning using information from multiple documents (Hotpot QA). Still, we agree with the reviewer that certain tasks/prompt designs may be less suitable for ContextCite. As we note in Appendix C.2, if the model outputs a chain-of-thought reasoning chain, then it is quite possible that only the initial facts would yield proper attributions. For example, for the output “I have 3 eggs. You have 4 eggs. Together, we have 3 + 4 = 7 eggs.”, ContextCite would be able to meaningfully attribution “I have 3 eggs” and “You have 4 eggs”, but not “Together, we have 3 + 4 = 7 eggs” because this statement follows from the previous part of the response and not directly from the context. Extending the core task of context attribution to chain-of-thought style reasoning chains is an interesting avenue for future work.

Examples of attributions.

We’ve provided an example attribution in the global response, and will include additional examples in the next revision.

We thank the reviewer for their feedback. We address each question individually below.

On technical novelty and connections to data attribution.

We agree with the reviewer that our work leverages the data attribution literature (e.g., datamodels, LDS); we acknowledge and discuss these connections in our paper. That said, understanding how a model uses information presented in its context is different (both conceptually and mechanically) from understanding how a model uses its training examples. Furthermore, some of the applications enabled by context attribution (e.g., helping verify correctness, see Section 5.1) don’t have clear data attribution analogues. Hence, despite the connections, we believe that it is valuable to study context attribution as its own task.

We also believe that it is a valuable empirical finding that a linear surrogate faithfully models how a language model uses its context (especially given the dependencies between sentences). A priori it is not clear that this design choice from the data attribution literature would work well for context attribution too.

Confusion over corroborative attribution and usage of GPT-4.

We would like to briefly clarify: the goal in Section 5.1 is to use the contributive attributions estimated via ContextCite to help verify the correctness of a generated statement. Our intuition is that if the contributive sources (i.e., sources which cause a model to generate a statement) do not also entail the generated statement, then the generated statement might be incorrect (because the model may have misinterpreted the sources or hallucinated). We use GPT-4 only to assess whether each generated statement is actually correct and not for any sort of attribution.

The reviewer is right that verifying correctness is closely related to corroborative attribution. In Section 5.1 though, we only focus on exploring whether contributive attribution methods (like ContextCite) can help in verifying the correctness of generated statements.

Effect of number of context sources on sample complexity.

The reviewer makes a good point that in general, if we have $n$ sources, we require $O(n)$ forward passes to learn an accurate linear surrogate model. As we discuss in Appendix C.1.1, in the case where the ground-truth model is sparse, however, we actually only require $O(k\log(n))$ forward passes where $k$ is the number of non-zero entries in the ground-truth model. In Figure 3(a), we illustrate empirically that even when the context is very long (hundreds of sources), the number of relevant sources (i.e., sources whose removal causes a probability drop beyond some threshold) is generally small ($<10$). As a result, in practice, just $32$ forward passes are enough to learn an accurate surrogate model even when there are hundreds of sources.

We thank the reviewer for their feedback. We address each question individually below.

Justification for linear surrogate modeling.

Our justification for using linear surrogate models for context attribution is purely empirical. A priori, we do not see any theoretical reason why linearity is the “right” design choice for context attribution. That said, across NLP tasks (summarization, knowledge extraction, reasoning over multiple documents), we empirically observe that a linear surrogate faithfully models how language models use their context (i.e., the surrogate model’s predictions match the actual log-probability drops, see Appendix B.1). Furthermore, by inducing sparsity, we are able to learn accurate linear surrogate models efficiently (with just $32$ context ablations, even when the context consists of hundreds of sources).

On non-linear surrogate modeling and sentence-level dependencies.

We agree that extending ContextCite to work with non-linear surrogate models is a promising avenue for future work. As the reviewer points out, surrogate model classes that factor in interactions among context sources (e.g., decision trees, linear models with interaction terms) could naturally account for the dependencies between context sources. Moreover, the expressivity of non-linear context attribution could help in capturing the effect of context ablations on model generations more accurately. In this work, however, we did not look into nonlinear variants of ContextCite for two reasons: First, as per our evaluation in Section 4, sparse linear surrogate models already result in accurate and sample-efficient context attributions. More sophisticated surrogate models may require many more context ablations to learn accurately. Second, linear models directly yield an attribution score for each source, making them interpretable. It would be less clear how to assign an attribution score to each source if the surrogate model were, say, a decision tree.