SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Models

We thank Reviewer PWPX for the constructive comments!

…different notions of "citation" targeted by ContextCite on the one hand and evaluated by LongBench-Cite (and similar benchmarks) on the other hand.

Thanks for pointing out the mismatch between the objectives of corroborative (sources that support a statement, e.g., LongCite & LongBench-Cite benchmark) and contributive context attribution (e.g., ContextCite). As you said, SelfCite applies contributive alignment (using context ablations) to a method for corroborative evaluation (LongBench-Cite). Our intuitions are:

1. Among citation candidates proposed by LongCite, the candidates actually “used” by the model are also more likely to “support” the statement—other candidates may just be semantically related. Although such support is certainly not always guaranteed, the two objectives are still aligned to some extent.
2. Current corroborative methods (LongCite) have a big room to improve, even by a contributive method, despite their discrepancy in goals. In other words, if LongCite were already near perfect, enforcing it to be “more contributive” may not help much.

We will discuss this nuanced point more clearly throughout our paper!

…why the text implies that best-of-N sampling has to be done "after generating the full response" (lines 154-155). Could it not also be done after generating each statement $r_i$?

Yes, BoN only needs each statement $r_i$ before sampling citation $e_i$. Our implementation here is mostly for convenience. We generate a full response to get $r_1, ..., r_S$ first, then re-sample each $e_i$, considering the weak dependence between $r_{>i}$ and $e_i$.

In re-sampling citation sequences in position $e_i$, are the future statements and citations $r_{i+1}, e_{i+1}$,… removed? The notation in Algorithm 1 suggests yes?

Yes, future statements aren't used when sampling citation $e_i$.

Lines 132-134, right column, "additional inference cost of generating candidates and re-ranking": This inference cost should be quantified…

We measured latency per example (8*A100 GPUs, batch size 1, model parallelism) on LongBench-Cite. Direct decoding of LongCite-8B vs. SelfCite SimPO shows similar latency. BoN sampling+reranking is ~7x slower. We'll add this to our paper.

What constitutes a statement $r_i$ in the response? How is the response divided into statements?

In LongCite-45k, statements are split based on semantic integrity in data generation, so fine-tuned models will naturally learn to produce statements. We'll clarify this in our paper.

Lines 112-113, right column, "we exclude the BoN candidates that cite more than 384 tokens in total": I believe this refers to the set of context sentences $c_{e_i^1}, ..., c_{e_i^m}$, not the sequence of identifiers $e_i^1, ..., e_i^m$…

Yes, the length limit was applied on the cited texts, not identifiers. We'll clarify this in our paper.

Equation (1): Were weighted sums of the probability drop and probability hold metrics also considered?

Good suggestion! We only tested 1:1 weights for simplicity but will explore this further.

Table 2: Are the numbers in the second Llama-3.1-8B-Instruct row better than the ones in the first Llama-3.1-8B-Instruct row because the second one is for answering with citations?

After carefully checking our experiments, we found the second Llama-3.1-8B-Instruct row (avg 71.7) was actually mistakenly taken from the ContextCite result, which uses greedy decoding and answering without citations, thus is not directly comparable.

We reran the experiments and show full results in the table below. The first Llama-3.1-8B-Instruct row in Table 2 of our paper should be updated with row (2) below (avg 68.9). Its original scores with "†" (avg 60.2) in Table 2 of our paper are taken from Table 3 in LongCite paper and thus have some prompt/implementation differences (they didn't open-source this part of code). We will update it to our own results for now. The second Llama-3.1-8B-Instruct row should be updated with row (4) (avg 63.3).

In summary, answering with citations hurts accuracy (68.9 -> 63.3), which is expected and consistent with the same trend from Table 3 in the LongCite paper (all non-LongCite models have such degradation when asked to answer with citations.) We'll update Table 2 of our paper.

The Limitations section could also note that SelfCite assumes access to the LLM's predicted probabilities…

Additional references on contributive attribution: …

We'll add these points and references to our paper. Thanks again for the valuable suggestions!

We thank Reviewer Cy66 for the constructive comments!

Ours vs ContextCite

The proposed method makes sense, but not entirely novel, ablation technique has been proposed by ContextCite

While inspired by ContextCite (CC)’s context ablation (L029, right column), our key contribution differs notably: SelfCite enables an LLM to directly generate its own citations, while CC does not.

CC is post-hoc, relying on external linear models trained from scratch for each new example, requiring heavy inference per example (32 to 256 ablated generations) after a response was produced.

In contrast, SelfCite directly teaches LLMs to produce accurate citations in responses with a few tokens in one pass. The context ablation signals become internalized capabilities of LLMs after SimPO; no context ablations needed at inference.

Extra distinctions:

• CC measures citation “necessity” by “prob drop”; SelfCite adds “sufficiency” by “prob hold” to catch missing citations.
• CC’s linear model assumes context sentences independently impact responses; SelfCite directly learns from nonlinear “prob drop/hold” rewards, handling sentence interactions.

CC’s details were in Appendix B due to space. We'll move them to the main text.

Answer Accuracy Drops?

Table 2, it seems answer accuracy drops on Llama-3 which is concerning. (avg 71.7 vs 64.6)

Good catch! We carefully checked our experiments and found the high score (avg 71.7) of Llama3 baseline under “Answering with citations” was mistakenly copied from our ContextCite (CC) experiment. We reran the experiments to get a corrected baseline of avg 63.3 in row (4) of the table below. Compared to this correct baseline, both our + SFT on CC (avg 64.6) and + SimPO (Ours) (avg 64.7) in fact show slightly higher accuracy. We apologize for the mistake.

Why is CC higher? (avg 71.7)

CC’s higher score (71.7) comes from:

1. CC uses greedy decoding; we follow LongCite [1] to use sampling (top_p=0.7; temp=0.95). See rows (1) vs (2): avg 71.7 vs 68.9
2. CC’s citations are post-hoc, so it’s “answering without citations”, making answer generation easier and more accurate. See rows (1) vs (3): avg 71.7 vs 64.8

The correct baseline to be used is Sampling + Answering with citations in row (4): avg 63.3, which is slightly better than the same results from LongCite ([1], Table 3, row Llama-3.1-8B, column C’s). And it confirms no accuracy drop after our fine-tuning (avg 64.6 & 64.7). We’ll update Table 2 in our paper.

[1] LongCite: https://arxiv.org/pdf/2409.02897

More RL algorithms?

…beneficial to investigate these RL algorithms...

Our main goal is to validate a novel "reward" for citation. Prior work ([2], Figures 4 & 5) shows Best-of-N (BoN) closely approximates the upper-bound scores of RL without training artifacts to make comparisons more confounded. Following established practices [3, 4], we used BoN as main evaluation, and further verified it using training-based alignment, SimPO, achieving the same improvement of BoN. While extra RL algorithms may offer improvements, we believe they wouldn’t qualitatively change our observations given BoN results.

We also acknowledge SelfCite doesn’t aim to boost answer accuracy (but not to decrease it either). Combining it with answer-matching rewards to jointly improve citations & answers is an exciting direction we want to explore in future work!

[2] Controlled Decoding from Language Models, Mudgal et al., ICML 2024

[3] Scaling laws for reward model overoptimization, Gao et al., ICML 2023

[4] Let’s Verify Step by Step, Lightman et al., ICLR 2024

More Benchmarks/Baselines?

…other benchmark datasets, such as ALCE... …both Huang et al., 2024 and Gao et al., 2023 have proposed … there are no comparisons in experiments.

Following your advice, we evaluated on ALCE and compared to Huang et al. 2024 & Gao et al. 2023. Due to space, please see our rebuttal to Reviewer WRpv. Spoiler: SelfCite outperforms baselines on ALCE even on cross-domain transfer!

More Discussion?

lack of detailed discussion on comparison against existing approaches such as Zhang et al., 2024 (LongCite) & Cohen-Wang et al., 2024 (ContextCite).

Comparison with ContextCite is at the top of this reply; will be added to our paper.

Comparison with LongCite is in our Section 5. Briefly, LongCite uses data from proprietary APIs for SFT only. SelfCite performs further alignment steps without external supervision.

We also made a table to contrast key distinctions among prior works; see our rebuttal to Reviewer ci8i.

We thank Reviewer WRpv for the constructive comments!

A Better Baseline: SimPO with NLI Rewards (Also for Reviewer Cy66)

…ContextCite baseline is not appropriate. … [1] has been proposed to leverage NLI models to measure citation precision / recall, which seems to bit a more appropriate baseline

We agree that ContextCite’s mechanism is much different from SelfCite, so we will change our framing and treat its scores mainly for reference. We follow your advice to adopt NLI rewards from Huang et al. 2024 [1] as baseline. For fair comparison, we reuse our SelfCite SimPO training pipeline (initializing from LongCite-8B + trained with LongCite-45k data), but only change the reward function to be the NLI-based citation recall/precision proposed in Huang et al. 2024 [1]. We ignore the correctness reward in [1] as we don’t have ground truth answers from LongCite-45k.

We compare this method (SimPO w/ NLI Rewards) with ours (SimPO w/ SelfCite) on both LongBench-Cite (table below) and ALCE (table in the next section). Both results show SimPO w/ NLI Rewards improves citation quality over LongCite (except for MultifieldQA & HotpotQA), but still consistently outperformed by SelfCite, further verifying the effectiveness of SelfCite. We will include this baseline in our final version paper.

Evaluation on Chunk-level Citation Benchmark ALCE (Also for Reviewer Cy66)

previous method [1] can be adopted to chunk-level citation. Therefore, it would be better to also include experiments are datasets such as ALCE [0].

We follow your advice to test our models on ALCE and show the results in the table below. We found that our baseline LongCite-8B already achieves much better citation recall/precision than the prompting method of Gao et al. (2023). The baseline “SimPO w/ NLI Rewards” (using the rewards from Huang et al., (2024) above) performs slightly better than LongCite-8B. Our method, “SimPO w/ SelfCite”, further brings substantial improvements over both baselines.

The bottom row is the best result from the supervised method of Huang et al. (2024). It differs from other model settings and was trained on in-distribution data. Its numbers are thus incomparable with the other rows and we only include them for reference. Specifically, the differences are:

1. They train the models only on the “in-distribution” training sets of QA datasets in ALCE, with the exact same chunk-level setting of ALCE, while SelfCite was trained on “out-of-distribution” LongCite-45k data with sentence-level citations.
2. They directly use the same NLI evaluator used in ALCE benchmark (google/t5_xxl_true_nli_mixture) to provide rewards for citation recall/precision, essentially optimizing the benchmark scores of ALCE directly.
3. They also do distillation from ChatGPT.

Despite the fact of a cross-domain & cross-setting transfer setting, SelfCite still achieves good performance much better than baselines: LongCite-8B & SimPO w/ NLI Rewards, showing its effectiveness. We will include this result in our paper.

How does SelfCite handle unfaithful answers?

are there cases where the generated answer is not faithful (and thus can't be supported by the context) and how does SelfCite deal with that?

We did sometimes (but not often) find that the answer can be a slight misunderstanding of the cited information, and SelfCite will still cite such text that it is based on. Stepping back, this is a common theme for all methods that post-hoc generate citations, where the prevailing philosophy is that “citations” are for the traceability and verifiability of answers, which enable a user to double-check the answer correctness easily, in case the answer is wrong. We will include this discussion in our final version paper.

We thank Reviewer ci8i for the constructive comments!

A Comparison Table (Also for Reviewer Cy66)

The paper should more explicitly differentiate SelfCite from previous work… A comparison table summarizing key differences could be helpful.

We follow your suggestion to make a table and point out the key differences among the previous work. We will include it in our final version paper.

A Toy Example of Necessity and Sufficiency Scores

The necessity and sufficiency scores are well-motivated, but additional theoretical justification or a toy example demonstrating their individual impact could strengthen the argument.

The necessity and sufficiency scores are designed based on human’s common preference: a citation has to be necessary and sufficient, which also matches the metrics of citation precision (necessary) and recall (sufficient) commonly exist in the evaluation benchmarks. Here we follow your advice to show a simple toy example demonstrating their individual impacts:

Document:

[1] Alice traveled to France in 2020.
[2] Bob visited the famous National Museum in Tokyo, Japan in 2019.
[3] Chloe visited the Louvre Museum in Paris, France in 2018.
…

Query:
"Which famous museum could Alice have visited?"

Response:
"Alice could have visited the Louvre Museum."

Citation Candidates:

• [1,2] (Incorrect):

  • Necessity: Probability drops (since removing [1] prevents “Alice traveled to Paris”). (✅ high necessity due to [1])
  • Sufficiency: Probability doesn’t hold. [1,2] alone cannot fully support “visited the Louvre Museum” since [2] is irrelevant and [3] is missing. (❌ low sufficiency)

• [1,3] (Correct):

  • Necessity: Probability drops more; removing [1,3] loses essential details (“Alice traveled to Paris,” “visited Louvre”). (✅ high necessity)
  • Sufficiency: Probability holds. [1,3] Fully supports the response. (✅ high sufficiency)

This toy example clearly shows the individual contributions of necessity and sufficiency scores.

Discussion on Efficiency

Since best-of-N sampling increases inference-time costs, a discussion on its efficiency trade-offs and potential ways to reduce overhead (e.g., pruning low-quality candidates early) would be beneficial.

We calculate the latency per example on the LongBench-Cite dataset. On average, direct decoding from LongCite-8B/SelfCite SimPO models have similar latency. When using SelfCite BoN, the sampling + reranking steps in total take roughly 7x longer time compared to the direct decoding methods. All experiments are done with 8*A100 GPUs with batch size 1 and model parallel.

Also, because we only sample the citation sequence, not the whole responses, the number of generated tokens is very limited, usually within 5~10 tokens. The strategy of pruning low-quality candidates early may not help, as they are mainly for saving time in generating long responses.

Latency of BoN is not a major concern

In fact, we do not have concerns on the longer latency or extra inference cost of BoN, because we also have the SelfCite SimPO model that can achieve the same performance of BoN in one pass generation, without any additional inference cost. If the users have the need for best efficiency, the best solution is to directly use our SimPO model, instead of using our BoN and trying to optimize BoN.

Exploring Hyperparameters

The effect of hyperparameters like N in best-of-N sampling and the choice of probability thresholds for necessity/sufficiency scores should be explored more systematically.

There are no “probability thresholds” for necessity/sufficiency scores in SelfCite. We use the raw probability changes (probability drop and probability hold) during context ablation directly as the reward. There is no need to tune any thresholds in our reward design.

For N in best-of-N sampling, as we mentioned in Line 217 (left column), after deduplicating repeated citation candidates, on average there are only 4.8 candidates (std=3.2) left per statement. It is due to the fact that we only do sampling within the citation sequences, and keep the statements in the response unchanged. When generating citations, usually only a limited number of relevant sentences can support the statements, resulting in a limited possibility of citations to be generated. Given the low diversity of citation candidates, increasing N to be larger than 10 would have a very limited impact on the BoN results.