Uncertainty Quantification in Retrieval Augmented Question Answering

Weaknesses

• Indeed, the usefulness of a passage $p$ is defined on whether the model can correctly answer the question with the passage. We would like to clarify the computation of the passage utility score. It is effectively defined as the mean of the accuracy and entailment score, with the accuracy being a binary value, hence, a value from the set {0, 1}, whereas the entailment score corresponds to the posterior probability of the entailment class, hence, a value in the interval [0, 1]. We further take the average because initially we wanted to have a final utility score also in the interval [0, 1]. However, it would also be possible to take the summation of both, in the end, the Utility Ranker is trained with a ranking objective. The only requirement is that the utility score should be able to order passages according to their usefulness. For instance, passages that lead to an accurate response and have high entailment should have higher utility that those that have midle entailment and lead to an inaccurate response.

• 2. Additional results and explanation for Section 5.3

We added results for TriviaQA and SQuAD which show a similar trend (Table below).

It is a coincidence about the same increase. If incorrect answers with $|R|=3$ happen to turn to be correct with $|R^{urank}|=3$ and the answer string matches the gold answers then will also add up the same increments as the LLM one. For this experiment (data and models) maybe one metric is enough to show the gain.

Questions

• 1. and 2)

$m$ in Equation 1 is a hyper-parameter set to 0.1 in all our experiments.

In Equation 2, we compute the BCE loss for each pair of passages $p_i$ and $p_j$ in $R$. $\sigmoid(u)$ means applied to $u_i$ and $u_j$. We will rewrite this en the new version of the pdf for clarity.

• 2. and 3) references will be fixed in the new version of the pdf.
• 4. and 5) Choice of $R$.

We selected a small $|R|$ (i.e., $|R|=3$) because we want to evaluate the re-ranking by the utility ranker (and show the differences w.r.t. the original retriever ranking) on few top passages. If the number of top passages increases and is large then it is less visible the effect of re-ranking. The goal of doing re-ranking, is to improve performance with the smallest number of input passages possible. For in production QA systems, the smaller the context the better both for cost and latency purposes.

There is no (to the best of our knowledge) a study about what values of $|R|$ should be used. However, we chose $|R|=5$ for our main experiments based on the following facts. First, we follow most of existing work on retrieval augmented QA that uses $|R|=5$ (e.g., [1], [2], [3]). Second, LLMs may exhibit poor behaviour when reading long contexts, thus ([4]), the smaller and more precise the set of passages the better. Finally, as mentioned in the previous paragraph and inspired by authors' knowledge of real scenario practices in industry products, the smaller the number of passages the better.

[1] Chain-of-note: Enhancing robustness in retrieval-augmented language model
[2] RECOMP: IMPROVING RETRIEVAL-AUGMENTED LMS WITH COMPRESSION AND SELECTIVE AUGMENTATION
[3] Self-RAG: Learning to retrieve, generate, and critique through self-reflection
[4] Lost in the Middle: How Language Models Use Long Contexts

• 6. We add results for a model of different family but similar size, i.e., Mistral-7B-Instruct-v0.3. See response to Reviewer trrB.

Dear Reviewer, thank you for your useful comments and recommendations. We have submitted a response addressing all your points and would like to have your acknowledgement/feedback and continue discussions. Kind regards, Authors.

Thank you for the acknowledgement and feedback on our response.

• RE:W1 we will upload a new version of the pdf.

• $|R|=3 experiment$.

We want to clarify that the primary goal of this experiment is to show that the ranking by the Utility Ranker is better that the original ranking by the retrieval system. As we do not have gold passage order annotations to directly compare, for instance with Hit@N metric, we compare this via end-task performance (i.e., retrieval augmented QA accuracy).

• We reported accuracy for PopQA and RefuNQ in the main paper, L301. We will include these results in Table 1 to make them more visible.

Thank you for updating the pdf and answering my questions. The paper does look much better now. I will raise my score.

Also, minor points:

• the legend of Figure 2 seems to be broken. The colors of utility ranker and p(true), and those of semantic entropy and MSP are the same. Maybe the coral/orange color bar is for utility ranker, and the khaki color is for semantic entropy? You can try create a shared dictionary for each handle and their value for each subplot by fig.legend(all_handles_labels.values(), all_handles_labels.keys()).
• please keep the updated pdf within 10 pages :).

We thank you for your careful consideration of our responses, looking at the updated pdf, and the constructive feedback. Your feedback helped improve the quality and clarity of our work significantly. We have uploaded a pdf version with Figure 2's legend fixed and with content within 10 pages.

Weaknesses

• We clarify questions below and will upload a new pdf version of the paper.

• Inherent ability of the QA models to abstain.

We did not explicitly instruct the QA models to abstain. It has been shown in previous work that LLM-based QA models instructed to abstain struggle with decisions on when they should or not refrain from answering [1]. That is, often abstain from answering when they should have provided an answer and generate a response when they should have abstained. Thus, to simplify the assessment of answer correctness, we did not instruct the models to abstain. In addition, following previous work [2], we treated the few observed abstentions as cases of answer uncertainty. We provide the percentage of abstentions out of the total incorrect ones for each model and dataset for completeness.

[1] Examining LLMs' Uncertainty Expression Towards Questions Outside Parametric Knowledge
https://arxiv.org/abs/2311.09731
[2] Detecting hallucinations in large language models using semantic entropy
https://www.nature.com/articles/s41586-024-07421-0#MOESM1

• Connection to IR evaluation.

We will include the discussion of the "Evaluating Retrieval Quality in Retrieval-Augmented Generation" (SIGIR 2024) paper.

Questions

• 1. Range of utility scores.

Yes, utility scores based on entailment and accuracy are in the interval [0, 1]. These values are used to rank passages (best --higher-- to worse --lower--). However, although the ranking loss enforces the order constrain, it does not guarantees that predicted utility values will be in this interval. That is why predicted utility scores are $<0$ and $>1$ in Figure 1. Additionally, to enforce the signal on accuracy we add the auxiliary binary cross-entropy objective.

• 2. To train the Utility Ranker we use the predicted model answer given just the passage $p$ because predictions are at passage level.
• 3. Equation 3, margin loss. There is a typo, parenthesis are around i and js. The pytorch function used is MarginRankingLoss.
• 4. We report ablation experiments on variations of the Utility Ranker training objective in the following Table. The different variants are trained with Gemma2-9B-Instruct as the target QA model on Natural Questions.

where $e$ and $a$ are the entailment and accuracy scores, and $L_B$ is $\mathcal{L}_{BCE}$.

• 5. The pre-trained model is used to initialise the encoder of the Utility Ranker. Thus, the Utility Ranker architecture consists of a BERT-based encoder with a pooling layer and two MLP layers stacked on top of the BERT-based encoder outputs. The output layer computes a utility score.
• 6. Your understanding is correct. The idea was to reflect that $\upsilon$ was trained on data from the QA model $M$. As the notation is indeed misleading, we'll change to $\upsilon_M$.
• 7. We will include these curves in the Appendix.
• 8. We inspected a sample of size 50 for our task, in additional experiment we also used another critic model which agreed almost perfectly with Qwen2-70B used throughout our experiments. [1] originally inspected a sample of size 840.

[1] Head-to-tail: How knowledgeable are large language models (LLMs)? https://aclanthology.org/2024.naacl-long.18/

• 9. We mean different levels of detail, i.e., different amount of information. In these cases, the correct answers will not be clustered (despite being correct) what leads to falsely observing variation.
• 10. We report OOD evaluation for all combinations of training and evaluation data (See Additional Results in response to Reviewer trrB).

Dear Reviewer, thank you for your useful comments and recommendations. We have submitted a response addressing all your points and would like to have your acknowledgement/feedback and continue discussions. Kind regards, Authors.

• Additional base LLM, Mistral-7B-Instruct-v0.3.

• Additional experiments on distribution shift.

The first column indicates the train data, the first row indicates the evaluation data. Results in the diagonal correspond to the Utility Ranker trained/evaluated in the same data.

Weaknesses

• Related Work + Baselines

Thank you for these related work references. We will include a thorough discussion in our paper. Below we comment on them.

A common observation on approaches [1, 2, 3] is that none of them is applied to retrieval augmented QA; but instead to Reading Comprehension (RC), i.e., the task of generating an answer based on a positive (i.e., supposed to contain the answer) context document. Moreover, these approaches look at prediction at a single passage. In our work, we focus on the generalisation of passage utility to predict answer uncertainty (error) for retrieval augmented QA with a set of input passages.

In relation to [1] and [2]. Their calibrator is trained to predict answer correctness (i.e., a binary classifier) from a context document based on shallow features (e.g., document length) plus QA model's output probabilities [1] or embeddings [2]. They assess the calibrator on distribution shift cases. In their scenario, all input documents are useful. In our scenario, the utility of retrieved passages is varied. Our calibrator will learn diverse causes of uncertainty. We show performance on in-distribution as well as OOD and adversarial QA settings. Interestingly, [1] observes that their approach does not capture unansarable questions while ours provides the best performance in these cases.

In relation to [3]. This work relies on NLI models (off-the-shelf and QA-fine-tuned) to evaluate correctness of QA models' generated answers. The NLI usage and evaluation method in their work differs from ours. They focus on evaluating RC, i.e., after generating the answer, NLI is used to verify that the answer follows from the document. Instead, we use NLI as a metric to rank retrieved (potentially imperfect) passages. Furthermore, we train a secondary model to predict passage utility given a passage and user question (without the QA model generating an answer).

Note, that there are two situations in our retrieval augmented LLM-based QA task in which only NLI verification is not enough. The quality of retrieved passages is not guaranteed. If the retrieved passage is related but misleading (e.g., contains a confounder entity), the answer produced by the QA LLM model can be entailed by the retrieved passage yet do not be the correct one. Second, given the amount of memorised knowledge in LLMs, there are cases where even the input passage does not entail the answer but the generated answer is still correct (i.e., the passage does not contain the answer but positively primes the model to generate the correct one). In our ablation experiments (see response to Reviewer gKP3), we show that using only entailment as a passage utility indicator in retrieval augmented QA helps but is not enough.

[1] Selective question answering under domain shift Amita Kamath, Robin Jia, Percy Liang
https://aclanthology.org/2020.acl-main.503.pdf
[2] Knowing More About Questions Can Help: Improving Calibration in Question Answering Shujian Zhang, Chenyue Gong, Eunsol Choi
https://aclanthology.org/2021.findings-acl.172.pdf
[3] Can NLI Models Verify QA Systems' Predictions? Jifan Chen, Eunsol Choi, Greg Durrett
https://aclanthology.org/2021.findings-emnlp.324.pdf

• Evaluating role of the ranking loss and entailment score

The significant novelty of our work lies in the prediction of answer uncertainty (error) for retrieval augmented QA with $|R|$ input passages and that we do this from individual passage utilities. In the ablation experiments (see response to Reviewer gKP3), we show the impact of the ranking loss and the usage of NLI scores for the ranking signal.

• Poor generalization to LLAMA3

We report results with an additional base LLM of similar size but different family, i.e., Mistral-7B-Instruct-v0.3. Development results in the 'Additional Results' post.

We will add results on the same family but different sizes for Gemma2 in the new version of the pdf (we are currently running these experiments).

Questions

• Equation in L162 is the average of the accuracy and entailment score.

• Generalisation experiments with more combinations of train/test data and models.

We report additional experiments with all combinations of train/test data for experiments on distribution shift in the Table in the 'Additional Results' post.

We will add results for Llama-3.1 on the adversarial QA datasets (PopQA and RefuNQ) in the new version of the pdf.

Dear Reviewer, thank you for your useful comments and recommendations. We have submitted a response addressing all your points and would like to have your acknowledgement/feedback and continue discussions. Kind regards, Authors.

Dear Reviewer,

We have uploaded a new pdf version of our paper with further details (with major changes as outlined in our general comment).

As the extended discussion phase ends in three days, we kindly request confirmation of receipt of our response and updated pdf and welcome any additional feedback.

Sincerely, The Authors

Weaknesses

• We clarify notations below and will upload a new pdf version of the paper.

• Experiments on various sizes.

We add an additional model of a different family but similar size, i.e., Misral-7B-Instruct-v0.3 (See response to Reviewer trrB). We will add variants of different sizes, i.e., 2B and 27B, for Gemma2.

• Llama3.1-8b-instruct, Utility Ranker on par with p(True) approach (Table 3).

Our approach is an alternative that performs comparable (and sometimes even better) to strong but more expensive methods. P(True) is not a simple probability of True approach. To work well (as reported in our paper), it requires as many in-context examples as possible and a question that is formulated based on ten samples (as proposed by [1]). Thus, it requires (i) generating ten samples with a potentially big LLM and a long retrieval augmented context plus (ii) the final forward with a huge prompt (in-context examples and the actual question). See cost of each approach in response to Reviewer NaAL.

[1] Language models (mostly) know what they know https://arxiv.org/abs/2207.05221

Questions

• $m$ in Equation 1 is a hyper-parameter set to 0.1 in all our experiments. In initial experiments we search with values 0.01/0.001 but results were not better.

• Yes, i and js in the equation are for each pair. In the new pdf version we rewrite the equation for clarity.

• The accuracy $a$ is the observed accuracy of the target QA model on input question $x$ and passage $p$, ($x$, $p$). To train the Utility Ranker we generate training data with the target QA model.

Dear Reviewer, thank you for your useful comments and recommendations. We have submitted a response addressing all your points and would like to have your acknowledgement/feedback and continue discussions. Kind regards, Authors.

Thank you for considering our response. We have uploaded a new pdf version of our paper with further details (with major changes as outlined in our general comment). We kindly ask you to let us know if your concerns have been addressed.

Weaknesses

• Utility Ranker to predict on multi-hop reasoning.

In our paper, we focused on short-form QA and single hop reasoning. However, as you correctly point out in multi-hop QA, no single passage will have high utility. In this task setting, we expect that passage utilities will be middle/low for relevant passages and much lower for irrelevant ones. Thus, several points follow from this. First, this case highlights the need to train the Utility Ranker with a smoother score like NLI. Second, while we use a simple passage utility aggregation function to predict answer uncertainty (error) for retrieval augmented QA, passage utilities could be used as features of an answer uncertainty predictor. Third, our Utility Ranker could be useful for rearranging input passages to improve QA performance in multi-hop QA. We will run further experiment on HotPotQA [1] and include results in the final version of the paper.

[1] HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering https://aclanthology.org/D18-1259/

• Reference utility score to train the Utility Ranker.

We would like to clarify that we rely on smooth values as we include the entailment score, i.e., the posterior probability of the entailment class. As for the aggregation functions, we focus on a simple function as already this shows that it is possible to obtain comparable performance to other more expensive uncertainty estimation methods. However, it would make sense to define more complex aggregation methods, e.g., training a confidence estimator based on the set of passage utilities together with other features like the Maximum Sequence Probability (MSP).

• No other re-ranking baseline.

The main evaluation of the Utility Ranker is extensive covering in-distribution, our-of-distribution, and adversarial QA in various datasets and models as well as ablation studies. The main focus of our work, is on predicting answer uncertainty (error). Thus, we evaluate this w.r.t. to strong baselines. As an additional experiment, we show that the Utility Ranker also brings added value to improve QA performance on top of the original ranking provided by the Information Retrieval (IR) system. Thus, we do not include baselines/comparison systems in this additional experiment. Nevertheless, we could include in the final version experiments with other re-ranking baselines for completeness.

Questions

• L200 argmax.

Yes, there is a typo, it should be max(.), i.e., take the maximum predicted utility.

• Inference time difference between approaches.

There is indeed a difference in latency at inference time. Below we detail the number of forward passes (and what type of forward call) required by each approach.

Where LLM-G means answer generation with a QA prompt ( $|R|$ passages and question), LLM-E means evaluation with a verification prompt (including as many in-context examples as possible, question, and candidate answers). Bert-F means an Utility Ranker forward on passage and question to obtain the passage utility score. size-of(R) means $|R|$.

Dear Reviewer, thank you for your useful comments and recommendations. We have submitted a response addressing all your points and would like to have your acknowledgement/feedback and continue discussions. Kind regards, Authors.

Dear Reviewer,

We have uploaded a new pdf version of our paper with further details (with major changes as outlined in our general comment).

As the extended discussion phase ends in three days, we kindly request confirmation of receipt of our response and updated pdf and welcome any additional feedback.

Sincerely, The Authors