LLM-Cite: Cheap Fact Verification with Attribution via URL Generation

We thank the reviewer for the insightful comments and feedback!

Restricted to Wikipedia: Although the authors demonstrate the proposed fact verification approach is effective, the experiments are only conducted on Wikipedia. There are concerns whether this method could be extended to beyond this limited knowledge base.

We have tried to address this in the general response.

Unfair baseline comparison: First of all, the authors did not use the same LLM for their method and baseline method, which is unfair. Additionally, for FactScore, the whole point of the paper is decomposing long texts into atomic facts and then evaluating factuality. However, due to the experimental setup, this decomposition part seems removed. Therefore, it may be unfair for the authors to say they “beat the current best method (FactScore) on Biographies”.

All the baselines and our method use the same NLI model. For P(True), we used gemma 7B and all result figures also include a comparison to our method with gemma 7B for URL generation. Compared with the same model, LLM-Cite is better than P(True) across all datasets while also providing attribution which is lacking in P(True).

We do agree that breaking down long texts into atomic facts is an important point in FactScore. What we are showing is that after breaking down into atomic facts, we can do URL-generation instead of retrieval and it performs equally well if not better, while having other benefits like cheaper and freshness. So the comparison is to the retrieval part in FactScore while keeping the other components the same (breaking down into atomic claims, and NLI). In fact, the error rate reported in table 3 in the FactScore paper is over atomic claims — exactly the same metric we report (1 - error rate) in our results.

Dataset issue: First, while the author tested on three datasets, most of these dataset are relatively small.

We do agree that our datasets are not very big — for FreshQA specifically, we used a smaller set since we manually wrote answers to make sure they are up to date.

Second, for the ASQA dataset, the author uses a NLI model to ensure data quality. However, as seen in Figure 4 right, the NLI model performance is only somewhere around 85%.

We want to clarify that the 85% number shown in Figure 4 is the end-to-end verification accuracy and not NLI accuracy — the 15% error includes cases where the URL generated was valid but not useful to verify the claim, and hence is not always NLI error.

Citation threshold: What does this mean? Does it mean > 0.6 is entailment?

Citation threshold is used by Check Grounding, the NLI model we use for verification – https://cloud.google.com/generative-ai-app-builder/docs/check-grounding. It corresponds to a threshold for the confidence above which we say that the claim is entailed.

We thank the reviewer for the insightful comments and feedback!

The paper's assumptions about URLs have limited generalizability. Although the paper clarifies that it can be applied to any web domain, intuitively, its application is primarily focused on verification that has a high degree of regularity in URL structure, such as Wikipedia. It is difficult to implement for real-world claim verification like PolitiFact or gossip, and it remains in an ideal stage.

We have tried to address the use of Wikipedia in the general response.

Factual verification claims often exhibit high semantic complexity and typically require multi-hop evidence for verification, which the proposed method clearly struggles to achieve.

This is a valid point and we will acknowledge it in the paper. However, it is also an orthogonal issue in the following sense. Our key point is that LLM-cite + lookup could be a drop-in replacement for search+retrieval for fact verification. Both approaches would struggle equally with verifying claims requiring multi-hop reasoning. Furthermore, the datasets we use in this work are realistic datasets which have been used in prior work for fact verification. For larger claims (e.g. long answers in Biographies), each long response is first broken down into atomic claims which are less likely to require multihop reasoning.

URLs are extremely sensitive to characters, and the paper does not handle invalid URLs reasonably and does not provide experimental results for valid URLs.

Section 4.4 gives a detailed analysis of when and how many of the generated URLs are invalid. We generate multiple URLs per claim, and there are close to 0% of claims where all of the generated URLs for the claim are invalid (Figure 4 left) — this means we always have some valid URL against which we verify the claim.

As a benchmark dataset for factual verification, why not validate on datasets like FEVER, HOVer, and FEVEROUS, which use Wikipedia as an external knowledge base?

Our motivation was to show that for existing factuality evaluation methods like FactScore, we can replace the retrieval part with URL-generation and it does equally well (if not better) — hence we used the same dataset as in FactScore. To show that URL-generation gives additional benefits such as verifying fresh claims, we decided to use the FreshQA dataset.

We thank the reviewer for the insightful comments and feedback! One general point we wanted to bring up, especially in regards to the comments about NLI is that we build off of existing well-established methods such as FactScore, and our main idea is to replace the retrieval with URL generation (which has advantages as we demonstrated) — we keep all other parts of the pipeline such as use of Wikipedia and NLI the same.

We try to address some of the specific concerns which were raised:

The assumption that generated URLs are always meaningfully related to the core content of the document from where the premises are to be fetched is not true by and large. It works for Wikipedia because the URLs are well-structured semantically.

LLMs generating URLs on Wikidata have a significantly higher probability of being linked with a valid URL because extensive entity linking has already been done. This, however, is not the case for many other web sources.

We have tried to address the concern regarding Wikipedia only in the general response.

There are several URLs that are not named according to the premise entities. In that case, those sources will never be fetched.

We have also tried to address this in the general response.

How to resolve contradictory entailment from premises belonging to different sources?

There can be many sources that are themselves false (particularly for the open Internet and also in cases of unverified Wiki pages). So assuming the premises to be true may lead to incorrect RTE.

That is a really good point, and we followed other works such as FactScore in making the assumption about contradictory premises (assumption 3 in their paper section 3.1). In our method, even if any source entails the claim, it will be verified and because LLM-Cite gives attribution, the user can determine if it’s a trustworthy source (e.g. maybe a user is less likely to trust a wikipedia page which was not thoroughly verified). In general, we envision a setting where the user gives a set of documents that they trust, and our method can be constrained to use those documents only.

A discussion on the prompt dataset (for the few-shot case) creation together with its source should be discussed.

Thanks for pointing this out, we will add more discussion on how we designed the few-shot prompt. We want to note that we did not actually do much prompt engineering — a simple, intuitive prompt with a few diverse examples already works well.

Line 253: What is citation threshold? I could not find the definition.

Citation threshold is used by Check Grounding, the NLI model we use for verification – https://cloud.google.com/generative-ai-app-builder/docs/check-grounding. It corresponds to a threshold for the confidence above which we say that the claim is entailed.

In the comparisons with the baselines and variants of LLM-Cite, what was the justification behind not keeping the model set fixed for all the experiments? I think this should be clear.

All the baselines and our method use the same NLI model. For P(True), we used gemma 7B and all result figures also include a comparison to our method with gemma 7B for URL generation. Compared with the same model, LLM-Cite is better than P(True) across all datasets while also providing attribution which is lacking in P(True).

In sections 4.1 and 4.2, an analysis of why the verification models perform better on model-generated claims as compared to human-generated claims is very important to me.

This observation comes from our difference performance on ASQA (model generated claims) and Biographies (human generated claims). However, we did not want to call this strongly as the two datasetsalso differ in other things such as domain, average length of query etc, so there could be potential confounders that affect our performance on the claims.

The key success of LLM-Cite depends on the NLI model (given that at least one valid URL is generated that points to a valid premise). Hence, a discussion on the accuracy of SOTA NLI models (with citation) and the rationale behind choosing the Check Grounding NLI API and Gemini-1.5-Flash should be included.

We will make sure to add more discussion around this in the paper. Check Grounding NLI model is a cheaper alternative to prompting larger models for entailment since it was specifically finetuned for NLI. However, we are consistent in using the same NLI model for all approaches (our method, and baselines) and don't expect trends to change with any other NLI model. We discuss this in section 4.4 where a stronger NLI model improves performance but is costlier than Check Grounding NLI model.

We thank the reviewer for the insightful comments and feedback! We try to address some of the concerns which were raised:

My main concern is the novelty of the paper.

Thanks for pointing out the relevant work (we were not aware of this work before)! We will make sure to cite it. Even though both our work and that paper rely on generating URL we do want to emphasize a few key differences: (a) we focus on the freshness aspect of the URLs, where we show that it can help verify fresh claims; in fact, we show that in a few cases the generated URL is beyond the knowledge cutoff of the LLM (e.g. Figure 1) ; (b) we focus on fact verification and evaluate on corresponding datasets and compare to relevant baselines (e.g. FactScore) whereas the goal of the other paper was the retrieval part in QA — thus our work shows the applicability of generating URLs for a different task too.

Wikipedia URLs follow an entity-based template of https://en.wikipedia.org/wiki/<entity_name> which simplifies the task of generating relevant URLs to mainly just identifying the relevant entity in the claim to be verified.

We have tried to address this concern in the general response.

While the authors show cost comparison, there is no analysis done on latency comparison.

We also briefly discuss latency in the paper (line 518 - 521). The estimated latency for LLM-Cite is small enough that it can be used in a production system too. We expect the total latency to be in the same ballpark as Google Search + NLI.

While Google Custom Search is expensive at 5$ per 1000 queries, there are cheaper alternatives like https://serper.dev which offer a pricing varying from 1 per 1000 queries.

We want to note that APIs like SerpAPI rely on scraping search results, which will be unintended use of Google Search and not compliant with enterprise requirements. Hence, we decided to use Google Custom Search for our experiments and for cost comparison.