Precise Information Control in Long-Form Text Generation

We thank reviewer 8i1m for the thoughtful assessment! We are encouraged that they like this paper, and that they find the manuscript “well-written,” and the experiments “thorough and well-motivated.” Thanks for catching the typos as well! Responding point-by-point:

“Check the definitions of intrinsic and extrinsic hallucinations…consider using the terms "faithful" instead of "intrinsic" and "factual" instead of "extrinsic.”

We agree that intrinsic and extrinsic hallucination have particular meanings in prior work [7,8]: (i) intrinsic denotes outputs that contradict the source, and (ii) extrinsic denotes outputs that cannot be verified from the source (and may or may not be true in reality).

To avoid confusion and align with reviewer suggestions, we'll revise our terminology to using “faithful” instead of “intrinsic”, and “factual” instead of “extrinsic.” This means that PIC evaluates faithfulness with respect to supplied verifiable claims; improving faithfulness also improves factuality. We'll update the text and expand the discussion in App. A.3.

“[Include] qualitative results showing (1) why a perfect match is so rare, especially in the PIC-Bench-Full setting, and (2) why instruction-following performance decreases [after SFT].”

For (1), we find that a perfect F1 is rare due to distortion errors (where the model rewords claims in ways that distort the original meaning) and fabrication errors (where the model “fills in the blanks” by introducing unsubstantiated information). Please see the categorical error breakdown in our response to reviewer pcWR; we'll also include qualitative examples in our revision. We also observe that claim recall tends to be lower than precision, particularly when the sample has many claims, which harms the F1. This is a known problem, as LMs face a lost-in-the-middle effect and struggle to attend to long contexts.

For (2), we notice that instruction-following performance decreases after SFT as the LM learns to optimize for PIC at the expense of following instruction constraints, e.g., not following word counts.

Here is an ExpertQA example (with a Prometheus score of 3/5). While this response provides a basic overview, it fails to abide by the pros-and-cons format required by the instruction:

Instruction: Give me an overview of the free granular synthesizer software with pros and cons for each option. [Claims omitted]

PIC-LM SFT Output: 
Here are some options for free granular synthesizers
1. Granulab - This program has many features including real-time audio processing
2. Argotlunar - Another feature-rich granulator
3. Hourglass - A good choice for those who want more control over their sound
4. Fragment - Great for collaboration or live performance
5. PaulXStretch - For advanced users looking to push the limits of time-stretching
Each one has its own strengths and weaknesses so it really depends on how you plan to use them.

“Doesn't [XSUM’s "extreme summarization"] conflict with the paper's focus on "long-form text generation"?”

While we use the data from XSUM, we do not adopt its one-sentence “extreme” summarization protocol, and instead prompt for standard summaries. We consider this a long-form generation task as we could extract a sufficient number of verifiable output claims; e.g., the PIC XSUM task has 30.6 avg. input claims, and PIC-LM 8B responses average around 21.8 claims. We’ll clarify the misleading task name in the revision.

“Why did you sample the y_perturb from θ_SFT, not superior models?”

We sampled $y_{perturb}$ using $θ_{SFT}$ as it’s a cheap proxy for API-based LLMs. We reuse SFT data for preference data construction, so these perturbed responses already exhibit strong PIC behavior as $θ_{SFT}$ has seen similar samples. To empirically validate the quality of $y_{perturb}$, we randomly sample 50 full and partial PIC samples from our preference data where the perturbed sample is preferred, and find high PIC scores (84.3% F1 for full PIC, 86.5% precision for partial PIC); these numbers are competitive with GPT-4o sampling. While not perfect, these signals are still valuable for preference tuning, as recent work (the Delta Learning Hypothesis [9]) suggests that the relative delta between chosen and rejected responses is more important than absolute response quality.

“[For using] the last L tokens for score computation…couldn't the length bias be mitigated by using a different denominator?”

Following [10], we compute the avg. per-token log-likelihood over the last L=20 tokens of each response to reduce computational cost and length bias; in this setting, the shorter response is chosen 49.0% of the time.

We ran a new ablation where we recreate preference data by computing the per-token log probability over the entire response instead. This results in a length bias with the shorter response preferred 60.2% of the time. Training on length-imbalanced preferences leads to worse avg. PIC performance (91.0% vs. 91.4% for full, and 92.6% vs. 93.7% for partial PIC) and Prometheus scores (3.89 vs. 3.92). Thus, we retain last-L-token scoring for its balance of efficiency and performance. Alternative normalization strategies, e.g., length-aware scoring, or an adaptive L, may improve robustness; we leave this to future work.

“The train and test sets do not overlap [and] it would be better…not to use FACTS and biography generation datasets for training.”

Thanks for raising this point! First, we emphasize that the PIC-LM training and PIC-Bench evaluation datasets are completely disjoint; there’s no overlap between training and test splits.

Second, our evaluation is designed to measure both in-distribution (ID) and out-of-distribution (OOD) generalization. We include EntityBios and FACTS in PIC-Bench to assess ID performance for PIC-LM, which had exposure to these tasks during training. The other tasks (along with a biomedical task in App. C.7) serve as OOD evaluations, allowing us to measure performance on unseen domains. PIC-LM shows good OOD performance (90.4% avg. OOD F1, competitive with GPT-4o at 90.3%). We'll make the ID/OOD distinction clear in the revision.

Lastly, EntityBios and FACTS contain significantly more claims (51 and 64 on average, respectively) and are more challenging than other domains. High‑quality instruction data with long, rich contexts is scarce and expensive to curate, so these domains are instrumental to improving PIC.

(Note that the ID/OOD PIC-LM numbers cited above reflect updated training results after stricter data quality control.)

“It would be helpful to compute the ChatGPT–Human agreement score.”

We ran human evaluations; please see our response to reviewer Dux5 for setup details. For claim extraction, the LLM-human agreement averaged 97.5%, with inter-annotator agreement at 94.1%. For claim verification, the LLM-human agreement is 81.0%, with substantial inter-annotator agreement (Fleiss kappa=.694).

“It would be better to exclude POPB-CF from the average calculation…[the] average score gap appears to come from this benchmark.”

Thanks for this perspective. PopB-CF is intentionally designed as a stress test: it modifies biographical claims to conflict with models’ parametric knowledge to assess whether LMs can follow explicit user instructions, even when doing so contradicts their internal priors. This scenario simulates realistic safety-relevant settings, such as updating outdated facts or enforcing user-specific overrides. In these cases, instruction-following behavior is desirable, even if it overrides a model’s factual default. To ensure a fair comparison, we agree it’s valuable to report both the average with and without PopB-CF. In our revision, we'll include both versions to make these tradeoffs transparent. Without PopB-CF, the top-3 highest avg. PIC F1s are GPT4o (93.7%), PIC-LM 8B (93.0%), and Llama 3.3 70B Inst. (89.9%), and PIC-LM 8B shows similar Prometheus scores to Llama 3.1 8B Inst. (3.94 vs. 3.99).

“[The] implications [that user-provided knowledge should supersede the LM's inherent priors] are not thoroughly discussed.”

We appreciate the reviewer’s concern. Our work aims to make LMs more controllable by giving users finer-grained command over their generations.

Stronger control over model outputs may indeed exacerbate misuse. PIC-LM is more capable of generating counterfactual or harmful content if such claims are passed as input, and may override abstention behaviors instilled via RLHF. However, this tradeoff should be contextualized. Uncontrolled toxic generation, arising from model hallucinations, can be harder to detect and filter. By contrast, PIC-LM makes the provenance of outputs attributable to user input, allowing developers to place safeguards upstream.

Our contribution follows work showing that safety alignment is shallow and brittle [11]. PIC-LM shifts the safety locus from the model’s parameters to auditable inputs. This can lead to more robust safety systems, such as pairing PIC-LM with a pre-hoc safety classifier (e.g., LlamaGuard) that screens for malicious claims. Therefore, while PIC-LM may not be the safest model in isolation, we see it as a promising building block for safety-aware systems where user-grounded control is critical.

We'll expand the limitations and broader impacts (App. A.4) in our revision.

Citations

[7] Maynez et al., 2020. On Faithfulness and Factuality in Abstractive Summarization

[8] Huang et al., 2025. A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions

[9] Geng et al., 2025. The Delta Learning Hypothesis: Preference Tuning on Weak Data can Yield Strong Gains

[10] Chuang et al., 2025. SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Models

[11] Qi et al., 2024. Safety Alignment Should Be Made More Than Just a Few Tokens Deep

We thank reviewer UZLz for the constructive feedback. We are encouraged that the reviewer finds the breadth of datasets covered in PIC-Bench to be “commendable,” and that our paper is “well-written [with] exhaustive experiments in several directions.” Responding point-by-point to their comments:

“[PIC] is very similar to RAG & attributable generation (without the need for explicit citations)...[full PIC] would be achievable in the attributable generation setting by modifying the input instruction prompt to direct the LM to include all source refs as well.”

Although PIC shares surface similarities with RAG and attribution, this framework is fundamentally about information control—ensuring that every output claim is traceable to user-provided input. This goal differs from that of RAG and attribution generation, both of which primarily aim to improve factuality. We show that PIC is surprisingly difficult for LMs; when provided with exactly which claims to include, even frontier LMs may drop, distort, or fabricate content. This fundamental control problem motivates a dedicated evaluation framework, benchmark, and training techniques.

While PIC, RAG, and attributable generation all require grounded responses, we note several key differences. First, PIC enforces full coverage of all input claims, while RAG allows for information omission if final answers are correct. Second, attribution systems often work at the sentence or passage level, while PIC operates at verifiable claim level, enabling a finer-grained detection of distortion, fabrication, and conflation. Prompting an LM to “include all sources” can only approximate this in a best-effort way, and cannot guarantee coverage of every input claim. Lastly, PIC has broad task coverage. While RAG and attribution generation primarily focus on QA, PIC is more universally applicable to a range of generative tasks.

Improvements on PIC may also benefit related tasks. We demonstrate strong factuality gains in our RAG experiment (Section 6.1), and leave attribution generation to future work. Notably, PIC-LM outperforms Self-Cite 8B, a strong attribution generation LM, across both partial and full PIC settings (our RAG and self-verification pipeline case studies).

“[PIC] expects claim extraction as a necessary step to be conducted from typically noisy input source material…This creates a large dependency on [the] efficacy of the claim extractor model, and assumes that only relevant claims will make their way into the input…[which seems] unrealistic.”

Thanks for pointing this out. For PIC-Bench evaluation and our RAG experiments, we use a claim extractor from [6] that decomposes a given source text into its constituent verifiable claims. We emphasize that this extractor is not prompt-aware, and does not filter claims based on user instruction.

We would also like to correct the assumption that PIC assumes that only relevant claims will be in the input. In partial PIC (which is used for RAG), the LM is fed a complete claim set that may contain noisy, irrelevant, or contradictory claims. The LM must select a relevant subset of claims to ground its output, without adding any unsupported claims. Concretely, it's the LM, not the claim extractor, that must decide which claims to incorporate.

We ran RAG experiments with retrieved passages processed as claims as PIC-LM is trained with this format. However, we conduct a new RAG ablation, where we omit the claim extraction step entirely and instead pass in raw retrieved passages to PIC-LM 8B (“PIC-LM 8B (Psg.-level)”). This setup corresponds to standard in-context RAG, and simulates noisy input without explicit claim structuring. Despite being trained only on structured claims, PIC-LM generalizes well to this unstructured setting and still outperforms Llama 3.1 8B Inst. by a significant margin (59.7 vs. 52.3 EM) on ASQA:

PIC-LM improves downstream RAG performance as it grounds on retrieved context more thoroughly. Please note that not all PIC use cases rely on an auxiliary claim extractor. For example, in our chain-of-verification pipeline experiments, the LM autonomously generates and self-corrects its claims to improve end-task factuality. We will incorporate these results and discussion into our revision.

“Were there experiments conducted by varying the prompts used for [PIC evaluation]?”

We ran a new ablation experiment with stronger, more explicit prompts and found PIC-Bench to be largely prompt-insensitive. We report absolute performance changes (%) for PIC-LM 8B and three baselines when evaluated using the original prompt (from the paper) versus an alternative prompt pair:

• Full PIC: “You are an assistant that strictly adheres to given instructions. Be sure to incorporate each provided input claim in your response, and do not introduce or omit any claims.”
• Partial PIC: “You are an assistant that strictly adheres to given instructions. Respond using any subset of given claims that you find relevant, but do not introduce any new claims.”

As shown, changes in PIC performance are relatively small across prompt variations, with most metric shifts <4%. These results indicate that PIC-Bench evaluations remain stable across reasonable prompt variations. We will include these results in our revision. While certain prompts may slightly improve absolute scores, we believe they are unlikely to affect the relative ranking between models, which is our primary evaluation focus.

“Could you elaborate on how the gold response was obtained (line 111)?”

The long-form gold response $y$ comes from the instruction-following dataset, $D_{IT}$. Many off-the-shelf instruction-following datasets (e.g., Alpaca) are formatted as instruction-response pairs $(I, y)$, which can be easily converted to the PIC format $(I, C, y)$, wherein $C$ is the set of extracted verifiable claims from the response $y$. We'll clarify this point in our revision.

“A few lines on the model used to compute support for eval (line 115) would be helpful.”

Thanks for the suggestion! We use an off-the-shelf claim verifier introduced by VERISCORE [6], which prompts an LLM (in our case, GPT4o-mini) to judge whether a verifiable claim is supported or unsupported by a corresponding set of evidence claims. Note that [6] has found substantial agreement between this claim verifier and human annotators; we also corroborate this across our PICBench tasks and baseline generations in our own human evaluation (please see our response to reviewer Dux5).

“Which instruction-following model is referred to in line 184?”

We take Llama 3.1 8B Instruct as the instruction-following model used to score responses for PIC-LM preference data creation. We choose this model because in practice, (1) it shows strong agreement with GPT-4o in our LLM-as-a-judge comparison (see Appendix C.5), and (2) as a relatively small, open LM, it enables efficient and cost-effective data selection.

“It's curious to see the [PIC-LM] SFT model degrade in performance w.r.t the same base model in F1 for AskH, ELI5, ExpertQA & FACTS.”

We thank the reviewer for pointing this out. AskH, ELI5, ExpertQA, and FACTS are the four long-form QA tasks on PIC-Bench, and PIC-LM SFT shows a drop in PIC performance compared to the base model Llama 3.1 8B Inst. on these tasks. We believe there are two reasons: first, there is a mismatch between the token-level cross-entropy loss used for SFT and the PIC objective, which emphasizes exact adherence to a constrained set of claims. During SFT, the model may learn to generate broadly informative responses that resemble typical QA-style outputs but deviate from strict PIC constraints, leading to improved recall but reduced precision, and thus lower F1.

The second factor is SFT training data quality. Table 20 (in Appendix C.4) shows that training only on filtered samples with high PIC scores improves both SFT and DPO performance. Since submission, we have experimented with raising the minimum PIC score threshold from 0.9 to 1.0 for all training examples except those in the biography domain; as shown below, training on more stringently filtered data improves PIC performance on long-form QA.

We hope our responses have adequately addressed your concerns. Please let us know if there are any other points we can clarify.

Citations:

[6] Song et al., 2023. VERISCORE: Evaluating the factuality of verifiable claims in long-form text generation.

We thank reviewer pcWR for their time and feedback! It is encouraging that the reviewer finds our PIC framework to bring “a novel perspective” by enforcing per-claim control, “[going] beyond prior work” and that our PIC-LM “delivers strong precision improvements” in concrete downstream use cases. We address their questions and suggestions below:

“PIC-LM is specialized to follow given facts religiously…[which may] affect its general language abilities or flexibility. [A] more extensive evaluation on non-PIC tasks would strengthen confidence in PIC-LM’s general usability.”

We did not evaluate PIC-LM on general language benchmarks as it’s not meant to be a standalone LM, but rather a highly faithful module within larger systems for grounded generation (Sect. 6). For example, using PIC-LM as a drop-in replacement in RAG or chain-of-verification pipelines boosts end-task factual accuracy (+14% on ASQA, +21% on Birthplaces, +6.9% on QamPARI).

Although PIC-LM is a specialized model (its alignment process is solely focused on PIC), for transparency, we report new results on standard benchmarks (MMLU and ARC-C using OLMES [1]; GSM8K using LM-Harness [2]; and AlpacaEval 2 [3]):

We emphasize that PIC-LM is an initial model to tackle PIC, which still remains an open problem. For example, it is possible to instill general-purpose LMs with better PIC abilities by mixing in PIC-formatted data during post-training. We leave this direction to future work, and will include the above results in our paper with a more in-depth discussion.

“PIC penalizes any additional (even correct) information; the paper lacks discussion or ablation on whether such rigid constraints harm user value in contexts where minor extra facts are tolerable…could the authors provide more discussion on this nuance?”

We acknowledge that PIC’s strict grounding requirement isn’t appropriate for every scenario (e.g., creative storytelling, open-ended reasoning). Rather, PIC is built around user-specified control: if the user explicitly instructs the model to only ground its response on given claims, the LM should comply, even if this means forgoing extra correct details. Enforcing these constraints is necessary in fast-moving, high-stakes domains (e.g., scientific consensus on infectious diseases), where plausible but unverified information pose serious risks. We will incorporate these points into the discussion in our revision.

“There is little assessment of the linguistic quality or user-perceived quality of [PIC evaluation] outputs…How do the PIC-LM outputs do in terms of coherence and style, beyond…required facts? …Include a small qualitative analysis or human evaluation of output quality.”

Thanks for the suggestion! We ran a new small-scale qualitative analysis by sampling 40 PIC-Bench instructions and comparing responses from Llama 3.1 8B Instruct and PIC-LM 8B along two dimensions: coherence and fluency. Following prior work [5], we define coherence as the degree to which the ideas in the text flow together logically (i.e., whether the sentences are on-topic, and follow a clear progression of thought). We define fluency to be how grammatically correct, smooth, and natural the text is.

For each output pair, we indicated a preference for Llama 3.1 8B Inst., PIC-LM 8B, or a tie along each dimension. All outputs were evaluated blindly in shuffled order, without revealing model identity, to reduce annotation bias. The results are as follows:

These results suggested that PIC-LM 8B outputs tend to be more coherent than Llama 3.1 8B Inst. Qualitatively, we find that PIC-LM generations are more concise and factual, whereas Llama generations are more verbose. Both model generations are similarly fluent. we'll include this experiment in our revision, and provide more qualitative examples of PIC-LM generation. Note that PIC-LM can also be steered along stylistic dimensions while preserving claim grounding. We demonstrate this capability in a style transfer experiment in Appendix D.2.1.

“It remains unclear how well [PIC-LM training] would scale to bigger or smaller models.”

We ran new PIC-LM training experiments starting from 1B, 3B, and 70B Llama 3 Instruct models, and found strong PIC-Bench improvements in all settings. Note that due to computational constraints, we trained the 70B PIC-LM with QLoRA fine-tuning. Please see the PIC-Bench results below:

“Consider providing a breakdown of remaining errors (missing vs. unsupported claims, topical patterns) and discuss how those insights can guide future mitigation strategies.”

Thanks for the suggestion! We manually examined 40 PIC-LM generations on PIC-Bench with imperfect scores (5 samples per PIC evaluation task). This sample has 1165 output claims in total, of which 68 response claims are marked as unsupported (affecting precision), and 63 input claims are marked as missing (affecting recall for the full PIC setting).

We provide a categorical error breakdown of the 68 unsupported response errors below:

Among the 63 missing claims, we observe no obvious qualitative differences, but a mild head-bias: 27% of drops occur within the first quarter of the list (vs. 22% in the final quarter), indicating that earlier claims are slightly more likely to be omitted, and corroborating the “lost-in-the-middle” context phenomenon. We'll add these error analyses to our revision. For mitigation strategies, we believe it would be beneficial to explore improved data selection during training, such as incorporating hard negatives with subtle distortion and fabrication errors. Additionally, adopting techniques from long-context modeling (e.g., attention re-weighting, recurrence, or segment-aware encodings) may help address position-based errors, such as the observed head-bias in missing claims.

“In the partial PIC setting, only a subset of provided claims should be used. It would help to know how the “relevant” subset is determined for evaluation.”

In the partial PIC setting, it is up to the LM to select the relevant subset from the input claims. PIC does not measure the quality of the answer, but just ensures that every claim in the answer is fully grounded. PIC is reference-free in that the output isn’t compared against a given gold answer.

To measure actual generation quality (e.g., if the response adequately satisfies the task instruction), we pair PIC evaluation with Prometheus, an LLM-based judge that scores outputs along a user-defined rubric and shows high agreement with human evaluations. As our PIC evaluation framework provides well-formed claims, we find that our Prometheus scores are relatively high (see Table 4).

Citations:

[1] Gu et al., 2025. OLMES: A Standard for Language Model Evaluations

[2] Biderman et al., 2024. Lessons from the Trenches on Reproducible Evaluation of Language Models

[3] Dubois et al., 2024. Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators

[4] Liu et al., 2023. Lost in the Middle: How Language Models Use Long Contexts

[5] Li et al., 2023. Contrastive Decoding: Open-ended Text Generation as Optimization

We thank reviewer Dux5 for their time and feedback! We are encouraged that Dux5 finds our paper “well-presented [with] good motivation,” that our PIC framework “can serve as a foundation for future intrinsic hallucination research” and is “precise…for a more fine-grained task,” and that our benchmark is “well-designed…[with] complete and comprehensive technical implementation details.”

Responding to the reviewer’s comment below:

"[T]he evaluation pipeline depends a lot on the LLM…[which] could result in failure to generalize to evaluation with different LLM judges."

We thank the reviewer for raising this important point. We acknowledge that evaluations that rely on a specific LLM (in our case, GPT-4o mini) may be susceptible to model-specific biases that affect generalizability. We address this concern along two fronts, with new experiments:

First, we explore changing the LLM backbone to GPT-4.1 and Claude 3.5 Sonnet when evaluating generations from the following baselines: Llama 3.1 8B Inst., PIC-LM 8B, and GPT-4o. We report the effect of different LLM judges on avg. PIC-Bench F1/precision. The results are as follows:

We find that PIC scores are largely judge-agnostic, with GPT-4.1 behaving slightly stricter than the other two judges. Across judges, PIC-LM 8B consistently achieves the highest precision (partial PIC metric), while GPT-4o attains the highest F1 (full PIC metric).

In practice, like most automatic LLM-based evaluation methods, we fix to one judge model for a fair comparison. Furthermore, note that employing GPT-4o mini as the LLM backbone costs roughly 7 USD for a full PIC-Bench evaluation run; based on API costs, GPT-4.1 costs roughly 13x more, and Claude 3.5 Sonnet is 20x more.

Second, our LLM judge of choice (GPT4o-mini) shows high agreement with human annotation. The LLM is invoked at two stages of our evaluation pipeline: claim extraction (decomposing text into verifiable claims) and claim verification (checking that a claim is supported by a set of verifiable claims). We conduct a pilot human evaluation experiment to assess agreement across PIC tasks and 3 baseline models (Llama 3.1 8B Instruct, PIC-LM 8B, and GPT-4o). We use 3 human annotators per component.

Claim Extraction Human Evaluation

For claim extraction evaluation, we sample 5 generations per PIC task and baseline model, leading to 120 samples in total. For each sample, we show the annotator the generated output and an extracted claim from that output.

Recall that for claim extraction, the LLM is prompted to extract high-quality, verifiable claims from a source task. Here we ask the annotator to rate the extracted claim on a 3-point ordinal scale (0-2, 0 being the best) along the following dimensions: faithfulness (0: definitely supported, 1: partially supported, 2: not supported), decontextualization (0: self-contained, 1: ambiguous, 2: context-dependent), and quality (0: fluent, 1: minor readability issues, 2: hard to read).

We report (i) LLM-human agreement (the average fraction of annotator ratings equal to the best score (0) for the LLM-extracted claims), and (ii) exact inter-annotator agreement (the fraction of items where all raters chose the same label).

We observe fairly strong LLM-human agreement (and substantial exact inter-annotator agreement) across all three axes, which confirms that GPT-4o mini is able to correctly extract high-quality, verifiable claims.

Claim Verification Human Evaluation

For claim verification evaluation, we sample 3 generations per PIC task and baseline, leading to 72 generations in total. For each sample, we provide the list of input claims, and a response claim that the LLM-based claim verifier has marked as either supported or unsupported. We ask the annotators, who are blind to model identity and the verifier’s label, to judge if the response claim is supported or unsupported by the input claims.

We found high average human agreement with the LLM (81.0%), and substantial inter-annotator agreement (Fleiss’ Kappa of 0.694).

Taken together, these results give confidence that our evaluation pipeline (using GPT-4o mini) is producing evaluations that are broadly consistent with human judgment. Finally, we emphasize that our PIC evaluation design is flexible and judge-agnostic: in practice, it’s easy to swap to a different LLM depending on one’s use case.