Dear Reviewer,

Thanks for your comments and questions. We are making a revision to the paper for better clarity, and we appreciate the opportunity to clarify and elaborate on key aspects of the paper here.

$**Regarding Pareto optimal Solution:**$

• Based on our claim in Theorem 1, the harmonizer model $h^*$ identified via our approach approximates one of the possibly many Pareto optimal solutions. Indeed, different scenarios may favor different solutions on the Pareto frontier. We achieve this diversity through the selection of different Pareto scalarizers $G$, as outlined in our paper. Specifically, we explored three distinct Pareto scalarizers and the non-Pareto Chebyshev scalarizer for comparative analysis. Our empirical results, as shown in Table 2, suggest that the quadratic scalarizer is particularly effective when paired with the BERT harmonizer model. This scalarizer emphasizes challenging examples where the LLM and weak sources exhibit significant discrepancies, as briefly discussed following Definition 2.

• We understand your concern regarding the need for reweighting among the LLM and weak sources. To address this, we provided two alternative strategies in Appendix B, designed to automatically adjust weights based on different scenarios, such as high dependency and significant differences in source accuracy. We hope these provide a good variety of Pareto optimality for different scenarios, addressing your concern.

$**On the novelty of using scoring function:**$

• We respectfully express our disagreement on "Finding scoring function from external weak sources and use them for calibration and correction has no novelty". Any calibration method, including early ones like Platt scaling, involves some form of scoring function. The challenge lies in deriving a robust scoring function with limited resources. Our approach uniquely tackles this by using weak supervision sources in the absence of ground truth labels, ensuring independence from the LLM and mitigating bias. We believe this is a reasonable approach to provide information that is independent from the LLM itself, which is essential in as a guardrail.

• Our work's novelty is threefold:

1. We are pioneers in using weak supervision sources for LLM calibration and correction.
2. To our knowledge, this is arguably the first attempt to estimate confidence levels using only weak sources, a concept previously unexplored. We are excited to report our results that may inspire future work.
3. Our methodology for aggregating weak sources is innovative and distinct from prior works focused on label weighting.

$**Responses to Specific Questions:**$

• Question 1. Theorem 1 tells us the criteria for the function $G$ that are beyond mere convexity; it must also be strictly increasing element-wise (as per Definition 2). An illustrative counterexample is the Chebyshev/max scalarizer. Though convex, it fails to meet the strictly increasing condition, hence does not provide the guarantees outlined in Theorem 1. Our experiments support this, showing its relative underperformance compared to other Pareto scalarizers.

• Question 2: Identifying hallucination and error in LLM is a broad and ongoing study area. In this study, we aim at solving the cases where the LLM response is mapped to a finite space, such as the categorical setting. While this doesn't encompass all LLM applications, it covers a substantial range including many important scenarios. This subset includes but but is limited to: GPT-4 evaluation on various exams, short answer QA evaluations, applications using LLM for structured outputs... For the example in Figure 1, one LLM response only maps to one answer.

• Question 3: We utilized the quadratic scalarizer in our experiments, as detailed in the 'Harmonizer Training' section of Section 4.

We hope this addresses your concerns and provides further clarity on our paper. We are eager to answer any additional questions and hope you will consider revising your score with these clarifications.

Best regards, Authors

(... Continued from the previous comment...)

5. $**Dynamic Prompting Iteration in Algorithm 2:**$ The POLAR-assisted dynamic prompting process in Algorithm 2 is typically executed once per instance. If we take the threshold $\delta = 0.5$, after one iteration of correction it must be below $\delta$ already. However, one can re-fit the harmonizer using the updated LLM response and do it again. In our experiments, we didn't observe additional significant improvement on performance, likely due to the information limit.

6. $**Additional References and Comparisons:**$ We have added the suggested recent references to our manuscript and discussed how our work relates to these developments in the field. Specifically, we have also included a comparison with another weak supervision model in our experimental evaluations (Table 1) to demonstrate the advantages of our approach.

7. $**Experimental Details:**$ The details behind the experiments have been expanded for greater clarity. This includes more information on the datasets used, the specifics of the implementation of the harmonizer, and the rationale behind the choice of our evaluation metrics. Explanations of the benchmark comparison methods are also expanded.

Thank you again for your time and valuable comments, which did put our work and paper into a better state! We believe these revisions have significantly strengthened the manuscript, addressing the key concerns you raised. Hope that the improvements in clarity, presentation, citation, and extra experiments adequately address your feedback and make the paper a strong candidate for acceptance.

Thank you once again for your valuable feedback, and we look forward to your further thoughts!

Best regards, Authors

Dear Reviewer,

Thank you for your thorough review and constructive feedback on our manuscript. We sincerely appreciate your acknowledgement of our contribution. The suggestions you made to improve our paper presentation are highly valuable, and we have made significant revisions to address your concerns, particularly regarding clarity and the specificity of our theoretical claims. Here is a summary of the key revisions and responses to your questions:

1. $**Clarity and Writing Quality:**$ We have carefully revised the manuscript to eliminate typos and grammatical issues. We have also restructured sentences for better clarity and coherence, particularly in sections where the methodology and experimental details are explained. This includes clarifying the role and the limitations of Propositions 1 and 2, and providing a more explicit explanation of our approach in the end of Section 3.3. Although there are many changes in the paper, for your time and convenience we want to highlight one example of modification for your reference. This is the part after Proposition 1 (distillation from imperfect source promotes calibration), and before Proposition 2 (noisy weak signals helps accurate LLM):

"(After Proposition 1) ... Muller et al. (2019) empirically demonstrated that label smoothing improves or at least doesn’t harm model calibration compared to standard supervised learning. Therefore, Proposition 1 suggests that access to ground truth labels is not necessary for obtaining well-calibrated outputs. If LLM errors were uniformly “random”, we could directly use it as a teacher model to distill a small network on the specific task for calibration.

However, LLM errors are rarely “random”. Deterministic functions typically err on specific types of inputs. Distilling a model solely from the LLM could lead to overfitting, thus unable to signal errors in the teacher LLM. Our solution is to integrate other weak supervision functions as external signals independent of the LLM. As each supervision source makes different types of errors on different examples, their ensemble approximates a “random” scenario... (Before Proposition 2)"

2. $**Theoretical Clarification:**$ Regarding Proposition 2, we have expanded the explanation of "weak independent signal ensemble" to clarify that these signals provide information independent of the LLM. We have also revised the notation for the function "$\psi(\Lambda, w)$" to "$\psi(\Lambda(x), w(x))$" to avoid confusion. Additionally, we now explicitly state why the weak supervision signal ensemble $\omega(x)$ is modeled with a continuous random variable in Proposition 2.

3. $**Section 3.3 Clarification**$ We now explicitly states in this section that our main theoretical results takes a leap from the inspirational propositions. We also added a brief summary for the theoretical part at the end of Section 3.3 to discuss the connections between the theoretical results and their limitations. Here is the content for your reference:

"Let’s make a brief theoretical summary here. If the LLM makes uniformly “random” errors, simply distilling a smaller network would give a calibration model. Since this is not usually the case, we incorporate multiple independent supervision functions alongside the LLM, fitting a harmonizer model to all simultaneously to prevent overfitting to the LLM. Theoretically, we approximate a harmonizer model that is Pareto optimal. However, due to the complexity of modeling the errors made by the LLM and supervision functions, we cannot offer further theoretical guarantees at this point. We empirically demonstrate the calibration capability of the harmonizer model in Section 4."

4. $**Pareto Optimization Scalarizer:**$ We have further clarified the concept of the Pareto optimization scalarizer in our framework. We clarified that while it does aggregate multiple supervision signals into a single harmonized output, this process is fundamentally different from a simple weighted sum because of Jensen's inequality. One example is the quadratic scalarizer, which automatically emphasizes on the harder examples with larger overall loss. The trade-offs between different supervision sources is handled in the Pareto optimal manner, thereby avoiding the pitfalls of traditional weighted sum approaches.

(To be continued...)

As shown in the table, for all three LLMs across all four tasks, the proposed POLAR score consistently outperforms other methods. Among the baseline methods, Snorkel, WeaSEL, and LLM distilled model can achieve top or close-to-top performance in some cases under specific metric, but lack the consistency to deliver stable calibration for different LLMs on different tasks. In comparison, the proposed POLAR score is consistently well-calibrated to the true error rate.

While WeaSEL shows commendable performance on specific tasks and LLMs (e.g., calibrating GPT-3.5 on the CDR relation extraction), it does not consistently deliver across all scenarios. Notably, it performs optimally with GPT-3.5 and in situations with a limited number of supervision functions. Compared to the Snorkel model, WeaSEL exhibits superior performance in most cases, likely due to its greater expressiveness and instance-specific weighting.

We hope this extra benchmark comparison strengthen the effectiveness of our approach. We appreciate any of your comments and questions, and your consideration for elevating the score.

Best regards, Authors

Dear reviewer,

We sincerely appreciate your insightful comments and feedbacks. We have undertaken a major revision of our paper, taking into account your valuable suggestions. Below are the key areas of improvement:

1. $**Enhanced Theoretical Section:**$

• We have revised the theoretical section for enhanced clarity, adding detailed explanations and discussions about the implications and limitations of our propositions and theorems. We acknowledge that our propositions are more inspirational in nature and may contain unrealistic assumptions (e.g., non-random LLM errors, independence of weak supervision sources, and an insufficient number of weak sources for CLT approximation). For main results, we provide theoretical guarantee with Theorem 1 free from these assumptions, and we demonstrate the empirical effectiveness of our approach.

• A new paragraph at the end of Section 3.3 now provides a summary and discusses the connection between the theoretical results and their limitations. Here is the added content for you reference:

"Let’s make a brief theoretical summary here. If the LLM makes uniformly “random” errors, simply distilling a smaller network would give a calibration model. Since this is not usually the case, we incorporate multiple independent supervision functions alongside the LLM, fitting a harmonizer model to all simultaneously to prevent overfitting to the LLM. Theoretically, we approximate a harmonizer model that is Pareto optimal. However, due to the complexity of modeling the errors made by the LLM and supervision functions, we cannot offer further theoretical guarantees at this point. We empirically demonstrate the calibration capability of the harmonizer model in Section 4."

2. $**Benchmark Experiments with WeaSEL:**$

• In response to your suggestion, we have included the End-to-end weak supervision paper in our citations. This paper presents an nice framework in weak supervision, assigning instance-specific weights in an end-to-end manner.

• We conducted benchmark experiments using the WeaSEL model and have included these results in our comparison Table 1. While WeaSEL shows commendable performance on specific tasks and LLMs (e.g., calibrating GPT-3.5 on the CDR relation extraction), it does not consistently deliver across all scenarios. Notably, it performs optimally with GPT-3.5 and in situations with a limited number of supervision functions. Compared to the Snorkel model, WeaSEL exhibits superior performance in most cases, likely due to its greater expressiveness and instance-specific weighting. For your convenience, we provide the extended comparison table in the next comment.

We want to thank you again for bringing the comments and suggestions. We believe these revisions substantially strengthen our paper, providing a more nuanced theoretical discussion and adding robust empirical comparisons. We hope this addresses your concerns and would be grateful for any further comments or suggestions. Your consideration in revising the rating is greatly appreciated.

Best regards, Authors

(... Continued from last comment...)

9. Error Rate Calculation in Baseline Methods: Yes, the error rate is indeed calculated by comparing the estimated class probabilities from the baseline model with ground truth labels.

10. Dynamic Self-Supervision Prompting Strategy: Yes, as long as the supervision functions are independent from the LLM (thus provide extra information), we can always improve the error rate for the high POLAR score examples. Once the error got corrected, because the POLAR score already dropped below the threshold, re-prompting doesn't further improve the results. This is consistent with the information limit available from the supervision functions.

We believe our revisions and responses address the key points of your feedback and strengthen the contribution of our paper. We hope that our responses and the improvements made will lead to a reconsideration of the rating.

Thank you again for your valuable input! We look forward to any further suggestions or questions you may have.

Best regards, Authors

Dear Reviewer,

Thank you for your insightful feedback and questions. We greatly appreciate your comments and have undertaken a comprehensive revision to address your concerns. Below is a summary of how we have addressed each point, and our response to the questions.

Response to Weaknesses

1. Applicability to Real-World Tasks with Non-Finite Output Spaces: We agree with you and recognize the limitation of our framework being applicable primarily to tasks with finite output spaces. As identifying and correcting LLM error is a challenging problem and an emerging field, we chose to be focus on and tackle a smaller problem. What we hope from our paper is not only the introduction of the proposed method, but also theoretical and empirical results that can inspire future work. In that sense, targeting a smaller problem helps us dig deeper.

2. Clarity on Theoretical Intuitions: We have revised the introduction to provide clearer explanations of the intuitions behind our approach. The two intuitions are proved in Proposition 1 and 2 respectively. In short, if the LLM error is uniformly random, the distilled model from it serves as a good calibration model. Introducing multiple external supervision sources diversifies the output, avoiding the distilled model being completely bias by the LLM and not able to signal its error.

3. Connection Between Propositions and Pareto Optimal Learning:

To clarify the relationship between Propositions 1 and 2 and Pareto optimal learning, we have added a detailed explanation in Section 3.2. We also added brief summary of the connections between the theoretical results, as well as their limitations. The content is below for your reference:

"Let’s make a brief theoretical summary here. If the LLM makes uniformly “random” errors, simply distilling a smaller network would give a calibration model. Since this is not usually the case, we incorporate multiple independent supervision functions alongside the LLM, fitting a harmonizer model to all simultaneously to prevent overfitting to the LLM. Theoretically, we approximate a harmonizer model that is Pareto optimal. However, due to the complexity of modeling the errors made by the LLM and supervision functions, we cannot offer further theoretical guarantees at this point. We empirically demonstrate the calibration capability of the harmonizer model in Section 4."

4. Label Smoothing: In Proposition 1, label smoothing is implicitly achieved by fitting to the imperfect LLM output. This technique is not adopted for training, but as a theoretical concept to justify our approach. We have made the explanation for this clearer in the revision to avoid confusion.

Response to Questions:

5. Presentation of Supervision Functions: We have revised the manuscript to provide a more detailed explanation of the supervision functions. In short, they are any programmatic supervision sources, such as keywords, regex, knowledge base, that outputs noisy heuristic labels on the input. We explained the choice of the supervision function in the experimental part, and also discussed their contribution to the overall calibration performance.

6. Objectives in Pareto Scalarizer: The difference between $G(\ell_0, \cdots, \ell_j', \cdots, \ell_m)$ and $G(\ell_0, \cdots, \ell_j, \cdots, \ell_m)$ is $\ell_j'$ and $\ell_j$. The requirement for $G$ is, for any $\ell_j' < \ell_j$, there must be $G(\ell_0, \cdots, \ell_j', \cdots, \ell_m) < G(\ell_0, \cdots, \ell_j, \cdots, \ell_m)$, i.e. strictly increasing (but not necessarily continuous).

7. Clarification on Pareto Scalarizer and Weighting Schemes: The Pareto scalarizer in our experiment is a nonlinear function aggregating the losses, thus different from a uniform weighting scheme. When performing SGD types of optimization, Jensen's inequality makes the nonlinearity take effect. For example, the quadratic scalarizer, which is the one we experimented to be most effective, automatically emphasize on the hard examples where the LLM and the other sources have larger disagreement. In our revision, we highlight its unique role in balancing multiple objectives in a harmonized manner.

8. Computation of POLAR Score Metrics:

• ECE: Split the examples into 10 bins according to their POLAR score, e.g. 0-10%, 10-20%, etc. For the examples in each bin, calculate the real LLM error rate using the ground truth labels. Compare the difference between the POLAR score and the true LLM error rate, and take expectation over all the examples.
• $R^2$: Ranks the LLM responses into POLAR score bins each with 100 examples, and plot the average POLAR score and true error rate for each bin. The R2 is calculated from the (POLAR - error rate) scatter plot.

(To be continued...)