BIRD: A Trustworthy Bayesian Inference Framework for Large Language Models

We sincerely thank reviewer VqFc for the valuable comments. Here we answer all your questions.

W1.Paper writing

We have addressed the minor writing issues in the updated version. While we agree that the writing for the general setting could be made more concise and that the algorithm can be explained in greater detail (many details have been added in Appendix A.3), we plan to address these aspects more comprehensively in the final version as structural changes to the writing require a relatively large amount of time.

We are displaying factors related to the decision making in Figure 2, and “the color of the vehicle” is not, so it does not appear in the figure.

W2. Ablation studies

We conduct a preliminary ablation study on the loss function for decision making, as detailed below, and demonstrate that both components of the loss function are crucial for the model's performance. We will include the complete ablation study in the final version.

We believe that assuming each value in an unobserved factor has an equal probability of being selected is an unbiased and neutral assumption, given the lack of information about which value might be chosen. However, we agree with the reviewer that further user interaction could be beneficial in determining if there are specific preferences regarding the possible values.

Q6. Decision-making involving more than two outcomes

We agree with the reviewer that testing decision-making scenarios with more than two outcomes is an important next step. BIRD can handle multiple outcome situations by hierarchically iterating pairs of outcomes within the decision-making framework. Additionally, future work could focus on deriving new guarantees, similar to Equation 3, for multiple outcomes, which could lead to the formulation of a new constrained optimization problem.

We sincerely thank reviewer ZTvP for the constructive suggestions. Here we address and clarify the questions you posed.

Paper writing and code release

We agree with the reviewer that the writing for the general setting could be more concise and that the method can be explained in greater detail, we plan to address these aspects more comprehensively in the final version as structural changes to the writing require a relatively large amount of time.

We will first clarify some of your questions. All components within the central box, including the internal factors and assignments in the conditional probability tables for a given scenario, are automated and inferred through LLM interactions without knowing the specific downstream conditions. Later, the computations of observations (i.e., the condition-factor mapping) are also done automatically with LLMs. We include an extra instance in Fig. 2 that supports the opposite outcome, illustrating that BIRD can reliably compute probabilities and make decisions under any condition within the same scenario.

We will release the complete codebase, including the code for all detailed methods of prompting the LLM, the learning algorithm, and the data.

Q1. Design choices of model parameters

We agree with the reviewer that the design choices for model parameters can be better elaborated. To address this, we have added detailed explanations of these design choices in Appendix A.3 and provide the same detailed explanation here.

For most of the chosen values, we adopt a neutral and unbiased assumption. Consequently, we assign equal weights $w_j = 1, j = 1, ..., N$ and assume $P(O_i) = 50\\%$ in Section 3.2. The value of $\mathbb{P}_{\rm init}(O_i|f_j)$ in Equation 5 Section 3.3 is assigned as it represents random initialization.

To determine the mappings of rankings to probabilities in Section 3.3, we consulted two psychology experts and adopted the Likert scale theory. The finalized mapping between verbalized probabilities and numerical probabilities reflects typical human behavior and effectively distinguishes between verbalized probabilities in an unbiased manner. Additionally, slight adjustments to the mappings (~5%) do not noticeably affect overall performance.

We randomly sample 128 instances from the LLM as training data to manage costs in Section 3.3. This represents the minimum number of instances required for effective training. Ideally, increasing the number of sampled instances would improve the model's alignment with the underlying LLM. All training hyperparameters for the algorithm are optimized using a grid search with a hold-out validation set from Plasma.

We also conduct a preliminary ablation study on the loss function for decision making, as detailed below, and demonstrate that both components of the loss function are crucial for the model's performance.

Q3. 'Unknown' predictions

We verify that 98% of the unknown instances stem from shortcomings in the factor generation method, as these instances cannot be mapped to any factor value. While our framework is theoretically robust, challenges in implementation may arise due to the lack of comprehensiveness or sufficient granularity in the generated factors. To address this, we plan to explore and develop more effective factor generation methods in future work.

At the same time, since BIRD does not rely on task gold labels to determine whether a prediction is unknown, we can leverage baseline methods (e.g., CoT) to generate predicted labels for these unknown cases. This approach ensures that our current comparisons, particularly regarding absolute performance gains, remain consistent even if we do not remove the unknown cases.

Q4. Spurious biases of LLMs

Language models operate as induction machines and thus may rely on spurious information about relevant concepts internally. This enables CoT-like inferences to make educated guesses that are correct in many cases but not necessarily for the right reasons to directly solve tasks. (as discussed in https://arxiv.org/pdf/2305.14825, https://arxiv.org/pdf/2311.09702, https://arxiv.org/pdf/2410.05229). BIRD provides an advantage by decomposing tasks into components with clearer individual objectives that leverage LLMs' strengths, such as abduction for factor generation, coarse estimation under complete information, and textual entailment. By structuring tasks in this way, each instance of LLM prompting becomes more controllable, mitigating the impact of spurious biases. Furthermore, BIRD incorporates a symbolic structure with explainable factors, encouraging more reliable and interpretable predictions.

We sincerely thank reviewer gTw6 for the valuable comments. Here we answer all the questions you posted.

W1. The training setting for the learning algorithm

We agree with the reviewer that the training setting for the learning algorithm can be more clearly described. We have revised and added additional detailed explanations of the learning algorithm in Appendix A.3 and provide the same detailed explanation here.

1. For a given scenario $S$, the number of learnable parameters is the total number of values for all the generated factors, i.e., $\sum_{j=1}^{N} \text{card}(\mathcal{F}_j)$.
2. All training instances for a scenario are drawn from the complete information space for the same scenario $S$ and possible outcomes $\\{O\\}$ and will share the same factors $\\{F_j\\}_{j=1}^N$.
3. LLMs generate the factors for a scenario without knowing the specific downstream conditions and their corresponding outcome labels, therefore the factors are the same in the process of fitting the Bayesian network and later inferences.

W2. Unknown instances

We verify that 98% of the unknown instances arise because these instances cannot be mapped to any factor value. While our framework is theoretically robust, challenges in implementation may stem from factor generation, as the generated factors may lack comprehensiveness or sufficient granularity. We plan to explore and develop more effective factor generation methods in future work to address this.

At the same time, since BIRD does not rely on task gold labels to determine whether a prediction is unknown, we can leverage baseline methods (e.g., CoT) to generate predicted labels for these unknown cases. This approach ensures that our current comparisons, particularly regarding absolute performance gains, remain consistent even if we do not remove the unknown cases.

Q1. Experiment settings for section 4.3

All experiment settings can be referenced in Feng et al. (2023)(https://aclanthology.org/2023.acl-long.671.pdf ), and we have also included them in Appendix A.9 in the updated version. The T5-large model and Patterntime (Zhou et al., 2021)(https://arxiv.org/pdf/2010.12753) are specialized temporal reasoning models, as outlined in Feng et al. (2023). Both models were specifically fine-tuned on temporal reasoning datasets using the original temporal training data from Feng et al. (2023). In contrast, the additional training data used in our paper focuses on commonsense reasoning. Consequently, the additional finetuning aims to create a more general reasoning model, which may result in slightly reduced temporal reasoning performance. However, our primary objective is to demonstrate that finetuning with BIRD-produced probabilities yields better results than finetuning with hard classification labels. We show that using BIRD-produced probabilities preserves temporal reasoning capabilities while enhancing general reasoning abilities.