C-LoRA: Contextual Low-Rank Adaptation for Uncertainty Estimation in Large Language Models

Thank you for your detailed and thoughtful feedback. We’re glad you found our method novel, well-motivated, and effective in both calibration and robustness. We appreciate your suggestions regarding uncertainty types, scaling, and visualization, as well as your insightful questions, which we address below.

W1. Limited discussion on epistemic/aleatoric uncertainty.

Our work primarily aims to enhance the model’s awareness of aleatoric (input-dependent) uncertainty, rather than to explicitly disentangle aleatoric and epistemic uncertainty.

We acknowledge that we do not provide a theoretical guarantee that C-LoRA can fully separate the two types of uncertainty. However, our architectural design, especially the contextual module, is intended to make the model more sensitive to variations in the input, thereby emphasizing aleatoric uncertainty.

That said, our method is still capable of capturing both aleatoric and epistemic components. As shown in our additional experiments (in answer to weakness 3), the ratio of aleatoric uncertainty in our model is consistently higher than in the baseline (on average, around 0.99 and 0.82 for contextual and BLoB, respectively), suggesting a stronger focus on modeling input-dependent uncertainty.

We appreciate the suggestion and will incorporate a more detailed discussion of this distinction and its limitations in the final version.

W2. Limited Scale of the Backbone LLMs.

It is true that our current experiments are conducted on the LLaMA2-7B model. We conduct our experiments using LLaMA2-7B due to its wide adoption and reasonable computational cost for extensive experimentation. Furthermore, most of the recent Bayesian PEFT methods employ LLaMA2-7B model in their experiments and we stick to the same model for reproducibility and fair comparison.

That said, our proposed C-LoRA is a model-agnostic framework that does not rely on any architecture-specific assumptions or scaling-related heuristics. Therefore, we believe the efficiency and effectiveness of our method can generalize to larger-scale models (e.g., 13B+), other model families (e.g., Qwen, Mistral), and more recent versions (e.g., LLaMA3.x).

We agree that extending experiments to additional backbones would strengthen our claims on generalizability. As it would require a significant amount of computational resources, we leave this for future work, and will clarify this limitation in the final version.

W3. Lack of intuitive, qualitative demonstrations.

EE and MI Scores (C-LoRA vs BLoB, M=10)

Due to the regulations of NeurIPS, we are not able to provide any visualization. Hence, to assess input-dependent uncertainty, we adopt the decomposition of predictive entropy (PE) as in [1] (Section 2.2) and [2] (Section 3.4), where the expected entropy (EE) captures data uncertainty (aleatoric), and mutual information (MI) reflects model uncertainty (epistemic). Note that this decomposition follows the relation PE=MI+EE.

In our analysis (as shown in the table), we compute EE and MI at $M=10$ across both in-distribution (OBQA) and out-of-distribution (ARC-C, ARC-E, Chem, and Phy) test sets. The model is trained on OBQA for all experiments. This setting allows us to better probe input-specific uncertainty, as OOD examples are more likely to elicit higher uncertainty levels that reveal how sensitively the model responds to unfamiliar or ambiguous inputs. We observe that the C-LoRA model consistently exhibits higher EE than BLoB across all datasets, while maintaining substantially lower MI. This suggests that the C-LoRA model is more attuned to data-dependent uncertainty, without attributing uncertainty to model parameters—indicating a more calibrated and stable epistemic profile. Particularly, the low MI suggests that most of the uncertainty in C-LoRA’s predictions comes from input variability, not model instability.

In general, the results in the table support our claim that C-LoRA is more effective at capturing input-specific uncertainty.

Q1. If each layer has an independent MLP?

While the architecture of the contextual module is consistent across layers, each layer has its own independently parameterized module. That is, the MLP used to predict the matrix is not shared across layers, and its parameters are learned separately for each layer during training.

Q2. Why does temperature scaling worsen the performance?

We indeed observe that, for OBQA and a few similar datasets, temperature scaling (TS) did not yield improvements in ECE, and in fact slightly worsened it, despite no meaningful change in average NLL as indicated in the following. This outcome, while unintuitive at first glance, has been observed in prior work and reflects the limitations of TS in some settings.

We tuned the temperature on a held-out validation set (not training data), so overfitting is unlikely. Since TS directly optimizes NLL, we verified that NLL before and after TS remained effectively unchanged (e.g., 0.59 for OBQA), suggesting TS reached a flat region in the NLL landscape but slightly altered the confidence distribution.

Also, temperature scaling minimizes NLL, which is a proper scoring rule, but does not directly minimize ECE, which is a binned, non-differentiable, and improper metric. Hence, if a model is already moderately calibrated, or if the logit distribution has specific local structure, TS may shift predictions in a way that harms ECE in certain bins.

Additionally, since our model is more sensitive towards data-dependent (aleatoric) uncertainty, it may already exhibit well-calibrated behavior in regions of high variability. This may further limit the effectiveness of a global post-hoc method like temperature scaling, which cannot adapt to such input-specific uncertainty patterns.

To find the temperature parameter, we used a single sample ($M = 1$) without Bayesian model averaging (BMA). While averaging over multiple samples ($M>1$) would constitute BMA, we did not adopt that here.

Similar effects are reported by Kull et al. [3], who show that TS can worsen classwise-ECE—for example, increasing cw-ECE from 0.296 to 0.327 on the car dataset and degrading mid-range calibration on balance-scale. These findings are consistent with our observations.

We emphasize that temperature scaling was used purely as a standard post-hoc tool to potentially improve the uncertainty quantification capabilities, and is not a component of the model itself.

Q3. How does C-LoRA scale with the test-time compute?

In the table, we report the mean performance among 3 random seeds when the model is fine-tuned on the OBQA task. We observe that increasing the number of samples $M$ at test time generally improves the calibration performance, as reflected in the decreasing ECE values. Specifically, ECE drops from 5.10 at $M=0$ to 3.55 at $M=4$, and continues to decline slightly with additional samples, reaching 3.67 at $M=50$. This suggests that aggregating predictions over multiple samples helps reduce miscalibration.

In terms of NLL, the deterministic prediction ($M=0$) yields the lowest value (0.53), indicating confident predictions with strong likelihood. Among the stochastic variants ($M\geq 1$), NLL remains relatively stable around 0.59–0.60, showing a slight decreasing trend with more samples and indicating some benefit from sampling.

Accuracy is highest at $M=0$ and slightly lower across other values of $M$, but remains fairly stable overall. This suggests that increasing $M$ primarily affects uncertainty estimation rather than core predictive accuracy.

Overall, as the number of test-time samples increases, calibration improves consistently, and likelihood shows marginal gains, while accuracy remains largely unchanged. This pattern is consistent with the fact that our method primarily models data (aleatoric) uncertainty, for which sampling has a more limited impact compared to approaches that also capture epistemic uncertainty.

Q4. Broken citation format.

We will fix the citation format to include the conference names (i.e venues).

We hope that we have addressed your concerns and look forward to the discussions if there are additional questions?

[1] Mucsányi, Bálint, et al. “Benchmarking Uncertainty Disentanglement: Specialized Uncertainties for Specialized Tasks” NeurIPS 2024

[2] Jishnu Mukhoti and Yarin Gal. “Evaluating Bayesian Deep Learning Methods for Semantic Segmentation”. arXiv preprint, 2018.

[3] Kull, Meelis, et al. "Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration", NeurIPS 2019

Thank you for your responses.

Thanks to the authors' efforts, many of my concerns have been addressed (Q1, Q3, and Q4). Here are the follow-ups for the other concerns and questions.

W1 & W3. Aleatoric and Epistemic Uncertainties.

I understand this decomposition itself might not be a main claim of the paper. However, when C-LoRA explicitly models the data uncertainty (AU) and makes a claim about it being the cause of the improvement, we need some sort of evidence. At present, the benchmarks used in the paper are mainly (i) single-answer questions which do not have data uncertainty (correct me if I'm wrong); (ii) for assessing the confidence alignment, but not hallucination detection, which might not reflect the claimed advantage.

Furthermore, I find the additional table of EE and MI confusing. If C-LoRA is able to model data/model uncertainty better, shouldn't we see a difference in them on IND (ARC-C and ARC-E) and OOD (Chem and Phy) datasets, e.g., higher model uncertainty on OOD data. But I don't see this trend in the table: C-LoRA produces the same level of uncertainties on all datasets. This brings questions to the main claim of the paper.

Importantly, C-LoRA having a lower MI than BLoB does not support this claim either. As the decomposition goes: $H[p(y|x)] = \mathbb{E}_{q(\theta|D,x)}[H[p(y|x,\theta)]]+ I(y;\theta|x)$ Lower MI, especially across all the datasets, only indicates that the approximate posterior of C-LoRA is close to the point estimation of the model, and somewhat downplays the importance of the Bayesian property of C-LoRA.

Could you please elaborate more on this issue?

W2. More LLM backbones.

This work [1] provides a full table on the reasonable baseline models, evaluated on Llama3.1-8B model. It could be a good start for incorporating this set of experiments for C-LoRA.

Q2. Issues of Temperature Scaling

Why don't you use multiple samples for temperature scaling? Keeping the number of samples the same for both training and testing could solve this degrade performance problem.

References

1. Shi, Haizhou, et al. "Training-free bayesianization for low-rank adapters of large language models." arXiv preprint arXiv:2412.05723 (2024).

Thank you for your constructive feedback. We appreciate that you found our introduction well-motivated, the model design novel, and the experimental evaluation comprehensive. We also value your detailed comments regarding the model formulation and writing clarity, as well as your observations on performance trade-offs. We address each of these points below.

W1.1 Motivation for the model design of $E$

The core motivation behind our design is to effectively capture data-dependent uncertainty in a scalable and stable manner. In principle, any of the components $A$, $B$, or $E$ can be made input-dependent by introducing a contextual module. However, conditioning either $A$ or $B$ on the input significantly increases the computational burden—both in terms of memory and compute—due to their dimensionality being linear complexity with respect to frozen weight matrix dimension which tends to be large. In our C-LoRA setting, for $B$ and $A$ LoRA adapters where $d\gg r$ (e.g., $d=4096$, $r=8$), this results in needing to predict a large number of mean and variance parameters per input ($O(d\cdot r)$), which hinders scalability and affects training robustness and stability. In contrast, the matrix $E$ operates in the lower $r\times r$ space (constant complexity w.r.t. $d$) and offers a computationally efficient and more stable avenue for introducing input-dependent behavior. By contextualizing $E$, we can model structured, input-specific variations in the adaptation layer without incurring the drawbacks of full contextualization. Therefore, our decision to restrict contextualization to $E$ is a deliberate tradeoff: it retains the ability to model data uncertainty (aleatoric uncertainty) effectively, while remaining computationally tractable and robust in practice. This strikes a meaningful balance between expressivity and efficiency.

W1.2 and Q2. Why $A$ and $B$ are not dependent on $x$?

Aligned with the previous answer, we specifically contextualize only $E$, and keep $A$ and $B$ deterministic and shared across inputs. This choice is driven by practical considerations: contextualizing $A$ and $B$, which are high-dimensional ($d \times r$), would introduce significant computational and memory overhead, and in our experiments, could reduce training stability. By modeling input-dependence through the lower-dimensional $E_x$, we strike a balance between efficiency and expressiveness—capturing aleatoric uncertainty while keeping the overall model lightweight and robust.

W1.3 and Q3. Why only $E$ is modeled in an autoregressive manner?

Here, by modeling $E_x^l$ in an autoregressive manner, we introduce an autoregressive distribution on $E_x^l$. As, $B$ and $A$ are learned deterministically, we do not consider them in an autoregressive fashion.

W2. Revision of the model writing

In section 3.3, the parameterization refers to the layer-wise auxiliary contextual module’s parameters. As mentioned on line 141, we parameterize the per-layer contextual module with a neural network with two fully-connected layers, collectively denoting their parameters by $\varphi$.

In line 154, for effective sampling from the distribution of $E_x^l$ we employ a widely known reparameterization trick from Bayesian Machine Learning literature.

We will edit these parts for improved clarity.

W3. Performance based on table 1 (improvement on ECE but potential task performance reduction )

We have discussed the trade-off between predictive performance and uncertainty estimation in the experiments section, supported by our empirical findings. That said, we would like to point out that C-LoRA consistently achieves strong performance across all three metrics—ACC, ECE, and NLL. Notably, in the low-sample setting ($M=0$), which is most relevant for practical efficiency, C-LoRA ($M=0$) outperforms BLoB ($M=0$) on both ECE and NLL across nearly all datasets, while maintaining competitive accuracy. Similarly, in the $M=10$ setting, C-LoRA performs on par or better than BLoB ($M=10$) in terms of NLL, and almost always outperforms all baselines on ECE, with comparable accuracy.

Moreover, in out-of-distribution (OOD) settings, C-LoRA achieves either the best or second-best scores in both ECE and NLL, demonstrating not only improved calibration but also robust generalization.

Thus, we acknowledge that C-LoRA may not always achieve the highest accuracy across all datasets; however, it consistently delivers strong uncertainty quantification (UQ) performance, as evidenced by the lowest or second-lowest ECE and NLL in most settings. We believe this reflects a favorable trade-off: while minor accuracy fluctuations (typically within 1–2%) can occur, these are well-compensated by substantial improvements in calibration and robustness, especially under distribution shift and limited inference budgets.

Q1. Why BNN limits UQ in few shot fine-tuning?

Conventional Bayesian neural networks (BNNs) typically learn a single, input-agnostic posterior distribution over model parameters. Accurately approximating both the mean and variance of that posterior typically requires large datasets, so in a few-shot LoRA scenario the posterior remains broad and prior-dominated. Because it doesn’t depend on the input, it produces uniform (“homoscedastic”) epistemic uncertainty across all $x$, failing to spike for novel or out-of-distribution inputs. In contrast, our C-LoRA injects stochasticity via input-conditioned latent variables, producing heteroscedastic, data-dependent uncertainty and delivering better calibration and predictive performance (as shown by lower NLL and ECE) even with scarce data.

Q4. How many parameters are manually set?

In our proposed work, we manually set only a limited number of hyper-parameters. These include the reduced dimension $r$, the architectural details of the contextual module such as the number of layers and hidden neurons, learning rates, and other similar training-related parameters. A list of these hyper-parameters along with their values is provided in the appendix (section D). Importantly, all other parameters of the model ($A$, $B$, and contextual modules’ weights) are learned automatically during training, without manual intervention.

We hope that we have addressed your concerns and look forward to the discussions if there are additional questions? Moreover, we kindly request that you reevaluate our work in light of our rebuttal.

Thank you for your detailed and insightful feedback. We are pleased that you recognize the motivation behind our approach and see value in the proposed method’s ability to represent aleatoric uncertainty, its robustness under distribution shift, and the empirical gains from contextual conditioning. We address your comments and suggestions point by point below.

W1. Emphasise how $\phi$ and $\varphi$ relate more clearly.

We want to clarify that $\phi$ represents all the parameters collectively, that is $\phi=\lbrace \theta, \varphi \rbrace$, where $\theta$ represents parameters of all the $B$ and $A$ adapters collectively and $\varphi$ represents parameters of all the contextual modules collectively. $q_\phi$ represents the inference network (encoder) in the amortized VI perspective. We will explicitly mention this for improved clarity.

W2. Whether the authors train $\theta$ in a discrete training phase before $\varphi$?

To clarify, the parameters $\theta$ and $\varphi$ are learned simultaneously in an interleaved manner, rather than in separate phases.

W3. The authors leave the use of diagonal Gaussian distributions throughout relatively unexamined.

While we did not explore diagonal Gaussian contextualization directly, we did conduct a complexity study (included in the appendix, section E) comparing diagonal versus full $E$ matrix parameterizations (detailed explanation of FE and DE is provided in Section 4.4 and Appendix section E). In this study, we found that using a full $E$ matrix led to significantly better uncertainty estimates (as measured by ECE and NLL) in both in-/out-of-distribution scenarios, while achieving similar accuracy compared to the diagonal variant. Based on these empirical results, we chose to focus on contextualizing the full $E$ matrix in the main experiments, as it offered more reliable performance across metrics of interest.

W4. Several nits.

We will address clarity improvements and nits underscored by you in the final version.

Q1. Is this sufficient scale to meaningfully learn to represent data uncertainty? When does $r$ become too small? Is this why you train C-LoRA for only 1500 iterations while the baselines for 5000?

Our choice of $r = 8$ adheres to the default settings in the PEFT library and LoRA paper, and is also adopted by BLoB. This ensures reproducibility and fair comparison with prior works. Moreover, low-rank adapters with small $r$ have been shown to offer a good balance between model capacity and training stability. The rank $r$ controls the number of stochastic parameters injected into the model. A larger $r$ increases expressiveness and the ability to model complex uncertainty, but can also lead to higher computational cost and potentially less stable training due to the increased stochasticity. Conversely, a very small $r$ can restrict the model’s capacity to represent uncertainty, especially in tasks with more complex latent structures. However, we acknowledge that the optimal value of $r$ can be task- and model-specific. Finally, we observed that C-LoRA converged reliably within 1500 iterations, and additional training did not improve performance. We attribute this to the lower-dimensional and structured nature of C-LoRA’s parameter space.

Q2. How does the fine-tuning time compare between vanilla LoRA and C-LoRA, and at inference time?

Training Time (in minutes)

Inference Time (in minutes)

We report both training and inference wall-clock times in the table where all experiments were conducted on the same hardware (NVIDIA A100 40GB). As shown in the table, the inference time of C-LoRA with $M=0$ is comparable to MLE (vanilla LoRA) across all datasets—e.g., 1.75 vs. 1.68 minutes on WG-S, and 6.8 vs. 4.55 minutes on BoolQ. This suggests that contextual modeling incurs minimal overhead at inference when no sampling is involved, while still producing well-calibrated predictions. For applications requiring more accurate uncertainty estimates, C-LoRA with sampling (e.g., $M=10$) leads to improved calibration with additional computational cost. Training time for both methods is comparable except for BoolQ. This longer runtime is likely due to its larger validation set, that increases evaluation overhead during training. This training cost can be reduced by adjusting the evaluation frequency. Finally, we note that while the total training time is comparable, the per-iteration runtime is higher for C-LoRA than for MLE (approximately 0.05 minutes vs. 0.02 minutes), due to the difference in the number of training iterations.

Q3. Whether it can be quantified that there is anything to be gained from a Bayesian treatment of parameters?

Our work focuses on aleatoric uncertainty, modeled via data-dependent variational distributions over the LoRA weights. This design is intentional: we aim to capture input-dependent uncertainty arising from ambiguous or multimodal outputs.

While placing distributions over parameters is often associated with epistemic uncertainty, the data-dependent nature of our variational posterior means the uncertainty varies with the input. This aligns more closely with aleatoric uncertainty, which reflects inherent variability in the data rather than model uncertainty. That said, placing a distribution over LoRA weights is complementary to the current approach and could capture residual epistemic uncertainty in these adapter parameters, as the backbone is frozen and most learnable capacity resides in the LoRA adapters.

We also compared against a data-independent variational posterior (BLoB in Sections 4.2, 4.3, and FE in Section 4.4) and found that the data-dependent version (C-LoRA) yields significantly better uncertainty estimates—suggesting aleatoric effects dominate in these experiments. Extending Bayesian modeling to the full network (e.g., the backbone) would be computationally nontrivial extension of our work and is beyond the scope of current work, but we agree it is a valuable direction for future work.

We hope that we have addressed your concerns and look forward to the discussions if there are additional questions.

Thank you for your thoughtful and constructive feedback. We are glad you found our approach novel, effective in uncertainty quantification, and robust across distribution shifts, and our experimental validations comprehensive. We also appreciate your questions and suggestions regarding model complexity, performance trade-offs, and prior sensitivity, which we address below.

W1. Code and data availability.

We understand the reviewer’s concern, but this year’s NeurIPS rebuttal guidelines do not permit providing any links to external pages including an anonymized code repository. We will definitely release our code upon acceptance. Additionally, we have included all relevant hyper-parameters, hardware specifications, and implementation details in the appendix (sections D and J).

W2. Marginally Lower Accuracy in Some Cases.

We have acknowledged the trade-off between predictive performance and uncertainty estimation in the experiments section where we provided discussions based on our results. We will further highlight this point in the introduction.

W3. Temperature scaling inconsistency.

Here, we conducted a deeper analysis using predictive entropy and mutual information to investigate different types of uncertainty as done in recent uncertainty disentanglement efforts. Particularly, we adopt the decomposition of predictive entropy (PE) from as in [1] (Section 2.2) and [2] (Section 3.4), where the expected entropy (EE) captures data uncertainty (aleatoric), and mutual information (MI) reflects epistemic uncertainty. Note that this decomposition follows the relation PE=MI+EE.

The reported results in the table show EE and MI computed at $M=10$ across both in-distribution (OBQA) and out-of-distribution (ARC-C, ARC-E, Chem, Phy) test sets, with the model trained on OBQA for all experiments. The proportion of aleatoric uncertainty out of total uncertainty captured by our model is consistently higher than the one captured by BLoB (on average, around 0.99 and 0.82 for C-LoRA and BLoB, respectively), demonstrating a stronger focus on modeling input-dependent uncertainty.

Since our model primarily captures aleatoric uncertainty, it is often well-calibrated in noisy or ambiguous regions. Temperature scaling, being a global post-hoc adjustment that does not account for input-dependent variability, may therefore have a limited effect or even slightly worse calibration in such settings. Hence, temperature scaling might need to be applied with some caution when input-dependent uncertainty modeling is involved.

Moreover, since temperature scaling optimizes NLL but not ECE—which is non-differentiable and bin-based—it may inadvertently harm calibration in certain regions. Prior work [3] has reported similar findings, where temperature scaling can degrade classwise-ECE (Appendix, section C.1).

Finally, we want to note that temperature scaling was used only as a standard post-hoc tool to potentially improve uncertainty estimation capabilities and is not part of our model.

EE and MI Scores (C-LoRA vs BLoB, M=10)

Q1. Could the authors provide more insights or a small ablation on how varying C (or the number of layers in the contextual module) might affect performance (both UQ and accuracy) and computational overhead?

In the original submission, we used a fixed architecture: a single hidden layer with 64 neurons (To clarify- In footnote on page 5, “two fully connected layers” denotes one hidden layer and an output layer in each contextual module). To explore the impact of model complexity, we conducted additional ablation experiments on the ARC-C dataset with $M = 0$, comparing the original model to its variants of varying complexity: 1) Simple: 1 hidden layer with 4 neurons, 2) Complex: 1 hidden layer with 100 neurons, and 3) Complex+: 2 hidden layers with 64 neurons each. Results (averaged over 3 random runs, on 1 A100 40 GB GPU) show that increasing the contextual module’s complexity leads to improvements in ECE and NLL, indicating better uncertainty quantification, at the expense of prediction accuracy. However, the original architecture offers strong predictive performance and reasonably good calibration with favorable NLL, making it a solid choice for the contextual module. That said, for tasks where better calibration and lower NLL are prioritized, a more complex architecture may be more suitable.

In terms of computational overhead, all models had similar training time (90–100 minutes), suggesting that the contextual module’s architecture has minimal impact on runtime, compared to the overall LoRA fine-tuning procedure in our setup.

Q2. Could the authors elaborate on scenarios or tasks where this accuracy difference might be a critical limiting factor for C-LORAs deployment, despite its superior UQ?

In many high-stakes applications—such as medical diagnosis, autonomous driving, financial risk modeling, and weather forecasting—well-calibrated uncertainty estimates are essential, and slight reductions in accuracy may be acceptable when they improve reliability and safety. However, in domains such as spam detection, malware classification, or other security-sensitive tasks, accuracy is often the top priority, and drop in performance may outweigh the benefits of better calibration.

We agree this trade-off is task-dependent and should be carefully considered when evaluating C-LoRA (and in general any uncertainty aware method) for deployment. We will include this in the conclusion and limitations section.

Q3. Could the authors discuss the sensitivity of C-LORA's performance to the choice of this prior? Are there alternative prior choices that might be more theoretically grounded or empirically beneficial?

We adopt a fixed Gaussian prior for $E_x$ which when paired with a Gaussian approximate posterior $q(E_x)$ enables a closed-form KL divergence, which reduces loss gradient variance and stabilizes training. Because our ELBO objective is estimated by Monte Carlo sampling (as NLL term does not have closed-form expression), we could also support alternative priors (e.g. Gaussian mixtures, Laplace, VampPrior, or normalizing-flow priors) as long as their KL is tractable or can be unbiasedly estimated. While richer priors might offer empirical benefits in some settings, they also introduce additional computational and design challenges. In practice, our contextual posterior $q(E_x)$ is highly expressive (parameterized by a neural network), so the Gaussian prior mainly acts as a light regularizer rather than a strict inductive bottleneck.

Moreover, prior work [4] has shown that Bayesian Model Averaging (BMA) is robust to the choice of prior scale and that posterior predictive behavior remains similar across prior families such as Gaussians, mixture of Gaussians, and logistic priors.

Finally, we want to point out that our aim is to compare inference and performance across methods under a consistent modeling setup. Since prior works- BLoB and LAP both use a simple Gaussian prior, this choice facilitates fair comparison.

We will include a brief discussion about the aforementioned points in the final version of the paper.

We hope that we have addressed your concerns and look forward to the discussions if there are additional questions.

[1] Mucsányi, Bálint, et al. “Benchmarking Uncertainty Disentanglement: Specialized Uncertainties for Specialized Tasks” NeurIPS 2024

[2] Jishnu Mukhoti and Yarin Gal. “Evaluating Bayesian Deep Learning Methods for Semantic Segmentation”. arXiv preprint, 2018.

[3] Kull, Meelis, et al. "Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration", NeurIPS 2019

[4] Izmailov, Pavel, et al. “What Are Bayesian Neural Network Posteriors Really Like?” ICML PMLR 2021