Do Bayesian Neural Networks Actually Behave Like Bayesian Models?

Thank you for your careful review and helpful feedback! We are glad you found our paper “engaging” and “well argued”. We hope our responses and new experiments below answer your questions and alleviate the concerns you raised.

The proposed alternative (martingale posteriors) is presented in a somewhat preliminary manner ...it remains unclear whether it truly resolves the issues identified with standard BNN algorithms.

While we agree that the paper could be improved by expanding on martingale posterior (MP) evaluations and will present new results on this shortly, we first wanted to discuss its role within our overall contributions. Our primary aim, as you eloquently explain, is to investigate the adherence of BNNs to the theoretical principles of Bayesian updating. Predictive coherence (Section 6) is one of these key principles. The MP framework is a novel tool we propose for this evaluation, which then naturally leads to our experiments in Figures 7 and 8 intended to identify the lack of coherence in existing approaches.

It is only after we have conducted these tests that the avenue of using MPs as a mechanism itself to potentially combat the problem becomes apparent. Given our original aims and the large amount of other content, we thus feel that fully exploring this new idea is not feasible within the scope of the current paper. Instead, we see this as a very exciting avenue for future investigation, adding significance to our work.

Nonetheless, we do plan to significantly expand on our discussion of the MP using the extra page in the camera ready (see also response to Reviewer 8qCC) and an exposition in the appendix for ML audiences, as you very rightly suggest.

The claim that [MPs] could be a worthwhile pursuit could be further substantiated …, further experiments for the [MP] could strengthen the paper’s concluding claim that MPs are worthy to pursue in Bayesian deep learning…

To provide more evidence for the generality of the MP findings, we have repeated our CIFAR experiments on the IMDB dataset: see here. The VI results are almost identical, strengthening the argument that the MP can provide performance gains in this setting. For SGLD, while the performance comparison of the MP is inconclusive, we still observe significant incoherence in uncertainty calibration, which this experiment was originally set up to test.

It would be valuable to compare [MPs] to the broader literature on alternative Bayesian approximations and uncertainty estimation techniques.

Thank you for raising this. We will add more discussions and references to the broader literature in Section 3 including non-standard “generalized” updates to Bayes’ rule (we assume this is what you mean by “alternative Bayesian approximations"), Gibbs posteriors, PAC-Bayesian methods, and non-Bayesian approaches like epistemic neural networks. Please let us know anything else you think would be important to discuss here.

Questions For Authors

1. The introduction states that BNNs “offer better predictive performance…than deterministic networks in multiple settings.” How do these claims hold up against deep ensembles?

Deep ensembles (DE) have also indeed been shown to offer better performance than deterministic networks, and we actively avoid taking a position on the relative superiority of BNNs and DE. Assessing non-Bayesian uncertainty quantification methods like DE under our Bayesian lens would certainly be an interesting future investigation.

2. The [MP] is defined without any conditions on the belief updating scheme … does this framework require specific assumptions to guarantee desirable properties?

We refer back to our initial discussion that the desirable property of MPs for the purpose of our experiment is to recover the initial posterior. Typically, they will not do so under approximate inference.

3. In the CIFAR-10 experiment, why does using the intermediary $\pi’$-predictive to impute labels lead to a well-defined [MP]?

The regularity assumptions needed for well-defined MPs are generally weak, requiring that the prior and updated belief densities exist, and BMA has finite expectation, clearly satisfied for classification where it is bounded. We will expand on this.

4. I’m not sure my reading of the CIFAR-10 experiment is correct.

The summary is correct. We exploit the property that sampling from $\Theta\sim\pi(\theta|y)$ is equivalent to imputing a random observation $Y'\sim p(y’|y)$ and then drawing the posterior sample $\Theta\sim\pi(\theta|y,y')$. Our experiments find that the samples obtained this way deviate from the original distribution in their BMA, as captured by our evaluation metrics.

We apologize we do not have the space to address all minor comments, but we will incorporate them into the paper.

Again, thank you for the review. Please let us know of any further questions or concerns.

Thank you for your careful review and helpful feedback! We hope that the responses and new results below answer your questions and alleviate any remaining concerns.

In a set of experiments like this, it would be nice to have a small version where exact inference is possible. This would help clarify which effects are due to HMC being treated as the gold standard and what is inherent to Bayesian inference itself.

Great suggestion! To provide this, we have performed an extra experiment with Bayesian linear regression: see here. To keep higher input dimensionality, we expand $x$ into 20 random Fourier features with a squared exponential kernel, approximating a (sinusoidal) Gaussian process regression. Exact inference demonstrates the desired coherent behavior. MFVI updating still shows coherence, despite its apparent lack of faithful posterior representation. For completeness, we have run all algorithms in the single-step setting and confirmed that HMC matches the true posterior. For this classical Bayesian setup, even our inexact approximate inference schemes exhibit Bayesian properties. This contrasts the BNN behavior we find at scale. We will expand on this running example to demonstrate the expected behavior with exact Bayesian updating through the paper.

The paper demonstrates several issues with approximate inference algorithms for BNNs but does not attempt to provide theoretical understanding of why these issues occur or what can be done to remedy them.

The primary objective of our paper is to highlight the discrepancy between the theoretical expectations and the actual behavior of BNNs. We thus believe it is natural that we focus on experimental evaluation rather than new theory. The theoretical reason for the issues is arguably quite straightforward: inexact inference causes substantial discrepancy in behavior from the true posterior. One could try to do theoretical analysis on this gap itself, but it would likely be challenging and prone to the same theory-practice gap we are investigating in the first place; we feel this is beyond the scope of what can be achieved in the current work.

In terms of remedies, these extremely challenging and fundamental issues at the core of the BNN methodology cannot feasibly be solved in a single paper. We see our work as the first step on this journey, highlighting and carefully documenting the issues, but we expect it to be many years before the community has managed to remedy them, if indeed they ever do.

Discussion about key takeaways is also relatively light…

Thank you for this feedback. We will exploit the extra page available for the camera ready to add more discussion on key takeaways. On the specific questions you raise:

Should practitioners avoid BNNs completely? Should they only be used in certain settings?

Definitely not! They have been shown to outperform point estimates and provide effective static uncertainty in multiple settings, which our results do not undermine. We simply argue that care is needed in our expectations and interpretation, especially outside these prediction settings where the pathologies we discuss might occur. In particular, we need to be very careful about empirical evaluation, as good static performance may not translate to other desired behaviors.

Is more research needed to develop inference schemes that respect e.g. coherence in sequential inference?

We would indeed argue that more research is needed in this direction. Moreover, in our opinion this should not be thought of as simply an inference scheme problem: entirely not Bayesian methods might in practice give more coherence under sequential inference. Even though the problems we raised are a direct result of inexact inference, improving inference schemes is not necessarily the solution: accurate inference may be unachievable, and strategies focusing on final behavior could be more effective. In short, this remains a highly unsolved problem with many possible future research directions.

Other questions we will discuss are Why does it matter that BNNs are not truly Bayesian? and If BNNs are not Bayesian, what are they?. Let us know if there are any other suggestions.

p. 3, l. 156: Are the sizes listed correctly here? 26 seems like a typo.

Well caught! Indeed this should say 256.

Are there any cases where the uncovered pathologies are actually beneficial?

There is never likely to be any direct benefit in the inconsistent behavior itself, but the deviations from the “correct” posterior may potentially sometimes be beneficial from a static prediction perspective. For instance, [1] found the calibration of VI to be consistently better than more faithful Bayesian approximations such as HMC.

Thank you again for your review and helpful suggestions!

[1] Izmailov et. al. (2021). What are Bayesian Neural Network posteriors really like? ICML

Thank you for your careful review and helpful feedback! We are glad you found our paper interesting, and hope that the responses and new results presented below help increase your confidence in backing acceptance of the paper.

I don't really understand why references on Bayesian inference […] are focused on Robins et al. and other citations of this sort [...]

Thank you for pointing this out. We agree and will update the references to some of the originals (e.g. [1,2]).

The BMA in Appendix A looks a bit trivial to me [...]

We appreciate this appendix indeed presents background that might appear elementary. It introduces terminology and makes our work accessible by detailing common concepts “often underexplained in Bayesian deep learning papers”, quoting reviewer KcGY. We will add a paragraph to better position this appendix. Thanks for highlighting it!

Why in Section 4, does it come back to the functional posterior predictive perspective? I don't see the point/utility here.

We think this is worth reiterating here. Pointwise uncertainty quantification (UQ) and BNNs improving UQ over SGD are of course well studied; the novel investigation here is to evaluate BNNs in dynamic settings. Indeed, VI, for example, produces good static results and uncertainty but its functional and uncertainty calibration properties deviate from Bayesian behaviors.

I am concerned about the assumption of heteroscedastic noise modelling in Sec. 4 [...] I'm a bit afraid that the use of such networks is somehow deteriorating the correctness of conclusions obtained from the empirical results there.

Thank you for pointing this out! It is true that the choice of observation model may have a significant impact on the performance and behavior of Bayesian models, and agree this is important to investigate. We highlight that there are relevant ablations already in our appendix (in Appendix F we examine model misspecification and in Appendix D we do a number of ablations on network architecture). To show, however, that our empirical findings are not merely an artefact of the difficulty in heteroskedastic noise calibration, we have repeated our experiment from Figure 3 with homoskedastic noise model: see here. We run inference with both a fixed noise variance, and placing an inverse Gamma hyper-prior on it for a fully Bayesian treatment. In both cases, the results agree with the conclusions of Figure 3.

How difficult/easy is satisfying that $\theta_N$ is a martingale?

$\theta_N$ is always a martingale under proper Bayesian updating regardless of architecture (subject to some very weak assumptions about the posterior existing). It will typically no longer be a perfect martingale under approximate updates, which is why we are using this as a test for adherence to Bayesian principles. We will add more high level discussion on this to the section to make it clearer.

...[Section 6] becomes somehow unclear due to the lack of space. The analysis for Fig 6 and 7 is super important, but results are presented in a quick manner with many details poorly discussed…

Thank you for this important feedback. We do agree the clarity has suffered here from needing to get things in length. We plan to use most of the extra page available for the camera ready to expand on and improve the clarity of this section. In particular, we plan to add more high-level intuition and explanation detail throughout, further interpret the results, and add more insights on the MP behavior itself. We have also repeated the experiments from Figures 7 and 8 with additional dataset, as discussed in the response to Reviewer KcGY. Please let us know if there is anything further you feel we should add.

The way the experiment is designed looks correct to me, but for instance, the intermediary pi’-predictive densities used to sample are also unclear (or I don't have a taste on how to obtain these in a reproducible way), and how the empirical MP is computed in L412 also doesn't give me a good feeling if applied to different BNN models.

The $\pi’$ distributions correspond to an initial trained BNN posterior. Our imputation follows the predictive resampling step from the original MP paper. We first draw a random setting of weights from the posterior $\Theta\sim\pi’(\theta)$, then generate a label from the predicted class probabilities $f_\Theta(x^*)$ in the case of classification (but it generalizes). For reproducibility, in our experiments we have taken sequentially increasing seeds. We will make updates to ensure this is clear.

Thank you again for your insightful review which has really helped us to improve the paper. Let us know if you have any further questions or concerns.

---

[1] Jeffreys, H. (1939). Theory of Probability. Clarendon Press.

[2] Savage, L. J. (1972). The foundations of statistics. Courier Corporation.

Thank you for your work in reviewing our paper. However, we believe there may have been some significant misunderstandings about our work and we are a little puzzled by both your summary of our work and the conclusions of your review, neither of which reflect our actual contributions.

For instance, the review claims the paper studies whether “different BNN posterior approximations adequately capture epistemic uncertainty by analyzing their behavior in both parameter-space i.e. weights and function-space i.e. predictive distributions”. However, our work is not about comparing how different posterior approximations capture epistemic uncertainty and we do not have any analysis or experimentation at all about parameter-space behavior.

The tradeoff between various approximate inference techniques in terms of their static predictive performance is indeed already well understood, as we acknowledge in the paper itself. As explained in, for example, the penultimate paragraph of our introduction, our work is instead about critically examining how well these BNN algorithms adhere to various important theoretical principles of Bayesian belief updating in dynamic settings, which are not in themselves often used to evaluate inference schemes. This goes beyond simply assessing predictive performance or understanding computational and expressivity trade-offs between different approximate inference schemes. The work shifts the focus to systematically examining the behavior of Bayesian updates, looking at functional consistency, coherency of sequential belief updates, and the self-consistency of predictive uncertainty.

Other reviewers recognize this perspective as significant empirical contributions not yet widely established within the community. We would therefore like to ask you to please reconsider your assessment.