Uncertainty-Aware Counterfactual Explanations using Bayesian Neural Nets

I thank the authors for the response. I will comment on individual answers.

This work investigates how the learning algorithm behind BNNs influences the explanations as compared to the learning algorithms behind MLPs and Ensembles. The fact that the CFX production is similar in BNNs/Ensembles is not a coincidence but in the design of the comparison. Adding softmax to the output layer provides a measure of aleatoric uncertainty. A BNN provides a natural measure of the epistemic (model) uncertainty in the prediction task. These two uncertainty measures are inherently different.

I thank the authors, I am aware of aleatoric and epistemic uncertainty and it clarifies this point for me. I am wondering whether this response will be reflected in the revised manuscript (?). The original version does not seem to do it. However, as commented by reviewer RyH5, there exist some prior works on Bayesian recourse (that I looked into briefly). I would expect a discussion around what separates the proposed method from such prior works in this domain. This seems to be missing entirely.

In general, CFX literature focuses on tabular datasets rather than vision, and accordingly, we also focus primarily on tabular data. The inclusion of MNIST was motivated by wishing to provide some visual exposition to our paper, as well as the fact that it is frequently included by other CFX literature. We would happily provide results on CIFAR in a later version of the paper but want to assert that vision is not the focus here.

I thank the authors for the response, but I am afraid that this statement is certainly incorrect. Papers such as [1, 2] use proper vision data sets for counterfactual explanations, just to name two examples from a plethora of works. As stated in my review, I only request that all claims are supported experimentally. If the claim about vision data sets could simply be removed, this would suffice to address the issue.

We believe that the CFX produced on BNNs do predominantly outperform those produced on MLPs and Ensembles; especially when considering the metric tradeoff discussed in the paper. If it would please the reviewer we are happy to change the word for ‘predominant’ or ‘frequently improved’ results.

I thank the authors. The proposed wordings seem appropriate.

Thank you for your positive remarks on our qualitative results. In producing these results we tuned the parameters to obtain the best explanation from each model. The result is that all explanations are somewhat interpretable; particularly for the Ensemble and BNN. With that said, we believe the differences are clear and important and that the generally-high quality of the explanations should not detract from the improvements the BNN-produced explanations exhibit. We agree that one of the qualitative figures could and should be moved to the appendix. Thank you for this suggestion, we will implement it in a future version of the paper.

I disagree that the differences in the qualitative results are clear, but I am aware that this is highly subjective. However, I believe that showing one example should suffice. I believe the space gained from this could be utilized more effectively for other relevant aspects that were brought up in the reviews.

I thank the authors for their efforts. Overall, I am still not convinced of the manuscript. The relevant aspects from my review have not been considered in the response. I will therefore keep my score.

[1] Augustin, Maximilian, et al. "Diffusion visual counterfactual explanations." Advances in Neural Information Processing Systems 35 (2022): 364-377.

[2] Rodriguez, Pau, et al. "Beyond trivial counterfactual explanations with diverse valuable explanations." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.

We thank the reviewer for their consideration and feedback, and we provide some clarifications on your points and questions below:

• This work investigates how the learning algorithm behind BNNs influences the explanations as compared to the learning algorithms behind MLPs and Ensembles. The fact that the CFX production is similar in BNNs/Ensembles is not a coincidence but in the design of the comparison.

• Adding softmax to the output layer provides a measure of aleatoric uncertainty. A BNN provides a natural measure of the epistemic (model) uncertainty in the prediction task. These two uncertainty measures are inherently different.

• We are happy to use ‘counterfactual explanation’ in place of ‘counterfactual’ and can see where the latter may be misleading. Thank you for this suggestion.

• In general, CFX literature focuses on tabular datasets rather than vision, and accordingly, we also focus primarily on tabular data. The inclusion of MNIST was motivated by wishing to provide some visual exposition to our paper, as well as the fact that it is frequently included by other CFX literature. We would happily provide results on CIFAR in a later version of the paper but want to assert that vision is not the focus here.

• We believe that the CFX produced on BNNs do predominantly outperform those produced on MLPs and Ensembles; especially when considering the metric tradeoff discussed in the paper. If it would please the reviewer we are happy to change the word for ‘predominant’ or ‘frequently improved’ results.

• Thank you for your positive remarks on our qualitative results. In producing these results we tuned the parameters to obtain the best explanation from each model. The result is that all explanations are somewhat interpretable; particularly for the Ensemble and BNN. With that said, we believe the differences are clear and important and that the generally-high quality of the explanations should not detract from the improvements the BNN-produced explanations exhibit. We agree that one of the qualitative figures could and should be moved to the appendix. Thank you for this suggestion, we will implement it in a future version of the paper.

• Regarding “we will focus” thank you for spotting this. We will make the correction to our paper.

We thank the reviewer for their consideration and feedback, and we provide some clarifications on your points and questions below:

• Summarizing the paper's contribution as “an empirical average over model predictions, rather than just a single model prediction” undermines the paper's true contribution. Firstly, BNNs allow minimizing the model’s uncertainty, which you cannot achieve in standard NN settings. Secondly, a formal definition of counterfactuals for BNNs was missing and our work is the first to formalize this definition. Thirdly, this work paves the way for more sophisticated definitions of counterfactuals that may use other techniques besides empirical averaging.

• Relation with ensembles. We believe that the connection with ensembling methods is rather tenuous. While both BNNs and ensembles are able to capture uncertainty, they do it in very different ways. Ensembling methods typically consider a finite set of models from which to sample, whereas BNNs represent a continuous probabilistic space of models from which we approximate the BNN with finite samples. While the AAMAS 2024 paper on argumentative ensembling proposed a very interesting approach to the problem, we believe the approach is of little relevance to what we proposed here. The authors of the AAMAS paper consider a finite number of models, each accompanied by a single counterfactual explanation computed for the single model. Then, they consider the problem of computing the largest set of models and counterfactuals that agree with each other. Our approach is completely different, as we are trying to compute a counterfactual that provably achieves the counterfactual class with high probability. The argumentative ensembling paper does not perform any kind of probabilistic reasoning to this end. For this reason, we see the two works as being radically different. 

• We also believe that our work has strong implications in terms of the robustness of counterfactuals, hence our experiments in Tables 2 and 3. However, our main focus here was not a new method to generate robust counterfactuals, rather we proposed the first approach to generate counterfactuals for BNNs. For this reason, we believe an extensive comparison with methods for robust counterfactuals/recourse is out of scope.

• Our intuition is that improved plausibility is a direct result of the BNN’s ability to better capture the data manifold.

We thank the reviewer for their consideration and feedback, and we provide some clarifications on your points and questions below:

• We do not fix a seed in our experiments and find that the results averaged over the 50 samples are consistent across different seeds. We have not reported standard deviations as this is unusual in the CFX literature. That notwithstanding, we agree that it would enhance our results to do so and will include this data in a future version of the paper. Thank you for this suggestion.

• The figures are intended to accompany the numeric results in a way which helps the reader understand where the numeric discrepancy originates (specifically for the MNIST data). We agree that alone these figures are insufficient to draw a conclusion from but we hope they provide better context to other results.

• We disagree that the “motivation behind CFX is to explain models that would be good at prediction”. Instead posit that the motivation behind CFX is to explain models that are used, regardless of their performance metrics. It is the case that for some tasks a practitioner may prefer a model which can naturally model epistemic uncertainty, such as a BNN, and still require explanations on this model. Moreover, as indicated in our results, the performance of the BNN is comparable with that of the MLP in the datasets we test on.

We thank the reviewer for their consideration and feedback, and we provide some clarifications on your points and questions below:

• Regarding network scale, we have used network architectures in line with those from previous papers on CFX. With that said, we would happily expand the network scale in our experiments for a future version of the paper.

• Previous work counterfactual explanations have highlighted some connections between plausibility, proximity and robustness by means of empirical evaluations (e.g., higher plausibility seems to correlate with higher cost, i.e. lower proximity). However, to the best of our knowledge, a full theoretical study on the interplay between these 5 properties has not been carried out yet. Sparsity could be incorporated by adding a dedicated loss term to the loss in Eq 8. Similarly, actionability could be incorporated by penalizing perturbations on features that are not considered to be actionable. Our experiments only focused on validity, plausibility and proximity as adding sparsity and actionability would complicate the optimisation problem further, potentially introducing additional dependencies in the problem that might hinder our exploratory analysis of the properties of counterfactuals in BNNs.

• We do not use gradient accumulation.

• When mentioning the limitations of deterministic NNs we mean that in general, they struggle to accurately capture the underlying data manifold. This is evidenced by their tendency to overfit and the presence of often-cheap adversarial attacks in their input space.

• $l$ is the index that identifies the output vector component (or more precisely, marginal posterior predictive probability). Admittedly, the index $l$ is missing in the first half of the definition. The correct definition is as follows:

• Definition (Probabilistic Counterfactual). Given a BNN $\\mathcal{B}$, an input $x$, with observed negative outcome, $\\mathcal{B}(x)\_l = 0$, a probabilistic counterfactual is an input $x\_c$ such that the output achieves the desired outcome i.e., $\\mathcal{B}(x\_c)\_l = \\mathbb{E}\_{w}$ f_{w \sim P(w)}(x) $\_l = 1$. Formally,$x\_c = \\argmin\_{x\_c} d(x, x\_c) \\quad \\text{s.t.} \\quad {\\argmax\_l\\mathcal{B}}(x\_c)\_l = 1.$

• Regarding the use of terminology, we agree that the word “unit” is not common in the BNN literature. Thank you for pointing this out. We have replaced it with marginal predictive probability.