Input-gradient space particle inference for neural network ensembles

Thanks for your review. We have updated our manuscript according to your comments and uploaded the new version to the system. Below we address the specific concerns.

The scale of experiments are quite small to show the effectiveness of FoRDE.

In our paper, we performed experiments using ResNet18 and PreActResNet18 architectures with CIFAR-10/100 and TinyImageNet datasets. These experiments have been designed to show effectiveness of FoRDE on scales comparable to earlier works on particle inference for neural networks [1, 2, 3] and neural network ensembles [4, 5].

The overall performance gain compared to other baselines looks quite marginal for the out-of-distribution datasets, especially for the TinyImageNet which is the largest dataset. And shows lower performance compared to the other baselines for the in-distribution datasets.

While our method achieve slightly lower performance on clean data compared to the baselines, we argue that the performance gain on corrupted data is much more substantial and is not trivial to achieve since we need not assume any prior knowledge of these corruptions, and FoRDE consistently outperforms the baselines on corruptions across different datasets. For instance, Table 1 and 2 in the manuscript shows that FoRDE-PCA achieves a +1.3% gain on CIFAR-100-C and +2.4% gain on CIFAR-10-C in accuracy compared to the second-best results.

Having empirical or theoretical evidence to demonstrate the effectiveness of FoRDE in enhancing input gradient diversity would be beneficial.

Providing empirical results that illustrate how the improved input gradient diversity effectively changes into enhanced functional diversity in deep neural network scenarios would be valuable.

To show that FoRDE indeed produces ensembles with higher input gradient diversity among member models, which in turn leads to higher functional diversity than DE, we visualize the input gradient distance and epistemic uncertainty of FoRDE and DE in Fig. 7 of Section D.1 in the Appendix of the revised manuscript. To measure the differences between input gradients, we use cosine distance, defined as $1-\cos(\mathbf{u}, \mathbf{v})$ where $\cos(\mathbf{u}, \mathbf{v})$ is the cosine similarity between two vectors $\mathbf{u}$ and $\mathbf{v}$. To quantify functional diversity, we calculate the epistemic uncertainty using the formula in [6], similarly to the transfer learning experiments. Fig. 7 shows that FoRDE has higher gradient distances among members compared to DE, while also having higher epistemic uncertainty across all levels of corruption severity. Intuitively, as the test inputs become more corrupted, epistemic uncertainty of both FoRDE and DE increases, and the input gradients between member models become more dissimilar for both methods. These results suggest that there could be a connection between input gradient diversity and functional diversity in neural network ensembles.

Additional hyperparameters for the kernel would be another burden for this method.

We did acknowledge this difficulty in our manuscript, and proposed setting the lengthscales based on the PCA of the training data which makes the model more robust against natural corruptions. For the revised version of the manuscript, we also included Section D.4 in the Appendix which shows that by choosing a lengthscale setting which is a linear combination of the identity lengthscales and the PCA lengthscales, FoRDE can achieve good performance on both clean and corrupted data. This can serve as a guideline for choosing the lengthscales when applying FoRDE on new datasets.

It is recommended to include an ethics statement and a reproducibility statement right after the main paper.

Thank you for your recommendation. We have included the ethics statement and reproducibility statement at the end of the revised manuscript, noting that these sections do not count towards the page limit according to the ICLR author guidelines.

[1] Z. Wang, T. Ren, J. Zhu, and B. Zhang, “Function Space Particle Optimization for Bayesian Neural Networks,” in ICLR 2019.

[2] F. D’Angelo and V. Fortuin, “Repulsive deep ensembles are Bayesian,” In NeuRIPS, 2021.

[3] S. Yashima, T. Suzuki, K. Ishikawa, I. Sato, and R. Kawakami, “Feature Space Particle Inference for Neural Network Ensembles,” in ICML 2022.

[4] A. Rame and M. Cord, “DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimation,” in ICLR 2021.

[5] T. Pang, K. Xu, C. Du, N. Chen, and J. Zhu, “Improving adversarial robustness via promoting ensemble diversity,” in ICLR 2019.

[6] S. Depeweg, J.-M. Hernandez-Lobato, F. Doshi-Velez, and S. Udluft, “Decomposition of Uncertainty in Bayesian Deep Learning for Efficient and Risk-sensitive Learning,” in ICML 2018.

Thank you for the positive review.

One thing I often worry about is that the experiments are performed only in the vision domain, on over-hygenic datasets. While I don't want to discount the amount of work needed to extend to other domains, projects like WILDS make this easier, and build confidence that demonstrated success isn't due to some quirk of CIFAR datasets.

Thank you for your suggestion. We were not aware of the WILDS project. We will certainly start using this project in our future works for more robust evaluations. Here, we followed previous works on Repulsive deep ensembles [1, 2] which focused mainly on benchmarking on vision datasets. We also provided experiment results on TinyImageNet, which is a bigger dataset with a higher number of classes (200 classes) compared to CIFAR.

One other complaint that to the authors' credit they highlight in section 3.5 is the cost of computing FoRDEs. At a 3x computational premium to DEs, the penalty paid in compute seems to be the largest drawback of FoRDE with respect to DEs. Do the authors have any ideas for reducing this burden? DEs themselves are expensive in both space and time to compute.

We are currently investigating methods to circumvent this computational drawback of FoRDE. One approach that we are studying is to replace the input-output Jacobian with the Jacobian-vector product. This approach can be implemented in Jax using the jax.jvp function, which efficiently calculates the Jacobian-vector product during the forward pass using the dual number approach [3]. In our preliminary experiments, we found that this new approach is almost as fast as plain DEs. We have added this discussion in Section 6 of the revised manuscript.

Regarding the motivation of the RBF kernel in the Choosing the base kernel paragraph of section 3.2, they are good arguments for using the RBF kernel, but are there others that were considered? As the authors suggest in section 3.3 and 3.4, choosing the length scales for RBF presents its own problem. Could this be circumvented by employing a simpler base kernel?

Here, we employed the RBF kernel following prior works on particle inference for neural networks [1, 2] and we did not consider other kernels for this task. While it is difficult to set the length scales for the RBF kernels, we also demonstrated that these length scales allow us to control the biases of the resulting ensemble, since we can improve the ensemble’s robustness against natural corruptions using the PCA length scale setting. For alternative kernels, we can explore the family of kernels on the unit sphere introduced in [4] since we compare input gradients on a unit sphere, which is an interesting future research direction. Another solution is to study dimensionality reduction techniques for input gradients before calculating the kernel, which also reduces the number of length scales to be set. We have added this discussion in Section 6 of the revised manuscript.

Again in section 3.3, is the median heuristic required? I’m a little unsure at the outset why this is the solution chosen over any others that would reduce the effect of the dominant eigenvalues.

Since we calculate the kernelized repulsion term using the RBF kernel, we need to use the median heuristic so that the RBF kernel does not vanish during optimization. To understand this, we can look at Eq. (14) in the manuscript. If the eigenvalues are large, then the square distance inside the exponential function of the RBF kernel can reach a large value during optimization, and thus the RBF kernel will quickly converge to 0 and the repulsion term will also reach 0. Therefore, we divide the square distance inside the exponential with a scalar estimated via the median heuristic so that the repulsion term does not vanish during optimization. Prior works [5, 6] also employ the median heuristic to avoid this vanishing problem when using the RBF kernel in particle variational inference.

[1] F. D’Angelo and V. Fortuin, “Repulsive deep ensembles are Bayesian,” In NeuRIPS, 2021.

[2] S. Yashima, T. Suzuki, K. Ishikawa, I. Sato, and R. Kawakami, “Feature Space Particle Inference for Neural Network Ensembles,” in ICML 2022.

[3] J. Bradbury et al., “JAX: composable transformations of Python+NumPy programs.” 2018.

[4] S. Jayasumana, R. Hartley, M. Salzmann, H. Li, and M. Harandi, “Optimizing over radial kernels on compact manifolds,” in CVPR 2014.

[5] Q. Liu and D. Wang, “Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm,” in NeuRIPS, 2016.

[6] C. Liu, J. Zhuo, P. Cheng, R. Zhang, and J. Zhu, “Understanding and Accelerating Particle-Based Variational Inference,” in ICML, 2019.

Thank you for the reviews. We have made changes to our manuscript based on your suggestions and uploaded the new version to the system. Below we address the specific concerns.

For this paper to be considered complete, it should not just acknowledge such ideal lengthscales but also offer experimental evidence of their practical identification.

We have included in Section D.4 in the Appendix of the revised manuscript anecdotal evidence that the ideal lengthscale setting lies somewhere between the identity lengthscales and the PCA lengthscales for the ResNet18 / CIFAR-100 experiments. Particularly, we train FoRDE under the lengthscale settings $\alpha * \mathrm{pca\\_lengthscales} + (1-\alpha) * \mathrm{identity\\_lengthscale}$, where we increase $\alpha$ from $0$ to $1$, and visualize the accuracy on both clean and corrupted data as a function of $\alpha$ in Fig. 11 in the Appendix. Fig. 11 shows that as $\alpha$ increases, FoRDE becomes more robust against corruptions, while exhibiting small degradation in performance on clean data. With $\alpha \in [0.1, 0.4]$, FoRDE achieves higher accuracy than DE on both clean and corrupted data, with $\alpha=0.4$ produces the best result.

The paper mentions the reasons for the ineffectiveness of weight-space repulsion: (1) "Typically repulsion is done in the weight space to capture different regions in the weight posterior. However, due to the over-parameterization of neural networks, weight-space repulsion suffers from redundancy." (2) "Weight-space repulsion is ineffective due to difficulties in comparing extremely high-dimensional weight vectors and the existence of weight symmetries (Fort et al., 2019; Entezari et al., 2022)." Could you provide a more detailed explanation of this?

Performance of an ensemble partly relies on the functional diversity of its members. The purpose of weight-space repulsion in ensemble learning is to find weight particles that are different from each other in the hope that they represent different functions in the function space. However, the weight posterior of a neural network contains weight symmetries, meaning that two different weight vectors can represent the same function [1]. If we average the predictions from these two weight vectors, it would be equivalent to using the prediction from one of those weight vectors, and we refer to this problem as redundancy. Additionally, modern neural networks contains large amounts of parameters (more than tens of millions of parameters) arranged into hierarchical structures with various components (such as non-linear activations, convolution, normalization and pooling layers), making the geometry of the weight posterior highly complex [2]. This complexity makes it difficult to define a meaningful kernel to measure the similarity between two weight vectors, which is needed to calculate the kernelized repulsion term. Previous works [3, 4] used the RBF kernel for weight-space repulsion and observed no improvement in performance compared to plain Deep ensembles.

[1] C. M. Bishop, Pattern recognition and machine learning. Chapter 5.1. 2006

[2] R. Entezari, H. Sedghi, O. Saukh, and B. Neyshabur, “The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks,” in ICLR, 2022.

[3] F. D’Angelo and V. Fortuin, “Repulsive deep ensembles are Bayesian,” In NeuRIPS, 2021.

[4] S. Yashima, T. Suzuki, K. Ishikawa, I. Sato, and R. Kawakami, “Feature Space Particle Inference for Neural Network Ensembles,” in ICML 2022.

Thank you for the suggestions which helped us further improve the manuscript, and we have uploaded the new version to the system. Below we address the specific concerns.

What corruption is considered? CIFAR-10/100-C datasets have several types of corruptions each of which has several level of severity of corruptions. No confidence intervals (+-) for corruption results.

We used all the available types of corruptions and all 5 levels of severity in CIFAR-10/100 and TinyImageNet datasets for evaluation. We have added these details in Section C.2 and C.3 in the Appendix of the updated manuscript. We also added figures containing the corruption results with confidence intervals for each level of severity in Section D.2 in the updated manuscript.

Section 3.1 doesn't address that the target distribution \pi is not available, or am I missing something?

You are correct. We have clarified that the target distribution $\pi$ is intractable (and hence approximated with the particle distribution in Eq. 6) in Section 3.1 of the revised manuscript.

It would help to clear some confusion of how Algorithm comes in place if steps in Algorithm would be linked to equations in Section 3. We have added clarifications in Section C.1 in the Appendix of the revised manuscript. Section 3.4. "However, in practice we found no performance degradation nor convergence issues in our experiments" - though the convergence issues can easily be observed, in order to see no performance degradation one would need to compare the performance with and without mini-batches. This experiment is not presented in the paper (including Appendix).

We wholeheartedly agree with this assessment. We however cannot perform the full-batch experiments due to resource limitations, since performing forward-backward passes on the entire 50000 training samples of CIFAR requires a large amount of GPU memory. One could compromise by varying the minibatch size, however it has been observed that large-batch training produces models with lower generalization performance than small-batch training [1], meaning that we would likely observe lower performance on large batch sizes compared to small batch sizes due to this phenomenon. Therefore, we have changed this statement to “However, in practice we found no convergence issues in our experiments'' in Section 3.4 of the revised manuscript.

Though the code is provided, some implementation details in text are missing. For example, ECE computation details such as a number of bins. Or details of OOD experiments: what portion of OOD data (CIFAR-100 for CIFAR-100 and vice versa) was used.

For all experiments, we used 15 bins to calculate ECE. For the OOD experiments, we calculated the epistemic uncertainty on the test sets of CIFAR-10/100 and CINIC10. We have added these details in Section C.2 and C.3 in the Appendix of the updated manuscript.

No reference for CINIC10 dataset.

We have added the reference in our updated manuscript.

[1] N. S. Keskar, D. Mudigere, J. Nocedal, M. Smelyanskiy, and P. T. P. Tang, “On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima,” in ICLR 2017.

We have added clarification in the current version of the manuscript available on the system. Specifically, we added the following sentence in the first paragraph of Section 5.2 in the main text:

For evaluations on input perturbations, we use CIFAR-10/100-C and TINYIMAGENET-C provided by Hendrycks & Gimpel (2017), which are datasets of corrupted test images containing 19 image corruption types across 5 levels of severity, and we report the accuracy, NLL and ECE averaged over all corruption types and severity levels (denoted cA, cNLL and cECE in Tables 1–4).