Scalable Ensemble Diversification for OOD Generalization and Detection

We thank the reviewer for writing the review. Below we will provide responses to the weaknesses (response to the weakness 1 will be added later once experiments are finished).

W2. The overall performance improvement of the proposed method seems to be marginal on 3 out of 5 of the domains in Table 2. Can you provide more analysis on the reason behind these results?

The primary objective of this paper is to introduce a methodology aimed at improving ensemble diversity.

Our results successfully demonstrate that HDR enhances ensemble diversity, as shown in Table 1.

OOD generalization represents a common application scenario in the field [a, b, c, d, e]. The results in Table 2 verify the usual wisdom that diversification helps OOD generalisation. While the improvement is not particularly pronounced, our findings suggest that the diversified ensemble remains a viable alternative to other ensemble methods, such as DE.

For OOD detection, however, diversification proves to be highly beneficial (see Table 3), achieving state-of-the-art performance across various benchmarks.

We believe the effectiveness of HDR is sufficiently demonstrated and that the contribution is significant enough for acceptance.

[a] Yoonho Lee, Huaxiu Yao, and Chelsea Finn. Diversify and disambiguate: Out-of-distribution robustness via disagreement. In International Conference on Learning Representations, 2023.

[b] Matteo Pagliardini, Martin Jaggi, François Fleuret, and Sai Praneeth Karimireddy. Agree to disagree: Diversity through disagreement for better transferability. In International Conference on Learning Representations, 2023.

[c] Teney, D., Abbasnejad, E., Lucey, S., & Hengel, A.V. (2021). Evading the Simplicity Bias: Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalization. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 16740-16751.

[d] Ross, A.S., Pan, W., Celi, L.A., & Doshi-Velez, F. (2019). Ensembles of Locally Independent Prediction Models. AAAI Conference on Artificial Intelligence.

[e] Teney, D., Peyrard, M., & Abbasnejad, E. (2022). Predicting is not Understanding: Recognizing and Addressing Underspecification in Machine Learning. European Conference on Computer Vision.

Dear reviewer eQEt, we would be grateful for an updated score if you have no further concerns. However, if you have any outstanding concerns, we would appreciate you sharing them now so that we have sufficient time to ensure the paper meets your expectations.

We thank the reviewer for the high-quality review of our paper and for providing a detailed list of weaknesses and questions. Below we provide responses to them (some responses will appear a bit later than others due to the time needed for experiments running and terminology revision):

W1. Explain design choice: agreeing and disagreeing on the same datapoints.

Yes, for some samples, the two objectives clash. Here's our formulation for sample $n$ and model pair $(m,l)$: $L_{HDR} = L_{main} + \alpha_n L_{div}$. When $\alpha_n > 0$, both terms are applied to the same sample $n$, leading to potential clash in objectives. We control the relative importance of the terms through $\alpha_n$: for harder samples, we make $\alpha_n$ greater, such that the relative weight of the diversification term is greater and vice versa.

What would have been cleaner is to control the weights like this: $L_{HDR} = (1-\alpha_n) L_{main} + \alpha_n L_{div}$, which is also possible indeed. We will run the corresponding experiments and post a separate response with them.

The membership status of each sample often shifts during training, particularly in the initial epochs. This dynamic behaviour is beneficial. The role of hard samples, those with higher $\alpha_n$ values, is to improve the models' ability to generate diverse outputs. If these hard samples change throughout training, it is important to adapt continuously to maximise diversification.

We will soon post a new response with the results of the progression of $\alpha_n$ values across datapoints to analyze ID/OOD membership fluctuations during training.

While the authors of divdis provided a code for parallel loss computation it can be seen that its complexity scales quadratically with the ensemble size (see line "marginal_p = torch.einsum("hd,ge->hgde", marginal_p, marginal_p)" in the code on page 16 [a]) as a consequence training an ensemble of big sizes on ImageNet is prohibitively expensive (for ensemble of size 50 one epoch takes more than 24 hours in our setup). Similar argument applies to A2D (quadratic complexity can be seen in line "for $i \in 0, \ldots , m − 1$ do" of algorithm 2 in page 14 in [b]). For this exact reason, we did not train ensembles of such size for DivDis and A2D and did not include them in Table 2.

Stochastic sum makes this loss scaling constant with respect to the ensemble size (diversification regularizer is computed for outputs of only two models for an ensemble of any size) and allows for training an ensemble of 50 models with 30 min per epoch. Shallow disagreement allows for further training speed up.

[a] Yoonho Lee, Huaxiu Yao, and Chelsea Finn. Diversify and disambiguate: Out-of-distribution robustness via disagreement. In International Conference on Learning Representations, 2023.

[b] Matteo Pagliardini, Martin Jaggi, François Fleuret, and Sai Praneeth Karimireddy. Agree to disagree: Diversity through disagreement for better transferability. In International Conference on Learning Representations, 2023.

What matters for diversification is not sourcing "OOD samples" - it's the hard samples that help (e.g. IN-R doesn't help A2D and DivDis in Table 2).

While OOD datasets might contain more informative hard samples than the training dataset, using them without proper filtering is not an option. Even within a different dataset than the training data, there exist samples that are close enough to in-distribution training data where models find it extremely difficult to propose plausibly diverse outputs (that is why baselines have low diversity in outputs even on OOD datasets, as can be seen in Table 1 - e.g. 1.04/5 unique predictions on average for ensemble of size 5 on IN-C-1).

That fact was not so pronounced with the previous approaches utilizing external OOD data. The possible reason is that they source the diversity-inducing OOD samples from the same distribution as the OOD evaluation set (e.g. waterbirds held-out set in [a, b]). This leads to confounding of whether the OOD dataset is helping with the diversification of the ensembles or it is leaking potential eval-set information.

Furthermore, even assuming that we decided to use OOD samples for disagreement, reliance on OOD data makes it unclear in practice which data should be chosen for each deployment scenario.

We have revised the terminologies and precise argumentation in §2.2 of the updated PDF.

[a] Yoonho Lee, Huaxiu Yao, and Chelsea Finn. Diversify and disambiguate: Out-of-distribution robustness via disagreement. In International Conference on Learning Representations, 2023.

[b] Matteo Pagliardini, Martin Jaggi, François Fleuret, and Sai Praneeth Karimireddy. Agree to disagree: Diversity through disagreement for better transferability. In International Conference on Learning Representations, 2023.

We thank the reviewer for the suggestion and will modify the revision with weights numbers and ablation study before 26.11

Dear reviewer sLK4, thank you once again for your valuable feedback and for posing thoughtful, actionable questions. If any of our responses met your expectations, we would greatly appreciate it if you might consider revising your score. Should you have any further questions, please don’t hesitate to reach out—we’d be delighted to provide additional clarifications.

We thank the reviewer for the thorough review of our paper and provide our response to the named weaknesses and answers below (some responses will appear a bit later than others due to the time needed for experiments running and terminology revision):

W1. Contribution is not significant: not clear how hardness measure helps OOD.

Our goal is to increase ensemble diversity to improve OOD generalisation and detection (§3.3, §3.4). Achieving this requires data points that encourage models to disagree. Previous work has used a held-out OOD dataset for model disagreement, where the OOD dataset actually comes from the same distribution as the OOD evaluation set (§4.2 in [a], §4.1 in [b]). This is a major limitation of earlier approaches.

We contribute our finding that for encouraging disagreement, what is important is not precisely the "OODness" but the "hardness" of the sample that invites diverse plausible hypotheses.

From the practical point of view, using hard samples is more effective than relying on OOD samples. It is resource-efficient because all data points come from the original training set. Empirically, it produces more diverse ensembles and leads to better OOD generalisation and detection. For example, disagreeing on hard training samples via HDR instead of OOD set of IN-R via A2D leads to the growth from 1.19 to 4.68 in average number of unique predictions (shown in Table 1 to verify better diversity); growth from 45.2% to 48.7% in ensemble accuracy on IN-R (shown in Table 2 to verify better OOD generalization); growth from 85.0% to 89.4% in AUROC with PDS (shown in Table 3 to verify better OOD detection).

[a] Yoonho Lee, Huaxiu Yao, and Chelsea Finn. Diversify and disambiguate: Out-of-distribution robustness via disagreement. In International Conference on Learning Representations, 2023.

[b] Matteo Pagliardini, Martin Jaggi, François Fleuret, and Sai Praneeth Karimireddy. Agree to disagree: Diversity through disagreement for better transferability. In International Conference on Learning Representations, 2023.

Thank you for the suggestion we will conduct the experiments on DomainNet and add them to the rebuttal before 26.11

The usage of stochastic sum is justified by the fact that gradient of the loss in eq. 6 computed for only a subset of models is an unbiased estimator of the full gradient computed for all models. Therefore, gradient steps will move the ensemble weights in the optimal direction in expectation and training with stochastic sum is a valid approximation of the full ensemble training.

In our experiments we used the smallest possible stochastic sum size of 2, as reducing it increases difference between ensemble members by diversifying the batches processed by each model during an epoch as well as reduces training time.

Below we will explain why stochastic sum gradient is an unbiased estimator of the full gradient of eq. 6 and why reducing stochastic sum size leads to additional diversification as well as provide empirical results supporting our choice of the stochastic sum size.

Stochastic sum is an unbiased estimator

A simplified version of equation 6 for one batch of data is:

$L = L_{main} + L_{div} = \frac{1}{|M|} \sum_{m \in M} L(m) + \frac{1}{|P_M|}\sum_{p \in P_{M}} G(p)$,

where $M$ is a set of all models in ensemble; $L(m)$ is loss computed for the $m$-th model output; p is a pair of models; $P_M$ is a set of all possible pairs of models; $G(p)$ is a regularizer computed for the outputs of p.

Let's imagine that on the current iteration we sampled a subset of models $I$. Then stochastic approximation of $\nabla L_{main}$ is:

$\overline{\nabla L_{main}} = \frac{1}{|I|} \sum_{m \in I}\nabla L(m)$

It is an unbiased estimate of the full gradient:

$E_{m \in M}[\overline{\nabla L_{main}}] = \frac{1}{|I|} \sum_{m \in I} E_{m \in M}[\nabla L(m)] = \frac{1}{|I|} \cdot |I| \frac{1}{M}\sum_{m \in M} \nabla L(m) = \nabla [\frac{1}{M}\sum_{m \in M} L(m)] = \nabla L_{main}$

Similarly, stochastic approximation of $\nabla L_{div}$ is:

$\overline{\nabla L_{div}} = \frac{1}{|I|} \sum_{p \in I}\nabla G(p)$

Which is also unbiased:

$E_{m \in M}[\overline{\nabla L_{div}}] = \frac{1}{|I|} \sum_{p \in P_I}E_{m \in M}[\nabla G(p)] = \frac{1}{|I|} \cdot |I| \cdot \frac{1}{|P_M|} \sum_{p \in P_M}\nabla G(p) = \nabla L_{div}$,

where $P_I$ is a set of all pairs of models from $I$.

Reducing stochastic sum size leads to better diversity

Smaller stochastic sums increase difference between ensemble members by diversifiying the subsets of the training data used to train each ensemble member. For example, with a stochastic sum of 2, each batch appears in the training subsets of only 2 models during one epoch. This is similar to bagging methods [a], where ensemble members benefit from training on different subsets of the training data.

Empirical results support the choice of stochastic sum size 2

Empirical results support the choice of stochastic sum size equal 2. As can be seen in Table 9 increasing stochastic sum size from 2 to 4 takes 8 times more time per epoch (453 vs 53 seconds) and results in worse OOD generalization performance across the board (e.g. IN-R accuracy drops from 48.1% to 46.8%) while giving only mixed gains on OOD detection (e.g. AUROC on IN-C-1 grows from 0.686 to 0.703 while AUROC on iNaturalist drops from 0.977 to 0.973).

Dear reviewer e4ou, thank you again for your feedback and for raising pertinent and actionable questions. If any of our response were satisfactory we would be extremely grateful if you could consider increasing your score. Please feel free to ask additional questions, we would be more than happy to provide further clarifications.