Dropout-Based Rashomon Set Exploration for Efficient Predictive Multiplicity Estimation

1. The biggest weakness of the approach concerns its limitations. As honestly discussed by the authors in Section 6 – Limitations, the fact that when the hypothesis space to explore is huge (that is when the predictor has many parameters to tune), the exploration that is done by the dropout approach is fairly limited. This is clearly related to Proposition 4, where an important value of M is necessary for layers having hundreds of neurons and an important value of k is necessary with complex neural networks.

   Indeed, when the hypothesis space is huge, exploring models in the Rashomon set, i.e., obtaining almost-equally-optimal models from the hypothesis space, is in general computationally infeasible. Despite dropout-based exploration is limited, it makes the exploration possible compared to other strategies such as re-training and AWP, which are much harder computational tasks as they requires repeated re-training and perturbation. We thank the reviewer for bringing this up, and will add a more thorough comparisons of the pros and cons among the three strategies, re-training, AWP, and the proposed one in Section 3 and 4.

2. Two scenarios could occur: the first one is that the hypothesis space to explore is relatively small. That is, it is fairly explored by the dropout method. But, even though a 30x to 5000x speedup over other approaches is seen, it is never defended that those other approaches are not scalable with small hypothesis space. Even though there is a huge speedup gain, if the other approaches are relatively fast (do not take hours to compute), then why favourising dropout? The second scenario is that the hypothesis space to explore is large. The time gain is then undermined by the limitation in the exploration. Plus, when it comes to large models, it is common to retrain only the classification head of the predictor, or to fix many layers; doing so could really fasten the retraining scheme, thus undermining the potential advantage of the dropout approach.

   We thank the reviewer for bringing up this practical concern. When the hypothesis space is relatively small, the re-training and AWP strategies could be fast; however, the dropout method could still be faster (despite that the speedup could be smaller) than re-training and AWP for the same reason—it does not require gradient computation and weight updating. Moreover, whether the proposed dropout method could explore the Rashomon set as effective as re-training or AWP when the hypothesis space is mall, is an interesting theoretical question and also requires numerical evidence.

   The second scenario is quite a common practice when fine-tuning a pre-trained model. Indeed, re-training and AWP could be significantly faster when only applied on part of the weights (e.g., the classification head). In this sense, the dropout method could also be applied on part of the weights in order to accelerate. The comparison of runtime performance among re-training, AWP, and the dropout method when applied on partial weights (also briefly discussed in Section 6 Future Directions.) is an interesting and practical research direction. Due to needlessness of updating weights, the proposed dropout strategy could still be more computationally efficient with the speedup varying depending on different complexity of the hypothesis space. Finally, as we have discussed in Section 6 Future Direction, the proposed dropout method could also be applied as a compliment to re-training, in order to leverage between efficiency and effectiveness of Rashomon set exploration. We will address this concern by adding a paragraph in Section 4 or 5 to further discuss the impact of the cardinality of the hypothesis space with some numerical examples.

Continued...

I thank the authors for their exhaustive response.

I don't feel like the responses (#1, #2, #4, #5, #6) actually answer my concern. For example, having literature comparing their approach to retraining (#4) might not be relevant here, for it depends on how these approaches resemble or not retraining on the point I raised in Weakness - Major - 3. Another example would be, response #5: sure, "[the goal of this paper is to] provides an alternative method to estimate predictive multiplicity metrics more efficiently", but in the article, what motivates its estimation is that "when predictive multiplicity is left out of account, an arbitrary choice of a single model in the Rashomon set may lead to systemic exclusion from critical opportunities, unexplainable discrimination, and unfairness, to individuals". Finally, response 6: even though the mass is concentrated around $\sqrt{d}$, it is uniformely distributed on each point at a given distance of the mean.

Response #3 did help me understand how a single empirical minimizer was shared by the approaches. I do feel like some of the details in Apprendix (some of the points I raised) deserve to be in the main part of the paper and not the appendices and am glad to know that some of that information will be transfered to the main part.

I thank again the authors for their honest responses; I will keep my score.

1. It is unclear what's the practical use of the propose method. It is fast, but it does not explore the Rashomon set well. For this reason, we can only mitigate predictive multiplicity as estimated by Dropout but not in general.

2. Following 1, it seems like additional experiments on whether the mitigation via Dropout also transfers to, say, AWP, is interesting.

   We address the reviewer's concerns in Weaknesses 1 and Weaknesses 2 together here. The good exploration of the Rashomon set, i.e., obtaining very diverse models with almost-equally-optimal performance, comes with the cost of efficiency (cf. e.g. [Xin et al., 2023], [Watson-Daniels et al., 2023]). With this in mind, the practical use of the proposed dropout method is to provide an alternative to the current re-training strategy that is not only more efficient but also bears with theoretical analysis (Proposition 1 to Proposition 4). Numerical results, e.g., in Figure 2, also show that the proposed method is comparable or outperforms the re-training strategy in terms of exploring the Rashomon set. Note that both re-training and the dropout methods give a subset (defined in (2)) of the true Rashomon set, and will all lead to an under-estimate of predictive multiplicity metrics. Therefore, higher estimates of the multiplicity metrics indicate a better exploration of the Rashomon set. To the best of our knowledge, it is the first time when the dropout method is connected and analyzed with notions of the Rashomon set and predictive multiplicity, making reporting predictive multiplicity for large-scale models possible.

   Moreover, the methods to mitigate predictive multiplicity discussed in Section 5 are not limited to the dropout methods. In fact, the ensemble method shown in Figure 4(a) work for any strategies that could obtain models in the Rashomon set. Since more models in the the ensemble leads to smaller multiplicity metrics, the proposed dropout method serves as an efficient way to obtain a huge amount of models.

   Mitigating predictive multiplicity, especially when selecting a model with the least predictive multiplicity from several given models, as shown in Figure 4(b), could also be achieved by the AWP. We will include an additional experiment in the Appendix of future revision. Lastly, please note that the re-training strategy can be combined with the dropout method to order to leverage the efficiency and efficacy of Rashomon set exploration, as already discussed in Section 6 Future Directions. We will summarize the discussion above and add a paragraph accordingly in Section 2.

3. The bounds in Proposition 2 and 3 only converge to 1 when $d\to\infty$, which does not seem like useful. See Q3 as well.

   We thank the reviewer for bring this up, and will add explanations and citations to support this assumption. The condition $d\to\infty$ is a regime commonly discussed in the asymptotic analysis of learning models such as over-parameterization [1], and the universal approximator theorem [2]. We would like to emphasize that the asymptotic behavior when $d\to\infty$ is based on the bound we derived in Eq. (5).

   [1] Cao, Y. and Gu, Q., 2020, April. Generalization error bounds of gradient descent for learning over-parameterized deep relu networks. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 34, No. 04, pp. 3349-3356).

   [2] Hornik, K., Stinchcombe, M. and White, H., 1989. Multilayer feedforward networks are universal approximators. Neural networks, 2(5), pp.359-366.

Continued...

We thank the reviewer for the feedback and encouragement! We address the questions point-by-point below.

1. Model augmented by Dropout could result in a fairly small search space of Rashomon set. I am not very convinced that this is a good idea in practice if our goal is to look for a better model that can address various reliability problem of predictive model. e.g. fairness etc. It maybe inspirational to see the movement of predictive multiplicity measurement, but I am wondering what is the practical meaning of it.

   We thank the reviewer to point out this confusion. Our goal is not finding models in the Rashomon set that comply with additional reliability problems such as fairness, but to efficiently estimate predictive multiplicity, despite that both involve exploring the Rashomon set, i.e., obtaining almost-equally-optimal models from the hypothesis space. As stated in the second paragraph in Section 1, our focus is estimating predictive multiplicity, a problem induced from the Rashomon effect, not the Rashomon effect stated in the first paragraph of Section 1. We agree with the reviewer that studying if the dropout models in the Rashomon set could lead to other reliability properties such as fairness is an interesting future direction.

   There is a trade-off between the efficiency and efficacy of exploring the Rashomon set and prediction multiplicity estimation. Note that re-training and our proposed dropout method all aim to approximate a subset of the entire (true) Rashomon set, and thus all estimates based on re-training or our method are under-estimation of the true multiplicity metrics. Despite that our method may only explore a small subset in the Rashomon set, it give an lower bound of the multiplicity metrics in an efficient manner, as clearly shown in Table 1. Moreover, the estimation using the dropout method also gives results comparable to re-training and AWP, as shown in the empirical evidence in Figure 2. Therefore, the proposed dropout strategy serves as yet another option (besides AWP and re-training) to explore the Rashomon set depending on the need of practitioners.

   We will summarize the discussion above and add it to the final revision.

2. The paper demonstrates the effectiveness of the proposed method on toy datasets that were used for decades. As the paper concerns the efficiency of existing methods, I am wondering if the authors can introduce more realistic tasks to show the effectiveness of the proposed method quantitatively. While COCO is good example, it is very qualitative without much statistic support.

   The datasets we used for evaluation throughout this paper includes 6 UCI datasets, CIFAR-10/-100, and the MS COCO dataset, ranging from small tabular data to large-scale image datasets. Indeed, the UCI datasets are small and have been used for decades; however, they are still widely used in predictive multiplicity literature, especially in the early stage of this new field. For example, the UCI Adult dataset is used in [Long et al., 2023] and [Kulynch et al., 2023]; the mammography dataset is used in [Watson-Daniels et al., 2023]; and other UCI datasets are also adopted in [Xin et al., 2022]. Note that all these literatures are very recent.

   Aside from the UCI datasets, we also evaluate the proposed method on image datasets such as CIFAR-10/-100 and the MS COCO, where the corresponding quantitative metrics including loss/accuracy changes and predictive multiplicity metrics are all reported in Sections E.2 and E.3, and also referred in the main text. We believe the datasets we used here are on par or more abundant than existing literature. We will re-emphasize the reasons (as above) why we select these datasets in the final version of the paper.

Continued...

We thank the reviewer for the feedback and encouragement! We address the questions point-by-point below.

1. In the experiments, the authors mentioned that "On the other hand, AWP outperforms both dropouts and re-training, since it adversarially searches the models that mostly flip the decisions toward all possible classes for each sample." I may miss some details of the method part, how can the proposed method to adversarially search the models since the dropout is random?

   We thank the reviewer for pointing out this confusion. We would like to further clarify that AWP and re-training are two existing baselines we compared against with our proposed dropout method. The AWP, re-training and ours are three different strategies to explore the Rashomon set, i.e., obtaining almost-equally-optimal models from the hypothesis space, and ours and AWP will not be applied at the same time. The AWP algorithm, as proposed in [Eq. 9, Hsu & Calmon, 2022], will adversarially perturb the output scores of a given sample to each class. For example, if there are $c$ classes, AWP maximizes each entries of $h\_w(x)$, i.e., $[h\_w(x)]\_i$ for each $i \in [c]$, under the constraint that the model after perturbation is still inside the Rashomon set. Therefore, for one sample, AWP requires to perform one perturbations (adversarial training) via SGD for each class (i.e., $c$ adversarial training in total for one sample), and if there is a dataset with $n$ samples, we need $n\times c$ adversarial training to estimate multiplicity metrics for the entire dataset. We can imagine that each of these $n\times c$ perturbations leads to a model in the Rashomon set that outputs the most inconsistent score for a given class per sample. On the other hand, the dropout method does not require to iterate over all samples and does not require gradient computations and updating model weights. In this sense, AWP is very different from the proposed dropout strategy.

   Due to space limit, we refer the readers to [Section 4, Hsu & Calmon, 2022] for more details of the AWP algorithm in the initial draft. We will add pseudo codes for the AWP in Appendix B, and a brief discussion in Section 2 to clarify the difference of the three strategies in the final version of the paper.

2. It seems the proposed method is only evaluated on in-distribution scenarios. Can it be applied to out-of-distribution data?

   We thank the reviewer for bringing this up. Indeed, the loss, accuracy and predictive multiplicity metrics reported in this paper are all evaluated on test data, i.e., unseen but in-distribution samples. The proposed method can also be applied for out-of-distribution (OOD) data. However, without special treatment (e.g., OOD generalization), the pre-trained model would have much higher loss $L(h_{w*}, \mathcal{D}\_{OOD})$ when fed with OOD data, leading to a larger Rashomon set (with the same $\epsilon$) and potentially much higher values of predictive multiplicity metrics (since multiplicity metrics are optimized over the Rashomon set; please also refer to [Defn 1, Hsu et al., 2022]). We will add a brief paragraph to include the discussion above in Section 3.

3. Are the uncertainty scores calibrated? In other words, are the confidence scores reliable?

   Thanks for the great question! We would like to further clarify that if a perfectly calibrated classifier assigns a 50% score to a sample (e.g., in binary classification), it does not necessarily mean that this sample has high multiplicity. A perfectly calibrated classifier is one whose predicted classes matches the true classes on average across samples (e.g., samples predicted to be 50% of one class have true outcomes matching that class 50% of the time). However, this does not necessarily translate to a (in)consistent set of predictions for a single target sample across equally calibrated classifiers. It may be the case that all calibrated models drawn from the Rashomon set assign the same 50% probability for that sample (no multiplicity). Conversely, some models may assign higher and lower confidence for that sample (high multiplicity) yet, on average, still be well-calibrated. Again, this happens because calibration (like accuracy and loss) is an average metric across all samples. We will add a brief discussion to include the response above in Section 4 in the final revision.

Thanks again for the review! We would be happy to provide more clarifications and answer any follow-up questions.