Selective Prediction via Training Dynamics

We thank the reviewer for their feedback on our work and address individual concerns below:

Added storage overhead for the intermediate models. Added inference cost for the prediction using the intermediate models compared to other selective prediction models that only require one forward pass through the architecture and the gating mechanism.

The reviewer is right that our approach requires storage of additional models and also has an increased inference time cost due to forward-propagating inputs through all models. However, deep ensembles, the best competing approach to date, also requires storage of multiple models as well as an increased inference time cost. Many past works from the selective classification domain [1,2,3,4] do not consider Deep Ensembles explicitly as a competing method. In summary, we observe that the added storage and inference time cost do contribute to better selective prediction performance.

References

[1] Feng, Leo, et al. "Towards Better Selective Classification." The Eleventh International Conference on Learning Representations. 2022.

[2] Huang, Lang, Chao Zhang, and Hongyang Zhang. "Self-adaptive training: beyond empirical risk minimization." Advances in neural information processing systems 33 (2020): 19365-19376.

[3] Liu, Ziyin, et al. "Deep gamblers: Learning to abstain with portfolio theory." Advances in Neural Information Processing Systems 32 (2019).

[4] Geifman, Yonatan, and Ran El-Yaniv. "Selectivenet: A deep neural network with an integrated reject option." International conference on machine learning. PMLR, 2019.

It is unclear which of the many intermediate checkpoints should be used for the inference stage.

See discussion below.

Have you plotted other baselines for the Figure 2 to see what impact these baselines have compared to SPTD?

We provide this extended experiment as part of the updated PDF in Figure 8. We see that all methods reliably improve over the SR baseline. At the same time, we notice that SAT and DE still assign higher confidence away from the data due to limited use of decision boundary oscillations. SPTD addresses this limitation and assigns more uniform uncertainty over the full data space.

How do you select which of the intermediate checkpoints should be used for the inference stage? [...]

Our method relies on first computing a very detailed approximation of the training dynamics and then subsampling (see Checkpoint Selection Strategy on page 8) and weighting the subsampled checkpoints using $v_t = (\frac{t}{T})^k$. You can think of the weighting as a prioritization to which part of the training dynamics to pay attention to. As we show below, our weighting is an appropriate choice for the problems we consider and outperforms alternative non-convex or uniform weighting strategies. We agree with the reviewer that other weighting strategies with potentially stronger performance might exist. However, these weightings might require deliberate distributional assumptions. Our work demonstrates that $v_t = (\frac{t}{T})^k$ with a flexible choice of $k$ enables SOTA selective classification performance across a wide range of datasets without much hyper-parameter tuning.

Have you tried other weighting schemes than (t/T)^k?

We have experimented with other weightings beside $v_t = (\frac{t}{T})^k$ with $k \in [0,\infty)$ but found that our choice of $v_t$ performs best across our experimental panel. This weighting encourages us to be particularly sensitive to late disagreements while at the same time incorporating diversity from models across the full training spectrum. We have updated the submission with a new experiment shown in Figure 13 in which we also consider concave weightings $k \in (0,1]$ as well as a uniform weighting assigning the same weight to all checkpoints. It is evident that our weighting choice performs better than other approaches. Finally, we want to remind the reviewer that, as described in Section 3.2, our choice for $v_t$ is not arbitrary but inspired by recent results from sample difficulty [1,2,3].

References:

[1] Baldock, Robert, Hartmut Maennel, and Behnam Neyshabur. "Deep learning through the lens of example difficulty." Advances in Neural Information Processing Systems 34 (2021): 10876-10889.

[2] Zhang, Chiyuan, et al. "Understanding deep learning (still) requires rethinking generalization." Communications of the ACM 64.3 (2021): 107-115.

[3] Jiang, Ziheng, et al. "Characterizing structural regularities of labeled data in overparameterized models." arXiv preprint arXiv:2002.03206 (2020).

Have you compared the inference cost of STPD with other methods (which only require one forward pass through the network and some gating mechanism)?

As is typical for ensemble-based methods, our method naturally incurs a higher inference-time cost compared to models that do not require access to multiple models. While an ensemble consisting of $M$ models incurs approximately $M$ times the inference cost of a model only relying on a single forward pass, we remark that ensemble-based methods (Deep Ensembles, Selective Prediction Training Dynamics, as well as their combination) provide significantly stronger selective accuracy at a fixed coverage level than single forward pass methods (see Table 1).

---

We hope that we have addressed the reviewer’s concerns and that the reviewer considers raising their score as a result.

We thank the reviewer for their feedback on our work and address individual concerns below:

Fail to cite some very related works on training dynamics (e.g., https://arxiv.org/pdf/2009.10795.pdf) as well as using training dynamics to analyze test sample (https://proceedings.mlr.press/v163/adila22a/adila22a.pdf)

We thank the reviewer for pointing us to these works. Here we clarify that while both suggested papers look at “training dynamics”, the quantities we are using are different: in both of the referenced papers the authors use variability while our metrics focus on convergence rates. Furthermore, note that variability is defined with respect to the correct label, whereas in selective prediction we do not know the ground truth label (and need to measure convergence in label prediction. To further set our method apart, we also demonstrate the applicability of our approach beyond classification and its composability with other selective classification approaches including composing on top of common ensembling techniques. Due to these key differences, we are confident that our work continues to be a valuable contribution to the selective prediction and uncertainty quantification communities. We have updated our related work section with a paragraph discussing this and other related training-dynamics specific works.

Considering the two works mentioned above, the novelty of the work seems less now. If the authors can come with a convincing argument on this, I would not be opposed to raising my score

The novelty in our method is the specific training dynamics metric we propose for selective prediction as well as our evaluation in conjunction with other approaches and on tasks beyond classification. The cited work uses variance of the logit output for the true label; while we do not have access to the true label, we can use the final predicted label as a proxy. Doing so, we present results in Figure 14 along with additional discussion in Appendix D.2.6 which shows that our metric (which emphasizes convergence) is more performative than the training dynamics metrics proposed in the cited work.

The numerical improvement over baseline (Table 1) seems very small.

We understand the reviewer’s concern about the small improvements shown in Table 1. At the same time, we would ask the reviewer to put the presented results into context:

• Many past works from the selective classification domain [1,2,3,4] do not consider Deep Ensembles explicitly as a competing method. Since we mainly consider our method as an alternative to DE, these improvements can appear small.
• Moreover, many past works [2,3] do not explicitly accuracy-align models at full coverage which can give the proposed methods an unfair head-start and as a result overestimate the method’s effectiveness. We make sure to compare all methods on an equal footing by disentangling selective prediction performance from gains in overall utility. We highlight the presence of accuracy alignment in the caption of Table 1.
• Finally, we highlight that our method’s advantages extend beyond the SPTD results reported in Table 1. This includes (i) ambiguity w.r.t the training stage; (ii) retroactive applicability; as well as (iii) composability with existing SC approaches.

References

[1] Feng, Leo, et al. "Towards Better Selective Classification." The Eleventh International Conference on Learning Representations. 2022.

[2] Huang, Lang, Chao Zhang, and Hongyang Zhang. "Self-adaptive training: beyond empirical risk minimization." Advances in neural information processing systems 33 (2020): 19365-19376.

[3] Liu, Ziyin, et al. "Deep gamblers: Learning to abstain with portfolio theory." Advances in Neural Information Processing Systems 32 (2019).

[4] Geifman, Yonatan, and Ran El-Yaniv. "Selectivenet: A deep neural network with an integrated reject option." International conference on machine learning. PMLR, 2019.

Distribution on g evaluation (Figure 4) is only done for the proposed method. If other baselines have the same pattern, the relative efficacy of the method becomes questionable

We provide an extended study on all methods in Figure 10. Since all methods are designed to address the selective prediction problem, they all manage to separate correct from incorrect points (albeit at varying success rates). We see that SPTD spreads the scores for incorrect points over a wide range with little overlap. We observe that for SR, incorrect and correct points both have their mode at approximately the same location which hinders performative selective classification. Although SAT and DE show larger bumps at larger score ranges for incorrect points, the separation with correct points is weaker as correct points also result in higher scores (i.e. longer blue tail) more often than for SPTD.

Have the authors try the method on OOD datasets? Can it reliably reject OOD samples?

The reviewer is right that the field of OOD detection is an important discipline in trustworthy ML related to SC. We therefore have already provided preliminary evidence in Figure 15 in the Appendix that our method can be used for detecting OOD examples (and also adversarial examples). While these results are already encouraging, we remark that adversarial and OOD samples are less well defined as incorrect data points and can come in a variety of different flavors (i.e., various kinds of attacks or various degrees of OOD-ness). As such, we strongly believe that future work is needed to determine whether a training-dynamics-based approach to SC can be reliably used for OOD and adversarial sample identification. In particular, a study of the exact observed training dynamics for both types of samples seems vital to ensure improved detectability. We remark that in our extended synthetic experiments in Figure 8, we observe that SAT and DE assign higher confidence away from the data due to limited use of decision boundary oscillations. SPTD addresses this limitation and assigns more uniform uncertainty over the full data space, enabling better OOD detectability.

---

We hope that we have addressed the reviewer’s concerns and that the reviewer considers raising their score as a result.

We thank the reviewer for their detailed assessment of our work and address individual concerns below:

First, the results are presented in a way that gives the impression that one can control the coverage. [...]

As part of our evaluation, we directly compute the accuracy/coverage tradeoff on the test set. This is consistent with how prior works evaluate their selective classification approaches [1,2,3,4] and allows us to highlight relative performance improvements of our method over competing works.

References

[1] Feng, Leo, et al. "Towards Better Selective Classification." ICLR. 2022.

[2] Huang, Lang, Chao Zhang, and Hongyang Zhang. "Self-adaptive training: beyond empirical risk minimization." NeurIPS 33 (2020).

[3] Liu, Ziyin, et al. "Deep gamblers: Learning to abstain with portfolio theory." NeurIPS 32 (2019).

[4] Geifman, Yonatan, and Ran El-Yaniv. "Selectivenet: A deep neural network with an integrated reject option." ICML, 2019.

Second, if I understand the main point correctly, the exploitation of the checkpoint model is mainly to help with uncertainty calibration. [...]

The reviewer is right that our method helps with uncertainty calibration. Although we think that the precise reason for the effectiveness of our particular method (in particular theoretical guarantees) should be part of future work, we do have some evidence that our method provides improved results due to diversity ensembling. Note that while model ensembling can be achieved in various different ways, many past works have found that a key ingredient to well-performing ensembling is sufficient diversity between ensemble members [1,2,3]. We do provide some empirical evidence for this connection in Figure 8 where we see that the decision boundaries considered by SPTD are significantly more diverse than the boundaries derived by DE or SAT.

References

[1] Kuncheva, Ludmila I., and Christopher J. Whitaker. "Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy." Machine learning 51 (2003): 181-207.

[2] Sollich, Peter, and Anders Krogh. "Learning with ensembles: How overfitting can be useful." NeurIPS 8 (1995).

[3] Morishita, Terufumi, et al. "Rethinking Fano’s Inequality in Ensemble Learning." ICML, 2022.

Third, does the tuning of the parameter k depend on test data?

Although the parameter $k$ can be tuned on a per-dataset basis, we found that on the datasets we consider the particular choice of $k$ is robust in the interval $[1,3]$ (see Figure 12). Ablating the parameter in this interval delivers comparable performance. We also consider even more choices for the weighting $v_t$ in Figure 13 and find that concave and uniform weighting schemes do not outperform the convex choice proposed in the main section of the paper.

Fourth, part of the claim is the proposed method can be applied on top of existing models. But is it always effective? Could we demonstrate such synergy between SPTD and other baselines? [...] Furthermore, I wonder how well a simple probabilistic method with explicit prediction uncertainty [...]

Table 1 already provides evidence that employing SPTD on top of DE yields improved performance. We also experiment with using SPTD on top of SAT and find that SPTD also improves performance in combination with SAT (see Figure 11). This result reinforces our claim that applying SPTD on top of existing approaches yields improved performance across various methods.

We are skeptical that additional results using BNNs or GPs would provide insightful alternate baselines. The reason for this is that Deep Ensembles, for which we provide results, have been found to dominate these methods [1,2]. Bayesian Neural Networks only allow for unimodal uncertainty quantification (approximate a single mode in the loss landscape) while Deep Ensembles allow for multimodal uncertainty estimation (approximate multiple modes in the loss landscape). Moreover, to the best of our knowledge, Gaussian Processes are rarely used as an uncertainty quantification method for high-dimensional image classification with large amounts of data. Inverting the Gram matrix is typically intractable in these use cases and approximations of the Gramm Matrix and its inversion reduce the quality of the provided uncertainty.

References

[1] Fort, Stanislav, Huiyi Hu, and Balaji Lakshminarayanan. "Deep ensembles: A loss landscape perspective." arXiv preprint arXiv:1912.02757 (2019).

[2] Ovadia, Yaniv, et al. "Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift." NeurIPS 32 (2019).

Last, will SPTD still be robust regarding checkpoint resolution if the model complexity increases? [...]

We have previously experimented with VGG architectures and have found our results to be consistent independent of the chosen architecture. We remark that ViT models have not yet shown to consistently provide better utility compared to ResNets on the datasets we consider [1] and hence opted against ViT experimentation in our work. To still showcase our method with larger model complexity, we show results for CIFAR-100 on a Resnet-50 architecture in Table 3. We observe that the effectiveness of SPTD carries over to this larger architecture.

References

[1] Zhu, Haoran, Boyuan Chen, and Carter Yang. "Understanding Why ViT Trains Badly on Small Datasets: An Intuitive Perspective." arXiv preprint arXiv:2302.03751 (2023).

How do you set the threshold? [...]

Across all experiments, $\tau$ is chosen to achieve a desired targeted coverage level. This is done by first computing the selection score $g$ across all data points, ranking the data points based on $g$ (effectively sorting them in ascending order), and then picking $\tau$ such that the first c% of test points are accepted. We don’t report these values for our experiments as they are method-dependent (different methods have different selection score distributions and therefore different thresholding values) and typically less interpretable than a specific targeted coverage level (for example, “accept 90% of data points” is often easier to understand than “accept all data points with selection score smaller than 0.5”). As described above and as is consistent with prior work, we compute the threshold on the test set directly.

I find this statement confusing: "checkpoint each model after processing 50 mini-batches of size 128. All models are trained over 200 epochs" [...]

The reviewer is right that we do indeed initially approximate the training dynamics on a very granular scale, leading to more than 1k checkpoints per dataset. As forward-propagating through > 1k checkpoints is evidently prohibitive, we show in the Checkpoint Selection Strategy paragraph on page 8 that subsampling 50, 25, or even 10 checkpoints still leads to strong selective classification. Especially for high targeted coverage (>50%), the performance across all checkpointing resolutions is indistinguishable from each other (Figure 5). This insight allows us to reduce the computational overhead of our method to the same cost as Deep Ensembles at test time while having a considerably less expensive training stage. Note that there is no need to compute the fine-grained trajectory and subsequently downsample to fewer checkpoints; this was merely done for our experiment to understand the limiting behavior of our approach. Practical implementations of our method can directly store checkpoints at a more coarse-grained resolution.

In addition, I wonder whether an explicit probabilistic method (such as BNN & GP) would be over-confident in the synthetic experiment. [...]

We provide an extended experiment as part of the updated PDF in Figure 8. We see that all methods reliably improve over the SR baseline. At the same time, we notice that SAT and DE still assign higher confidence away from the data due to limited use of decision boundary oscillations. SPTD addresses this limitation and assigns more uniform uncertainty over the full data space. We also provide a result using Bayesian linear regression in Figure 9 which gives results comparable to deep ensembles.

---

We hope that we have addressed the reviewer’s concerns and that the reviewer considers raising their score as a result.

Thank you for the detailed response. Your response has addressed many of my points but some remains.

First, I still think the authors' response to the uncertainty calibration is a bit weak given that it is one of the main point of the paper. Please do consider generate more diverse experiments to support this point in the next revision of this paper. But, I meant this as a constructive feedback only and I can take the current response under the scope of this rebuttal.

Second, I am now quite concerned with the statement that "As described above and as is consistent with prior work, we compute the threshold on the test set directly" -- if I understand this correctly, that means the authors have peeked into the test set to configure the learning algorithm. This seems to align with my point earlier that ultimately, we actually do not have a way to control the coverage. I suppose this has to be computed based on some statistics on the training set somehow.

Can the authors please elaborate more on this without deferring to prior work? It would even be better if the authors can explain the rationale here & highlight particularly how this algorithm will be used in a scenario where we do not know the test set in advance?

In addition, what will be the result if the authors use a threshold computed on the training dataset instead?

In a different discipline, it is generally not acceptable to configure the prediction algorithm based on statistics of the test set so I really have to press on this. Please address the above points.

We thank the reviewer for their positive assessment of our work and address individual concerns below:

The novelty of the method is limited, the ideas of re-using past checkpoints to form an ensemble can be found in e.g. [1]

We thank the reviewer for alerting us to this relevant work! We remark that past work on ensembling (including [1]) considers averaging the predictions (and/or computing second moments) to get a better estimator of a model’s prediction (and/or uncertainty). Instead we investigate using checkpoints/ensembles as a means to check if a datapoint follows temporal patterns typical of datapoints we should accept from an example difficulty viewpoint, leading to novel aggregation schemes for ensembles. To state in other words, although [1] also uses training dynamics implicitly by constructing an average checkpoint ensemble, the goal in [1] is not selective prediction but boosting overall accuracy. Our approach focuses on selective prediction which requires us to derive a different aggregation scheme than [1]. We also demonstrate the applicability of our approach beyond classification and its composability with other selective classification approaches including composing on top of common ensembling techniques. Due to these key differences, we are confident that our work continues to be a valuable contribution to the selective prediction and uncertainty quantification communities. We have updated our related work section with a paragraph discussing this and other related training-dynamics specific works.

The results for SPTD and Deep Ensemble (DE) are both relatively close to one another and it would be nice to derive conditions under which one method is expected to be better than the other.

We understand the reviewer’s concern about the small improvements shown in Table 1. We also agree with the reviewer that deriving theoretical conditions for the effectiveness of SPTD would be interesting future work. At the same time, we would ask the reviewer to put the presented results into context:

• Many past works from the selective classification domain [1,2,3,4] do not consider Deep Ensembles explicitly as a competing method. Since we mainly consider our method as an alternative to DE, these improvements can appear small.
• Moreover, many past works [2,3] do not explicitly accuracy-align models at full coverage which can give the proposed methods an unfair head-start and as a result overestimate the method’s effectiveness. We make sure to compare all methods on an equal footing by disentangling selective prediction performance from gains in overall utility. We highlight the presence of accuracy alignment in the caption of Table 1.
• Our current intuition for the strong performance of SPTD (and DE+SPTD) is based on increased diversity. Note that while model ensembling can be achieved in various different ways, many past works have found that a key ingredient to well-performing ensembling is sufficient diversity between ensemble members [5,6,7]. We do provide some empirical evidence for this connection in Figure 8 where we see that the decision boundaries considered by SPTD are significantly more diverse than the boundaries derived by DE or SAT.
• Finally, we highlight that our method’s advantages extend beyond the SPTD results reported in Table 1. This includes (i) transparency w.r.t the training stage; (ii) retroactive applicability; as well as (iii) composability with existing SC approaches.

References

[1] Feng, Leo, et al. "Towards Better Selective Classification." The Eleventh International Conference on Learning Representations. 2022.

[2] Huang, Lang, Chao Zhang, and Hongyang Zhang. "Self-adaptive training: beyond empirical risk minimization." Advances in neural information processing systems 33 (2020): 19365-19376.

[3] Liu, Ziyin, et al. "Deep gamblers: Learning to abstain with portfolio theory." Advances in Neural Information Processing Systems 32 (2019).

[4] Geifman, Yonatan, and Ran El-Yaniv. "Selectivenet: A deep neural network with an integrated reject option." International conference on machine learning. PMLR, 2019.

[5] Kuncheva, Ludmila I., and Christopher J. Whitaker. "Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy." Machine learning 51 (2003): 181-207.

[6] Sollich, Peter, and Anders Krogh. "Learning with ensembles: How overfitting can be useful." Advances in neural information processing systems 8 (1995).

[7] Morishita, Terufumi, et al. "Rethinking Fano’s Inequality in Ensemble Learning." International Conference on Machine Learning. PMLR, 2022.

It is unclear how the performance of SPTD is tied to optimization noise. Especially, regression experiments use full-batch gradient descent, how would the results evolve when using smaller batches?

As per the reviewer’s suggestion, we have run our regression experiments using mini-batches and find that the results were statistically indistinguishable from the results with full-batch gradient descent. Although the results are the same for the regression experiment, we do hypothesize that the randomness added by SGD is helpful in yielding more diverse intermediate models. As discussed above, diversity is a desirable property for ensemble models which in turn enables improved selective prediction performance.

The values for $v_t = (\frac{t}{T})^k$ seem a bit arbitrary.

We note that the weighting $v_t$ is chosen to reflect sample difficulty patterns as described by past work [1,2,3]. As we already describe in Section 3.2, prior work has found that easy-to-optimize samples are learned early in training and converge faster. Our convex weighting is inspired by this insight as well as the convergence patterns derived in Figure 6. We have experimented with other weightings beside $v_t = (\frac{t}{T})^k$ with $k \in [0,\infty)$ but found that our choice of $v_t$ performs best across our experimental panel. This weighting encourages us to be particularly sensitive to late disagreements while at the same time incorporating diversity from models across the full training spectrum. We have updated the submission with a new experiment shown in Figure 13 in which we also consider concave weightings $k \in (0,1]$ as well as a uniform weighting assigning the same weight to all checkpoints. It is evident from this experiment that our weighting choice performs better than other approaches.

References:

[1] Baldock, Robert, Hartmut Maennel, and Behnam Neyshabur. "Deep learning through the lens of example difficulty." Advances in Neural Information Processing Systems 34 (2021): 10876-10889.

[2] Zhang, Chiyuan, et al. "Understanding deep learning (still) requires rethinking generalization." Communications of the ACM 64.3 (2021): 107-115.

[3] Jiang, Ziheng, et al. "Characterizing structural regularities of labeled data in overparameterized models." arXiv preprint arXiv:2002.03206 (2020).

How would your method compare to using conformal-based selective prediction [2, section 5.5]?

The conformal-based selective prediction method in [2] is equivalent to softmax response (SR); they pick a threshold on $\hat{P}(x) = \max(\hat{f}(x))$. In particular, following typical conformal prediction methods, they use a calibration set to pick the threshold, while we (aligning with past selective prediction work) picked the threshold that gave a specific coverage/accuracy on the test set (as to compare this with other methods). In other words, conformal prediction in [2] deals with how to pick the threshold for softmax response, while our work shows there is a metric that gives a better signal for correctness than SR. Future work could try to apply conformal prediction to our SPTD metric as a means to pick the threshold.

---

We hope that we have addressed the reviewer’s concerns and that the reviewer considers raising their score as a result.

We thank the reviewer for their feedback on our work and address individual concerns below:

The proposed idea lacks novelty as it is very similar to using ensembles of models. The difference here is that the ensembles are generated on a fixed schedule from the training dynamics.

See discussion for next point.

Checkpoints are chosen based on a fixed schedule which can correspond to models of bad performance. A better approach is to follow the approach from [Huang et al. 2017] which constructs an ensemble by choosing points of good performance using a cyclic learning rate. Using good checkpoints removes the need for the complicated weighted aggregation of the disagreement functions as all the snapshots are good models.

We thank the reviewer for alerting us to [Huang et al. 2017]! We remark that past work on ensembling (including [Huang et al. 2017]) considers averaging the predictions (and/or computing second moments) to get a better estimator of a model’s prediction (and/or uncertainty). Instead we investigate using checkpoints/ensembles as a means to check if a datapoint follows temporal patterns typical of datapoints we should accept from an example difficulty viewpoint, leading to novel aggregation schemes for ensembles. To state in other words, although [Huang et al. 2017] also uses training dynamics implicitly by constructing an average checkpoint ensemble, the goal in [Huang et al. 2017] is not selective prediction but boosting overall accuracy. Our approach focuses on selective prediction which requires us to derive a different aggregation scheme than [Huang et al. 2017]. We also demonstrate the applicability of our approach beyond classification and its composability with other selective classification approaches including composing on top of common ensembling techniques. Due to these key differences, we are confident that our work continues to be a valuable contribution to the selective prediction and uncertainty quantification communities. We have updated our related work section with a paragraph discussing this and other related training-dynamics specific works.

The proposed method has several hyperparameters including the number of checkpoints and the weights to calculate the selection function $g$ from the disagreement function $a$.

We want to clarify that our method operates in the same number of hyper-parameters (checkpointing resolution, weighting parameter $k$) as other competing approaches. While SR is a hyper-parameter free method, Deep Gamblers (reward, pre-training duration), SAT (pre-training duration, SAT momentum for moving average), as well as Deep Ensembles (number of members, aggregation weighting) require tuning 2 hyperparameter, just like SPTD.

Several choices are not clear as described in the following section.

We address these concerns below.

How was $\tau$ chosen?

Across all experiments, $\tau$ is chosen to achieve a desired targeted coverage level. This is done by first computing the selection score $g$ across all data points, ranking the data points based on $g$ (effectively sorting them in ascending order), and then picking $\tau$ such that the first c% of points are accepted. We don’t report these values for our experiments as they are method-dependent (different methods have different selection score distributions and therefore different thresholding values) and typically less interpretable than a specific targeted coverage level (for example, “accept 90% of data points” is often easier to understand than “accept all data points with selection score smaller than 0.5”). Note that this procedure of reporting coverage instead of $\tau$ is the default way to evaluate selective prediction approaches [1,2,3,4].

References

[1] Feng, Leo, et al. "Towards Better Selective Classification." The Eleventh International Conference on Learning Representations. 2022.

[2] Huang, Lang, Chao Zhang, and Hongyang Zhang. "Self-adaptive training: beyond empirical risk minimization." Advances in neural information processing systems 33 (2020): 19365-19376.

[3] Liu, Ziyin, et al. "Deep gamblers: Learning to abstain with portfolio theory." Advances in Neural Information Processing Systems 32 (2019).

[4] Geifman, Yonatan, and Ran El-Yaniv. "Selectivenet: A deep neural network with an integrated reject option." International conference on machine learning. PMLR, 2019.

"Checkpoint each model after processing 50 mini-batches of size 128", how many checkpoints are chosen?

Like $\tau$, the total number of checkpoints is also dataset-dependent and we report the exact values for the different datasets in Table 2 in the Appendix. Note that in the paper we first aim to derive a very detailed characterization of the training dynamics, leading to hundreds of checkpoints for each model, but then downsample from this full set of checkpoints to obtain a more tangible algorithm that is still performative. We discuss this selection in our Checkpoint Selection Strategy paragraph on page 8. As discussed there, subsampling 10-25 checkpoints is enough for performative selective prediction on the datasets we consider. Moreover, when particularly targeting the high coverage spectrum (>50%), we find that SPTD provides SOTA selective classification performance across a wide range of checkpointing resolutions.

In Table 1, the name of the baseline is SAT but in the text, it is mentioned that the baseline is SAT with Entropy Regularization (ER) and Softmax Response (SR) Selection from [Feng et al. 2023]. Which one is the baseline? If the latter, then please update the table as SAT and SAT+ER+SR are 2 different methods.

We thank the reviewer for their attention to detail and have updated our draft with their suggestion.

Why not add SelectiveNet as a baseline for the regression task?

We have not benchmarked SelectiveNet on neither the classification nor the regression task since ensemble-based methods (which we do include in our comparison) dominate SelectiveNet in our classification experiments. We have now also ran SelectiveNet (SN) on the regression task and observe in our updated Figure 7 that SelectiveNet outperforms parametric outputs but does not reach selective utility levels obtained by DE or SPTD.

How were $g$ and $\tau$ chosen for the Deep Ensembles (DE)?

Consistent with the Deep Ensembles paper [1], we choose $g(x) = \frac{1}{M} \sum_{m \in [M]} \max f_m(x)$. Note that $f_m(x)$ corresponds to the full $C$-class logit output and $\max f_m(x)$ extracts the maximum logit value. $\tau$ is chosen as described above.

References

[1] Lakshminarayanan, Balaji, Alexander Pritzel, and Charles Blundell. "Simple and scalable predictive uncertainty estimation using deep ensembles." Advances in neural information processing systems 30 (2017).

What is the intuition of SPTD performing better than DE for some coverages? It seems counter-intuitive as DE consists of high-performing models vs SPTD which has fixed checkpoints that do not have to be high-performing.

Although we believe that future work should discuss this question in more detail, our current intuition for the strong performance of SPTD (and DE+SPTD) is based on increased diversity. Note that while model ensembling can be achieved in various different ways, many past works have found that a key ingredient to well-performing ensembling is sufficient diversity between ensemble members [1,2,3]. We do provide some empirical evidence for this connection in Figure 8 where we see that the decision boundaries considered by SPTD are significantly more diverse than the boundaries derived by DE or SAT.

References

[1] Kuncheva, Ludmila I., and Christopher J. Whitaker. "Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy." Machine learning 51 (2003): 181-207.

[2] Sollich, Peter, and Anders Krogh. "Learning with ensembles: How overfitting can be useful." Advances in neural information processing systems 8 (1995).

[3] Morishita, Terufumi, et al. "Rethinking Fano’s Inequality in Ensemble Learning." International Conference on Machine Learning. PMLR, 2022.

How are the disagreement function, $g$, and $\tau$ chosen for DE+SPTD?

For DE+SPTD, we first compute SPTD for multiple models $m \in [M]$ as follows: $g_{m}(x) = \sum_{t} v_t a_{m,t}(x)$. Note that $a_{m,t}(\cdot)$ now corresponds to the disagreement at time $t$ for model $m$. In a second step, we combine all SPTD scores into a single DE+SPTD score: $\frac{1}{M} \sum_{m \in [M]} g_{m}(x)$. This procedure is informally described in the Accuracy/Coverage Trade-off paragraph on page 7. Once again, $\tau$ is chosen as described above.

---

We hope that we have addressed the reviewer’s concerns and that the reviewer considers raising their score as a result.

I would like to thank the reviewers for their detailed responses to all the comments and their additional experimental results. I still have concerns about the paper.

Regarding the General Response

Reason for better performance (especially compared to ensembles): Being an ensemble method, our work benefits from diversity between individual members. Preliminary experimentation confirms that our approach yields models that are significantly more diverse than fully converged models from Deep Ensembles.

I do not agree with the authors that SPTD is likely to have more diversity compared to DE. One missing point here is how the checkpoints are chosen. If there are 25 checkpoints, are those generated from the last 25 epochs of training? Or are those selected at random by subsampling the large number of checkpoints? If the former, then the models will be similar. If the latter, then there are no guarantees on the quality of the checkpoints. On the other hand, DE and Snapshot Ensembles (SE) [Huang et al. 2017] rely on visiting different parts of the search space by either using random restarts (for DE) or by increasing the learning rate close to local optima (for SE). This should yield more diverse models than SPTD that uses models that are close in the optimization landscape.

Choice of $v_t$ : We note that our proposed convex weighting is inspired by results from example difficulty as well as our observed convergence patterns.

DE uses uniform weighting and there is no need to choose neither the functional form of the weights nor the hyper-parameters $k$. This makes DE (and other ensembles methods) more practical to use.

Regarding the response to my questions:

the total number of checkpoints is also dataset-dependent and we report the exact values for the different datasets in Table 2 in the Appendix.

For SPTD, only $T$ and $k$ are reported, not the number of checkpoints. Could you please clarify how many checkpoints were used?

We want to clarify that our method operates in the same number of hyper-parameters

There is also the choice of the weighting function which is arbitrary. The results shared in Figure 13 confirms the importance of the weighting function which can be dataset dependant.

Additional questions:

1- Why do all the methods have identical performance at 100% coverage when the performance at 100% should only depend on the training method used and should not affected by the selection criteria. For example, SR and SAT+SR+ER both score 77.6% accuracy for StandfordCars, although they should have different training mechanisms. [Feng et al. 2023, Table 3] reported a 5% performance gap for the 100% coverage between SR and SAT+SR+ER.

2- Based on the question above, could you please explain how the SR baseline was implemented? Which training method was used, loss function,...,etc?

3- For regression (Figure 7), could you please share the Mean Square Errors comparison?

References

• Huang, Gao, et al. "Snapshot ensembles: Train 1, get m for free." ICLR 2017.
• Feng, Leo, et al. "Towards Better Selective Classification." The Eleventh International Conference on Learning Representations. 2023.