Deep Anti-Regularized Ensembles

• Abstract: I think you should introduce the concept of regularization reducing weights before stating the term "large weights network?" because at this point, the reader has no way of knowing the meaning of this statement until reading further.

• If you propose setting the tau validation loss parameter based on the validation loss of a trained model, then the method requires a complete training of a model from scratch before even beginning to train the proposed model. By definition, this training cannot be done in parallel.

• Below Equation 6, the text refers to "Assumption 4" which has not been properly defined. If the equations are meant to be assumptions, then they should be properly labeled as such.

• If $s^2$ is supposed to denote the input feature variance as stated before equation 7, and implied in the $X^\top X$ before equation 4, then the inputs features should be defined as being mean centered.

• Before Corollary 1.1, the following is said: "We see the importance of choosing a strictly concave function like the logarithm..." Where do we see this? Nothing has been demonstrated up until this point which would lead to this conclusion. If this is an insight gained from the specific details of the proof (which I had not read at this point in the paper), then something more useful should be said here.

• Before equation ten, the following statement is made: "To avoid this negative effect, the loss function is set to the mean squared error, scaled by the numberof classes." Could you be more specific about why this choice was made? Why should one scale the one hot vector with the number of classes? Wouldn't this mean that for a very high number of classes, the high magnitude outputs of a anti regualized ensemble would become hard to distinguish since K is also a high value?

• What are the red and green markers on the top row of plots in figure 2? They do not have a label.

• The presentation in Table 2 is confusing to read. It is not immediately clear whaty the accuracy and dataset columns are showing. One must deduce that for instance the SVHN, CIFAR100, and Accuracy columns under CIFAR10 are showing OOD Detection AUROC for SVHN and CIFAR100 and Accuracy on CIFAR10, is this correct?

• In the last paragraph before section 5: "We train 5 networks in each ensemble and repeat the experiments 5 times." If those are the only models which are trained, then how do you learn the $\tau$ parameter?

• Theoretical Analysis in Sec 2.3 - I found this section to be very unclear relative to the rest of the paper. While I think it is possible that there is an interesting motivation in this line of reasoning, I found it unconvincing in its current form. I would highly recommend that the authors re-write this section entirely to make their argument much more clear. I list some of the issues I encountered below:

  • The claim that this setting offers valuable insights into the behavior between layers in a neural network seems to not necessarily be true. Anything proved in this section and the intuitions developed may not directly carry over due to important differences (e.g. linear regression to a target optimizes a different loss than between two hidden layers of a neural network, this section seems to not account for interaction terms which would surely exist in a neural network, …). I would suggest that linear regression could be valuable in itself without needing to claim that it is a good model for hidden layers of a neural network.
  • Eqns (4), (5), and (6) were quite unclear to me. Are these purely assumptions/definitions? If so it might be better to use the correct notation “:=“. Particularly eqn (6) is unclear, would the solution to this arg max not simply be $\sigma = \infty$ without further assumptions? Are we maximizing the elements of the vector? The problem setup and message that the authors wish to communicate is unclear.
  • The connection to theorem 1 is not explicit. Are the authors saying that because eqn (6) takes its current form this means it is optimal in some way? It is not clear to me what is being shown here.
  • In this section and throughout the rest of the paper I would recommend being more clear in distinguishing between scalars and vectors by using bold font for the latter and for matrices.
  • Is the expectation over the training data?
  • On line 5 of paragraph 2, why is there an $n$ term multiplied by the loss function?
  • The statement $\theta$ centered in $\theta^*$. What does this mean? Or do the authors mean “centered at”?
  • Typo: “loosing in generality” -> “loss of generality”.

• Experimental design - I found that the experiments contained too many moving parts and resulted in it being difficult to conclude that the method was sufficiently isolated. I would have liked to have seen much more stripped-back experiments that provide clear ablated evidence of the authors' claims around their method. Furthermore, I would have liked to have seen more evidence around the claims that (a) some neurons activate infrequently on training data but more frequently on OOD data, (b) the weights associated with these neurons grow more than others, and (c) this results in higher uncertainty without simply resulting in degenerate predictions. I note a number of additional components that were at play in the experiments below.

  • The conformalization procedure used (page 8) adjusts the intervals produced by the authors' method by using a calibration set to guarantee marginal coverage in distribution. Since conformal quantile regression simply uses the provided intervals as an initialisation for its procedure it would seem that the performance gains could be due the the conformalization rather than the authors' method. It is unclear why the authors' relationship would make a particularly good initialization for this method and this is not discussed.
  • The pretrained ResNet used in the classification experiments would seem to undermine the random initialization aspect of ensembling upon which the method claimed to rely. Are the weights frozen or trainable?
  • Using multiple heads on a single base model seems to make this no longer an ensembling approach and also contradicts some of the ensemble-based motivation earlier in the paper.
  • The experiments seem to generally evaluate some form of coverage on OOD data. However, it is possible that this could be achieved by causing each ensemble member to perform degenerately but independently on OOD data. This may result in high variance predictions but would seem not to be useful in practice. I think these experiments need to provide some measure of error on OOD or at least in distribution test set data to ensure this is not the case.
  • The fact that a custom loss function was required and that the more standard softmax was found to be ineffective.
  • The inclusion of the $K$ term in this loss function seems to be another introduced heuristic.

• Related works - The authors seem to have missed [1] which introduced a very similar approach to the cited work of Pagliardini et. al. I would also suggest that [2] should be cited as an even earlier work that made an explicit connection between ensemble performance and diversity before the works cited already.

• Custom loss function - If we remove the $K$ term and replace the addition with a subtraction, the custom loss function proposed in eqn (10) appears to be exactly the multiple output MSE of the entire ensemble decomposed into the individual losses and the diversity between them as originally derived in [2]. Furthermore, it appears by adding (rather than subtracting) the second term the authors are, in fact, minimizing diversity rather than encouraging it as it appears they intended. This may actually explain why it performs more favorably as recent work in [3,4] have investigated this joint objective in the case of deep ensembles and found that discouraging diversity in this way results in better performance. Could the authors comment on the apparent contradiction between their penalisation of diversity in this objective and their goal of encouraging diversity on OOD data?

• Name of method - While I appreciate that regularisation is often performed by shrinking or sparsifying network weights, the more general idea of regularisation is of reducing the size of the hypothesis space using some inductive bias. In my view, the proposed method is doing exactly that by seeking solutions that maximize the magnitude of weights associated with neurons that activate more frequently on OOD data. Therefore, I would suggest that naming this method as “anti-regularisation” may be somewhat of a misnomer and doesn’t accurately reflect what the method actually does. A more accurate name would be something closer to “anti-shrinkage”.

[1] Lee, Y., Yao, H., & Finn, C. (2022, September). Diversify and disambiguate: Out-of-distribution robustness via disagreement. In The Eleventh International Conference on Learning Representations.

[2] Krogh, A., & Vedelsby, J. (1994). Neural network ensembles, cross validation, and active learning. Advances in neural information processing systems, 7.

[3] Abe, T., Buchanan, E. K., Pleiss, G., & Cunningham, J. P. (2023). Pathologies of Predictive Diversity in Deep Ensembles. arXiv preprint arXiv:2302.00704.

[4] Jeffares, A., Liu, T., Crabbé, J., & van der Schaar, M. (2023). Joint Training of Deep Ensembles Fails Due to Learner Collusion. arXiv preprint arXiv:2301.11323.

• In networks with normalization layers, e.g. BatchNorm, the network would become scale-invariance, then increasing the weight in intermediate layers would not change the output. The proposed method could therefore be less effective.

• The paper only conducts experiments with fully-connected layer models. I could not find discussions in the manuscript for this choice.

• The proposed method requires using MSE loss for classification, which is not a typical choice.

• The empirical results are not appealing enough: In table 2, when the in-distribution data is CIFAR-10, DARE has better AUROC in SVHN but worse AUROC on CIFAR100 compared with baseline approaches. Therefore it is unclear to me whether one should use DARE for problems of CIFAR-10 level complexity.

• The presentation of the results in table 2 is not very clear: If I understand correctly, the 1,2 and 4, 5 columns are AUROC and 3,6 columns are for accuracy? This should be revealed more clearly in the table caption.

The main weaknesses of this paper are the limited set of experiments and mixed results. ResNet-32 on CIFAR-10 with only the final layer ensembled provides limited data. It's a fine experiment to include in the paper, but a larger-scale experiment with a fully-ensembled bigger model on a bigger dataset (even on ImageNet, which is not that large anymore) would provide much more information on the viability of this approach and whether it provides enough benefit. Concretely, looking at Table 2, both in-distribution accuracy and OOD detection is mixed for DARE, including worse results for the important near-OOD case of CIFAR-100 vs CIFAR-10. As noted in the limitations section, this paper is also missing comparisons to Bayesian neural nets (rank-1 BNNs and Prior networks could be nice experimental comparisons here). In the current environment, experiments with language models would also be extremely useful. Overall, the experimental setup is limited and the results are not that compelling in their current state.