Generative Classifiers Avoid Shortcut Solutions

We really appreciate your comments and suggestions! We answer your questions below:

Comparison to classifier with pretraining and data augmentation: although pretraining does not seem to improve the proposed model, it does have a big importance for standard classifiers

To be clear, we didn’t find that “pretraining does not seem to improve the proposed model”; we believed it would be difficult to fairly match the strength of pretraining across all methods, and had decided to train all models from scratch instead. Based on your suggestion, we’ve now added new results to Appendix A.7 that show pretrained discriminative models do not achieve the same “effective robustness” (better OOD accuracy for any given ID performance) as even a generative classifier trained from scratch.

Was [data augmentation] included in the discriminative models reported in the paper?

We train all image models in this paper (discriminative and generative) with the same set of standard augmentations: color normalization, random horizontal flip, and RandomResizedCrop. We’ve clarified this in our section on training details (Appendix B.1.2). We’ve also added results with stronger augmentations for discriminative models on several new benchmarks (see next point), and find that they do not close the “effective robustness” gap with generative models.

Does the proposed model scale to multi-class classification problems?

Yes, generative classifiers do work on multi-class classification problems! Our current results on FMoW show that generative classifiers work on 62-way classification. To be thorough, we’ve now added results on the Living-17 and Entity-30 distribution shifts from the popular BREEDS benchmark [1]. These datasets have 17 and 30 classes, respectively. We find that generative classifiers again display “effective robustness” compared to discriminative models. We also use the results reported in the paper as strong discriminative baselines, which include stronger techniques such as adversarial training or stronger augmentations (RandErase, gaussian noise, or stylization). Our generative classifiers achieve better OOD accuracy than any of these interventions. Thank you for your suggestion to try additional multiclass datasets!

Please let us know if you have any further questions!

[1] Santurkar, S., Tsipras, D., & Madry, A. (2020). Breeds: Benchmarks for subpopulation shift. arXiv preprint arXiv:2008.04859.

We really appreciate your comments and suggestions! We answer your questions below:

I think some recent papers about the superiority of generative classifiers should be mentioned in this paper

Thank you for the recommendations! The two papers on improving adversarial robustness with generative classifiers are definitely related; we do note that previous work has found that adversarial training has not improved robustness to distribution shift [1,2,3], so it’s not expected based on these two papers that generative classifiers would be better at distribution shift. We’ve added citations to these papers and made a comment to readers about this, thanks!

In line 322, why do you choose epoch 5? Can this affect the final results?

We previously chose epoch 5 in order to avoid any “warmup effect” at the very beginning of training. In practice, we double checked, and there is no qualitative difference in these plots if we use epoch 1 vs epoch 5.

In line 341, the increase of gradient norms makes me confused. It should decrease during the training because the gradients will converge to 0.

We plot the average of the per-example gradient norms $\frac{1}{n} \sum_i ||\nabla_\theta \mathcal L (x_i, y_i)||$, not the norm of the average gradient $||\frac{1}{n} \sum_i \nabla_\theta \mathcal L (x_i, y_i)||$. For generative models, the latter quantity converges to 0, but the per-example gradient norms do not (since there is always a way to increase the likelihood of a single data point). This is good, because it means that each datapoint always provides useful learning signal throughout training, as opposed to the learning signal rapidly decaying to 0 (which happens for discriminative models).

In line 364, the paper claims that parameter count does not seem to be responsible for the performance of the generative classifier. Does this mean, when considering downstream discriminative tasks, we can not enjoy the scaling law of generative foundation models?

This is a really good question! To summarize Section 5.3, we found that increasing discriminative model size does not improve performance. This limitation for discriminative models has also been previously observed and theoretically characterized [4]. Based on your suggestion, we’ve now added scaling results for generative classifiers in Appendix A.4 and Fig. 13, and we observe that generative classifiers can benefit a lot from scaling up model size. We really appreciate your comment, and think that this finding that “generative classifiers benefit more from scaling than discriminative classifiers do” is quite interesting!

[1] Taori, R., Dave, A., Shankar, V., Carlini, N., Recht, B., & Schmidt, L. (2020). Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems, 33, 18583-18599.

[2] Santurkar, S., Tsipras, D., & Madry, A. (2020). Breeds: Benchmarks for subpopulation shift. arXiv preprint arXiv:2008.04859.

[3] Miller, J. P., Taori, R., Raghunathan, A., Sagawa, S., Koh, P. W., Shankar, V., ... & Schmidt, L. (2021, July). Accuracy on the line: on the strong correlation between out-of-distribution and in-distribution generalization. In International conference on machine learning (pp. 7721-7735). PMLR.

[4] Sagawa, S., Raghunathan, A., Koh, P. W., & Liang, P. (2020, November). An investigation of why overparameterization exacerbates spurious correlations. In International Conference on Machine Learning (pp. 8346-8356). PMLR.

Regarding model size in the experiments, table 2 is also on CivilComments so that only covers one dataset. Moreover, on this dataset you performed worse than the discriminative models on the ID while you performed better than the other models on 3/4 of image datasets. This shows that the CivilComments behavior might not transfer. Also, parameter count isn't an exact comparison.

Figure 4 in our paper shows that scaling up the discriminative model does not improve performance at all on Waterbirds. Based on your suggestion, we’ve also added Fig. 12 to the Appendix, which shows the same conclusions on another 2 of our datasets. Finally, the CivilComments results in Table 2 should be a fair comparison, since the discriminative and generative models there have the same parameter count and architecture. Overall, we are quite confident that discriminative model size is not a confounder in our experiments.

In two of the datasets, waterbirds and FMoW you can see in Fig. 2 that the ID and OOD seem to be very highly correlated when looking over all models. Looking at these plots, especially FMoW, it is not clear at all that the better OOD performance isn't just because your base model had better ID performance and not because of some innate property of generative models.

We’re happy to clarify this! Previous work has found that ID and OOD accuracy are strongly correlated across almost all distribution shift benchmarks [5,7]. This phenomenon is called “accuracy on the line” — models that do better ID will do predictably better OOD. The most interesting previous finding is that all interventions (changing model architecture or size, augmentations, adversarial training) that affect OOD accuracy only do so by affecting ID accuracy and moving the model along the line. Only adding more diverse training data (which reduces or eliminates the distribution shift) has been found to move models vertically above the line [5,12]. This is called “effective robustness,” and it indicates that these models are truly more innately robust to distribution shift than the baseline models that lie on the line.

Our finding is that generative classifiers are the first approach to show “effective robustness,” and we observe it on most of our datasets (CelebA, CivilComments, and Living-17 and Entity-30 in our new results in Fig. 14). Generative classifiers don’t always have effective robustness (as you pointed out on Waterbirds or FMoW and as we wrote on L302). This agrees with the predictions from our “generalization phase diagrams” — generative classifiers will sometimes just be better both ID and OOD, based on the properties of the data. Overall, we think it’s exciting that generative classifiers are the first method to show effective robustness without using extra training data.

L28 you write "Ever since AlexNet (Krizhevsky et al., 2012), classification has mainly been tackled with discriminative methods" but this isn't accurate as discriminative models have been the standard approach even before AlexNet, just mainly SVMs/decision trees/boosting etc.

Thanks for pointing this out — we’ve modified this to be more clear that we’re referring to classification with neural networks.

Please let us know if you have any further questions!

Thank you for your feedback! We’ve incorporated your suggestions into our paper, and answer your questions below:

The main issue is limited novelty. Using generative models for classification isn't a new idea, and previous works already explored their robustness to OOD generalization and to adversarial attacks (and also the use of diffusion models as classifiers. For example Zimmermann et al "Score-Based Generative Classifiers", Grathwohl et al "YOUR CLASSIFIER IS SECRETLY AN ENERGY BASED MODEL AND YOU SHOULD TREAT IT LIKE ONE", Chen et al "Robust Classification via a Single Diffusion Model" and Fetaya et al "Understanding the Limitations of Conditional Generative Models". The paper should also add them as related works.

Thanks for the pointers to these papers — we’ve added them to our related works section along with a short discussion. However, we’d like to point out the novel contributions in our paper:

• We show that generative classifiers are particularly robust to distribution shift. This has not been found in previous work. Zimmerman et al. [1] found that their generative classifier performed worse on distribution shift than discriminative models. Grathwohl et al. [2] and Chen et al. [3] only found improved adversarial robustness for generative classifiers, but had no results on distribution shifts. Fetaya et al. [4]’s results go in the other direction and indicate that generative classifiers may not be more adversarially robust in the first place. Finally, it’s been well-documented in the distribution shift literature that adversarial robustness has not translated to any robustness to distribution shift [5,6,7]. For these reasons, it was not clear from previous work that generative classifiers would be more robust to distribution shift, and this is our most important contribution.
• We perform extensive experiments to understand why generative classifiers have better out-of-distribution generalization than discriminative classifiers (Section 5.3, Section 6, Appendix A.1, A.2, A.4, A.6).
• We provide better understanding of the inductive biases of generative classifiers in a simplified Gaussian setting, which has not previously been done for generative classifiers. This understanding helps us understand when generative classifiers are better for distribution shift problems (not always!). Previous work has focused [8] has extensively analyzed this for discriminative models, but none have analyzed OOD generalization for generative classifiers.

the observation that "As σ increases, the core feature becomes less reliable, and the generative classifier uses the spurious and noise features more" is quite expected and doesn't give any added value. This section also is only relevant to generalization so it mirrors the results in Ng&Jordan 2001.

Our observation that generative classifiers prefer reliable features is intuitive, but it has an important implication for OOD generalization. Consistent features are the ones that distributionally robust classifiers should use, and generative classifiers have the inductive bias to prefer using them! In contrast, discriminative classifiers have no such preference and thus suffer more from distribution shift. Furthermore, Ng and Jordan, which we cite, only analyze the convergence rate of the ID accuracy for Naive Bayes. It provides no insights into OOD generalization or the properties of the data that affect robustness.

the feature you name $x_{spu}$ is not a spurious feature as it has a direct causal relationship by the label (as you designed it so). It just isn't as useful for discrimination as the "core" feature.

In our illustrative setting, we call $x_{spu}$ a spurious feature because it cannot, by itself, be used to classify the label 100% accurately. This is intended to model features like background color, which are correlated with the label at some high rate, but do not always hold (especially under distribution shift). Previous distribution shift literature with a similar Gaussian data setup has consistently called this a spurious feature [9,8,10,11]. We’re happy to add more text to clarify this in the paper!

We hope we have addressed all your questions. If there are any other concerns remaining that prevent you from accepting the paper, please let us know!

Thank you for your feedback! We’ve incorporated your suggestions into our paper, and answer your questions below:

It is unclear to me how the conclusions documented on the rather simplistic and well controlled toy-problem could be translated to more complex practical problems under realistic scenarios.

You’re right that the simple setting is quite different from the reality of complex distribution shifts. However, we do still think that there are interesting takeaways from our illustrative setting. First, the biggest hallmark of a good toy setting is that it makes useful predictions for the real world — and we do see that predictions here are borne out by neural nets on real benchmarks. In particular, the “generalization phase diagrams” in this illustrative setting show that different data properties can induce (a) discriminative models are better ID and OOD, (b) discriminative models are better ID but generative models are better OOD, or (c) generative models are better both ID and OOD. However, it almost never predicts a scenario where discriminative models do better OOD, but generative models do better ID. And indeed, this matches our neural network results in Section 5. Second, our toy setting also makes it easy to control various data properties (strength of spurious correlation, ease of overfitting, consistency of core feature) and understand how much a model relies on each kind of feature. Finally, it could be a useful setting for theorists in the future to derive generalization bounds or more principled algorithms with more favorable inductive biases.

Can you comment and clarify how you ensure the fairness of the comparisons?

For a completely fair comparison, we trained all discriminative and generative models from scratch. This is because using an ImageNet-pretrained checkpoint, as is done in the original Sagawa paper, completely removes the need for feature learning on many of these tasks. For example, Waterbirds becomes trivial, since pretraining on ImageNet requires the model to differentiate between dozens of bird species, in a setting where the background is no longer a strong spurious correlation.

To ensure that the discriminative baseline is as strong as possible, we use the official code from Koh, Sagawa et al. [1] and we sweep over 3 model sizes (ResNet-50, ResNet-101, and ResNet-152), 4 learning rates, and 4 weight decays. For this reason, we are confident that this represents a fair comparison. We have added more information to Appendix B.1 to clarify this.

In page 5 you speak about "class-balanced accuracy". I am not sure how this is defined.

Class-balanced accuracy refers to computing the accuracy for each class independently, and then uniformly averaging those per-class accuracies into a single number. This reduces the effect of class imbalance in the validation data, which has previously been found to exacerbate models’ susceptibility to spurious correlations.

Could not the evolution of the gradient norm in Figure 3 simply suggest that the discriminative classifier learns faster?

In some sense, yes, and this is the problem! The discriminative classifier quickly “achieves its objective” of minimizing the discriminative loss by learning a shortcut solution on the majority examples where the spurious correlation holds. The discriminative objective can be easily cheated because it does not require models to learn the core features that will be consistent across distribution shifts. In contrast, the generative classifier has a harder objective that takes longer to learn, but this does encourage the model to eventually learn and use the core features.

Section 6.3 Figure 6 - you fix $\sigma_{noise}^2=0.36$. Why particularly this value? Is this somehow representative of the behavior? Would the graphs look similarly at different noise levels?

Good question! We used $\sigma_{noise}^2 = 0.36$ to illustrate a setting where SVM and LDA classifiers achieve roughly the same in-distribution accuracy, yet LDA achieves much better out-of-distribution accuracy. We thought this was particularly interesting because it corroborates our “effective robustness” results on CelebA and CivilComments in Figure 2, as well as new results on 2 additional datasets that we’ve added in Fig. 14.

As we previously showed in our “generalization phase diagrams,” increasing $\sigma_{noise}$ hurts SVM’s performance, and decreasing $\sigma_{noise}$ improves SVM’s performance. Based on your suggestion, we’ve added equivalents of Fig. 6 to the Appendix (Fig. 11) to highlight these two cases.

Section 6.3 Figure 6 uses the feature scale B=1? Or some other value?

Yes, we use feature scale B=1 for the Figures in this section.

We hope we have addressed all your questions. If there are any other concerns remaining, please let us know!