EvA: Erasing Spurious Correlations with Activations

W1. Since the original DFR paper uses ResNet50 as the architecture, it would be a good sanity check to compare results in that same setting. The authors provided results for ResNet50 in Appendix E.2, but they mention that, to mimic real-world scenarios, they tuned hyperparameters based on Mean Accuracy instead of Worst Accuracy. This might be debatable, as the setting where DFR is applicable assumes an unbiased validation set. Additionally, I wonder if the setup is consistent between ResNet18 in the main paper and ResNet50 in the appendix for the Waterbirds dataset. Could you clarify whether hyperparameters for ResNet18 were tuned based on Mean Accuracy or Worst Accuracy?

Thank you for the question. For both ResNet18 and ResNet50 in Tables 5 and 7, hyperparameters were tuned based on Worst Accuracy. Thus, the comparison with DFR is fair, as both EvA-C and DFR are evaluated using an unbiased validation set in Table 5. But we provide additional results to empirically verify that EvA-E does not rely on Worst Acc as shown in Table. 4.

W2. Overall, since most existing papers evaluate their methods on ResNet50 while this paper primarily uses ResNet18 (this makes the numbers not comparable to those reported in other papers; in general, the numbers are much lower for all methods than those in the literature potentially due to the architecture), and some baselines (e.g., JTT, SSA [1], AFR [2], and [3]) are not included in the main table, Table 5, it is hard to compare the proposed method to the state of the art and to assess whether it truly stands out among existing techniques. For example, I wonder if the proposed method can outperform JTT on the Waterbirds dataset when ResNet50 is used, which has a reported worst-group accuracy of 86.7%, a relatively high value compared to the numbers in this paper obtained on ResNet18.

Thank you for pointing this out. Table 4 presents results using the ResNet50 architecture, as described in Appendix E.2. JTT achieves a worst-group accuracy of 86.7%, while EvA-E achieves 86.6% when tuned for Worst Accuracy. This demonstrates that, even with group annotations available, our method is competitive with prior approaches. Furthermore, when group annotations are unavailable, our method outperforms JTT by 23.3%. We will clarify this in the revision.

W3. Perhaps a detailed discussion and comparison with [1] is necessary, given that it also achieves the same goal: lightweight reweighting of the last layer without group labels.

Thank you for highlighting this relevant work. The main contribution of AFR lies in its construction of a targeted reweighting dataset during the initial model training phase, which requires knowledge of the training details from the first stage. This enables AFR to achieve high worst-group accuracy. In contrast, our method does not rely on prior training information and instead identifies spuriousness at the feature level. Notably, our method could potentially complement AFR, enhancing its performance. We will include a detailed comparison in the revision and explore this integration in future work.

W1. Contrary to the claim in Line 93, the performance of the proposed method is not the state-of-the-art in comparison with some methods [1,2]. For example, with ResNet50 as the backbone, EvA-E has a worst-group accuracy of 86.6% (Table 7), while [2] achieves 89.1%.

Thank you for providing additional references, especially recent works. In many cases our method is competitive with [1] and [2]. For instance, on CelebA, for worst accuracy, EvA-E achieves 82.7%, compared to 80.1% for [2], and 73.3% for [1] when no extra label is available. Additionally, our method is competitive, particularly when considering computational and data efficiency. We will incorporate a discussion of these methods in the revision.

W2. The unbiased dataset $D_{unbiased}$ is first introduced in Line 156; however, it is unclear what $D_{unbiased}$ can be considered as "unbiased". Based on the descriptions in Line 230 and Line 312, $D_{unbiased}$ is selected from the validation set. In some datasets, such as CelebA, the validation set also has gender bias. If $D_{unbiased}$ is defined as a set of group-balanced data where each class of samples distribute equally across different spurious features, then group annotations are required to obtain $D_{unbiased}$, contrary to the claim in Line 50-51. Further clarification on this point would be beneficial.

Similar to DFR (Kirichenko et al., 2022), we refer to $D_{unbiased}$ as a set of group-balanced data. This is not contrary to the claim in line 50-51, since EvA-E does not require annotations and only EvA-C requires a small set of group-balanced data.

W3 In Eq. (4), why the unknown test distribution $\phi_{test}^{ik}$ can be approximated via $\phi_{unbiased}^{ik}$? In some datasets such as CelebA, the test set is also biased. It would be helpful to further clarify this point.

Thank you for this correction. To clarify, the unknown test distribution refers to a perfect group-balanced dataset with unlimited samples, as the goal is to improve worst-group and mean accuracies across all groups. We will revise the notation in the paper to make this clearer.

Q1. Why spurious features often lead to high confidence predictions?

Thank you for the question. This phenomenon, referred to as simplicity bias, has been documented in prior works, such as Tiwari & Shenoy (2023) and Xu et al. (2019). Models tend to prefer features with lower noise, which are often spurious. FTT (Ye et al., 2023) further supports this, showing that models prioritize features that are easier to learn. In Section 3.4, Theorem 2 formally demonstrates that when spurious features exhibit lower noise than core features, the evidence energy (and thus confidence) is higher. Intuitively, models are more confident on the features that are more prevalent and easier to learn from the training data.

Q2. Can the proposed method be used for other model architectures beyond convolutional neural networks?

We have evaluated our method on ViT-B using ImageNet-9 [a], which involves background spurious correlations in real-world datasets. Our method improves the baseline model’s accuracy from 87.9% to 88.7%, requiring no additional data or labels and only 10 minutes for hyperparameter tuning while DFR is harmful when performing on original dataset and only achieves 87.1%. We will include further results in the revision.

[a] Xiao, K., Engstrom, L., Ilyas, A., & Madry, A. (2020). Noise or Signal: The Role of Image Backgrounds in Object Recognition. arXiv preprint arXiv:2006.09994.

W1. the experiments establishing the correlation between their two measures are based only on one class of one of the datasets

Thank you for your comment. We respectfully disagree; our results demonstrate the correlation across all six classes of the BAR dataset, providing a broader evaluation on real-world images compared to other synthesized datasets with only two classes such as Waterbirds as suggested by [a], as shown in Appendix Figure 6.

W2. experiments are only based on small toy settings with strong spurious correlations

Thank you for your feedback. Our experimental settings are consistent with established benchmarks used in prior works, such as JTT (Liu et al., 2021), DivDis (Lee et al., 2023), and FTT (Ye et al., 2023), all of which utilize the same datasets. Furthermore, the BAR dataset [a] was specifically introduced to address the lack of real-world benchmarks with complex spurious correlations. To address your concern, we have also included experiments on ImageNet-9 for additional analysis, as detailed in our response to Q4.

W3. the method is strongly inspired by OOD methods, but apart from a brief mention in the conclusion, this is not discussed in the main paper

Thank you for highlighting this point. Indeed, our work is inspired by OOD methods. To clarify, the evidence energy is fundamentally motivated by the observation on lines 172–173: “based on the observation that there is a high probability of spurious correlations for network predictions with high confidence or prediction logits.” The first paragraph of the conclusion will be revised to more accurately reflect this motivation as follows:

“To address spurious correlation, we focus on each channel of the deep feature and formally define spurious indicators—using consistency when unbiased data is available and evidence energy otherwise. Our findings show that erasing potential spurious features and reweighting core features can mitigate spurious correlations, regardless of additional information availability.”

While our method and OOD approaches are related through free energy, we have focused on the energy perspective. We will correct this in the revision and add more discussion with OOD works.

Q1. In Table 5, how many runs were performed to compute mean and std, and was the same erase ratio used in all of the EvA runs?

All results are reported as the mean and standard deviation over ten runs (lines 280–281). The erase ratio is tuned based on the validation dataset. Empirically the erase ratio selected is almost the same across different runs given the same dataset and backbone. For example, on ResNet-18 with Waterbirds debiased by EvA, the erase ratio of EvA-E is stable at 50%.

Q2. Section 4.4 (and Fig. 3 c) discuss the effect of the erase ratio on the worst accuracy. How does this hyperparameter influence the mean accuracy?

The mean accuracy remains stable initially but drops significantly as the erase ratio increases. For EvA-E on the Waterbirds dataset, we report the mean accuracy using the same model as in Figure 3c. The accuracy values are 97.3%, 97.5%, 96.9%, 97.1%, 96.9%, 94.2%, 90.7%, and 84.1% for progressively higher erase ratios. Notably, a clear decline in mean accuracy is observed beyond a certain threshold of the erase ratio. This observation will be included in the revised version.

Q3. In Table 7, mean acc and worst acc are reported for CelebA but unbiased/conflicting acc in Table 5. The results on Waterbirds suggest that your method results in a better worst acc but worse mean acc for the ResNet-50 than for the ResNet-18: Does this pattern hold on CelebA?

Thank you for your insightful observation. Regarding the evaluation metrics in Table 7, they are consistent with the JTT paper [b]. For the unbiased/conflicting accuracy, the metrics align with the SiFER paper [c] to ensure comparability with the results in the original publications.

In terms of worst-case accuracy and mean accuracy, this pattern is clearly observed in the CelebA dataset. ResNet-50 achieves higher worst-case accuracy but similar or lower mean accuracy compared to ResNet-18. This occurs because the tuning objective is explicitly optimized for worst-case accuracy, while the mean accuracy on both the Waterbirds and CelebA datasets is inherently biased. Consequently, the mean accuracy of ResNet-50 can only be comparable to or lower than that of ResNet-18.

[a] Nam, J., Cha, H., Ahn, S., Lee, J., & Shin, J. (2020). [b] Liu, E. Z., e.t., al. JTT (2023). [c] Tiwari, R., & Shenoy, P. (2024).

Q4... it should be quite feasible to evaluate EvA-E on ImageNet scale spurious correlations benchmarks (e.g. [1],[2]) ... Can you provide empirical evidence or arguments that your evidence energy measure can still be used for detection and mitigation in such settings?

We clarify that both BAR and CelebA involve real images without any toy data, and the spurious features in BAR are particularly complex, as the backgrounds are varied without specific patterns. To address your concern, we provide results on the ImageNet-9 [d] Background Challenge, a common benchmark for spurious correlations in real-world datasets. Using ViT-B, our method improves the baseline accuracy from 87.9% to 88.7% without additional data or labels, requiring only 10 minutes for hyperparameter search. In contrast, DFR reduces accuracy to 87.1% when only original data is available. These results support the effectiveness of our approach in more realistic settings.

[d] Xiao, K., Engstrom, L., Ilyas, A., & Madry, A. (2020). Noise or signal: The role of image backgrounds in object recognition. arXiv preprint arXiv:2006.09994.

W1. ... the feature erasure in eq. 9 should technically not be achieving what it claims to achieve. By turning full dimensions on/off, it could be - (i) partially turning off some core features that could be in superposition with a spurious feature; and (ii) not fully turning off spurious features, since it may be in superposition with some core feature dimension which is not turned off.

Thank you for your insightful observation regarding the features. Our method detects spuriousness at the channel level, where each “deep feature” corresponds to a channel in the embedding space, as detailed in lines 147–154. In Appendix A.1, we clarify the distinction between spurious/core deep features (features learned by the neural network and represented as individual channels in the embedding space) and spurious/core components (the actual parts of the input data that are inherently spuriously correlated with the target within the dataset). Notably, spurious/core components are intrinsic to the input itself, while spurious/core deep features represent the neural network's learned representations of those components.

We acknowledge that a spurious feature can indeed be a superposition of both spurious and core components. Actually, our Theorems 1 and 2 are developed under this premise and demonstrate that our indices prioritize the erasure of more spurious components when removing a spurious feature, albeit with trade-offs, such as inadvertently erasing some core components. While directly targeting the full representation could improve precision, it would also significantly increase computational complexity. We greatly appreciate your suggestion and recognize it as a promising direction for future work.

W2. ...How do the authors account for this imbalance when applying the distance metric $d$?

Thank you for your question. The difference in the size of the unbiased dataset and biased dataset will not affect the computation of consistency theoretically since the Wasserstein distance computes the difference between two distributions. However, if we understand correctly, the concern is that a limited unbiased dataset may not sufficiently represent the true distribution of ideal unbiased data, potentially skewing the consistency measure d. In these cases, evidence energy may be a more reliable proxy for reducing spurious features. For example, as shown in Table 3, with an unbiased dataset of only 16 samples, EvA-E outperforms EvA-C. In practice, however, EvA-C remains effective even with small unbiased datasets. For instance, in the CelebA dataset, where the unbiased dataset comprises only 0.8% of the biased dataset (as detailed in Table 1), EvA-C still demonstrates strong performance, as seen in Table 5.

W3. ... However, whether consistency and evidence energy are really the best metric to look at when it comes to performing this selection needs to be either theoretically or empirically established ... erasure mechanisms.

Thank you for the suggestion. We have compared our method with Lasso Regression, as used by DFR (Appendix E.4), and with other potential spurious indicators (Appendix Figure 6). These comparisons highlight the effectiveness of our spurious indicator, demonstrating that its performance is not solely due to sparsity. Additionally, we provide further results of random feature dropout on the BAR dataset using the same protocol as described in the experimental section, achieving an accuracy of only 61.2%.

W4. Table 5 shows that the gains from EvA are more significant in the absence of extra unbiased training data. The authors should provide a discussion on why their method might have a stronger advantage over SOTA when unbiased training data is not present.

We mainly compare the strength of our methods in the absence of extra unbiased training data in terms of computation efficiency and data efficiency introduced in Section 4.2 and Appendix E.2. In terms of the improvement over accuracy, we have a separated discussion in Appendix. E.1 (Table 6) on how our methods improve over previous baseline in terms of each classes.

W5. When extra training data is not available, how does one perform the hyperparameter search for ϵ? Additionally, when training data is available, although a single pass of EvA is computationally inexpensive, the hyperparameter sweep over ϵ may be time-consuming. Discussing these, especially, the former case of selecting ϵ in the absence of an unbiased train subset is necessary.

When an extra unbiased dataset is unavailable, we use the biased validation dataset for hyperparameter search, focusing on mean accuracy rather than worst accuracy (as shown in Table 4). In practice, since embeddings are pre-extracted for linear layer fitting, scanning all possible erase ratios is computationally efficient. As noted in lines 411–412, the total runtime for Waterbirds, including searching on 90 candidates of hyperparameter, is only 10 minutes. We will clarify this in the revision.