Formalizing Spuriousness of Biased Datasets using Partial Information Decomposition

We thank the reviewer for the review and appreciate the positive feedback on our work. We have made several edits to the main paper (marked in blue), including reorganizing the Main Results, including a new Fig. 3 to highlight the nuanced insights from PID, and performing additional experiments for automatic segmentation (Appendix A), tabular datasets (Appendix B), and other findings for the previous experiments (Appendix C.1).

(W1) On manual identification of core and spurious features

We acknowledge the reviewer’s concern regarding the potential limitation of manual identification of core and spurious features. In fact, there are extremely limited works that attempt to automatically separate core and spurious features without any other information. However, we have now incorporated automatic segmentation for separating the core and spurious features (see Appendix A), which does not rely on pre-defined training classes or explicit labels for spurious features or manual feature identification. The automatic segmentation utilizes open-vocabulary semantic segmentation model CLIPSeg[1] and performs segmentation of several objects. Then, one can at least choose the most relevant objects as “core,” e.g., “bird” object in Waterbirds using partial knowledge about the supervised classification goals. The remaining regions of the image can be chosen as “spurious.” To further demonstrate the practical applicability of our framework, we extend our framework for explainability on a tabular dataset where one might be interested in understanding and interpreting the nuanced statistical dependencies of a specific feature, e.g., “gender” with respect to other features (see Appendix B).

(W2) On validating the framework on NLP tasks

We appreciate the reviewer’s suggestion to explore NLP tasks for validating our framework. While this paper focuses primarily on image classification, the underlying principles of our approach can be extended to other domains, including NLP. We have extended our framework to tabular dataset (see Appendix B). Investigating its application in NLP tasks would be an interesting direction for future work to further demonstrate the generalizability of our framework.

[Reference]

[1] Timo Lüddecke and Alexander Ecker. Image segmentation using text and image prompts. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 7086– 7096, 2022.

APPENDIX B: ADDITIONAL EXPERIMENT 2: TABULAR DATASET (also Q3)

To further demonstrate the practical applicability of our framework, we extend our framework to explainability on a tabular dataset where one might be interested in understanding and interpreting the nuanced statistical dependencies of a specific feature, e.g., “gender” with respect to other features. We studied the popular Adult dataset where the spurious feature is gender, and several other features are taken as the core. Dimensionality reduction is only performed on the core feature set because it is high-dimensional. Our technique is helpful in isolating biases in tabular datasets in this setting prior to training any model. Going beyond existing literature, we can pinpoint the nature of spurious dependencies and evaluate the quality of the dataset prior to model training.

APPENDIX C.1: ADDITIONAL RESULTS

As per the reviewer’s comment, we have calculated the Intersection over Union (IoU) scores for the entire test dataset as well as for the minority test group of the Waterbird dataset (see Table 2).

(Q2) On discussion of suggested references

We include discussion on references [1] and [2] in the Related Works and can expand further if required.

A recent work [2] looks into the problem of spurious correlations through the mathematical lens of separability of the spurious and core features under mixture of Gaussian assumptions. [2] also assume that the core and spurious feature split is given, and they define measures of separability to formalize the reliance on spurious feature by a model.

Reference [1] (which was referenced before as well) looks into the problem of improving model training to reduce reliance on spurious features as opposed to pre-emptive dataset evaluation which is the focus of our work. We note that [1] also provides interesting insights on how the noise in the core feature plays a role in a model’s reliance on the core features.

In contrast to these related works, our work isolates four specific types of inherent statistical dependencies in the dataset that provides a more nuanced understanding and dataset quality evaluation, going beyond solely identifying a model’s reliance on a specific feature. For instance, PID helps us isolate and understand four different types of statistical dependencies: when $F$ or $B$ is indispensable, when either $F$ or $B$ can be used, and when both $F$ and $B$ are required to be used jointly by the optimal classifier. PID provides a formal way of explaining when one feature is more informative than another, invoking Blackwell Sufficiency, which to the best of our knowledge, has not been explored in the context of understanding spurious correlations.

(Q4) On Redundant Information in Spuriousness

As we have mentioned, PID helps us isolate and understand four different types of statistical dependencies: when $F$ or $B$ is indispensable, when either $F$ or $B$ can be used, and when both $F$ and $B$ are required to be used jointly by the optimal classifier. Specifically, redundant information captures inherent redundancies across features, i.e., ambiguous scenarios where an optimal predictor may pick either the core or spurious features without preference since either might suffice. This is different from the case where spurious feature $B$ is indispensable. Redundancy hints at the possibility that there are optimal predictors possible that would not rely on the spurious feature; hence it is not included in the spuriousness measure.

Nonetheless, should one wish to account for this type of statistical dependency as well, the measure of spuriousness can be modified to include this term as well.

On Minor Edits

We have incorporated these edits in the main paper.

[Reference]

[1] Ye, H., Zou, J., & Zhang, L. (2023, April). Freeze then train: Towards provable representation learning under spurious correlations and feature noise. In International Conference on Artificial Intelligence and Statistics (pp. 8968-8990). PMLR.

[2] Wang, Y., & Wang, X. (2024, April). On the Effect of Key Factors in Spurious Correlation: A theoretical Perspective. In International Conference on Artificial Intelligence and Statistics (pp. 3745-3753). PMLR.

We thank the reviewer for their review and comments. We have made several edits to the main paper (marked in blue), including reorganizing the Main Results, including a new Fig. 3 to highlight the nuanced insights from PID, and performing additional experiments for automatic segmentation (Appendix A), tabular datasets (Appendix B), and other findings for the previous experiments (Appendix C.1). Below, we provide our responses to the comments, grouping some of them together.

(Q1 & W1-4) On the use-cases of the proposed method and relation to worst-group accuracy (WGA)

We would first like to clarify that our proposed technique is a dataset quality evaluation and explainability technique as opposed to model evaluation measures like worst-group accuracy (WGA) and SHAP/IoU. In essence, our goal is to develop explainability measures that are “pre-emptive” or “anticipative” of spuriousness using just the dataset before training the model since training a model can be expensive. The goal of our experiments is to show broad agreement between our anticipations from the dataset before training any model and the post-training behavior of actual models (when trained regularly to optimize performance without doing anything else specifically targeted towards avoiding spurious features).

Our main novelty lies in distilling four types of statistical dependencies involving the core and spurious features, that goes beyond just saying whether a model relies on the spurious feature or not (also see new Fig.3 in the updated paper). To this end, we leverage the information-theoretic framework called Partial Information Decomposition (PID), with roots in statistical decision theory, which has not been studied in this context before. In particular, PID helps us formalize four types of scenarios as follows:

$Uni(Y:F|B)>0$ captures scenarios where the feature $F$ is indispensable for prediction of $Y$ using Blackwell Sufficiency, and a Bayes optimal classifier will strongly rely on F.

$Uni(Y:B|F)>0$ captures scenarios where the feature $B$ is indispensable for prediction of $Y$ using Blackwell Sufficiency, and a Bayes optimal classifier will strongly rely on B.

$Red(Y:F,B)>0$ captures scenarios where either $F$ or $B$ suffice, and hence an optimal classifier could rely on either $F$ or $B$ or even both (the optimal classifier might not be an exclusive one, so it is ambiguous whether $F$ or $B$ is picked).

$Syn(Y:F,B)>0$ captures an interesting scenario less explored in existing works. This is when both $F$ and $B$ are needed together by the optimal classifier, and neither of them suffice alone.

We acknowledge that identifying core and spurious features is challenging, and in fact, there is very limited work which attempts to address this problem automatically without using any other information. Existing reweighting techniques [1] assume that the group information is available if not the exact pixels corresponding to the core feature. Some works [2] assume that an adversarially-robust model can sometimes identify the core features though with little interpretability. Since our work is focused on dataset evaluation/explainability prior to actual model training, we offer a workaround when the exact core/spurious features (pixels) are not known or even the group information is not available.

APPENDIX A: ADDITIONAL EXPERIMENT 1: AUTOMATIC SEGMENTATION OF FEATURES

Specifically, for supervised learning tasks, one might have some partial knowledge of what the core features should be even if we do not know the exact pixels, e.g., in Waterbirds, we know that the core feature is the “bird” object even if we do not know their exact pixels. In such scenarios, our proposition is to use automatic segmentation techniques (Open-Vocabulary Semantic Segmentation) such as CLIPseg [3] to at least perform object detection and then choose the most relevant objects as the core. Then, the regions of an image not associated with the “core” objects can often be considered spurious. Thus, our proposed technique can be applied in conjunction with such automatic segmentation techniques to interpret the nature of spuriousness and the anticipated behavior of the optimal predictor even without prior knowledge of the exact core and spurious feature split. See Appendix A for additional experiments on this setup.

Continued in next: TABULAR DATASETS

[Reference]

[1] Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint arXiv:1911.08731, 2019.

[2] Sahil Singla and Soheil Feizi. Salient imagenet: How to discover spurious features in deep learning? arXiv preprint arXiv:2110.04301, 2021.

[3] Timo Lüddecke and Alexander Ecker. Image segmentation using text and image prompts. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022.

We thank the reviewer for their positive opinion of our work. We have made several edits to the main paper (marked in blue), including reorganizing the Main Results, including a new Fig. 3 to highlight the nuanced insights from PID, and performing additional experiments for automatic segmentation (Appendix A), tabular datasets (Appendix B), and other findings for the previous experiments (Appendix C.1).

(Q1) On the choice of hyperparameter $\gamma$

We appreciate the reviewer’s suggestion. This constant $\gamma$, which lies in the range (0, 1), determines the weight given to the clustering loss and is treated as a hyperparameter. In our framework, we adopted the standard value of 0.1 for $\gamma$, which is commonly used in prior implementations of this clustering technique [1]. We have included this in Appendix C.3.1.

(Q2) On the threshold of $M_{sp}$ for identifying spurious feature

We appreciate the reviewer's comment. Our proposed measure of spuriousness $M_{sp}$ is designed to quantify the relative degree of spuriousness within a dataset. It is not intended to provide an absolute threshold for identifying individual features as spurious. However, by comparing $M_{sp}$ across different variants of a dataset, we can determine which dataset variant exhibits a higher or lower degree of spuriousness. In summary, a higher $M_{sp}$ ​ value indicates a more spurious dataset as compared to another prior to model training.

(Q3) On a typo in line 266

The typo has been corrected in the main paper.

(Q4) On $Z_3 \bigoplus N$

$\bigoplus$ represents modulo-2 addition, which corresponds to the elementwise exclusive OR (XOR) operation on binary values. In the specific example discussed in the paper, $Z_3$ and $N$ are Bernoulli random variables that independently take values of 0 or 1, each with a probability of 0.5. The operation $Z_3 \bigoplus N$ thus performs an elementwise XOR on these random variables.

[Reference]

[1] Xifeng Guo, Xinwang Liu, En Zhu, and Jianping Yin. Deep clustering with convolutional autoencoders. In Neural Information Processing: 24th International Conference, ICONIP 2017, Guangzhou, China, November 14-18, 2017, Proceedings, Part II 24, pp. 373–382. Springer, 2017.

We thank the reviewer for their review and appreciate the positive opinion about our work. We have now included the standard deviation for our results in Appendix C.3 Table 5. We have also included several new experiments in Appendix A, B, C and streamlined our writing in the Main Results section. We are also happy to incorporate any further suggestions that the reviewer might have.

We thank the reviewer for their review. We have made several edits to the main paper (marked in blue), including reorganizing the Main Results, including a new Fig. 3 to highlight the nuanced insights from PID, and performing additional experiments for automatic segmentation (Appendix A), tabular datasets (Appendix B), and other findings for the previous experiments (Appendix C.1).

(W1) On the separation between core and spurious features

We acknowledge the reviewer’s concern regarding the assumption that core and spurious features can be separated as foreground and background. We agree that it is not always possible to separate the core and spurious features, particularly when spurious features, such as rotation or color, influence the entire image rather than specific regions. In fact, there is extremely limited work that attempts to address this problem without any additional information. We emphasize that our approach does not aim to explicitly label spurious features. Nonetheless, our strategy is applicable in several use-cases as a measure of dataset quality evaluation/explainability prior to expensive model training (without knowing the exact split of the core/spurious features apriori).

APPENDIX A: ADDITIONAL EXPERIMENT 1: AUTOMATIC SEGMENTATION OF FEATURES

In supervised classification tasks, we can approximately identify the core features by utilizing their association with the target label, e.g., “bird” object in Waterbirds dataset. To achieve this, we employ automatic segmentation using CLIPSeg [1], which isolates the foreground objects from the background without relying on pre-defined training classes or explicit labels for spurious features. We can choose the relevant objects as our core, e.g, “bird” object, and the remaining image as spurious. We can then compute the measure of spuriousness using this approximate split between core and spurious features and evaluate the quality of the dataset/interpret the nuanced dependencies using PID.

Observe that there is a negative relationship between dataset spuriousness $M_{sp}$ and post-training metrics like the worst-group accuracy, as observed in the previous experiments. Hence, $M_{sp}$ quantifies spuriousness in this regard as well, thus providing pre-emptive insights into the dataset quality prior to training.

APPENDIX B: ADDITIONAL EXPERIMENT 2: TABULAR DATASET

To further demonstrate the practical applicability of our framework, we extend our framework for explainability on a tabular dataset where one might be interested in understanding and interpreting the nuanced statistical dependencies of a specific feature, e.g., “gender” with respect to other features. Specifically, we apply our method to the Adult [2] dataset, where the task is to predict whether an individual’s annual income exceeds 50k. We can choose a subset of features as the “core.” Since the core feature is high-dimensional, we employ k-means clustering to reduce dimensionality and discretize the features. Using our estimation module, we calculate PID values involving the core features, spurious features, and the target label (see Appendix B). For this set of experiments, the calculated $M_{sp}$ ​consistently exhibits a negative trend with worst-group accuracy, reinforcing the effectiveness of our approach.

In conclusion while disentangling causal and spurious features remains a significant challenge in OOD tasks, our method is still applicable to several use-cases for evaluating dataset quality without knowing the exact split of the core and spurious features apriori.

[Reference]

[1] Timo Lüddecke and Alexander Ecker. Image segmentation using text and image prompts. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 7086– 7096, 2022.

[2] Barry Becker and Ronny Kohavi. Adult. UCI Machine Learning Repository, 1996. DOI: https://doi.org/10.24432/C5XW20.

(Q1) On clustering and discretization

We appreciate the reviewer for the relevant inquiries. The number of clusters is chosen as a hyperparameter through empirical validation. We tested different cluster numbers (e.g., 5, 10, 20) and analyzed the resulting PID values to observe variance which is low in our chosen range. The chosen number of clusters (10 in our case) ensures a balance between computational efficiency and loss of information. Excessively large clusters were avoided to prevent overfitting and also avoid high computational cost. The results are given below:

Discretizing the distribution is necessary for numerical purposes because the current PID estimation tools and definitions are designed for discrete data. There is limited work on handling high-dimensional continuous data, e.g., [1] assumes Gaussian input random variables. Since our target variable is binomial, we cannot assume Gaussian distribution necessarily. Discretization causes some loss of information. To address this, we employed a self-learning approach using an autoencoder, which helps preserve as much information as possible during dimensionality reduction. This approach minimizes the impact of discretization while enabling us to leverage PID estimation effectively.

(Q2) On reporting $I(Y;B)$

We thank the reviewer for the suggestions. Notably, $I(Y;B)$ is the sum of $Uni(Y:B|F)$ and $Red(Y:F,B)$. We have now added the comparison of our proposed measure with this measure and other measures in Appendix C.1 Table 3.

[Reference]

[1] Praveen Venkatesh, Corbett Bennett, Sam Gale, Tamina Ramirez, Greggory Heller, Severine Durand, Shawn Olsen, and Stefan Mihalas. Gaussian partial information decomposition: Bias correction and application to high-dimensional data. Advances in Neural Information Processing Systems, 36,

Additional Clarification (Q2, W3 and W4)

Since our measure is a pre-emptive dataset evaluation measure prior to training, the goal of our experiments is to see how our anticipations from dataset agree with post-training model performance metrics like worst-group accuracy, SHAP, IoU, etc. Notably, they are not mathematically the same thing – our measures anticipate spuriousness from dataset based on how the Bayes optimal classifier(s) should behave, and the empirical measures show how specific models actually behave (when trained regularly on that dataset without doing anything specific for spuriousness). Nonetheless, we observe an interesting correlation with our anticipative dataset quality measure and post-training model performance metrics on a broad range of experimental setups, further validating the efficacy of our approach.

Also observe that for CelebA dataset, the candidate measure $I(Y;B)$ does not exhibit a negative correlation with the worst-group accuracy. However, our proposed measure aligns with this post-training trend, highlighting its effectiveness as an indicator for relative dataset quality.

I thank the authors for their response. However, several concerns still remain and I do not feel this paper is ready for publication at the moment. Hence, I cannot support acceptance.

W1 : I agree that the core and spurious features cannot be separated without additional information. However, this also means this measure is limited to only a few datasets and cannot be used broadly for all datasets.

I do appreciate the tabular experiments by the authors though.

W2: Thank you for adding the error bars. However, I would also expect error bars for Msp and other measures.

W3 + W4: I respectfully disagree that the contribution is significant. Firstly, there are no theoretical guarantees that the proposed measure must negatively correlate with OOD Generalization. The paper proposes two candidate measures which are demonstrated to be unsuitable in some cases. But it not clear why this measure is suitable in all cases. Secondly, there is not a significant contribution in methodology. PID is calculated based on existing methods.

Q1: Thank you for your answer. However, I suspect for high-dimensional data discretizing it would be a significant loss in information. This also relates to W4 since I believe this is a weakness in the methodology.

Q2: Thank you. The numbers between the different measures do not seem to be very different in the empirical experiments and even I(Y;B) seems to negatively correlated with OOD Generalization. Also, is there any reason measures like I(Y;B|F) etc have not been considered?

Also, I would recommend to improve the aesthetics of the plots in Figure 1 (although this does not affect my rating).