A Differentiable Rank-Based Objective for Better Feature Learning

Thank you for carefully reviewing our paper. In the following, we aim to address your concerns.

There is a disconnect between the theoretical foundation and its practical application, leading to a potentially vacuous theorem. The assumption that the coefficient (\beta) of the Softmax function is infinite is impractical, as it is only set to values like 5 in the experiments.

Your feedback highlights a crucial aspect of difFOCI. Our theorem establishes that in the infinite $\beta$ regime, we converge to the estimator from [1], which in the infinite $n$ limit recovers the original measure of conditional dependence suggested by [2]. However, and we should have clarified this more carefully, there are two key points:

1. Theoretically, our estimator is more versatile as it provides a generalization of the estimator in [1].
2. In practice, we benefit from $\beta$ being finite as increasing it will result in softmax transitioning to a hardmax, causing the gradients to zero out. Therefore, in practice we want $\beta$ finite.

The point that we initially intended concerns difFOCI’s robust performance to the selected value of $\beta$. Provided that it is set within a reasonable range - not too large to zero out gradients or too low as in this results in softmax yielding an uniform distribution - difFOCI demonstrated robust results across various values of this hyperparameter. We demonstrate this in Table 3 below on MetaShift dataset (you can find additional example in Appendix J): tuning $\beta$ yields minor performance gains, although these are mostly not statistically significant. Also, extreme $\beta$ values cause the estimator $T(X_G, f_\theta(X))$ to equal a constant (as mentioned before), which degrades it yielding standard ERM performance (these are marked in italic in the Table 3. below):

We are interested in knowing if you find the results convincing or believe additional explanation is necessary. We appreciate bringing this to our attention as further clarification was indeed needed.

Symbol definitions are ambiguous, as discussed in the questions below.

We address these below - thank you for detailed attention to our paper!

The definition of the rank ( r ) in Line 096 is unclear. What is meant by ( Y_{(j)} \leq Y_{(i)} )?

Consider a dataset where the $X_i$ and $Y_i$ values have no ties. Arrange the data pairs $(X_1, Y_1), (X_2, Y_2), \ldots, (X_n, Y_n)$ such that the $X$ values are in ascending order: $X_{(1)} \leq X_{(2)} \leq \ldots \leq X_{(n)}$. This yields $(X_{(1)},Y_{(1)}),..., (X_{(n)},Y_{(n)})$. Now, since there are no ties among the $X_i$ values, this ordering is unique. Therefore, we define $r_i$ as the rank of $Y_{(i)}$, the count of $j$ such that $Y_{(j)} \leq Y_{(i)}$. In other words, $r_i$ represents the position of $Y_{(i)}$ when $Y_i$’s are sorted in ascending order. Thank you for raising this - we have also added this clarification to the manuscript (as well as the case in which there are ties in the data).

Line 076 declares (\sigma) as a Sigmoid function, yet it denotes the Softmax function. Could the authors clarify this discrepancy?

We apologize - we meant the Softmax function and have amended this accordingly.

The comparison of the proposed method focuses on conventional methods such as LDA and PCA. It would be beneficial to include comparisons with more state-of-the-art methods.

We appreciate your feedback: accordingly, we have conducted 5 additional experiments. As shown in Table 1 (in the main response), we performed experiments on both text and image datasets of varying sizes, architectures and included additional three baseline comparisons with state-of-the-art methods: Just Train Twice, Mixup and Invariant Risk Minimization. The experiments illustrate difFOCI’s compatibility with contemporary architectures, such as ViT-B or BERT, which allow difFOCI to achieve competitive performance.

We hope that our answers and newly conducted experiments addressed your concerns. Please let us know if you have any remaining questions.

---

[1] Azadkia, M. and Chatterjee, S., 2021. A simple measure of conditional dependence. The Annals of Statistics, 49(6)

[2] Dette, H., Siburg, K.F. and Stoimenov, P.A., 2013. A Copula‐Based Non‐parametric Measure of Regression Dependence. Scandinavian Journal of Statistics

We sincerely appreciate the time and effort you dedicated to reviewing our paper. We were encouraged by your recognition of difFOCI's potential. In response to your constructive feedback, we undertook a thorough revision of the message some sections convey and conducted several further experiments.

Waterbirds dataset is a small and simple dataset and a successful method should be tested on other datasets besides it. How many seeds were used? Was the same protocol used to select hyperparameters for the baselines and the proposed method? The benchmark used in [A] can be used to evaluate the proposed method more rigorously. The results of the method will be more reliable if multiple datasets from [A] were used, together with similar hyperparameter selection and evaluation.

This is a very valid critique - thank you for pointing us to the relevant paper. In response, we incorporated all datasets from [A] (except ImageNetBG and Living17 as they do not contain attribute data). We are still waiting for access to medical data in [A] (as the datasets are not publicly available) and will include them once we obtain access.

Our findings are presented in Table 1 (in main response), with experimental procedures outlined in Appendix G (together with the Waterbirds hyperparameter configuration). To ensure consistency and comparability, we exactly replicated experimental configuration used in [A], including random search over same hyperparameter distribution, same train-validation-test split, and reporting averages with std.s across 3 random seeds. We would like to express our gratitude to the reviewer for recommending this paper, as their extensive codebase (with various architectures, datasets and algorithms) helped us in obtaining meaningful results in a timely manner. We also added this acknowledgement to the paper.

W2. For the fairness experiments, some additional explanations are needed.

W2.1 Are the explicit features Xs a subset of X?

The data is as follows $(X, X_s, y)$, and therefore $X$ does not contain $X_s$ . For example, in ASCI dataset $X=\\{\text{occupation, working class, place of birth, etc.}\\}$, $y=$income and $X_s=\\{\text{race, sex}\\}$. When we use difFOCI (dF3) to optimize $T(y, f_\theta(X) | X_s)$, where $X_s$ is only used in the conditioning part. We have clarified this in the paper.

W2.2 The authors state that “the predictor should be independent of one or more sensitive features”. How is this enforced? By maximizing T (Y , θ ⊙ X | Xs ) the model would learn to predict the target when we know Xs. Is this right? Can the authors explain this?

W2.3 Is there a measure to evaluate if the model relies on sensitive features? At the moment the results only report performance, and it is not clear if these results incorporate sensitive features or not.

These are valid points: upon reflection, our assertion about difFOCI achieving statistical independence was an overstatement. We have removed any mention of statistical independence, toning down the assertion. This was an oversight on our part - we apologize for this. Our original motivation for $T(y, f_\theta(X)|X_s)$ was empirical, as preliminary experiments demonstrated promising performance.

Specifically, we trained two NNs, first on $(X,y)$ without $X_s$ and second using difFOCI with $T(y, f_\theta(X)|X_s)$. We used the final layers of the two NNs to predict $X_s$ and, as can be seen in Table 2 (in main response), difFOCI significantly reduces predictability of $X_s$, sometimes to chance level, without significantly impacting accuracy on y (sometimes even slightly improving it). We have included this in the manuscript. Finally, we would like to point out that difFOCI’s primary contribution lies in its regularizer form (dF2). We clarified this in the manuscript, noting (for the fairness Section 5.3): “This section, while not the primary focus of our contribution, offers a complementary illustration of the difFOCI objective's versatility through a heuristic example. We found that this form (dF3) preserves the performance of the chosen parameterization while significantly reducing its predictivity of the sensitive attribute.”

In the context of Waterbirds, group refers to the combination of class and spurious variable (background). So X_G should be denoted as a spurious variable, not a group variable.

Thank you for pointing this out, we corrected the manuscript.

Once again, we would like to thank you for your constructive feedback - we believe that addressing your critiques helped us significantly strengthen the contribution of difFOCI’s.

We thank you for your thorough feedback. In your review, you raised several valid points.

1. The real world datasets seem to be out-dated, where the latest one was released in 2019. It would be more convincing to presents results on more recent are more complex datasets, such as those in WILDS benchmark.

This is a valid concern - we acknowledged this and expanded our study to five new datasets, out of which two date to 2022 and another one includes a WILDS dataset.

2. This paper only considers one model architecture, i.e., ResNet-50. With the increasing usage of Transformer-based models, it is also important to show the effectiveness on more complex models.

The newly added experiments encompass multiple modalities (images and text) and incorporate transformer architectures: ViT-B pretrained with DINO or CLIP and BERT pretrained on English corpus and Wikipedia. The datasets contain up to 300k samples and up to 60 classes. As can be seen from Table 1 (in the main response), difFOCI exhibits robust performance across all datasets.

3. Simply showing the improved performance on worst group accuracy does not sufficiently suggest the decrease reliance on the spurious features. Note that, the overall performance of the proposed method shows consistently lower performance (2 - 4%) for average accuracy. It would be suggested to conduct the synthetic experiments in [1].

This is an important point we would like to address. With the new datasets, we found this decrease in average accuracy to be dataset-dependent; the new experiments reveal an improvement in average accuracy (as well as worst-group) on four out of five datasets (Table 1).

The provided reference is an interesting study of simplicity bias through concatenation of MNIST and CIFAR datasets. While difFOCI does not directly extend to this due to the absence of sensitive/group attributes, your observation on solely reporting improved worst-group accuracy remains valid. We hope to alleviate this concern with the additional experiments presented in Table 2 (in the main response). Here, we trained two NNs to predict $y$: one on the whole dataset $(X,y)$ but without sensitive attributes $X_s$ and one on features obtained through difFOCI (dF3) - the experiment is detailed in Appendix G. Afterwards, we use the last layers of these two NNs to predict the sensitive attribute $X_s$. From Table 2, we see that the network trained on features obtained with (dF3) has a significant reduction in predictability of the sensitive $X_s$, sometimes down to chance level. Interestingly, training on difFOCI’s features does not compromise overall task performance on $y$ (in fact, sometimes we even observe a slight gain in accuracy).

4. Another concern is that the experiments are conducted on the case where the train test set share the same distribution. Given that the feature selector is a trained NN, it is interesting to show if such a method maintains its performance of feature selection where the train test set have a different distribution. For example, CIFAR10 vs CIFAR10.1, DomainNet-real vs DomainNet-Sketch? as in [2].

This is an interesting angle that we have not previously considered. We thank the reviewer for the suggestion: this motivated our decision of including NICO++ dataset in our evaluation, which is, as its authors state, ‘specifically designed to facilitate out-of-distribution (OOD) generalization in visual recognition’ [1] and is often characterized as a distribution-shift dataset (e.g., see [2,3]). In Appendix K, we also added experiments for feature selection on CIFAR10/10.1 and DomainNet (Real vs Sketch, Clipart vs Sketch and Sketch vs Quickdraw) showcasing difFOCI consistently maintains its performance. We hope that these prove difFOCI's effectiveness in handling distribution shifts.

We thank the reviewer for the thoughtful suggestions. The four points raised helped us in strengthening our paper, and we hope that the reviewer will find the updated version to be a substantial improvement.

---

[1] Zhang, X., He, Y., Xu, R., Yu, H., Shen, Z. and Cui, P., 2023. Nico++: Towards better benchmarking for domain generalization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 16036-16047).

[2] Gulrajani, I. and Lopez-Paz, D., 2020. In search of lost domain generalization. International Conference on Learning Representations, 2021.

[3] Zhang, X., He, Y., Xu, R., Yu, H., Shen, Z. and Cui, P., 2023. Nico++: Towards better benchmarking for domain generalization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 16036-16047).

Dear authors,

Thank you for your efforts. My concerns have been addressed by the added experiments. I will increase my score to 6.

Best regards,

Reviewer cLoX

Thank you for the encouraging feedback, we are glad that you found the motivation clear and the applications well chosen.

Please discuss the scalability and generalization potential of difFOCI.

We agree with your concerns regarding the scalability of our method. In response, we conducted further experiments on five datasets of varying sizes (up to 300k data samples), complexity (up to 60 classes) and modalities (text and image). In addition to ResNet-50, we incorporated attention-based architectures (e.g., BERT and ViT-B) utilizing various pretraining strategies (e.g., DINO and CLIP). We also added Just Train Twice, Mixup and Invariant Risk Minimization as additional baselines. As illustrated in Table 1 (in main response), difFOCI demonstrates competitive performance, both in terms of average accuracy and worst-group accuracy.

The fairness study is interesting; however, the dataset and task here are relatively simple. Can difFOCI be applied in large-scale settings?

To address this, we chose CivilComments (~250k instances) as one of the added datasets. The task concerns prediction of toxic language, with sensitive attributes related to race, sexual orientation, or religion. The dataset is commonly used for fairness evaluations, e.g., in [1,2]. Across all approaches on CivilComments, difFOCI+ERM yields highest overall accuracy, while difFOCI+DRO results in best worst group accuracy. We hope that this provides further evidence of difFOCI’s potential to contemporary fairness applications.

The visualization in Figure 2 is not very clear. What is the main takeaway from this figure? Additional visualization examples would be helpful.

The purpose of this figure is to provide a qualitative example, complementing quantitative analysis in Table 3, visually demonstrating that a difFOCI-trained model ignores spurious features (background), focusing on the relevant features (foreground). Without difFOCI, we can see from Figure 2 that models heavily rely on background, while with difFOCI this problem is effectively resolved. We’ve updated the caption of Figure 2 to summarize this key message. Note that this is not specific to this single example: in Appendix E1, we present 10 randomly-selected visualizations showing similarly strong effects.

Thank you for your feedback. We made corresponding revisions to the manuscript, including the addition of scalability and fairness experiments. We would appreciate your assessment of whether these changes have adequately addressed your concerns, and welcome any further feedback or questions you may have.

---

[1] Dige, O., Arneja, D., Yau, T.F., Zhang, Q., Bolandraftar, M., Zhu, X. and Khattak, F., 2024, November. Can Machine Unlearning Reduce Social Bias in Language Models?. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track (pp. 954-969).

[2] Villate-Castillo, G., Lorente, J.D.S. and Urquijo, B.S., 2024. A systematic review of toxicity in large language models: Definitions, datasets, detectors, detoxification methods and challenges.