Mitigating Biases in Blackbox Feature Extractors for Image Classification Tasks

We thank the reviewer for the thoughtful comments. We address the concerns below:

Deeper analysis of the approach. We refer the reviewer to the ArcFace paper [6] for details on the margin loss. Here, we show how the adaptive nature of the margin loss aids in the learning of the bias-conflicting samples. As shown in [6], the margin loss increases the angle between a sample's MLP feature ($\hat{f}$) and its ground truth weight vector. The degree of increase depends on which cluster the sample belongs to, and the inverse frequency of the ground truth class in that cluster. Let $x$ be an image in the bias-conflicting group, $m$ be its corresponding margin penalty (eq 2 and L248 in the main paper), and $\theta$ be the angle between the weight vector corresponding to the ground truth class and the feature $\hat{f}$. As $x$ belongs to a bias-conflicting group, the penalty $m$ is high (assuming that the clusters obtained from the features are good approximations of the different groups in the training set), which means that the angle between the feature vector and its weight vector increases considerably in the softmax term inside the margin loss in eq 4 (main paper) (recall that $\cos(\theta+m) < \cos(\theta)$ for $m>0$). Thus the loss value rises for the bias-conflicting samples (owing to the logarithm and the negative sign in eq 4). For the bias-aligned samples, as $m$ is lesser (as it belongs to the majority class in the cluster), the degree of increase in the loss is lower than those for the bias-conflicting samples. This is how the adaptive margins help mitigate biases.

For further evidence on the usefulness of the adaptive margin loss, we plot the training loss for each group (summed across all epochs and normalized by group size) in Waterbirds and CelebA between an ERM model and our method in Figure 2 in the rebuttal pdf. While both models have higher losses for the minority groups (for majority groups, the loss values are negligible and similar), the proposed method's minority loss values are much higher than that of the ERM model.

Computational Complexity of competing methods See the below table for the analysis of computational complexity.

Application to other pretrained models In the main paper, we show the performance of our model on self-supervised pretrained model like CLIP, and demonstrate the effectiveness of our method in Table 4. Our method can further be applied to any classification based task if a pretrained feature encoder is present. For example, if one wants to run it for a text-based classification task, they only need the features from a pretrained model (e.g., BERT). No other adjustment is necessary. It is to be noted that the method only relies on the angle between the feature vector and the ground truth class' weight vector. As an application, we run our method on the CivilComments dataset [a] using the BERT model pretrained on BookCorpus and Wikipedia datasets (https://huggingface.co/google-bert/bert-base-uncased), and report the results below. We find that both Contrastive Adapter and our method far outperform the ERM scores.

Clarification for the clustering task Kindly see the general response section.

Scalability of approach to extremely large datasets and models Since the method involves clustering the features, if the dataset is extremely large, one can randomly sample a small percentage of it to do the clustering. For example, on clustering only 10% randomly sampled images from celeba, we find that the worst group accuracy is $81.11$%, whereas the average group accuracy is $85.6$%. The scores become $80.55$% and $85.6$% respectively when the clustering is performed only on 1% of the images (Note that scores reported in the paper are $81.61$% and $86.04$% respectively). On the other hand, for extremely large models, the requirement is to be able to load the model into a GPU memory. Our method adds negligible overhead owing to the addition of only an adapter and a classifier layer. We will add these points to the limitations section in the camera ready version.

Exploring the method impact on model interpretability We thank the reviewer for this valuable suggestion. We intend to investigate this as a future work.

[a] Borkan, Daniel, et al. "Nuanced metrics for measuring unintended bias with real data for text classification." Companion proceedings of the 2019 world wide web conference. 2019.

Dear Reviewer oZ3K,

Can you have a look at the rebuttal and see if your concerns have been addressed?

Best regards Your AC.

We thank the reviewer for the thoughtful comments. We address the concerns below:

Clarifications about Table 1: Table 1 is solely meant for analyzing the nature of the different feature encoders, wherein we can see that different feature encoders may have different levels of awareness of the target labels. If a feature encoder is highly target-aware, even removing all bias-conflicting samples from the training data does not hamper the worst group accuracies to a large extent. On the other hand, if the feature encoder is bias-aware, the worst group accuracies are low even for the original dataset -- these are the scenarios that require explicit bias mitigation. Note that the scores in this table are those of the ERM method, obtained by varying the proportion of the bias-conflicting samples, and is not related to the proposed approach.

Assumption on alignment of the bias in the pretrained features and downstream dataset: Please see the general response section.

Clarifications on the presentation in Section 3: We apologize if the presentation of this section was not clear. We explain the main highlights of the section below and will make the presentation clearer in the camera-ready version upon acceptance.

• We begin by formally defining the existence of biased groups in deep learning datasets, explaining how model behavior differs for different groups (section 3.1).
• As mentioned above, section 3.2 analyzes the nature of different feature encoders, and highlights that the bias-aware feature encoders require explicit mitigation.
• In section 3.3, we show how existing methods (designed for full finetuning) perform inconsistently for different datasets in the proposed problem setting, while Contrastive Adapter [50] (designed for such frozen encoders) performs decently, but is highly time consuming. This motivates the requirement of dedicated research in the direction of the proposed problem setting.
• Section 3.4 describes our method, where we first motivate amplifying the bias, and then mitigating it through the clustering-based margin loss approach, after exploring a number of simple baselines involving weighted losses.

We can clarify further if the reviewer still has doubts.

Clarifications on selection of weight decay value: In the paper, we have shown that high weight decay leads to a reduction in the bias-conflicting group performances in the training set, thereby helping us detect the bias easily. The value of $\lambda$ is chosen using the validation accuracy in the mitigation stage in practice from a range of high values (0.01-1). Details are present in Appendix A.4. When $\lambda$ is sufficiently high, we find that the training accuracy collapses (~50% for Waterbirds and CelebA, <30 % for ColorMNIST-0.995). This fall can be clearly seen in Figure 2c in the main paper for ColorMNIST-0.995 for $\lambda \geq 0.5$. This helps us eliminate a few values of $\lambda$ in our specified range during the amplification stage itself. For Waterbirds and CelebA, the fall is typically seen at $\lambda \geq 3$ for the ResNet-18 backbone. This happens as the model’s learning is impeded with extremely high weight decay, which is not the goal of our approach. Recall that the purpose of this step is to learn the bias-aligned samples well, not the bias-conflicting samples. Hence, in the paper we suggest picking $\lambda$ lower than this threshold. We stick to the search space of (0.01-1) as we find that the scores remain similar for $\lambda \geq 1$ for both datasets (see below).

Clarifications on the clustering algorithm: Please see the general response section.

Dear Reviewer wKPv,

Can you have a look at the rebuttal and see if your concerns have been addressed?

Best regards Your AC.

We thank the reviewer for the positive comments and feedback and address the specific concerns below.

Effects of starting from weight decay $\lambda=0$ in Figure 2 of main paper: We have shown the effects starting from weight decay = 0 on the different group accuracies for Waterbirds, CelebA and ColorMNIST-0.995 in Figure 1 in the rebuttal pdf, where we see that the lower the weight decay, the higher the training accuracy, because of overfitting to all the groups.

Performance of different models for varying degree of bias-conflicting samples This is a great suggestion, and likewise we train our method on ColorMNIST-0.999 (Table 1, rebuttal pdf), and two versions of Waterbirds and CelebA. For Waterbirds, we show performance of different methods with a) no bias-conflicting samples, b) 50% bias-conflicting samples, where the bias-conflicting samples constitute two groups: waterbirds on land and landbirds on water (Table 2, rebuttal pdf). We show the scores for similar versions of CelebA, where the bias-conflicting samples constitute the blond males (Table 3, rebuttal pdf). As predicted by the reviewer, with a decrease in percentage of bias-conflicting samples, the scores of all methods drop. In Waterbirds, for the case of no bias-conflicting samples, the drop in performance is substantial, as is for ColorMNIST-0.999 (see Table 1, rebuttal pdf). For all other cases, we find Co-Ada and our method to perform decently. It is to be noted that our method outperforms all others in all the explored cases.

Standard Deviation for competing methods and other tables We will rectify this in the camera-ready version, upon acceptance. We apologize for not showing the standard deviation values in all cases to ensure ease of readability. We show the values for some of the competing methods below (for ResNet-18 backbone).

Dear Reviewer qb1H,

Can you have a look at the rebuttal and see if your concerns have been addressed?

Best regards Your AC.

We thank the reviewer for recognizing our contributions and appreciating the comprehensive experiments. We address the concerns below:

Assumption on alignment of bias of the downstream dataset with that in the pretrained model. Please see the general responses.

Table 8: Scores of ViT-B on Waterbirds: Please see the general responses.

Limitation on exacerbation of biases. We thank the reviewer for pointing out this limitation, where after bias-amplification, if the debiasing module fails, the method may end up exacerbating the bias instead of mitigating them. We agree that this is a limitation in general for most of the existing debiasing methods, and attackers can leverage this for all methods that involve a bias amplification stage. We will mention this in the camera ready version.

Dear Reviewer pwPo,

Can you have a look at the rebuttal and see if your concerns have been addressed?

Best regards Your AC.

Thank you for your response.

The authors have adequately addressed my primary concerns, and I have no further questions. I will maintain my previous rating, but conditioned to 'bias exacerbation' revision.