Post-Training Recovery from Injected Bias with Self-Influence

We thank reviewer dWNy for the insightful comments. We address the reviewer's comments below.

---

[Q1. Unclear definition in Section 2.1.]

Our intention in describing our problem setting was to define the correlation between the task-relevant signal $s$ and the task-irrelevant bias $b$. We have revised our paper to take into account mild spurious correlations, as in $P(B=b_{y}|S=s_{y}) = p_{y}$, where $p_{y} \geq \frac{1}{C}$.

---

[Q2. Reported accuracies in Figure 1.]

The reported accuracies are the accuracies (precision) in detecting bias-conflicting samples within the training dataset.

---

[Q3. Trend of precision w.r.t. the bias-conflicting sample ratio.]

For bias ratios 1,2,5%, the detection precision of SI shows an increasing trend, as the absolute quantity of bias-conflicting samples increases (hence raising the lower bound of the precision). In the extreme bias case of 0.5%, the model fails to learn any task-related features due to the extreme scarcity of bias-conflicting samples. This phenomenon is observed in Figure 1(a) (now renamed to Figure 1(b)) (which is averaged over multiple runs - we removed their error bars due to graphical overlaps), where the detection precision does not decrease over epochs but plateaus, indicating that the model failed to learn task-relevant features even when trained for a sufficiently long time (200 epochs). For this specific case, the capability of SI to detect noisy label samples takes effect, resulting in the detection performance shown in Figure 1(e) (now renamed to Figure 2(a)).

---

[Q4, Q5. Consistency of detection performance in Figure 1, and the extreme 0.5% case.]

In extreme scenarios, like 0.5%, the task-related features of bias-conflicting samples significantly counteract the dominant features, notably malignant bias features, of the model thereby easily separable by SI. However, as shown in Figure 1(b) (now renamed to Figure 1(a)), there is a notable decline in the detection performance of SI as the ratio increases since the model has learned more of the task-related features of bias-conflicting samples for classification. To address the reviewer's concern, we revised the mentioned part to enhance clarity.

---

[Q6. Novelty of the counterweight CE loss.]

The focus of previous literature was to effectively learn under highly biased datasets by amplifying the presence of bias-conflicting samples. To this end, their main concern was to amplify the signal of bias-conflicting samples when training the model. However, we observe and raise the issue that such strategies underperform under relatively weak bias due to the under-utilization of the dataset as a whole for learning task-related features. Under this scenario, adding a CE loss aids the model in learning such features by utilizing the whole dataset. For SePT, we separately average the CE loss and the pivotal set loss before summation, so that it is agnostic to the relative size of the pivotal set and the balance between the two terms is maintained. We revised our paper to clarify this formulation.

However, naively adding the CE loss to the formulation of previous methods such as [1, 2, 3] can cause detrimental effects to the learning process. Since they attempt to debias the model in the training phase, applying the CE loss destabilizes their learning process and injects bias, especially in the early stages of training where the model is most susceptible to bias[1]. In this sense, the naive introduction of the counterweight CE loss to previous works is not straightforward. However, SePT utilizes the CE loss during the post-training recovery phase, which is more stable compared to direct intervention in the training stage.

---

[1] Nam et al. Learning from failure: De-biasing classifier from biased classifier. NeurIPS 2020.

[2] Lee et al. Learning debiased representation via disentangled feature augmentation. NeurIPS 2021.

[3] Hwang et al. Selecmix: Debiased learning by contradicting-pair sampling. NeurIPS 2022.

We thank the Reviewer coZv for constructive feedback. We address the reviewer's comments below.

---

[W1. Connection of influence functions to bias and fairness.]

First, we briefly explain the concept of using self-influence (SI) functions to identify mislabeled samples. A simple approach to estimate the influence of a single data point on a model's prediction is Leave-One-Out (LOO) retraining, which involves excluding a data point from the trainset, retraining the model, and assessing the impact by comparing the model's prediction before and after the exclusion. However, retraining for each data point is highly time-consuming work. To address the issue, Influence functions $\mathcal{I}(z_i,z_j)$ are proposed to approximate the LOO retraining to compute the influence of a data point. Typically, $z_j$ is sourced from the validation set when available. If not available, $z_j$ is set equal to $z_i$ which is called self-influence (SI). A higher SI score indicates more difficulty in prediction $z_j$ itself when excluding from the trainset. In other words, the more $z_i$ counteracts the dominant features of the pre-trained model, the higher the SI scores it is measured. In this context, SI has been previously used to identify mislabeled samples in the trainset.

This property of SI establishes a connection for its usage in detecting bias-conflicting samples. Nonetheless, as shown in Figure 1, naively adopting SI for biased datasets generally fails due to the distinct nature of bias-conflicting samples compared to mislabeled samples. Bias-conflicting samples, containing task-related features but under-prioritized in training, differ from the dominant feature but do not counteract them. Consequently, models can learn task-relevant features in the late stage of training [2], leading to a reduction in the gap between the self-influences of bias-conflicting samples and those of bias-aligned samples. Therefore, we propose Bias-Conditioned Self-Influence (BCSI) to restrict the pre-trained model from learning task-related features of bias-conflicting samples by using Generalized Cross Entropy (GCE), and training the model for only five epochs based on the findings of Frankle et al.[1] the primary directions of the model's parameter weights had already been learned during iteration 500 to 2,000.

Please note that to enhance readers' understanding, we provide a more comprehensive description in Section 3.1.

---

[1] Frankle et al., "The Early Phase of Neural Network Training", ICLR, 2020.

[2] Nam et al. "Learning from failure: De-biasing classifier from biased classifier." NeurIPS 2020.

Thank you for your valuable feedback. We address the reviewer's concerns below.

---

[Q1,2. W1,2.] How self-influence is used in detecting bias-conflicting samples, its limitations, and the motivations for our modifications.

We agree with the need for a more detailed explanation to enhance readers' understanding of BCSI (Bias-Customized Self-Influence), and we provide a more comprehensive description in Section 3.1.

The key point is that bias-conflicting samples possess correct task-related features, unlike mislabeled samples. Therefore, we have to restrict the model to learn the features of bias-conflicting samples to achieve better separation.

Specifically, as we mentioned in the last paragraph of Section 2.2, self-influence is used for measuring the extent to which a sample conflicts with the dominant feature utilized by the pre-trained model for classification. However, as demonstrated in Figure 1, self-influence struggles to identify bias-conflicting samples in biased datasets. This mainly stems from the difference between the relationship of normal and mislabeled samples, and that of bias-aligned and bias-conflicting samples. While both mislabeled and bias-conflicting samples contrast the dominant features of the pre-trained model, they differ in nature.

In the case of mislabeled samples, which are erroneously labeled as their name implies, they strongly counteract the dominant features of the pre-trained model thereby separable by self-influence. On the other hand, bias-conflicting samples, containing task-related features but merely under-prioritized in training, differ from the dominant features but do not counteract them. In other words, in a noisy labeled setting, the mislabeled sample's feature is incompatible with the dominant feature, whereas, in a biased setting, the bias-conflicting sample's feature is not only compatible but ideally, both should be utilized. This characteristic of bias-conflicting samples makes it harder for SI to separate bias-conflicting samples. As shown in Figure 1(b), as the ratio increases, detection performance declines since the model has learned more of the task-related features of bias-conflicting samples for classification.

Motivated by this insight, we aim to restrict the model from learning task-related features to achieve better separation. To this end, we propose BCSI to effectively identify bias-conflicting samples. As mentioned in the third paragraph of Section 3.1, we used Generalized Cross Entropy (GCE) to train the model to become more biased, focusing less on the features of bias-conflicting samples. Furthermore, based on the findings of Frankle et al.[1] that the primary directions of the model's parameter weights had already been learned during the iteration 500 to 2,000, we train a model for only five epochs for masking the model focuses more on bias. Our method exhibits superior performance compared to conventional self-influence in general, as demonstrated in Figure 1 and Appendix A.

---

[W1.1. Elaboration on Figure 1.]

We have adjusted adjust the structural arrangement, including Figure 1, to improve readability. We will also provide a brief response to the questions.

The terminology of bias-aligned and bias-conflicting is mentioned in Section 2.1 right above Figure 1. We will add a brief reference to the corresponding section in the caption of Figure 1. The legend in Figure 1(a) (now renamed to Figure 1(b)) denotes the proportion of bias-conflicting samples in the dataset. We added this information to the caption of Figure 1. Figure 1(a) illustrates that the detection accuracy of bias-conflicting samples via self-influence(SI) decreases as we use SI on biased models trained for longer epochs. Figure 1(b)-(h) (Figure 1(e)-(h) are now renamed to Figure 2(a)-(d)) demonstrates that our BCSI outperforms naive SI(save for the extreme case of 0.5%) as well as other metrics such as loss and gradient norm in identifying bias-conflicting samples.

---

[1] Frankle et al., "The Early Phase of Neural Network Training", ICLR, 2020.