Diffusing DeBias: Synthetic Bias Amplification for Model Debiasing

We thank the reviewer for the comments and for recognizing as strengths the novelty of the idea and the quality of the experimental phase including a large ablation analysis. We will provide in the following a punctual response to the raised concerns. References, when applicable, follow the same numbering as in the submitted paper.

W1. This work has marginal novelty ...

As also noted by Reviewer 89yi, our main novelty is the usage of diffusion models for amplifying bias, obtaining synthetic samples that can be used for training a bias amplifier instead of using the original training set. Unlike existing works employing data augmentations (e.g., BiaSwap [21] and DFA [27]) or adversarial approaches to synthesize bias-conflicting samples (e.g., BiasAdv (Lim et al, CVPR2023)), to the best of our knowledge, this work is the first to propose the usage of diffusion models for actually generating bias-aligned samples, which are to be used for training an auxiliary biased model. Most notable, our approach solves by construction the problem of training set memorization, which affects the proper learning of an effective auxiliary biased model.

Existing methods often employ bias-annotated validation sets or complicated heuristics [34, 43, 51] for deciding when to stop such training. As a bias-annotated validation set is hard to obtain in real-world applications, we argue that solving this problem is a fundamental novelty of this work. This is also supported by the new performed experiments (please, also refer to the first answer to Reviewer 89yi).

Regarding the debiasing recipes, we exploit a popular end-to-end and a two-stage approach to provide evidence on how the proposed method is generally applicable, and how it can potentially be used as an add-on for existing debiasing approaches, providing significant improvements over the original methods. Furthermore, in this rebuttal, we included additional experiments empirically showing how diffusion models indeed amplify existing biases, and how this actually helps our debiasing recipes.

In the end, we would like to respectfully point out that our work is much more than a simple combination of two known methods. Rather, it has been conceptualized to specifically target known challenges in unsupervised model debiasing, successfully leveraging often overlooked properties of widely used tools such as Diffusion Models in a novel way, and our intuition is also supported by the strong empirical performance.

Regarding Section 3.2, introducing DDPMs and CDPMs (accounting for less than a page, including figures) is intended as a background section necessary to provide the basics that are useful for better understanding the main concepts of the work.

W2 (and Q1). More evidence for using diffusion model is recommended: Authors utilized the diffusion model to amplify the bias ... It is recommended to further investigate how the diffusion model behaves when trained on biased dataset. ... Such exploration is essential ...

We thank the reviewer for having raised the need of additional analyses to strengthen our contributions. To further highlight that the conditional diffusion model is capturing (and amplifying) the biases of the training set, we manually performed a resampling of the Waterbirds dataset to obtain alternative and less extreme bias ratios ($\rho = 0.90$, $\rho = 0.80$, $\rho=0.70$), to be used for training our CDPM. From each resulting model, we synthesized 1000 images per class for each $\rho$, with two independent annotators manually labeling bias-aligned/conflicting samples, thus estimating the degree of correlations $\rho$ in the generated samples. We provide this quantitative analysis in the following table.

As it can be observed, for the original $\rho=0.95$ the CDPM amplifies the bias, and the resulting synthetic training set presents a $\rho$ of 0.99. At the same time, the constructed less biased versions of the datasets also result in the diffusion model amplifying the presence of spurious correlations. We thank the reviewer for suggesting this analysis, and we agree that it adds further value to our work and allows for a better understanding of our main hypotheses and results.

W3 (and Q2). Limited discussion on the generated samples: There is limited discussion on the generated bias-aligned samples. ...

Please refer to the answer to Reviewer 4iDf, where we provided additional experiments to evaluate the impact of generated image quality over the final debiasing performance.

In summary, in these experiments, we empirically showed that even with an image resolution of 32x32 pixels or with CDPM trained for less iterations (thus increasing the FID of the generated images), we still obtain compelling debiasing performance, as the generated synthetic images still present dominant bias attributes that are effectively captured by our bias amplifier.

For the reduced resolution setting (32x32 pixels instead of 64x64), when employing our Recipe I (BA + G-DRO) on Waterbirds data, we report a WGA of 90.09% (<1% than the original result of 90.81%), thus measuring a negligible impact on the final performance. In reducing the number of CDPM training iterations from 200k down to 70k, we can observe how, with synthetic images presenting a much higher FID, we are still able to reach SOTA competitive results.

We thank the reviewer for the comments, and for appreciating our main idea, its flexibility to be used in both 2-step and end-to-end debiasing methods, as well as the clarity of the exposition. We will now provide a punctual response to the raised concerns. References follow the same numbering as the submitted paper.

W1 [Quality of generated images] ... and W2 [Computational complexity] ...

In our experimental stage, we include an extensive and diverse set of benchmark datasets for model debiasing, and we are not aware of a truly large-scale benchmark with proper annotations allowing for bias-mitigation analyses (e.g., equipped with a test set with subgroup annotations). Moreover, we would like to highlight that our performance gain is not negligible, given that we are able to achieve SOTA results across quasi-realistic, real-world data, and multiple bias benchmark datasets, while most of the other (bias) unsupervised debiasing methods struggle in at least one of these settings [31].

As per the resolution of the synthetic generated data, we indeed highlight that we generate images at a resolution of 64x64 pixels to reduce computational time, regardless of the quality of the generated data (as measured by FID or other metrics). However, this parameter is not fundamental for our method as we are not interested in perfect fidelity or high quality appearance, but rather in capturing the bias cues in place of the semantic attributes and, as such, we do not mind if some image artifact may be introduced. Rather, this is a pro of our approach, as we are not constrained to generate high-quality, perceptually highly realistic images.

To further support this statement, we performed an experiment on Waterbirds data, computing the Fréchet Inception Distance (FID) metric to evaluate the quality of generated images. Specifically, we trained our CDPM on Waterbirds using a pixel resolution of 32x32, obtaining a FID of 58.11, while our original generated images at 64x64 showed a FID of 39.03 (the lower, the better). Exploiting these generated images, we then run our Recipe I (BA + G-DRO): the obtained WGA results 90.09 +/- 0.55% over three independent runs, less than 1% lower than our reported main result of 90.81 +/- 0.68 %. The fact that even a low resolution of 32x32 is sufficient to capture the existing shortcuts should further be a reasonable evidence that the quality of the obtained generations is secondary to the final aim of our paper.

As an additional analysis on this matter, we performed new experiments exploiting BFFHQ. Specifically, in our original experiments, we trained the CDPM for 200k iterations, obtaining a FID of 22.60. In these additional experiments, we instead utilize intermediate training iterations, from 70k to 170k. As expected, the images generated with these models show a higher FID, and our aim was to assess how this impacts the final results. In the following table, we provide the FID of the generated images and the final resulting Conflicting Accuracy when applying Recipe I (BA+G-DRO).

As we can notice, even if the FID increases by 36% over the original results, when employing 170k training iterations, the final conflicting accuracy is comparable to the one corresponding to images obtained with a model trained for 200k iterations.

Furthermore, it is worth noticing that we achieve competitive results with respect to the state of the art, even when employing 170k iterations, requiring approximately 9.5 hours of training time.

When training the model for 70k iterations only (FID = 41.82), we obtain an average conflicting accuracy of 67.40% (with the best run reaching a 69.2% conflicting accuracy), but requiring less than four hours for training the CDPM. We believe that these results confirm that even if the quality of images decreases, bias is still present in the generated images, and is effectively captured by our BA.

W3. DDB with less biased dataset. Most experiments are conducted on datasets with high bias severity, which facilitates the generation of strongly bias-aligned samples. It remains unclear ...

We performed dedicated experiments to clarify this matter. Please refer to the answers we provided to reviewers EHbf and w3kR.

In summary, in the first reply, we report experiments on multiple versions of Waterbirds data, which we resampled to have $\rho=0.90$, $\rho=0.80$, $\rho=0.70$. In this experiment, we show how our method is always capable of greatly improving over the ERM baseline. In the second response, we also measure the bias correlation found in the generated images when the CDPM is trained on the same resampled versions of Waterbirds data with less extreme bias, confirming how the presence of bias-aligned samples is always amplified in the resulting synthetic generations.

[W4. Comparisons with image-generation studies. Although the authors reference image-generation-based debiasing methods such as Biaswap ...

We thank the reviewer for pointing us to other methods for the performance comparison, as well as other works employing generative models to countermeasure generalization failures in image classification (like ActGen).

Regarding the comparison with BiaSwap [21], we provide here a direct comparison between our two debiasing recipes and BiaSwap on two shared benchmark datasets (BAR and BFFHQ).

As evidenced by the significant improvement brought by our approach, the intuition of amplifying bias through bias-aligned synthetically generated images is effective and brings more advantages than relying on bias-amplifying loss functions for the auxiliary model (as commonly done in the literature, including BiaSwap, where the GCE [53] is employed), or by attempting to inject synthetic bias-conflicting characteristics into bias-aligned samples (also done in BiaSwap).

For the other suggested reference, ActGen, it does not address the bias mitigation problem and adopts different experimental protocols and data, preventing a direct comparison. Nevertheless, we will incorporate it in the Related Work section of the final version of our manuscript, as improving generalization in challenging downstream tasks through synthetic image generation is a topic linked to ours.

We thank the reviewer for the insightful comments and for recognizing our main idea as original and refreshing. We will now provide a point-to-point response to the raised concerns. References follow the same numbering as the submitted paper.

W1. The paper does not appear to include experiments that directly assess the effectiveness of the Bias Amplifier (BA) when trained solely on bias-aligned samples generated by the diffusion model. A comparison between a BA trained on the original training dataset and one trained on synthetic bias-aligned samples—using an otherwise identical debiasing setup—would be valuable. ...

We thank the reviewer for the comment. Employing Waterbirds (as done with the other ablation studies), we performed the suggested experiments as follows. First, we train the BA on the original dataset, then we employ it for our two Recipes, as follows:

• Frozen BA + LfF (Learning from Failure [38]), that is our Recipe II but with the BA trained on the original real training images (not the diffusion-generated synthetic ones). In this case, we obtain a Worst Group Accuracy (WGA) of 78.45 +/- 2.61 over three runs, with an average decrease of 13.11%. Furthermore, when comparing these results with the original LfF, whose auxiliary biased model is not frozen as the reviewer correctly noted, we got a WGA of 78.00%, which is in line with the result of our new performed experiment, still much less than the performance of our proposed approach.

• With the same BA trained on the original images, we applied our Recipe I, i.e., training GroupDRO with the group annotations obtained as in the Section 3.3.1 of the submitted paper. Here, the results confirm the problem of memorization that we highlighted in our paper. In fact, training the BA on the original images (same setting of our original experiment, which is 50 epochs) results in 100% training accuracy, hence with bias-conflicting samples totally memorized. As such, when mining the group pseudo-labels, we end up with two empty sets (corresponding to conflicting in class 0 and class 1). These results confirm that, indeed, memorization is a critical issue when training biased auxiliary models. Nonetheless, to still obtain a somewhat meaningful comparison, we trained the bias amplifier on the original images for only one epoch (as done in JTT [34]), obtaining a final WGA of 79.43 +/- 1.92, again, with a significantly lower mitigation performance than exploiting the BA trained with our original approach.

We will include these results in the final version of our paper, and we thank the reviewer again for suggesting these comparisons to strengthen and add value to our contribution.

W2. The performance table presented in Table 1 is not well organized, and nearly half of the cells lack reported performance values.

We agree with the reviewer that Table 1 should be improved. The lacking values are due to benchmark methods missing some of the considered datasets. Hence, we plan to divide Table 1 per dataset to have a better organization and readability.

W3. To the best of my knowledge, the training set of BAR consists solely of bias-aligned samples [1], which—as the authors note—makes it suitable for training a robust bias amplifier. ...

We personally inspected the original version of the BAR dataset (from Nam et al., 2020 [38]), and we can confirm that a few bias-conflicting samples are present in the training set (for instance, see samples with original filenames in the training folder diving_70, diving_89, diving_99), with an imbalanced distribution with respect to the classes.

However, as the original version of BAR does not provide bias annotations, we performed additional experiments, employing the BAR version introduced in Lee et al., 2023 [28]. Here, two versions with full bias annotation of BAR are provided (ρ = 0.95 and ρ = 0.99). In the rebuttal time, we managed to perform an experiment on this dataset utilizing the variant with $\rho=0.95$. Here we report the results obtained with our DDB-I (Recipe I), comparing against two SOTA methods on this dataset.

These results confirm the effectiveness of the proposed approach, and we are confident that they can have solved the reviewer's doubts.

We remain at disposal for further information or clarifications on that.

We thank the reviewer for the comments and for appreciating our empirical performance and the flexibility of the integration of the bias amplifier model. We will now provide a point-to-point response to the raised concerns. References follow the same numbering as the submitted paper.

Training diffusion models is expensive (~14 hours on an A30 GPU for BFFHQ at 64×64 resolution), which may limit scalability.

We agree that this is a limitation of our proposed approach, and in fact, we explicitly declared the training time as a possible limitation of our work. However, we also would like to point out that training a diffusion model has nowadays to be considered acceptable and in line with current standards in computer vision and deep learning computational requirements. It is also worth to notice that our reported computational times are referred to a basic implementation of CDPM, as tackling the training time of diffusion models is secondary to our main objective, which aims at showing how a CDPM can be effectively used to generate bias-aligned samples to replace the original training set for training biased auxiliary models, in order to solve by construction some critical open challenges in debiasing scenarios, such as memorization of the few bias-conflicting samples, and when to stop the training of such auxiliary models.

Nonetheless, we reported a new experiment on Waterbirds dataset, where the resolution was reduced to 32x32, obtaining a comparable WGA (90.09%, with respect to 90.81% of the original experiment using 64x64 image resolution with Recipe I), with the training time reduced to 3.2 hours (from an initial training time of roughly 14 hours), showing a significant reduction of the computational time with a negligible decrease of the performance. Please refer to the first response to Reviewer 4iDf for more details on this experiment.

The method's effectiveness may diminish when spurious correlations are weak (ρ < 0.95) or when multiple subtle biases coexist.

In this work, we employ the main benchmark datasets utilized in the image classification model debiasing community. Specifically, all the works on model debiasing that we reference in our paper and benchmark against, exploit a minimum degree of correlation ρ >= 0.95, as the main focus is on mitigating bias in models when the training distribution is predominantly biased. As such, more than a design choice is a convention adopted in this community for benchmarking debiasing algorithms (e.g., see [28, 33, 22, 18, 17, 46, 40]). Furthermore, we can reasonably expect that the generalization of a naively trained ERM model improves naturally when trained on a less severe biased dataset (with ρ < 0.95).

However, we agree that it is interesting to evaluate our DDB approach on a less-biased dataset, and thus, we perform a dedicated set of experiments, resampling the original Waterbirds dataset to obtain three alternative versions with lower amount of training bias-aligned samples (i.e., ρ=0.90, ρ = 0.80, and ρ=0.70). Therefore, we trained both a vanilla model (as a baseline) and our DDB with Recipe I (i.e., bias amplifier + G-DRO), reporting the obtained results in the following table:

As we can see, DDB is still capable of obtaining compelling performance, with a significant improvement over the ERM model. Furthermore, the ERM model test accuracy increases when ρ decreases, as expected.

Additionally, in the submitted version of the paper, we provided a dedicated experiment (DDB impact on unbiased dataset, already present in the submitted version) to evaluate the performance of DDB when no bias is known to be present at all (ρ=0), utilizing CIFAR-10, and obtaining about +1% in accuracy with respect to the ERM.

Finally, regarding multiple biases, we reported the results obtained on UrbanCars, which is a reference benchmark for testing multiple biases in the debiasing community. Specifically, it is constructed to have two different bias attributes (i.e., background and co-occurring objects). The obtained results (following the metrics introduced in the original paper proposing the dataset [31]) show how DDB is actually capable of handling more than one bias. For the latter case, with Recipe I (BA + G-DRO) we manage to reduce the generalization gap with respect to both bias attributes to less than 2%. In contrast, most of the other unsupervised debiasing methods struggle to mitigate model dependency on both.

We hope that this response clarifies the reviewer’s doubts, and we will include the additional results on Waterbirds in the final version of our manuscript.

Limited Real-World Scalability Demonstration: Most experiments are on controlled benchmark datasets. Less is said about generalization to large-scale or long-tail distributions.

Connected to our previous answers above, in this work we employed the main benchmark datasets utilized in the image classification model debiasing community (e.g., [28, 33, 22, 18, 17, 46, 40]), which are not large scale. To test for real-world datasets, we utilized BFFHQ and ImageNet-9/A. In this sense, we are not aware of a truly large-scale dataset (e.g., million sized) employed for evaluating debiasing methods. About long-tail problems, we argue that evaluating such problems, even if interesting, is not in the scope of this work.

Since synthetic data are sampled from biased distributions, the generated data might lack semantic diversity or introduce synthetic artifacts

We apologize for our lack of clarity on this point, and we thank the reviewer for raising this comment. Indeed, bias is by definition a spurious correlation between target labels and data. In a certain sense, biased (a.k.a. bias-aligned) images may be seen as "stereotyped", with respect to the bias attributes they share.

As such, our initial hypothesis was that, when learning the density of the training distribution with a conditional diffusion model, the bias may dominate the generated images' semantic attributes. Since our final aim is to exploit the generated images to train a bias amplifier, having "stereotyped" images may actually be beneficial for amplifying the bias.

Regarding artifacts, we expect that, when bias attributes are present, a trained model ends up to be strongly relying on bias shortcuts for making its predictions, giving less importance to semantic attributes and potential randomly introduced artifacts.

The reported empirical results support our intuitions, as the usage of DDB makes both our recipes better than the original counterparts, and competitive with existing SOTA methods.