Bias Analysis in Unconditional Image Generative Models

1. Some of the discussion in the method section (sec. 3) seems to be redundant or not tied to the paper. For example, what is the purpose of introducing $P^{ideal}$? Although it is canceled in the final equation, I don't think it is necessary to introduce such a term because, intuitively, $|{P^{gen} - P^{val}}|$ itself is sufficient to measure the bias shift. Introducing extra and probably unnecessary assumptions may overcomplicate the method and lead to confusion. Also, Section 3.1 also introduces a definition of "conditional bias," which is not discussed or studied in the rest of the paper. What is the purpose of introducing this definition?

2. The paper claims (L142) that "pre-trained models introduce their own biases, rendering the predicted labels unreliable for accurate bias evaluation." However, I disagree with this argument. I agree that the pre-trained model may be biased, but this reason does not invalidate them for performing attribute classification. Such a classifier serves as the expert in labeling attributes so that the most important criterion, if not the only criterion, should be classification accuracy. If any pre-trained models have outstanding attribute classification performance on the training/val dataset, I don't see why they shouldn't be used. Further, those pre-trained models can be finetuned on the training dataset (which this paper did) for an even better classification performance on specific datasets (e.g., CelebA), which can only benefit the bias shift analysis.

3. Further, the accuracy of the classifier is not sufficient for the analysis. Although the accuracies of the majority of attributes are 90%+, there is still a considerable amount of attributes on which the classifier performs unsatisfying. This fact is critical to the analysis, considering the listed subjective attribute examples are placed in the lower portion of the performance list. Lower accuracy may suggest higher analysis noise and larger ABS measuring error. Since the ABS for non-subjective attributes and subjective attributes are ~1% and 3-5%, the classifier with 91.7% (lowest attribute: 68.34%) or 90.5% (lowest attribute: 71.65%) accuracy is not good enough.

4. Further, the classifier is trained on the training set and directly applied to both training, valid, and generation sets. However, unlike training and valid sets are sampled from the same distribution, the generation set may have a different distribution than the original dataset. Thus, the classifier may suffer from distribution shift and/or visual domain generalization challenges, so the classifier may not be reliable on the generation set. This issue can further weaken the paper's analysis and conclusion.

5. Although the attempt to split the attribute into subjective and non-subjective groups is interesting, I am not convinced that the splitting method used in the paper (decision boundary-based) is valid. The decision boundary is closely connected with classifier accuracy, which can be further connected with analysis noise and measuring errors. Thus, those attributes with unclear boundaries are more likely to have higher ABS errors. Additionally, this splitting may not match human's definition of "subjective." Those "subjective attributes" to human definition (e.g., wearing glasses) may be easier to be classified so that they may have clearer boundaries. However, there is no guarantee of this, and the paper also does not have a complete list of subjective and non-subjective attributes to verify.

As a preliminary, I have reviewed this paper before for the NeurIPS SafeGEN workshop. I have reread this version of the paper, and my opinion has not changed.

Here's my concerns:

• Calling attributes subject vs non-subjective is very strange. For example, "Pale skin" or "male" in CelebA being a non-subjective attribute is surprising. I would be convinced if there were a user study to validate these attributes are similarly subjective to humans, but as it stands I'm not convinced.
• The raw bias shift is strikingly small. The subjective logits on figure 5c look extremely similar between the synthetic versus real data. Especially when comparing 5c to 5e, it's surprising that Male landed in non-subjective and Smiling did not.
• Using the same dataset for training the generator and classifier is very problematic: it's self-contamination. The bias studied in this paper could come from: the dataset itself, the generator's training/architecture, or the classifier's training/architecture. Given the generator and classifier are mapped to the same data distribution, the inherit biases are muddled between the two. I would have liked to seen dataset splits where half the data is used to train the generator and half the classifier. That would improve the self contamination issue substantially.
• Finally, the actual take-aways from the paper are fairly limited. Assuming my previous point were address, the fundamental why question is not answered: why are some attributes represented more/less in the synthetic distribution. It is somewhat useful to know that some attributes are, but I would be very interested to know how to predict which attributes would be over/under represented by just training a classfier.

1. Bias Definition: The paper's definition of bias, which only measures differences in attribute occurrence probabilities, may be overly simplistic. Bias typically encompasses more complex dimensions including social prejudices and systemic discrimination. Oversimplified Metrics: The Average Bias Shift (ABS) metric may be too reductive as it:

2. ABS doesn't consider correlations between attributes and ignores the varying social impact weights of different attributes

3. The 0.01 threshold for subjective/non-subjective classification lacks justification, and there are no ablation studies on threshold selection. The data-driven categorization approach may overlook inherent social and ethical implications of attributes

4. Relying on a single classifier may introduce classifier-specific biases. Figures 4 and 5 show relatively low accuracy (90% or below) for many attributes, questioning the reliability of the pre-trained classifier. It would be better to consider using Large Language Models as supplementary evaluators (Just a suggestion, no need to add experiments).

5. The current model selection appears dated, primarily relying on ADM (2021) and BigGAN (2019) for experiments. They may not reflect the latest advances in generative modeling. The paper would benefit significantly from validating the proposed framework on more recent architectures, such as Stable Diffusion, DiT, PixArt-α for diffusion models, and StyleGAN3 for GANs.

1. As this paper presents a bias evaluation framework for image dataset, it needs to be compared with other evaluation framework, i.e. compare with [1]. How is the presented framework differ with the [1]?

2. Limitations of this evaluation framework should be discussed in the paper.

References:

[1] Wang, Angelina, et al. "REVISE: A tool for measuring and mitigating bias in visual datasets." International Journal of Computer Vision 130.7 (2022): 1790-1810.