Improving Fairness and Mitigating MADness in Generative Models

In response to weakness 1, we plan on running this test and will report back.

In response to weakness 2, we added a new section to the appendix of the rebuttal revision (Section K) which includes generated images for some of the FID values provided.

In response to weakness 3, our experiments were designed to show that our method works on 1) different generative models (VAE, GAN, GMM), 2) different datasets (MNIST, CIFAR, etc), and 3) architectures of varying complexity, from simple parameter estimation to BigGAN

In response to weakness 2, the spike in MADness at iteration 4 occurs once the generations diverge enough from the training data. We do not expect a linear increase in FID, and indeed madcow iteration vs the distance from the estimated to the true parameters is nonlinear for many distributions, as shown in Figure 4. Also note that FID still increases, even with PLE; this collapse is observed for all generative models, see https://arxiv.org/abs/2307.01850 “Our primary conclusion across all scenarios is that without enough fresh real data in each generation of an autophagous loop, future generative models are doomed to have their quality (precision) or diversity (recall) progressively decrease” We can hope to stabilize it and slow this down but this is observed for all generative models.

In response to weakness 3, please note that we do train our method with a variety of other datasets beyond CIFAR-10, which includes multiple networks trained on MNIST in Section 5.1, GMM data in Section 5.2 (trained with a deep network).

In response to claimed weakness 4, we describe theoretically why unbiased estimation improves fairness in lines 76-83, which has been reproduced here for the reviewer’s convenience: “When MLE produces biased estimates of the parameters (as it often does), the parameterized distribution becomes even more concentrated around existing high-probability events . Since probability distributions must integrate to 1, increasing the frequency of some events events comes at the expense of decreasing the frequency of others. The other events in this case are those that are less frequent, or belong to minority classes. As one can see in the first row in Figure H.3, biased maximum likelihood estimates eventually collapse towards the mode(s) of the data and thus will underrepresent data away from the mode. As a result, unbiased estimators will generate distributions that more accurately represent the frequency of minority events.”

In response to question 1, we added a new section in the rebuttal revision (Section I) that explains how our definition of fairness can easily be extended to data belonging to multiple classes. It introduces a few new metrics for evaluating the between-class scores and frequencies, including a new metric that reports the frequency of generated data (separate from the quality of such data)

In response to weakness 1, we agree that training on diffusion models is important, however the expense of sampling from diffusion models means future work is needed to adapt our method to diffusion models. We explain this in the conclusion, which reads, “Since hypernetwork training involves sampling the generative model to evaluate the penalty, future work is needed to allow tractable training of diffusion models, which are expensive to sample from.” Additionally, in a new section of the appendix, we explain how this method could still provide a benefit to diffusion models without this overhead, which reads “There is likely still a benefit to using hypernetworks for training diffusion models due to the averaging operation that occurs, which is known to convexify the loss function, see See Averaging Weights Leads to Wider Optima and Better Generalization, https://arxiv.org/abs/1803.05407)”

In response to weakness 2, our experiments were designed to show that our method works on 1) different generative models (VAE, GAN, GMM), 2) different datasets (MNIST, CIFAR, etc), and 3) architectures of varying complexity, from simple parameter estimation to BigGAN. While our work can be applied to LLMs directly, it is out of the scope of our current paper for space reasons.

In response to question 1, we say GANs are trained with MLE because, as Goodfellow describes, “[t]the training objective for D can be interpreted as maximizing the log-likelihood for estimating the conditional probability” (See https://arxiv.org/abs/1406.2661). We cite Vapnik’s connection to empirical risk minimization and log likelihood maximization with maximum likelihood estimation on line 53.

In response to question 2, yes this was a typo and has been fixed in the rebuttal revision, thank you for pointing out.

In response to question 3, the assumption of linearity for S(M) was to illustrate that the loss incurred for data belonging to the majority class contributes more to the overall loss than that incurred for data belonging to the minority class. If the reviewer thinks this illustration is potentially confusing in its assumption of linearity, we can remove this equation.

Your question about line 316 and whether R_Fair should be less than or equal to |CMaj|/|Cmin| is one we thought about as well. RFair<|CMaj|/|Cmin| refers to a case where the model’s performance does not scale linearly with class frequencies, as the score disparity between majority and minority data would actually be less than expected from the ratio of frequencies. On one hand, one can certainly argue that RFair<|CMaj|/|Cmin| is unfair to majority datapoints, the same way RFair<|CMaj|/|Cmin| is unfair to minority datapoints. However, in the context of optimization, data belonging to the same class is likely close in feature space. As a result, decreasing the loss from data from one class will improve the performance of data in that class on average more than those not belonging to this class. This can cause saturation, and the terms that contribute to the loss most eventually become data that is not as well represented. This smaller distance between points of the same class explains why RFair can be pushed below the imbalance ratio without the model itself being biased. If the number (not the quality) of minority datapoints to majority points generated (classified by an external oracle) differed from the ratio, this would be problematic and indicative of bias.

In response to Question 5, we explain how our hypernetworks are trained in Section H.2 and in Section 4 (The formatting of the equations was getting messed up when trying to copy it in the comments) We also plan on providing our code upon publication (the Github link has been removed during the blind section of the review process).

In response to the first weakness, we added a new section that describes how our work differs from existing fairness methods (the papers suggested refer to fairness and not bias as we are describing it). This can be found in Section 1.2 of the Rebuttal Revision, and is copied here below:

While recent work has looked at improving fairness in generative models, our work differs conceptually in its focus on removing statistical bias. By removing statistical bias, we avoid over-representing data belonging to majority classes without needing to specify any protected attributes or classes. Other approaches are either restricted to a single model type or require data labeled with protected attributes. For instance, FairGAN (Xu et al., 2018) proposes a variant of GANs that requires labeled data with protected attributes, and can only be used for training GANs. Choi et al. (2020) proposes a method that uses two datasets in situations when a smaller dataset may better represent the population ratios, but does not address bias in the learning process itself.

The gradient clipping approach suggested by Kenfack et al. (2022) seeks to improve fairness by biasing the dataset towards uniformity. They write that their goal is “to improve the ability of GAN models to uniformly generate samples from different groups, even when these groups are not equally represented in the training data.” This differs from our model in that 1) it actively biases the model to favor more uniform generation of points with different classes to achieve fairness, and 2) requires data labeled with protected attributes, which may not be feasible to expect. Finally, Rajabi and Garibay (2022) suggest a method for generating tabular data whose statistics match a reference dataset. This approach also relies on explicit labels of the protected attribute in the training dataset and is restricted to GANs. Our method can be used for any generative model and requires no labels or information about the protected attribute

In response to the second weakness, our experiments were designed to show that our method works on 1) different generative models (VAE, GAN, GMM), 2) different datasets (MNIST, CIFAR, etc), and 3) architectures of varying complexity, from simple parameter estimation to BigGAN.

In response to the third weakness, training a generative model with hypernetworks allows bias to be removed via parametric bootstrapping, as described in section 3.2. How parametric bootstrapping removes bias is found in Hall’s 1992 book The Bootstrap and Edgeworth Expansion. The relationship between bias removal or correction and fairness is described in Section 1.2. The relevant section reads:

.Recent work has shown that generative models carry and often amplify unbalances present in training data (Zhao et al., 2018). When MLE produces biased estimates of the parameters (as it often does), the parameterized distribution becomes even more concentrated around existing high-probability events3. Since probability distributions must integrate to 1, increasing the frequency of some events comes at the expense of decreasing the frequency of others. The other events in this case are those that are less frequent, or belong to minority classes. As one can see in the first row in Figure H.3, biased maximum likelihood estimates eventually collapse towards the mode(s) of the data and thus will underrepresent data away from the mode. As a result, biased estimators will learn distributions where majority-class data is overrpresented and minority-class data is underrepresented, while unbiased estimators will learn distributions that more accurately represent the frequency of minority events. While recent work has looked at improving fairness in generative models, our work differs conceptually in its focus on removing statistical bias. By removing statistical bias, we avoid over-representing data belonging to majority classes… The scope of our work does not consider cases where the original training data is biased, as this knowledge could only come from additional (training) data or an oracle. Appropriate data collection (e.g. random sampling) is needed to ensure training data is unbiased. Instead we look at whether the model learns a biased representation from the given data.

For your comment about lambda, we added a section (Section J in the rebuttal revision) which expands on our choice of lambda = 0.1, which we chose empirically via ablation experiments. The relevant section reads:

“Our choice of λ = 0.1 was based on ablation experiments for the GMM example shown in Section 5.2. These experiments show that λ = 0.1 is the best choice; it performs better than MLE in the low-data regime and has less of a negative impact for larger samples than λ = 0.0 and λ ≥ 10. Note that hyperparameter training provides an advantage over MLE even when the PLE penalty is zero, as shown in Figure 6). We believe this benefit comes from averaging the weights from different batches, which is part of hyperparameter training shown in Equation 6. Work by Izmailov et al. (2019) has shown that averaging in weight space (as is done with evaluating the hypernetwork on batches of data) leads to better generalization and wider optima. As shown in the following figures, this averaging accounts for some (but not all) of the benefit of using our hypernetwork approach for training generative models. Furthermore, our choice of hypernetwork architecture described in Section H.1 implicitly applies the PLE penalty during training. Below is a plot of training epoch versus estimated empirical bias, for both a naive hypernetwork architecture which features no averaging (Figure 11) and our proposed hypernetwork architecture (Figure 12).

For both of these experiments, our hyperparameter λ was chosen to be 0, so the empirical bias is not explicitly minimized. However our chosen hypernetwork structure implicitly minimizes the empirical bias during training to a certain extent. Increasing the value for λ penalizes the empirical bias more, leading to models that are even more fair to datapoints belonging to minority classes and further stabilizing self-consumed estimation.”

In response to the fourth weakness about the relevance of the appendix, each section of the appendix is referenced in the main body of the paper. We reference Appendix A, which discusses hybrid Bayesian and Freqnetist methods when discussing the connection of our work to parametric bootstrapping in Section 2.1, as Efron’s work also connects parametric bootstrapping to hybrid Bayesian and frequentist methods. Section B is referenced in a footnote in Section 1.2 for readers unfamiliar with Kolmogorov’s axiomatization of probability theory, which is used for statistical estimation. Our theory in Section 1.2 refers to events which rely on this axiomatization. Appendix C provides an overview of generative models, and is referenced in Section 2.1 when discussing generative model training and unbiased estimation. Appendix D describes Maximum Likelihood in more detail, which forms the basis for the motivation for our method. Section E is cited both in Section 5.2 illustrating our method for closed-form solutions for distributions and in Section 4 to motivate our hypernetwork architecture. Sections E and F are both used to find closed-form solutions to distributions shown in Figure 4 in Section 5. Section G shows our method is asymptotically unbiased, illustrating our claim about bias. Section H describes our implementation details for Figure 4 in Section 5 and provides more details on our hypernetwork implementation described in Section 4. The additional appendix sections have been added in response to other reviewer comments and were not present in the initial submission.

For your comment on line 334, we added an additional sentence explaining our choice of digits 3 and 6. The goal of the experiment is to show the effect of our method on the minority class, which depends primarily on the frequency of occurrence and not the class itself. Any two digits from the dataset could be chosen as long as the training frequencies differ by the reported amount. We thus chose these digits at random.

For the final weakness, you claim, “The paper highly depends on hypernetworks, which increase the complexity of the method and its practical usage.” As we describe in lines 487-489, the computational overhead is minimal. We believe this is in part due to the weight averaging which convexifies the loss function (See Averaging Weights Leads to Wider Optima and Better Generalization, https://arxiv.org/abs/1803.05407). The benefits of increased fairness and robustness to MADness justify the additional overhead (with the overhead being minimal, as stated in our paper). For example, the FairGAN paper you referenced above justifies training additional discriminators based on the increased performance on minority data.

We have fixed all typos mentioned, thank you.

I thank the authors for their detailed response. However, the rebuttal mostly stops at repeating what was mentioned in the paper without addressing most of the concerns raised. I highlight a few of the issues below.

our experiments were designed to show that our method works on 1) different generative models (VAE, GAN, GMM), 2) different datasets (MNIST, CIFAR, etc), and 3) architectures of varying complexity, from simple parameter estimation to BigGAN.

The issue was to demonstrate the robustness of the solution by performing the same set of experiments across different datasets and generative models. Using a single and different dataset to evaluate each aspect of the work does not make the conclusions sufficiently robust and might hide some limitations.

In response to the third weakness, training a generative model with hypernetworks allows bias to be removed via parametric bootstrapping, as described in section 3.2 [...] The scope of our work does not consider cases where the original training data is biased, as this knowledge could only come from additional (training) data or an oracle. Appropriate data collection (e.g. random sampling) is needed to ensure training data is unbiased. Instead we look at whether the model learns a biased representation from the given data.

It remains unclear to me how the proposed method can mitigate bias and improve fairness. Furthermore, if the original training data is not biased, which is rarely the case, it is unclear what form of representation bias the paper aims to study. Furthermore, the paper studies the effect of the proposed method on the minority class and randomly underrepresents certain classes on the MNIST dataset, which makes the dataset biased. This makes the statement "The scope of our work does not consider cases where the original training data is biased, as this knowledge could only come from additional (training) data or an oracle" inconsistent with the methodology of the paper; please clarify if I missed some part of the work.

The benefits of increased fairness and robustness to MADness justify the additional overhead (with the overhead being minimal, as stated in our paper). For example, the FairGAN paper you referenced above justifies training additional discriminators based on the increased performance on minority data.

The authors should provide compelling arguments/proofs on how the proposed method increases fairness and also justify why the method assumes the training data is unbiased. As of now, I maintain my score.

In response to weakness 1, the accessibility of our work is important because we want to ensure people can understand it, however the motivation for our method does come directly from estimation theory. Sections B-F in the appendix provide a background on topics from estimation theory relevant to our contribution.

In response to weakness 2 and questions 1 and 4, about our choice of $\lambda$, we added a section (Section J in the rebuttal revision) which expands on our choice of lambda = 0.1, which we chose empirically via ablation experiments. It reads:

“Our choice of λ = 0.1 was based on ablation experiments for the GMM example shown in Section 5.2. These experiments show that λ = 0.1 is the best choice; it performs better than MLE in the low-data regime and has less of a negative impact for larger samples than λ = 0.0 and λ ≥ 10. Note that hyperparameter training provides an advantage over MLE even when the PLE penalty is zero, as shown in Figure 6). We believe this benefit comes from averaging the weights from different batches, which is part of hyperparameter training shown in Equation 6. Work by Izmailov et al. (2019) has shown that averaging in weight space (as is done with evaluating the hypernetwork on batches of data) leads to better generalization and wider optima. As shown in the following figures, this averaging accounts for some (but not all) of the benefit of using our hypernetwork approach for training generative models. Furthermore, our choice of hypernetwork architecture described in Section H.1 implicitly applies the PLE penalty during training. Below is a plot of training epoch versus estimated empirical bias, for both a naive hypernetwork architecture which features no averaging (Figure 11) and our proposed hypernetwork architecture (Figure 12).

For both of these experiments, our hyperparameter λ was chosen to be 0, so the empirical bias is not explicitly minimized. However our chosen hypernetwork structure implicitly minimizes the empirical bias during training to a certain extent. Increasing the value for λ penalizes the empirical bias more, leading to models that are even more fair to datapoints belonging to minority classes and further stabilizing self-consumed estimation.”

In response to question 2, FID still increases, even with PLE; this collapse is observed for all generative models, see https://arxiv.org/abs/2307.01850 “Our primary conclusion across all scenarios is that without enough fresh real data in each generation of an autophagous loop, future generative models are doomed to have their quality (precision) or diversity (recall) progressively decrease” We can hope to stabilize it and slow this down but this is observed for all generative models.

In response to question 3, for Figure 4, the MLE for all six distributions collapses. The reason PLE slightly diverges from the true value for a few of the plots (in particular the standard normal) is due to the variance dropping to zero for a few degenerate runs. A new section has been added to the appendix describing Figure 4 in H.3 to clarify, which reads: The upper-left and upper-middle sub-figures show PLE estimated parameters slope down slightly. This is due to the fact that for a few runs, the variance goes to zero and cannot “recover'” via the multiplicative constant added in the chosen form. These degenerate runs bring the overall average down slightly, as there is no analogous degeneracy for large values. In essence, for the few estimates of the variance that are near zero, the result becomes clipped. This is sometimes described as variance collapse or model collapse in the literature (See pages 8 and 17 of https://arxiv.org/abs/2307.01850)

In response to question 4, section J in the rebuttal revision (as mentioned above) contains more details on how $\lambda$ is chosen and why our specific hypernetwork choice is beneficial, especially in comparison to naive hypernetworks.
Regarding the comment on our choice of the hypernetwork architecture, as described in Section 4, our design choice is based on the following three requirements: (i) permutation-invariance with respect to input data, as the generative model weights should not depend on the ordering of input data; (ii) the ability to estimate the generative model weights given an arbitrary number of input data, which is essential for batch training; and (iii) reducing computational overhead, which we achieve in Equation (6), since the inner sum is embarrassingly parallel and the outer network is a collection of independent fully connected layers that get evaluated in parallel. We agree that future work can study the impact of different hyperparameter architectures for overall performance.

In response to weakness 3, we want to be as consistent as possible, so please provide specific details about what is inconsistent in our results so that we may better address them and have a clearer paper. Thank you.