FairDD: Fair Dataset Distillation via Adversarial Matching

W2: Scalability to Other Protected Attributes: The paper primarily discusses FairDD’s effectiveness concerning attributes like gender and race. However, it is unclear how well this method generalizes to other protected attributes or more nuanced groups within a PA, especially when there are multiple attributes with intersecting biases. An exploration of such scenarios would enhance the robustness of the approach.

Thanks for your insightful comments. We agree that testing more PAs could further help demonstrate the generalization of our method. We regard blond hair as the protected attribute and attractive as the target attribute, resulting in CelebA$_h$. As illustrated in Table, FairDD+DM obtains 7.76% $DEO_M$ and 6.02% $DEO_A$, outperforming DM by 9.25% and 3.54%. Accuracy has also been improved.

For more nuanced groups, we perform a fine-grained PA division. For example, we consider gender and wearing-necktie as two correlated attributes and divide them into four groups: males with a necktie, males without a necktie, females with a necktie, and females without a necktie (CelebA$_ {g \ n}$). Similarly, we consider gender and paleskin, and divide them into four groups (CelebA$_ {g \ p}$). Their target attribute is attractive. As shown in Table, FairDD outperforms vanilla DD in the accuracy and fairness performance on these two experiments. The performance on CelebA$_ {g \ n}$ is improved from 57.50% to 25.00% on DEO$_M$ and 52.79% to 21.73% on DEO$_A$. Accuracy is also improved from 63.25% to 67.98%. Similar results can be observed for gender and paleskin. Hence, FairDD can mitigate more fine-grained attribute bias, even when there is an intersection between attributes.

W3: Potential Dependency on Original Dataset Quality: The framework assumes that PA groups in the original dataset are balanced enough to train FairDD effectively. In real-world applications, where some PA groups might be underrepresented, this assumption could limit FairDD’s effectiveness. The authors could address how FairDD performs under different levels of dataset imbalance.

Thank you for your insightful comments. Your suggestions have inspired us to further study the effect under more biased scenarios. Specifically, we keep the sample number of the majority group in each class invariant and allocate the sample size to the remaining 9 minority groups with increasing ratios, i.e., 1:2:3:4:5:6:7:8:9. We denote this variant CMNIST$_ {unbalance}$ This could help create varying extents of underrepresented samples for different minority groups. Notably, the least-represented PA groups account for only about 1/500 of the entire dataset, which equates to just 12 samples out of 6000 in CMNIST$_ {unbalance}$. As shown in Table 3, FairDD achieves a robust performance of 16.33% DEO$_M$ and 9.01% DEO$_A$ compared to 17.04% and 7.95% in the balanced PA groups. A similar steady behavior is observed in accuracy, which changes from 94.45% to 94.61%. This illustrates the robustness of FairDD under different levels of dataset imbalance.

W1: Limited Practicality Discussion: While FairDD’s focus on fairness is commendable, the framework’s real-world applicability could be affected by computational demands introduced by adversarial matching. The authors could discuss the added computational overhead and resource requirements, especially when scaling to larger datasets.

Thank you for pointing out your concerns. To begin with, we want to clarify that current DDs in DMF complete their training once the total iteration number is reached. For consistency, our experiments use the same hyperparameters, including the total iteration number and batch size. Consequently, the training time in our experiments is independent of the dataset scale.

Actually, we provided the analysis of computational overhead compared to the vanilla methods in Appendix C of the submitted version. There, we evaluate the impact of the number of groups on training time (min) and peak GPU memory consumption (MB) because FairDD performs fine-grained alignment at the group level.

Here, we further supplement the overhead analysis with respect to image resolutions. We conduct experiments on CMNIST, CelebA (32), CelebA (64), and CelebA (96) on DM and DC at IPC=10. DM and DC align different signals, which would bring different effects.

As illustrated in Table, it can be observed that FairDD + DM does not require additional GPU memory consumption but does necessitate more time. The time gap increases from 0.42 minutes to 1.79 minutes as input resolution varies (e.g., CelebA 32 × 32, CelebA 64 × 64, and CelebA 96 × 96); however, the gap remains small. This can be attributed to FairDD performing group-level alignment on features, which is less influenced by input resolution. Notably, although CMNIST and CelebA (32 × 32) share the same resolution, the time gap is more pronounced for CMNIST (e.g., 3 minutes). This is attributed to CMNIST having 10 attributes, whereas CelebA (32 × 32) has only 2 attributes. These indicate that FairDD + DM requires no additional GPU memory consumption. Its additional time depends on both input resolution and the number of groups, but the number of groups more significantly influences it.

As for DC, FairDD requires additional GPU memory and time. Since FairDD + DC explicitly computes group-level gradients, the resulting gradient caches cause FairDD + DC to consume more memory. The small additional consumption is acceptable given the large performance gains in fairness. Additionally, the time gap is relatively larger than that observed between DM and FairDD + DM. Similar to DM, the group number is the primary factor contributing to additional time consumption compared to input resolution.

W1: Including Vision Transformer architectures as backbone networks would further demonstrate the method's generalizability.

Thank you for your comments. Although the Vision Transformer (ViT) is a powerful backbone network, to the best of my knowledge, current DDs, such as DM and DC, have not yet utilized ViT as the extraction network.

We conducted experiments using 1-layer, 2-layer, and 3-layer ViTs. As shown in Table, vanilla DM at IPC=10 suffers performance degradation in classification, dropping from 25.01% to 18.63%. Moreover, as the number of layers increases, the performance deteriorates more severely. This suggests that current DDs are not directly compatible with ViTs.

While FairDD still outperforms DM in both accuracy and fairness metrics, the observed improvement gain is smaller compared to results obtained on convolutional networks. Further research into leveraging ViTs for DD and FairDD is a promising direction worth exploring.

W2: Examining its performance on more challenging datasets like CIFAR100 or ImageNet would strengthen its practical applicability.

Thanks for your suggestions. We created CIFAR100-S following the same operation as CIFAR10-S, where the grayscale or not is regarded as PA. Due to the time limit, we supplemented CIFAR100-S on DM at IPC=10. DM achieves the classification accuracy of 22.84%, and the fairness of 69.9% DEO$_M$ and 25.37% DEO$_A$. Compared to vanilla DM, FairDD obtains more accurate classification performance and mitigates the bias to the minority groups, with 27.30% DEO$_M$ and 15.54% DEO$_A$ improvement.

W3: Including visual examples from the CelebA dataset in the supplementary material would help readers better understand the fairness improvements achieved by the proposed method.

Thank you for your insightful comments. We have supplemented the visualizations in Figure 13 in the Appendix of the revised version. The target attribute is attractive, and the protected attribute is gender. The top subfigure shows the initialized synthetic dataset, where the first row is dominated by males, and the second row is dominated by females. The middle subfigure displays the synthetic dataset generated by vanilla DM, which inherits the gender bias. In comparison, the images highlighted by red circles in the last subfigure are transferred from females (majority) to males (minority). This indicates that FairDD effectively mitigates bias by balancing the sample number across PA groups.

W4: The term "adversarial matching" could benefit from additional clarification, as the current mathematical formulation doesn't explicitly show adversarial operations. A brief explanation of this terminology would enhance the paper's clarity.

Thanks for your suggestion. We have supplemented the illustration at line 199 in the revised version: Compared to vanilla DDs, which simply pull the synthetic dataset toward the whole dataset center that is biased toward the majority group in the synthetic dataset, FairDD proposes a group-level adversarial alignment, in which each group attracts the synthetic data toward itself, thus forcing it to move farther from other groups. This "pull-and-push" process prevents the synthetic dataset from collapsing into majority groups (fairness) and ensures its class-level distributional coverage (accuracy).

Q2: How robust is this method to the availability of the spurious/group labels? E.g., if a method like JTT [a] is employed to get pseudo labels for the bias attribute, how would the performance change in terms of fairness?

Thanks for your insightful comment. We agree that evaluating the robustness of spurious group labels could provide more insights. We randomly sample the entire dataset according to a predefined ratio. These samples are randomly assigned to group labels to simulate noise. To ensure a thorough evaluation, we set sample ratios at 10%, 15%, 20%, and 50%. As shown in the table, when the ratio increases from 10% to 20%, the DEO$_M$ results range from 14.93% to 18.31% with no significant performance variations observed. These results indicate that FairDD is robust to noisy group labels. However, as the ratio increases further to 50%, relatively significant performance variations become apparent. It can be understood that under a high noise ratio, the excessive true samples of majority attributes are assigned to minority labels. This causes the minority group center to shift far from its true center and thus be underrepresented.

JTT [5] is an efficient approach for generating pseudo labels when we do not need to consider the specific attributes a sample belongs to. In other words, JTT only provides a binary label indicating whether the sample is biased according to loss ranking. However, in our study, we focus on fine-grained attribute bias, which requires identifying the specific attributes from multiple attributes for each sample. Consequently, JTT cannot be applied to our research. To provide attribute-level signals, we choose an unsupervised clustering method DBSCAN. Specifically, we do not have any group labels and use DBSCAN to cluster the samples within a batch. The clustering label is regarded as the pseudo group label. From Table, FairDD achieves 94.77% accuracy, and 12.38% $DEO_M$ and 6.80% $DEO_A$. This demonstrates the potential of FairDD combined with an unsupervised approach when group labels are unavailable.

Q3: What if the original dataset is group balanced first, and then the traditional distillation losses are applied? Would that automatically help reduce the bias?

Thanks for your insightful comments. We synthesized a fair version of CelebA, referred to as CelebA$_ {Fair}$. The target attribute is attractive (attractive and unattractive), and the protected attribute is gender (female and male). In the original dataset, the sample numbers for female-attractive, female-unattractive, male-attractive, and male-unattractive groups are imbalanced. To create a fair version, CelebA$_{Fair}$ samples the number of instances based on the smallest group, ensuring equal representation across all four groups. We tested the fairness performance of FairDD and DM at IPC = 10, as well as the performance of models trained on the full dataset. As shown in Table, vanilla DD achieves 14.33% $DEO_A$ and 8.77% $DEO_M$. In comparison, the full dataset achieves 3.66% $DEO_A$ and 2.77% $DEO_M$. DM still exacerbates bias with a relatively small margin, and this is primarily due to partial information loss introduced during the distillation process. FairDD produces fairer results, achieving 11.11% $DEO_A$ and 6.68% $DEO_M$.

W1: For the ColorMNIST and CelebA, the DEO_M is often comparable to that of the original dataset, showing that the distilled data still may follow the bias of the original dataset, though it hasn't alleviated the bias.

Thank you for your insightful comments. We attribute the effect of fairness metrics to two primary factors:

1. Data: More balanced data generally facilitates model fairness trained on it.

2. Inductive bias of model: The inherent bias of the model itself toward the input data.

In our work, we focus on generating PA-balanced data. However, PA-balanced data does not necessarily guarantee a fairer model. When the model has an inductive bias toward common patterns shared across PA groups for TA recognition, the importance of PA-balanced data becomes less significant.

Additionally, since our model uses condensed samples compared to the original dataset, this sometimes results in the partial loss of important patterns critical for TA recognition. The extent depends on the specific DD algorithm. Therefore, sometimes, in a certain dataset and metrics, the model trained on the whole dataset may have a good fairness performance.

Combining these two factors, although the models trained on CMNIST and CelebA perform well in terms of fairness, this does not hinder our approach from successfully mitigating the bias present in the original dataset from a data perspective, as demonstrated in Figure 3 of our manuscript.`

W2: One big issue is that it is not clear if the reported scores are statistically significant as no std was reported.

Thank you for your feedback. The results in our paper are the averaged values across three runs. In the revised version, we have supplemented the results with the standard deviation, and the final results are presented in the format of mean ± standard deviation in Tables 21-25.

W3: The only real image dataset for which the analysis was done is CelebA.

Thank you for pointing out your concerns. We have supplemented another dataset namely UTKFace, commonly used for fairness. It consists of 20,000 face images including three attributes, age, gender, and race. We follow a common setting and treat age as the target attribute and gender as the protected attribute. We test DM and FairDD + DM with the same parameters, the results in Table show that our method outperforms the vanilla dataset distillation by 16.1% and 8.92% on the DEO$_M$ and DEO$_A$. Similar results are observed at IPC = 50.

Q1: The proposed loss function currently considers all groups, but does not consider their cardinality. Would upweighting the minority groups benefit the loss further, where the weights can be inversely proportional to the group size?

Thank you for your insightful comments. We denote the model with inversely proportional weighting as FairDD$_ {inverse}$. Our experiments on C-FMNIST and CIFAR10-S at IPC=10 reveal that FairDD$_ {inverse}$ suffers significant performance degradation, with $\text{DEO}_M$ increasing from $33.05\%$ to $56.60\%$ and $\text{DEO}_A$ rising from $19.72\%$ to $35.13\%$ in terms of fairness performance metrics. Additionally, there is also a decline in accuracy for TA.

We attribute this degradation to the excessive penalization of groups with larger sample sizes. The success of FairDD lies in grouping all samples with the same PA into a single group and performing the group-level alignment. Each group contributes equally to the total alignment, inherently mitigating the effects of imbalanced sample sizes across different groups.

However, penalizing groups based on sample cardinality reintroduces an unexpected bias related to group size in the information condensation process. This results in large groups receiving smaller weights during alignment, placing them in a weaker position and causing synthetic samples to deviate excessively from large (majority) groups. Consequently, majority patterns become underrepresented, ultimately hindering overall performance.

Thank you very much for taking the time to review our paper. After proofreading your comments, we think that we must clarify the scope and contribution of our paper:

Our scope: Our work explores a new field that bridges fairness and dataset distillation, aiming to mitigate the unfairness of condensed datasets while preserving their accuracy (condensation fairness). Importantly, our framework, including DDs in DMF, does not update the model parameters. Instead, it uses randomly initialized neural networks (or trained with few epochs) as non-linear feature transformations

However, your comments primarily discuss fairness in the context of model fairness, which refers to training a model that outputs fair logits under class-imbalanced datasets. This approach places emphasis on the model itself and does not consider the process of information condensation. These two concepts—condensation fairness and model fairness—have fundamentally different emphases. Unfortunately, you seem to ignore this and stand on the side of model fairness when evaluating our approach to condensation fairness. Hence, we kindly ask you to reconsider the contribution of our paper, taking into account the distinction between these two types of fairness.

Our contribution: Our paper is the first to reveal bias inheritance and exacerbation during dataset distillation. To tackle this critical issue, we propose an effective approach that significantly mitigates bias in the condensed datasets.

Q1: Is the weighted centroid matching objective in Eq. 4 correct? It might be insufficient, as two groups of samples with entirely different distributions can still have the same centroid. Could you clarify the distribution matching objective better?

Thank you for pointing out your concerns. Eq. 4 is equivalent to Eq. 3 which unified the loss function of DDs in DMF. We rewrite Eq. 3 as Eq. 4 to highlight that the majority groups will dominate the alignment between the original and condensed datasets, leading to bias inheritance in the synthetic dataset.

Centroid matching is widely used in the dataset distillation field. Representative methods such as GM [a] and DM [b] utilize centroid matching to align the distributions between the original and synthetic samples. Especially when the distance metric D is MSE, Eq. 3 is the commonly used distribution alignment approach MMD (Maximum Mean Discrepancy), whose effectiveness has been demonstrated in numerous studies.

Distribution matching treats the embeddings as signals to be aligned. These methods use the same randomly initialized network to extract embeddings from both the original and synthetic datasets, and then employ MMD to measure the distributional discrepancy between them. The corresponding gradients are used to update the synthetic dataset.

Reference:

[a] Zhao, B., Mopuri, K., & Bilen, H. (2020). Dataset Condensation with Gradient Matching. ArXiv, abs/2006.05929.

[b] Zhao, B., & Bilen, H. (2021). Dataset Condensation with Distribution Matching. 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 6503-6512.

W1: Although the results are improved it’s hard to see the ingenuity in the approach, it would be great if the authors could please clarify the differences.

Thank you for raising your concerns. We have summarized the following differences between our work and [1] and [2]:

Different Scopes

References [1] and [2] focus on improving the classifier fairness trained on class-imbalanced datasets. These methods fall under the field of model fairness, which aims to regularize the model to learn fair representations. However, they do not address information condensation, which is a key objective in our work.

In contrast, our work explores a new research field, condensation fairness, which aims to mitigate the bias of condensed datasets. Condensation fairness encompasses two goals:

1. Guarantee the information of original datasets to be distilled into the condensed datasets.

2. During this process, both the bias inherent in the original dataset and the bias exacerbated by vanilla dataset condensation should be mitigated simultaneously.

Different target

References [1] and [2] primarily target class-level fairness, addressing class imbalance caused by differing sample sizes across classes, similar to the long-tail learning field.

Our work, on the other hand, mitigates attribute bias instead of class-level bias. Attributes and classes describe the object from different aspects. Our objective is to mitigate attribute bias while preserving class information.

Different usage

References [1] and [2] produce tailored fair classifiers, which often have limited applicability to other architectures.

Our method condenses a large dataset into a smaller and fair dataset. Once the fair condensed dataset is obtained, it can be reused to train diverse models across different architectures.

To address the common confusion about the term "Adversarial Matching", we have replaced it with "Synchronized Matching" (highlighted in orange) in the revised manuscript. We sincerely thank all reviewers for their constructive comments.