Unsupervised Lifelong Learning with Sustained Representation Fairness

Thank you for your constructive comments. Below, we address your comments one by one in a Q&A fashion.

Q1: Abstract: „Like human who tends to prioritize simpler tasks ...“: Is there any reference which claims that human tend to prioritize simpler tasks? Looks kind of obvious but there needs to be a scientific study which shows this.

A1: This idea was firstly coined as 'starting small' in [1], stating that human beings can only learn complex and abstract concepts from assembling more intuitive understandings over simple objects. For example, students who want to learn calculus should start with basic arithmetic. This idea was later applied to the learning of neural networks by [2] and is known as curriculum learning.

[1] Elman, Jeffrey L. "Learning and development in neural networks: The importance of starting small." Cognition 48, no. 1 (1993): 71-99.

[2] Bengio, Yoshua, Jérôme Louradour, Ronan Collobert, and Jason Weston. "Curriculum learning." In Proceedings of the 26th annual international conference on machine learning, pp. 41-48. 2009.

Q2: The datasets used for evaluation are quite simple once’s - tabular data only. It’s shown that AdaBoost is why more accurate at tabular data so in general why should one use DNN here?

A2: We use an elastic DNN in this work because its depth can serve as an indirect metric to measure the distance between two tasks. We appreciate your suggestion and conduct experiments on the vision datasets. Please find out the answer in A2 of Reviewer 8gZZ due to character limitations.

Q3: Are there any kind of loopbacks studied?

A3: Our design will not incur loopback and was widely used in the literature, like [3]. This is because not all the predicted pseudo labels are integrated in the retained dataset; rather, only those predicted with high confidence and more likely to be correct labels are integrated. To bolster the confidence of these samples, we implemented a margin mechanism. More specifically, a classifier has been designed to determine the membership of each sample, distinguishing whether it originates from the learned retained dataset or a new task. Only those samples from new tasks that exhibit a probability exceeding 80 percent (a margin commonly used across most datasets) of belonging to the retained dataset, as determined by the classifier's sigmoid layer prediction, are actually added to the retained dataset

[3] Sohn, Kihyuk, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A. Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. "Fixmatch: Simplifying semi-supervised learning with consistency and confidence." Advances in neural information processing systems 33 (2020): 596-608.

Q4: data distributional shifts (Barrett et al.).“: Missing year in the reference A4: We will correct this typo in camera ready.

Q5: what exactly are the tasks in the dataset? This is not really well explained in the text.

A5: Each task represents a distinct, non-overlapping subset of the original dataset. For instance, in the case of the adult dataset, individuals (instances) are categorized into 12 separate groups (tasks) based on their occupations, with examples including Tech-support, Sales, and Farming-fishing, among others. The rationale behind selecting occupation, as inspired by [4], is to ensure a varying distribution of P(Y|P) across different tasks. This approach leads to a dynamic proportion of males and females classified as having high incomes within various occupational categories. Once a model has effectively learned from a specific group of people (task), it is crucial to sustain its performance across other groups (tasks).

[4] Le Quy, Tai, Arjun Roy, Vasileios Iosifidis, Wenbin Zhang, and Eirini Ntoutsi. "A survey on datasets for fairness‐aware machine learning." Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 12, no. 3 (2022): e1452.

Q6: There is a resent paper which uses stochastic quantized representations [2], this can ensures fair representation also after a distribution shift.

A6: Thanks for bringing this paper into our radar. We scrutinized the paper and found that it focused on a different learning problem from ours. Specifically, the paper [5] proposed an alternative de-biasing function through the random (stochastic) projection of discriminative feature representations. However, there is no evidence to show that such a de-biasing function can hold invariant across different tasks, each of which has a distinct demographic distribution. Thus, their proposed method cannot be directly adapted into our lifelong learning context. We will add this discussion in our camera ready.

[5] Cerrato, Mattia, Marius Köppel, Roberto Esposito, and Stefan Kramer. "Invariant representations with stochastically quantized neural networks." In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 6, pp. 6962-6970. 2023.

Thank you for your constructive comments. Below, we address your comments one by one in a Q&A fashion.

Q1: The elastic network design .., making the EFRL ..., why not have a section ...?

A1: We respectfully point out that the elastic network and EFRL are the same, detailed in the Introduction (Page 2) and Section 2.2 (Page 3) with intuition and technical specifics, respectively. The presentation follows your suggested structure.

Q2: The task identification term ..., Why similarity between ..., unclear what "detriment ..." means?

A2: We respectfully point out that our work focuses on lifelong adaptation of fair representations to unlabeled tasks, not just single-task learning via an adversarial network. Challenges stem from unseen negative adaptation, owing to a lack of labels and demographic shifts, potentially leading to error build-up. The EFRL network addresses this by using depth information to select $\mathcal{T}_i$, the task with a distribution closest to $R^{(0)}$. '...detriment...' implies that choosing a suboptimal model depth can lead to negative model transfer between tasks, adversely affecting the fair representation learning process.

[1] https://proceedings.mlr.press/v37/ganin15.html

[2] https://openaccess.thecvf.com/content_cvpr_2018/html/Pinheiro_Unsupervised_Domain_Adaptation_CVPR_2018_paper.html

Q3: It would be interesting to see how ....

A3: Theorem 1 indicates our algorithm's performance is similar to an optimal-depth adversarial network, which is infeasible in practice, with comparable cumulative loss bounded by $O(\frac{\ln(L)}{1-\beta})$. FaDL serves as the baseline and our results show that FaDL fails to generalize fair representations across different tasks, underperforming our algorithm by 7.6% in accuracy on average.

Q4: "This strategy ..." is quite hard to understand.

A4: Divergent demographic distributions means that P (Y | P) varies across different tasks at various extends. We display this for the Adult dataset only in the table below due to the character limitations. "Incrementing fair representation" extracts latent features for learned data $R^{(i-1)}$ and new task $\mathcal{T}_i$. This process is efficient due to a similar demographic distribution between $R^{(i-1)}$ and $\mathcal{T}_i$, achieved by starting with similar tasks and leaving distinct tasks for later. Gradually, $R^{(i-1)}$ incorporates various aspects of prior tasks, aligning with diverse task distributions. The feature extractor's parameters are reused, enhancing the incremental nature of the representation process.

Distribution of P(Y=1|P=1) / P(Y=1|P=0) across all tasks in Adult.

Q5: The paper provides results on tabular datasets only.

A5: Please find the answer in A2 of Reviewer 8gZZ due to character limitations.

Q6: The claim of being unsupervised..., On a similar note, some baselines may do not rely on task labels.

A6: We agree that coining our regime as purely “unsupervised” is not rigor. However, given its reliance on labeling a single task in a multi task context, it is less burdensome than semi-supervised approaches requiring labels for all tasks, distinguishing our approach from those like FaDL and FaIRL, which necessitate full label information.

Q7: When exactly are the predictions ...? Is this consistent ...? How Q measures ...?

A7: In each round, $\mathcal{T}_i$ is chosen for prediction based on similarity to $R^{(i-1)}$. The process stops when adversarial training converges, marked by reaching a maximum number of epochs. The task with the lowest Q value is then chosen. All competing models follow the same setup. Tasks are split into training and testing subsets where testing subsets are used to calculate accuracy and statistical parity after learning on training subsets. A high Q suggests either uniform weight distribution across layers or significant weights in deeper layers, implying a deep network. Conversely, a low Q indicates that shallow layers are more important.

Q8: Where is the loss L_FaRULi defined?

A8: $\mathcal{L}_{FaRULi}$ is the cumulative loss of all layers over T rounds. Details can be found in Appendix 1.1.

Q9: "negative model use" need to be defined earlier on in the paper

A9: The term is first introduced on Page 2 (Introduction) with citing papers. It's akin to "negative transfer" [3], which means a large difference between learned and new tasks makes reusing the trained model be less effective than starting with a new model for the new task.

[3]https://openaccess.thecvf.com/content_CVPR_2019/html/Wang_Characterizing_and_Avoiding_Negative_Transfer_CVPR_2019_paper.html

Final, we appreciate your suggestions and will refine our language and correct typos for better clarity.

Dear Reviewer,

We wanted to check if you had the opportunity to review our rebuttal and the revised manuscript (rebuttal revision) for your reference. We are committed to addressing any concerns or questions you may have about our work. If there are any aspects of the rebuttal or the manuscript that require further clarification, or if you have any new questions, please do not hesitate to let us know. We are eager to provide any additional information that might help in your evaluation of our paper.

Thank you once again for your valuable insights and we look forward to your response.

Best, Authors

Thank you for your constructive comments. Below, we address your comments one by one in a Q&A fashion.

Q1: Are the statistics calculated from Table 1 correct? I fail to map why it is the case that "On average, our model surpasses three leading competitors by 12.7% in prediction accuracy and by a substantial 42.8% in terms of ensuring statistical parity"?

A1: We respectfully point out that only FaDL and FaIRL are our rival models, representing the state of the art in the FRL context. Compared to FaDL and FaIRL, our proposed approach demonstrated significant performance improvement in terms of prediction accuracy by 20.4% and statistical parity by 44.4%. The other methods including UnFaIRL, FaMTL, and ULLC are not direct competitors. Specifically, UnFaIRL without optimizing the learning order is the variant of our proposed approach, crafted for ablation study. FaMTL and ULLC represent the performance upper bound in our context. Namely, FaMTL learns all tasks with all labels at once, where the fairness constraint is enforced in such offline multi-task learning setting, thus it always presents the best accuracy performance. While ULLC can learn tasks in sequence with only one labeled task (like our algorithm does), it does not impose fairness constraints. We thus can observe that ULLC slightly outperforms our algorithm in accuracy, but is inferior to ours in the fairness metric by substantial margins.

Q2: Can the proposed algorithm be generalized to high-dimensional data like images in addition to the studied tabular data?

A2: Yes, we followed [1] to construct an MNIST image dataset, with the protected feature being the background color of each digit type. Specifically, MNIST is divided into five tasks, each containing ten digits from 0 to 9. These tasks differ in two aspects. First, the class distribution P(Y) varies across tasks, with the largest number of images representing a certain digit type differing from one task to the others. Second, we designated red and green colors as the protected features, and the conditional P(Y|P) also varies. For instance, Task 1 may have more digit "1" in red and digit "9" in green, while the other tasks may exhibit reversed patterns. Here, we intentionally designated Task 3 ($\mathcal{T}_3$) as the most distant task, in which its conditional P(Y|P) differs significantly from all the other four tasks. Intuitively, a fair classifier should not establish a superficial correlation between color and digit, thus accurately predicting an input image regardless of its background color. We compared our FaRULi model with FaDL, and the results are presented in the table below. $\mathcal{T}_1$ is selected as the initial task with full labels. From the results, we observed that our model outperforms FaDL by 3.4% and 0.148 in Accuracy and Equalized odds (EO) on average, respectively. Moreover, FaDL performs better in the first two tasks $\mathcal{T}_1$ and $\mathcal{T}_2$ (average accuracy = 92.5%, average EO = 0.242) but cannot generalize to other tasks. We extrapolate that the function to debias color from digit type learned in $\mathcal{T}_1$ and $\mathcal{T}_2$ by FaDL cannot be adapted to other tasks, even if $\mathcal{T}_3$ has slightly different P(Y|P). In contrast, our model maintains good performance on $\mathcal{T}_1$, $\mathcal{T}_2$, $\mathcal{T}_4$, and $\mathcal{T}_5$, all of which have different conditional P(Y|P). On the most distant $\mathcal{T}_3$, our model achieves comparable accuracy performance as FaDL. This aligns with our assumption that the model cannot generalize to a highly dissimilar task, and our model resolves the negative transfer issue by prioritizing $\mathcal{T}_2$, $\mathcal{T}_4$, and $\mathcal{T}_5$ and achieving significantly better EO performance on them compared to FaDL.

[1] Bahng, Hyojin, Sanghyuk Chun, Sangdoo Yun, Jaegul Choo, and Seong Joon Oh. "Learning de-biased representations with biased representations." In International Conference on Machine Learning, pp. 528-539. PMLR, 2020.

Dear Reviewer,

We wanted to check if you had the opportunity to review our rebuttal and the revised manuscript (rebuttal revision) for your reference. We are committed to addressing any concerns or questions you may have about our work. If there are any aspects of the rebuttal or the manuscript that require further clarification, or if you have any new questions, please do not hesitate to let us know. We are eager to provide any additional information that might help in your evaluation of our paper.

Thank you once again for your valuable insights and we look forward to your response.

Best, Authors

Thank you for your constructive comments. Below, we address your comments one by one in a Q&A fashion. Due to characters limitations, we are only reporting parts of the experimental results in your questions.

Q1: Can the fairness constraint ..., such as equalized odds (EO)?

A1: Yes, our proposed framework is general, allowing for various fairness metrics. Based on your suggestion, we've implemented EO, presenting parts of results as follows:

We can observe that the empirical superiority of our proposed FaRULi algorithm is invariant to the fairness metric, and our algorithm outperforms its two competitors FaDL and FaIRL in both datasets with 0.031 and 0.087, respectively.

Q2: Reporting P(Y|P). Please clarify what "more divergent ..." and "incrementing ..." mean?

A2: Due to character limitations, please find our answers in A4 of Reviewer kvtG.

Q3: Types of bias in ML models,

A3: Yes, we're aware of the survey and have cited it. Our focus is on "fair representation learning (FRL)," not machine learning bias. Existing FRL studies, like [1], mainly address deriving fair data representations from unfair data. We'll emphasize our sub-domain and differentiate it from other biases in camera-ready.

[1] https://dl.acm.org/doi/abs/10.1145/3278721.3278779

Q4: Supervised with or w/o fairness-constrained baselines.

A4: Results of Adult are shown as below.

Q5: Fairness regularization constants

A5: Yes, our experiments on the Law School dataset show that increasing $\lambda_1$ decreased accuracy from 91.6% to 4.7%, demonstrating that strict fairness constraints can lower accuracy and affect fairness. Both statistical parity (SP) and EO values fall to zero, with the learner classifying all instances as Y=0 to avoid unfair outcomes.

Q6: The dynamics of ... be conjectures.

A6: We respectfully disagree, referencing studies in traditional BP [2-3] and adversarial learning [4]. These show that while deeper neural networks offer more representation capacity, they're also more sensitive to initializations, potentially losing correlation with shallow features. Our backbone architecture is built upon [4], and our EFRL network design can enjoy the same merit as the elastic network in [4] does. We will further add experimental results as [3] did to support the intuition behind our design.

[2] https://arxiv.org/abs/1605.07648

[3] https://link.springer.com/chapter/10.1007/978-3-319-46493-0_39

[4] https://dl.acm.org/doi/abs/10.1145/3442381.3449839

Q7: Please clarify the correctness ...

A7: Due to the character limitations, please find our answers in A9 of Reviewer kvtG.

Q8: Could the authors either compare ..., or provide some ...here?

A8: We argue that traditional distance metrics are unsuitable due to two reasons: they struggle with high-dimensional data [5] and require predefined thresholds to distinguish "close" from "faraway" tasks, which aren't feasible in lifelong learning. Our experiments on high-dimensional image datasets are detailed in A2 of Reviewer 8gZZ.

[5] https://kops.uni-konstanz.de/entities/publication/7cac4320-01b6-4403-b38a-ba87d550bc0a

Q9: "..., a comparable performance ..." which seems like...

A9: We'll revise our statement: If a shared subspace between $\mathcal{T}_i$ and $R^{(0)}$ is found, generalization from fully labeled $R^{(0)}$ to unlabeled $\mathcal{T}_i$ is possible, proved by many UDA studies [6-7]. However, such $\mathcal{T}_i$ may not always be feasible due to significant task differences or starting with an outlier task. Therefore, we will adjust our wording to "comparable performance on $\mathcal{T}_i$ is more likely" to cover these possibilities.

[6] https://proceedings.mlr.press/v37/ganin15.html

[7] https://openaccess.thecvf.com/content_cvpr_2018/html/Pinheiro_Unsupervised_Domain_Adaptation_CVPR_2018_paper.html

Q10: how the replayed pseudo-labels ensure fairness?

A10: We apologize for any confusion. Here, "fairness" means setting up a fair experimental benchmark, not fairness in machine learning. It's about equal conditions for all compared methods, which we'll clarify in camera-ready.

Final, we will streamline our language for ease of understanding.