Conditional Support Alignment for Domain Adaptation with Label Shift

We greatly appreciate the reviewer's positive evaluation and constructive comments and suggestions. In the following, we provide the response to your main comments.

Explanation for mitigating the accumulative error by using pseudo-labels from CDAN

Theorem 1 motivates us to minimize the conditional support divergence between the source and target domain, in order to minimize the target error risk. However, as mentioned in our main paper, due to the absence of labels in the target domain, the proposed CASA method resorts to using pseudo-labels on the target domain. Based on several works that have pointed out the error accumulation issue in naively adopting pseudo-labels under data distribution shift [], we utilize the entropy conditioning technique proposed in [1] to mitigate this issue. Intuitively, we reweight the importance of each sample in Eq. 9 and Eq. 10 by the quality of their pseudo-labels, which is measured by pseudo-labels' entropy $w_{i} = 1 + e^{-H(g(x_{i}))}$ [1]. Since the prediction's entropy is closely related to model's error under domain shift [2], this reweighting scheme allows the model to prioritize easily transferable samples, and lessen the issue of error accumulation under domain shift. On the other hand, more advanced techniques on generating high-quality pseudo-labels [3][4][5] can be utilized in our model, and potentially help achieve even higher results. Investigating closely the effects of different pseudo-labels methods is outside the scope of our paper, and we leave this exploration to future works.

$$L_{d}(\phi) = - \frac{1}{n_S}\sum_{i=1}^{n_{S}} w_{i}^S\ln \left[G(x_i^S)\right]-\frac{1}{n_T}\sum_{i=1}^{n_{T}} w_{i}^T \ln \left[1-G(x_i^T)\right]$$ $$L_{align}(f) = \frac{1}{n_S}\sum_{i=1}^{n_{S}} w_{i}^S d\left(G(x_i^S),\{G(x_j^T)\}_{j=1}^{n_T}\right)$$ $$+\frac{1}{n_T}\sum_{i=1}^{n_{T}} w_{i}^T d\left(G(x_i^T),\{G(x_j^S)\}_{j=1}^{n_S}\right)$$

Comparison with more SOTA methods: SHOT, BIWAA, and CoVi

Please refer to the additional experimental results mentioned in our global response. As expected, those new results show that our proposed CASA consistently outperforms others by a large margin.

[1] Long et al. Conditional adversarial domain adaptation. Advances in neural information processing systems, 31.

[2] Dequan Wang et al. “Tent: Fully test-time adaptation by entropy minimization”. In: arXiv preprint arXiv:2006.10726

[3] Jian Liang, Dapeng Hu, and Jiashi Feng. “Do we really need to ac- cess the source data? source hypothesis transfer for unsupervised domain adaptation”. In: International Conference on Machine Learning. PMLR. 2020, pp. 6028–6039.

[4] Hong Liu, Jianmin Wang, and Mingsheng Long. “Cycle self-training for domain adaptation”. In: Advances in Neural Information Processing Systems 34 (2021), pp. 22968–22981.

[5] Qiming Zhang et al. “Category anchor-guided unsupervised domain adaptation for semantic segmentation”. In: Advances in Neural Information Processing Systems 32 (2019)

Dear reviewer,

Thank you once again for taking the time to read and review our submission. We would be happy to address any remaining questions before the discussion period ends today.

Best regards,

Authors.

We greatly appreciate the reviewer’s valuable comments and constructive suggestions. We address the main concerns as follows.

Comparison to other relevant theoretical works

We acknowledge the reviewer's concern over the lack of comparison with existing theoretical results on the generalized label shift problem. Our paper previously only focused on comparing the derived theoretical results with the existing literature on adversarial domain adaptation [3] [4], and adversarial support alignment [1] in section 2.3. To provide a more comprehensive treatment of existing approaches to domain adaptation with generalized target shift, we discuss the result of Theorem 1 and other relevant theoretical results in Section B of the updated Appendix. Thus, we refer the reviewer to this section in the revised paper for more detailed discussion.

Tractability of the induced constants in Theorem 1

We agree with the reviewer that those constants might be computationally intractable. Nevertheless, in the theoretical development of our method, we assume the existence of those constants to obtain the exact theoretical upper bound of the target error. In fact, those induced constants remain unchanged in the training process of our CASA, whose main goal is to reduce the divergence term $D_{supp}^c(P^S_{Z|Y},P^T_{Z|Y})$. In particular, on the additional induced quantity $\delta$, the appearance of additional term is due to the mild assumption ${f \in F}_{\epsilon}$ where $\epsilon$ be greater than 0. In fact, if we impose a more stringent assumption by setting $\epsilon$ to be precisely $0$ (indicating models making perfect predictions on labeled source data), these quantities will vanish. Please refer our updated Appendix for further details

Is conditional SSD in Def.4 a metric on conditional distribution?

The conditional symmetric support divergence (CSSD) $D_{supp}^c(P^S_{Z|Y},P^T_{Z|Y})$ is not a proper metric on conditional distribution; instead it serves a support divergence on conditional distribution. This is similar to [1] where their proposed symmetric support divergence is similarly categorized as a support divergence. The definition of support divergence is closely related to Chamfer divergence [1][2], which has been shown to not be a valid metric, since it does not satisfy the triangle inequality. In response to reviewer's suggestion, we have added Proposition 1 and the additional analysis to this on in the updated Appendix. Our approach distinguishes itself by targeting the alignment of supports in class-wise conditional distributions rather than those in margin distributions. Intuitively, two distributions $P_{Z|Y}, Q_{Z|Y}$ with the same label space can have $D^c_{supp}(P_{Z|Y}, Q_{Z|Y}) = 0$, if and only if their conditional supports are equal, i.e. $supp(P_{Z|Y=y}) = supp(Q_{Z|Y=y}) \forall y$.

We sincerely hope that you can reconsider the review score. Please let us know if there are further things you would like us to address.

Best regards,

Authors

[1] Shangyuan Tong et al. “Adversarial Support Alignment”. In: arXiv preprint arXiv:2203.08908 (2022)

[2] Haoqiang Fan et al. A point set generation network for 3d object reconstruction from a single image. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 605–613, 2017

[3] Shai Ben-David et al. “Analysis of representations for domain adaptation”. In: Advances in neural information processing systems 19 (2006)

[4] David Acuna et al. “f-domain adversarial learning: Theory and algorithms”. In: International Conference on Machine Learning. PMLR. 2021, pp. 66–75.

We greatly appreciate the reviewer’s valuable comments and constructive suggestions. We address the main concerns as follows.

Novelty clarification

We would like to emphasize that our main contribution includes a novel conditional SSD-based domain adaptation bound in section 2.3. The main motivation for our proposed method is to utilize discriminative features during adversarial domain alignment to improve performance under different levels of label distribution shift. Based on the novel target error bound in section 2.3, we then propose the corresponding training scheme that empirically yields superior results on benchmark datasets under the setting of label distribution shift. We provide a detailed comparison of our proposed error bound to other relevant theoretical results on the problem of unsupervised domain adaptation, in Remark 3, and in our response to Reviewer UE75. The corresponding loss terms in Eq. 7, 8, and 10 are directly motivated by our proposed bound in Theorem 1. Furthermore, we provide an analysis of these three loss terms in section 3.3 and confirm the merits of optimizing each of these loss terms in our ablation study.

Similarity between CSSD and pair-wise graph alignment

We thank the reviewer for this suggestion and would appreciate it if the reviewer could point out which specific work on cross-domain graph alignment he/she is referring to. However, as pointed out in [1], the support alignment loss is closely related to the Chamfer divergence, which also has been used extensively in 3D point cloud modeling [4, 5] and learning document embedding [6]. Different from these works, ours is the first work that proposes using the conditional support divergence within the problem domain of domain adaptation with generalized target shift. Furthermore, we have provided a comprehensive discussion of our novel alignment loss in section 2.3, and the novel optimization scheme of this loss term in section 2.4.3 in our main paper.

More performance analysis

We thank the reviewer for the suggestion of clarifying the competitive performance of CASA over severe label shift. However, we would like to point out that, in fact, CASA's performance does not improve as alpha decreases, as Reviewer XqJP has claimed. In particular, Tables 1-3 show that more severe levels of label shift also degrade CASA's performance mildly on all of the benchmark datasets. More importantly, Tables 1-3 show that CASA performs competitively with other baselines under alpha={None, 10.0}, and CASA attains the highest results under other smaller values of alphas. There are several reasons for this phenomenon. Since the objective of minimizing CSSD does not significantly degrade the model's performance under large label shifts, as we already discussed in Remark 3 and in our response to Reviewer UE75, CASA is more robust to large label distribution shifts. On the other hand, as support divergence is an extremely relaxed form of distribution divergence [1], aligning the supports of source and target features may not perform as well as aligning these distributions directly when there is no or mild label shift. We have visualized the feature distributions and studied the performance of CASA, in comparison to 2 other baselines CDAN and ASA in Figure 3 and Section 3.3. Compared to CDAN, the Wasserstein distance between source and target features of CASA is much larger, at a value of 0.65. This helps explain the superior performance of CASA over CDAN and other similar distribution-alignment methods under severe label shift of small alpha values [2,3].

We really hope that you can reconsider the review score. Please let us know if you would like us to do anything else.

Best regards,

Authors

[1] Shangyuan Tong et al. “Adversarial Support Alignment”. In: arXiv preprint arXiv:2203.08908 (2022)

[2] Zhao et al. On learning invariant representations for domain adaptation. In International Conference on Machine Learning, pp.7523–7532. PMLR, 2019

[3] Johansson et al. Support and invertibility in domain-invariant representations. In The 22nd International Conference on Artificial Intelligence and Statistics, pp. 527–536. PMLR, 2019.

[4] Haoqiang Fan et al. A point set generation network for 3d object reconstruction from a single image. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 605–613, 2017

[5] Trung Nguyen et al. “Point-set distances for learning representations of 3d point clouds”. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021, pp. 10478–10487

[6] Matt Kusner et al. From word embeddings to document distances. In International conference on machine learning, pp. 957–966. PMLR, 2015

Dear reviewer,

Thank you once again for taking the time to read and review our submission. We would be happy to address any remaining questions before the discussion period ends today.

Best regards,

Authors.