Tackling Dimensional Collapse toward Comprehensive Universal Domain Adaptation

We thank the reviewer for the constructive feedback and are glad to hear that our research problem resonates with real-world scenarios and that the observed dimensional collapse is found to be intriguing. The reviewer’s main concerns lie in the novelty of using SSL and the lack of experiments on partial domain adaptation (PDA) and open set domain adaptation (OSDA). We address these concerns below.

Novelty of using SSL to address the dimensional collapse issue.

There are concerns regarding the novelty of using SSL to address the dimensional collapse issue. The authors should further clarify this approach and compare it with previous SSL methods used in domain adaptation.

We refer the reviewer to the "Missing References" section of Reviewer tc8f, where we discuss the distinction between our work and the literature on self-supervised learning for domain adaptation.

Why SSL mitigate DC?

The claims and evidence in general are clear and convincing. But why SSL works in addressing dimension collapse may need more explanation.

Existing insights from the SSL literature offer useful intuitions. Contrastive learning frameworks (e.g., AlignUniform, SimCLR) demonstrate that negative samples and the uniformity loss serve as repulsive forces to the alignment loss, helping to prevent dimensional collapse. On the other hand, non-constrastive methods like SimSiam and Barlow Twins show that using asymmetric architectures and encouraging feature decorrelation can promote diverse representations across dimensions, thereby mitigating collapse. We will incorporate these insights into the latest version of our paper.

Experiments on PDA and OSDA

Thank you for the suggestions! We have now included the results for PDA and OSDA below.

• PDA (21/10/0, Office31)

*: from "LEAD: Learning Decomposition for Source-free Universal Domain Adaptation" CVPR'24

These results indicate that UAN+SSL significantly outperforms previous methods in PDA settings. Interestingly enough, the scenario corresponds to $\pi_s=0.68$, which is close to the extreme UniDA setting, and all prior domain-matching methods underperform the SO approach, reinforcing our observation.

• OSDA (0/10/11, Office31)

*: from "Subsidiary Prototype Alignment for Universal Domain Adaptation" NeurIPS'22

The result suggest that incorporating SSL does not significantly affect performance. In the setting where $\pi_s=0$, the critical challenge lies in identifying target-private classes rather than domain matching, which may explain why SSL does not substantially influence the performance.

Ablaion study is only conducted on a subtask of the small Office31 dataset

We have included a more comprehensive ablation study on the entire Office-31 dataset, which shows that alignment alone provides limited benefit, uniformity leads to a slight performance improvement, and combining both yields the best results.

Thank you for the constructive feedback. We address the raised concerns below.

Novelty and contributions

While SSL has been explored in various DA contexts, we respectfully argue that our contribution lies beyond using SSL for UniDA.

Prior SSL for UniDA works typically rely on pretext tasks (e.g., rotation, jigsaw), which are ineffective in mitigating DC and fail under extreme UniDA settings, as shown in our comparison table below (Comparison with other SSL pretext tasks). In contrast, we use both contrastive learning and non-contrastive learning methods, which are explicitly designed to preserve representation diversity and combat dimensional collapse (DC) [1]. To the best of our knowledge, we are the first to apply these techniques in UniDA specifically to address DC, making our use of SSL novel in both motivation and application.

We also highlight that identifying the previously overlooked challenge of extreme UniDA is itself a key contribution, as it draws attention to the limitations of current benchmarks and provides a foundation for systematically addressing the problem through improving representation quality.

Ablations on ViT backbone

We initially excluded this analysis because it is not commonly included in UniDA literature, even in recent works (e.g., LEAD, MLNet), except for [2], which specifically investigates ViT architectures. The following tables present the performance of ViT on both extreme UniDA and general UniDA settings for DomainNet. The results follow a similar trend to those observed with CNNs: improvements are more pronounced in the Extreme UniDA setting compared to the standard UniDA case.

• Extreme UniDA

  	P2S	P2R	S2P	S2R	R2P	R2S	Avg
  UAN	44.69	59.18	40.96	57.44	39.70	36.07	46.34
  UAN + SSL	51.57	61.12	48.24	58.22	42.95	37.49	49.93 (+3.59)

• General UniDA

  	P2S	P2R	S2P	S2R	R2P	R2S	Avg
  UAN	44.38	62.58	34.11	53.31	51.34	41.62	47.89
  UAN + SSL	46.25	62.81	38.74	54.76	53.26	42.11	49.65 (+1.76)

Lack of theoretical insights in support of the proposed method

We agree that understanding why SSL mitigates DC in extreme UniDA is a valuable theoretical pursuit. While seminal SSL methods (e.g., SimCLR, SimSiam, Barlow Twins) were initially driven by intuition and empirical success, later works sought theoretical explanations [3, 4]. We believe that our philosophical insights and empirical results lay a strong foundation for future theoretical study.

Comparison with more recent methods

Thank you for highlighting these baselines. We were aware of them but initially excluded them due to differing settings. LEAD uses a source-free setup without target data during training; we now include its results on DomainNet, where it underperforms compared to several baselines under the extreme UniDA setting. [2] uses a ViT backbone, making it incompatible with our CNN-based setup. It was also excluded in the MLNet (AAAI 2024) paper, and its code is unavailable, preventing reproduction within our limited timeframe.

Comparison with other SSL pretext tasks

We compare three different approaches to SSL in DA:

1. pretext tasks: Minimize domain gap via auxiliary tasks like jigsaw puzzles or rotation prediction (Bucci et al., Xu et al.).
2. prototype alignment: Align target samples to source/target prototypes using entropy minimization (DANCE).
3. Contrastive & non-contrastive: The focus of our paper—these methods promote representation diversity to avoid collapse, using contrastive losses (AlignUniform) or asymmetric architectures without negatives (SimSiam).

This suggests that contrastive and non-contrastive methods, which explicitly tackle DC, are effective in extreme UniDA—unlike previously explored SSL approaches, which struggle in this setting.

Results on PDA and OSDA

We refer the reviewer to the "Results on PDA and OSDA" section of Reviewer sBWy.

[1] “Rethinking The Uniformity Metric in Self-Supervised Learning”, ICLR 2024

[2] “Universal Domain Adaptation via Compressive Attention Matching”, ICCV 2023

[3] “Understanding Dimensional Collapse in Contrastive Self-supervised Learning”, ICLR 2022

[4] “How Does SimSiam Avoid Collapse Without Negative Samples? A Unified Understanding with Self-supervised Contrastive Learning”, ICLR 2022

We would like to thank the reviewer for the constructive feedback. The primary concern raised is whether the utilization of unlabeled target data during training is still consistent with the Extreme UniDA setting. We address this concern below:

Concern about the utilization of target data, which makes the setting no longer extreme UniDA.

We appreciate the reviewer’s perspective and acknowledge that different lines of work may interpret or label these setups differently. However, our approach remains squarely within the unsupervised domain adaptation (UDA) paradigm—specifically, universal domain adaptation (UniDA). In this framework, both label sets (label shift) and underlying data distributions (covariate shift) can differ between source and target domains—aligning with the first category the reviewer mentioned—and having access to unlabeled target data during training is a standard assumption.

We will clarify in the revised manuscript that having unlabeled target data during training is not an additional assumption outside of domain adaptation; rather, it is intrinsic to UDA and UniDA. Removing all target data during training would place the problem under settings like domain generalization [1], zero-shot transfer [2], or test-time adaptation [3], which focus on generalizing without any prior exposure to the target domain. In contrast, both UDA and UniDA explicitly require access to unlabeled target data, and every baseline PDM method we compare (e.g., UAN, CMU, UniOT) operates under this same premise.

Therefore, while we recognize that terminological nuances may vary, our work consistently follows the established UniDA framework. We do not shift toward a purely supervised or semi-supervised learning problem on the target domain; instead, we remain entirely within the standard assumptions of UniDA by incorporating unlabeled target data during training.

[1] Li et al., “Learning to generalize: Meta-learning for domain generalization”, AAAI 2018

[2] Radford et al., “Learning Transferable Visual Models From Natural Language Supervision”, ICML 2021

[3] Wang et al., “Tent: Fully test-time adaptation by entropy minimization”, ICLR 2021

Interchangeable usage of UniDA and UDA

Thank you for your helpful suggestion. We agree that the interchangeable use of UniDA and UDA may cause confusion. While our primary focus is on Universal Domain Adaptation (UniDA), we include key references from the Unsupervised Domain Adaptation (UDA) literature to provide necessary context. In the revision, we will clarify this distinction and streamline the discussion of UDA works to maintain focus.

We thank the reviewer for the constructive feedback and are glad our analysis strongly motivates the proposed approach. Below, we address the raised concerns.

Missing references

We appreciate the reviewer’s feedback on our discussion of SSL for DA. While prior work primarily employs SSL to minimize domain gap—either through pretext tasks or by aligning low-uncertainty samples with class prototypes [2, 3, 4]—our focus is on preventing dimensional collapse (DC) in extreme UniDA. This different goal leads us to adopt SSL paradigms specifically designed to counteract collapse by enhancing feature diversity. Our methods include both contrastive (AlignUniform) and non-contrastive (SimSiam, Barlow Twins) approaches that promote uniformity and decorrelation to mitigate DC [5], rather than simply using pretext tasks for domain matching. We also present performance comparisons on extreme UniDA in the table below (see "Comparison with other SSL pretext tasks" of reviewer yX6d), showing that pretext-based and prototype-alignment methods fall short of the performance achieved by approaches specifically targeting DC.

The above content will be included in Section 4 for a more comprehensive comparison in our latest version. We now discuss how our approach differs from the specific works [1, 3, 4].

Kundu et al. [1] propose "sticker-intervention," a pretext task that improves domain preservation over traditional SSL tasks. We will include it in the related work.

Zhu et al. [3] propose a framework that is highly similar to DANCE [2], comprising prototype alignment (referred to as adaptive memory in [3]) and an entropy separation module. As [3]'s code is unavailable, we use DANCE as a proxy given their similar design. DANCE struggles in extreme UniDA (see Table 3–5), making it a reasonable reference point. We will include a discussion of [3] in the related work section.

SPA [4] introduces an add-on module that performs adaptation at the mid-level layers, as these layers are shown to exhibit lower negative transfer. Their method leverages a Bag-of-Words-inspired pretext task to learn distinct visual word prototypes and promotes prototype alignment via entropy minimization. Their work mainly improves domain matching, which is orthogonal to our work that tackle DC. As their module is designed to be an add-on, it can serve as a complementary component to our method. Since the code is not publicly available, we currently include their method only in the related work discussion and will add it as a baseline once the code is released.

[1] “Concurrent Subsidiary Supervision for Unsupervised Source-Free Domain Adaptation”, ECCV 2022

[2] “Universal Domain Adaptation through Self-Supervision”, NeurIPS 2022

[3] “Self-supervised Universal Domain Adaptation with Adaptive Memory Separation”, ICDM 2021

[4] “Subsidiary Prototype Alignment for Universal Domain Adaptation”, NeurIPS 2022

[5] “Rethinking The Uniformity Metric in Self-Supervised Learning”, ICLR 2024

Report common-class accuracy and target-private accuracy separtely in Figure 2.

Thank you for the suggestion! Due to character limitations, we were unable to include the full tables here, but we summarize the observed trends below:

• H-score: The H-score decreases as $\pi_s$ increases, as shown in Figure 2. Additionally, UniOT and UAN perform worse than SO under high $\pi_s$.
• Common-class accuracy: This also decreases with increasing $\pi_s$. UniOT performs slightly worse than SO at high $\pi_s$, while UAN shows a clearly lower performance than both.
• Target-private accuracy: Accuracy on target-private classes declines as $\pi_s$ increases for all methods. Both UniOT and UAN yield slightly lower scores than SO.

While the analysis in Figure 4 is interesting and informative, it would be more complete with the inclusion of more recent methods.

We agree that more recent methods should be included. Since LEAD is a source-free method and does not use a domain-matching loss, we have instead included MLNet (AAAI'24), which employs mutual nearest neighbors (MNN) for domain matching. This approach is conceptually similar to our distance-based baseline, but its mutual filtering enforces a stricter alignment.

• Avg. error rate of importance weight function during training

  	UAN	CMU	Energy	Distance	MNN
  $\pi_s=0.25$	0.29	0.30	0.33	0.26	0.24
  $\pi_s=0.75$	0.59	0.53	0.51	0.49	0.41

  The MNN baseline shows a significant improvement. However, it is still far from the threshold (0.15–0.2), as shown in Figure 4 (a), required to outperform the Source-Only baseline.

Is the self-supervised loss from Sec. 4.2 the same as the uniformity loss from Sec. 4.1?

The self-supervised loss refers to alignment + uniformity loss, where alignment encourages class-wise aggregation and uniformity prevents DC. We’ll clarify this in the revision.