Pick and Adapt: An Iterative Approach for Source-Free Domain Adaptation

Q1: About the assumption of treating pseudo-labels directly as ground truth labels and i.i.d sampling from target domain.

Response: We respectfully disagree "treating pseudo-labels directly as ground truth labels does not fit SFDA tasks". "Treating pseudo-labels directly as ground truth labels" has been widely applied in source-free domain adaptation, early work such as the Source Hypothesis Transfer (SHOT) [1] utilizes a clustering-based approach to improve the quality of pseudo-labels for the unlabeled target dataset. Subsequently, it places trust in these pseudo-labels and applies cross-entropy loss to optimize the model. AdaContrast [2] also leverages weakly augmented unlabeled target data to obtain pseudo-labels. These pseudo-labels are then trusted as ground truth labels and combined with strongly augmented data for cross-entropy loss during the model update process. SFDA-DE [3] similarly leverages a source domain pre-trained model to assign pseudo-labels to unlabeled target domain data, and trust their pseudo-labels to optimize model during the training process. GPUE [4] employs a clustering-based pseudo-label refinement process to obtain higher-quality pseudo-labels. It then places trust in these pseudo-labels and utilizes cross-entropy loss to update the model.

Regarding the assumption that both $D_{tl}$ and $D_{tu}$ are i.i.d sampled from the target domain, our approach aligns with similar insight found in many data selection based source-free domain adaptation papers. For instance, SHOT++ [5] divides the target dataset into two subsets—namely, the low-entropy subset and high-entropy subset—based on the entropy values of the model's predictions. These subsets can be treated as two distinct distributions due to their differing distributions alone with the entropy values. Proxy-based Mixup (ProxyMix) [6] also adopts a strategy of splitting the unlabeled target dataset. It achieves this by selecting the $N$ nearest samples in the target dataset for each source class prototype (derived from the source pre-trained classifier weights) as the labeled subset, while the remaining samples constitute the unlabeled subset. These two subsets are considered from both the target domain and the source proxy domain. Therefore, it's reasonable to assume $D_{tl}$ and $D_{tu}$ are i.i.d sampled from the target domain.

[1] Jian Liang et al. (2020). “Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation.” In ICML.
[2] Dian Chen et al. (2022). “Contrastive test-time adaptation.” In CVPR.
[3] Ning Ding et al. (2022). “Source-free domain adaptation via distribution estimation.” In CVPR.
[4] Mattia Litrico et al. (2023). "Guiding pseudo-labels with uncertainty estimation for test-time adaptation." In CVPR.
[5] Jian Liang et al. (2021). “Source data-absent unsupervised domain adaptation through hypothesis transfer and labeling transfer.” In IEEE TPAMI.
[6] Yuhe Ding et al. (2023). “Proxymix: Proxy-based mixup training with label refinery for source-free domain adaptation.” In NN.

Q2: Ablation study for C-sampling and T-sampling.

Response: Thanks for the valuable suggestion. We adopt the adaptation tasks A$\rightarrow$C and Re$\rightarrow$Cl from OfficeHome and DomainNet, respectively. As depicted in Table 1, it is observed that the I-sampling strategy outperforms both C-sampling and T-sampling across various adaptation tasks. More detials, please refer to Appendix A.8 in our revised version.

Table 1: ablation study for C-samping and T-sampling.

Q4: Sensitivity of trad off $\rho$ in Eq.13..

Response: Thank you for the valuable suggestion. We investigated the performance of our method across various values of $\rho$ ranging from $0.1$ to $1.0$ on the adaptation task Re$\rightarrow$Cl from DomainNet. As indicated in Table 7, our method demonstrates lower sensitivity within the $\rho$ range of 0.3 to 1.0. More detials, please refer to Appendix A.9 in our revised version.

Table 7: sensitivity study for $\rho$ in Eq. 13 with adaptation task Re$\rightarrow$Cl on DomainNet dataset.

Q5: The connection between the proposed risk bound and the modules in our method.

Response: Our proposed method closely aligns with our theoretical analysis. In particular, the generalization bound (Equation 3 in revised version) primarily comprises three components: population risk, domain divergence between labeled and unlabeled target data, and the size of the target labeled subset. Drawing inspiration from these three components, our proposed method incorporates three key elements.

Firstly, we propose an innovative target data picking strategy aimed at selecting high-quality target data with pseudo-labels, effectively mitigating the impact of noise associated with pseudo-labels on population risk. This strategy involves splitting the target data into labeled and unlabeled subsets, enhancing the robustness of the SFDA framework.

Secondly, we introduce a semantic domain alignment approach between labeled and unlabeled target data through nearest neighbors voting. Our novel nearest neighbors voting strategy refines the quality of predicted semantic information for unlabeled target data, incorporating a well-designed metric to indicate the certainty of the refined pseudo-labels. This alignment process focuses on associating unlabeled target data with high certainty to labeled target data, thereby reducing the domain divergence between the two.

Lastly, we integrate the target data picking and model training into a cohesive framework. This framework systematically selects unlabeled target data with high-quality pseudo-labels, progressively incorporating them into the labeled target data subset. This iterative process effectively increases the size of the target labeled subset, consequently minimizing the SFDA generalization bound.

Q6: If using the intersection between C-sampling and T-sampling, why it degrades to single sampling when the other set is empty?

Response: Thanks for pointing out this point. What we try to express is that our I-sampling is the intersection between C-sampling and T-sampling, which is influenced by both C-sampling and T-sampling. If one of them becomes empty, I-sampling would be the empty for the corresponding category. We have revised this point in the updated version.

Q7: In Figure 3 (b)(c), why accuracy of pseudo labels decrease from 90% during adaptation?

Response: Thank you for your valuable feedback. Figures 3(b) and (c) illustrate the accuracy of pseudo labels during the $r$-th round. Specifically, (b) represents the accuracy of pseudo-labeled target data selected at the $r$-th round, while (c) displays the accuracy of all pseudo-labeled target data at the $r$-th round. During the $r$-th round, the selected pseudo-labeled target data generally exhibit higher-quality pseudo labels compared to the unselected target data obtained through C/T/I-sampling. Consequently, the accuracy of the pseudo-labeled data chosen during the $r$-th round tends to be higher than that of the overall pseudo-labeled data in the subsequent $r+1$-th round. This is because, in the $r+1$-th round, the selected pseudo-labeled target data is likely to have lower-quality pseudo labels compared with these selected in $r$-th round, as it originates from the remaining unlabeled target data. As a result, the accuracy of pseudo-labels would gradually decrease with the selection of more pseudo-labeled data.

Q8: Typo error and the place of the Table 4.

Response: Thank you for the valuable suggestion. We address the typo error and ensure that Table 4 is placed on the main page in the revised version.

Q3: compare the picking and adaption strategies separately with other well-known picking and adaption strategies.

Response: Thank you for the valuable comment. We have integrated picking and adaptation strategies from two state-of-the-art methods, namely SHOT++ [1] and AaD [7]. We denote our methods as "our-picking" + "our-adaptation". Specifically, we utilize the picking strategy from SHOT++, referred to as "SHOT++-picking", and the adaptation strategy from AaD, denoted as "AaD-adaptation." We evaluate the performance of "SHOT++-picking" + "our-adaptation" and "our-picking" + "AaD-adaptation" on adaptation tasks, namely A$\rightarrow$C, A$\rightarrow$P, and A$\rightarrow$R, using the Office-Home dataset. More details, please refer to the Appendix A.7 in our revised version.

In Table 3 We observed that adopting the picking strategy from SHOT++, which uses entropy as the metric to select target data with entropy values larger than the average entropy values over the entire dataset as pseudo-labeled data, results in "SHOT++-picking" + "our-adaptation" significantly underperforming our results. This phenomenon can be attributed to two reasons. Firstly, "SHOT++-picking" heavily relies on the entropy value, which may struggle to distinguish between confident and extremely sharp predictions. Secondly, "SHOT++-picking" selects pseudo-labeled data only once, and the large number of selected data leads to a low quality of pseudo labels. Therefore, it demonstrates the advantage of our picking strategy compared to the picking strategy from SHOT++. Combining "our-picking" with "AaD-adaptation," we observe that it also underperforms compared to our approach. This is mainly due to "AaD-adaptation" blindly trusting the predicted semantic information of the neighbors, which can lead to negative clustering when these predictions are not very accurate. Additionally, the reliance on inaccurate local clusters can result in suboptimal discriminative representation learning. Therefore, it proves the advantage of our adaptation strategy over the adaptation strategy from AaD.

Q4: About learning bounds, data sampling, and curriculum learning.

Response: Thanks for the valuable comment. For learning bounds: while we acknowledge the resemblance between our theorem and its proof to those presented in [8], it's crucial to note that [8] specifically addresses the generalization bound for traditional domain adaptation, which may not be directly applicable to the context of source-free domain adaptation (SFDA). Drawing inspiration from [8], we extend our efforts to establish a novel generalization bound tailored for SFDA. Building upon this bound, we subsequently propose a solution to address the challenges posed by the SFDA problem. For data sampling and curriculum learning: while data sampling strategies and curriculum learning have been extensively studied, the joint integration of these approaches to effectively address the source-free domain adaptation (SFDA) problem remains an intriguing solution. This is particularly noteworthy, guided by our proposed SFDA generalization bound

[8] John Blitzer et al. (2007). “Learning bounds for domain adaptation.” In NeurIPS.

For neighbor aggregation, there are 3 related papers suggested by the reviewer, i.e., Neighborhood Reciprocity Clustering (NRC) [4], Source Hypothesis Transfer(SHOT) [5], Semantic Consistency on the Nearest Neighborhood (SCNNH) [6], and Attracting and Dispersing (AaD) [7]. NRC aims to align each target data with its neighbors using carefully designed affinity values. However, the semantic information of the neighbors may not always guarantee consistency. This misalignment can have adverse effects, particularly in challenging adaptation tasks. SHOT employs a clustering-based method to enhance the quality of pseudo labels for the target dataset. It utilizes all unlabeled target data with pseudo labels to estimate class centroids, subsequently leveraging these centroids to refine the pseudo labels. However, SHOT encounters difficulties in distinguishing between different classes when class boundaries are exceptionally vague, and the distributions of different classes closely overlap. Additionally, constructing centroids for each class using all target data poses challenges, as not all predicted semantic information is accurate, leading to potential adverse effects on centroid construction due to misclassified data. SCNNH shares a similar concept with SHOT, utilizing estimated cluster centroids to derive semantic information. The distinction lies in the approach to obtaining pseudo labels. While SHOT directly assigns pseudo labels using the estimated centroids, SCNNH combines semantic information from a data point's neighbors to determine pseudo labels. AaD shares a similar concept with NRC by clustering target data to its neighbors. The key distinction is that AaD not only clusters target data with its neighbors but also endeavors to push the target data away from samples that potentially belong to different classes.

Our neighbor aggregation strategy aims to enhance the quality of estimated semantic information for unlabeled target data, setting it apart from NRC and AaD, where neighbor aggregation primarily serves to achieve local clustering. In comparison to SHOT and SCNNH, which use estimated target class centroids for refinement, our approach diverges in three key aspects. Firstly, instead of refining pseudo labels through estimated centroids, we leverage the semantic information of neighbors. Secondly, to address the potential negative influence of neighbors with low-quality semantic information, we establish a reliable neighbor space by selecting the top $\rho\%$ of target data per class to form a trustworthy set. Each target data's neighbors are then chosen from this reliable set, mitigating the adverse effects that might arise from using all target data as the pool for neighbor selection, which is common in SHOT and SCNNH. Finally, to further mitigate the impact of noisy label information from the refined pseudo labels, we introduce a metric that gauges the certainty of the refined label information using entropy and probability. Table 2 summarizes the component-wise differences between our method and others.

Table2:A component-wise comparison of the neighbor strategy in our method and other related methods, i.e., NRC, SHOT, SCNNH, and AaD, reveals distinctive features and characteristics.

[4] Shiqi Yang et al. (2021). “Exploiting the intrinsic neighborhood structure for source-free domain adaptation.” In NeurIPS.
[5] Jian Liang et al. (2020). “Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation.” In ICML.
[6] Song Tang et al. (2021). “Nearest neighborhood-based deep clustering for source data-absent unsupervised domain adaptation.” In arXiv.
[7] Shiqi Yang et al. (2022). “Attracting and dispersing: A simple approach for source-free domain adaptation.” In NeurIPS.

Q1: Difference between exisiting methods and ours.

Response: We sincerely thank the reviewer for providing insights into related works from different perspectives, including data sampling and neighbor aggregation strategies. We would like to clarify the differences between these methods and ours as follows. More details, please refer to the section: Related Work in revised version.

For data sampling, there are 3 related papers suggested by the reviewer, i.e., Source Hypothesis Transfer (SHOT++) [1], Proxy-based Mixup (ProxyMix) [2], and Black-Box Model Adaptation by Domain Division (BETA) [3]. SHOT++ utilizes the entropy values of predictions to partition the unlabeled target dataset into two subsets, treating them as a labeled subset and an unlabeled subset. More precisely, it selects the top $t_k$ smallest entropy values for each class $k$. A significant challenge lies in the fact that the labeled subset constitutes half of the entire dataset. Additionally, it follows a two-step process, first splitting the data and then performing adaptation. ProxyMix relies on the weights of the source domain pretrained classifier as the source class prototypes. It then searches for the $N$ nearest samples in the target domain for each source class prototype (referred to as C sampling in our paper). Importantly, ProxyMix performs this selection only at the beginning and subsequently proceeds with the training. BETA categorizes the unlabeled target dataset into two subsets: an easy-to-adapt subset and a hard-to-adapt subset. This is achieved by calculating probabilities for each target data. The high probability target data are chosen as the easy-to-adapt subset, and their pseudo labels are set as the ground truth labels. The remaining data form the hard-to-adapt subset. Importantly, BETA conducts this selection only at the outset and then continues with the training process.

Our selection strategy differs significantly from the aforementioned methods in several key aspects. Firstly, our approach involves a continual selection strategy. In each round, we exclusively choose target data with high-quality pseudo labels. In contrast, the methods mentioned above perform a single-time selection. Secondly, our method relies on semantic information from both the pretrained model (C sampling) and the target domain (T sampling) data to pick the reliable samples, where the reference methods are only based on semantic information from the source domain pre-trained model. Lastly, our selection process is integrated with model training, enhancing our method's ability to pick target data with high-quality pseudo labels. Conversely, the reference methods follow a two-step framework, conducting data selection in the initial step and subsequently proceeding with model training. In this initial step, they tend to select a larger number of target data, trusting their pseudo labels, which may result in lower-quality selected target data. Table 1 summarizes the component-wise differences between our method and others.

Table1: A component-wise comparison of our proposed picking strategy to other related methods, such as SHOT++, ProxyMix, and BETA, reveals distinctive features and characteristics.

[1] Jian Liang et al. (2021). “Source data-absent unsupervised domain adaptation through hypothesis transfer and labeling transfer.” In IEEE TPAMI.
[2] Yuhe Ding et al. (2023). “Proxymix: Proxy-based mixup training with label refinery for source-free domain adaptation.” In NN.
[3] Jianfei Yang et al. (2022). “Divide to adapt: Mitigating confirmation bias for domain adaptation of black-box predictors.” In ICLR.

Q2: About the baselines in DomainNet.

Response: Thank you for the valuable suggestions. In the case of the challenging DomainNet dataset, widely recognized in domain adaptation, we have opted to directly cite baseline results from the state-of-the-art method, GPUE [1]. Notably, DomainNet has only recently been employed for source-free domain adaptation tasks, resulting in a limited number of evaluated methods. However, our method demonstrates significantly improved performance compared to GPUE, underscoring the efficacy of our approach.

Q3: About the adaptation time.

Response: Thank you for the valuable suggestion. We evaluated the iteration time for the A$\rightarrow$C adaptation task in OfficeHome. Using a ResNet50 model with a batch size of $64$, our method incurs a cost of 0.52 second for each iteration. In comparison, ERM at the source pretrain stage costs 0.23 second per iteration, SHOT costs 0.31 second, and AaD costs 0.49 second. Overall, the adaptation time of our method is comparable to the best-performing competitors.

Q1: More dataset for ablation study.

Response: Thanks for the reviewer's valuable suggestion. We further adopt the adaptation tasks, i.e., Re$\rightarrow$Cl, Re$\rightarrow$Pa, and Re$\rightarrow$Sk, from DomainNet to do the ablation study. The results can be found in Table 1. More detials, please refer to Section 4.3: Analysis and Discussions in our revised version.

Table 1: Ablation study of sub-modules in our proposed method on Office-Home and DomainNet datasets with six domain tasks. The three adaptation tasks i.e., A$\rightarrow$C, A$\rightarrow$P, and A$\rightarrow$P come from Office-Home. The remaining three adaptation tasks Re$\rightarrow$Cl, Re$\rightarrow$Pa, and Re$\rightarrow$Sk are from DomainNNet.

We also evaluate the sensitivity of hyper-parameters in our method on adaptation task, i.e., $Re\rightarrow Cl$, on DomainNet. Namely, the length of the round in Table 2, the ratio of selected labeled data in Table 3, the ratio of the selected reliable data in Table 4, and the number of top-k data for labeling in Table 5. We adopt the state-of-the-art source-free domain adaptation method, i.e., GPUE [1], as the baseline. More detials, please refer to Appendix A.6: Hyper-Parameter Analysis in our revised version.

Table 2: illustrates the impact of the length of round on adaptation task Re$\rightarrow$Cl on DomainNet dataset.

Table 3: illustrates the impact of the ratio of labeled data on adaptation task Re$\rightarrow$Cl on DomainNet dataset.

Table 4: illustrates the impact of the ratio of reliable data on adaptation task Re$\rightarrow$Cl on DomainNet dataset.

Table 5: illustrates the impact of the Top-K for neighbors on adaptation task Re$\rightarrow$Cl on DomainNet dataset.

[1] Mattia Litrico et al. (2023). "Guiding pseudo-labels with uncertainty estimation for test-time adaptation." In CVPR.

Q1: Data sampling related works.

Response: We appreciate the reviewer's insight regarding the effectiveness of training with "Training with "key samples" found by data sampling techniques can improve performance". The recommended works align closely with the thematic direction of our method. We incorporated these methods into our revised version. More details, please refer to the section: Related Work in our revised version..

Posterior Sampling-based Outlier Mining (POEM) [1] and Adaptive Outlier Optimization (AUTO) [2] both address the Out-of-Distribution (OOD) detection problem. POEM strategically identifies the most informative outlier data from an extensive pool of auxiliary data points by selecting samples lying on the OOD decision boundary. In contrast, AUTO introduces an in-out-aware filter designed to assign pseudo labels to data. The Importance Sampling method for Domain Adaptation (ISDA) [3] and Domain Adaptation via Cross-Domain Mixed Sampling (DACS) [4] focus on unsupervised domain adaptation. ISDA proposes a novel loss function incorporating feature-norm and prediction entropy considerations to selectively identify data with significant information for effective domain alignment during the training process. On the other hand, DACS involves selecting half of the classes in a source domain image, and then cutting out the corresponding pixels to paste them onto an image from the target domain. In contrast to the aforementioned methods, our approach is distinct in its goal of selecting unlabeled target data with high-quality pseudo-labels. Specifically, our data sampling involves a gradual selection process throughout the training. In each round, we exclusively opt for target data with high-quality pseudo-labels.

[1] Yifei Ming et al. (2022). "Poem: Out-of-distribution detection with posterior sampling." In ICML.
[2] Puning Yang et al. (2023). "Auto: Adaptive outlier optimization for online test-time ood detection." In arXiv.
[3] Xuemiao Xu et al. (2019). "Unsupervised domain adaptation via importance sampling." In IEEE TCSVT.
[4] Wilhelm Tranheden et al. (2021). "Dacs: Domain adaptation via cross-domain mixed sampling." In WACV.

Q4:About the number of samples have been sampled.

Response: We appreciate the reviewer's insightful recommendation for further analysis of our method. In our experiments, we have set the maximum ratio of labeled data to be $0.5$ across all datasets. Should the labeled data ratio reach $0.5$, subsequent picking operations will not be executed. Additionally, once a data in the target dataset is selected, it is excluded from the unlabeled subset, ensuring that no data is sampled multiple times.

Q2: Ablation studies with different dataset.

Response: Thanks for the valuable suggestion. We further adopt the adaptation tasks, i.e., Re$\rightarrow$Cl, Re$\rightarrow$Pa, and Re$\rightarrow$Sk, from DomainNet to do the ablation study. The results can be found in Table 1. More detials, please refer to Section 4.3: Analysis and Discussions in our revised version.

Table 1: Ablation study of sub-modules in our proposed method on Office-Home and DomainNet datasets with six domain tasks. The three adaptation tasks i.e., A$\rightarrow$C, A$\rightarrow$P, and A$\rightarrow$P come from Office-Home. The remaining three adaptation tasks Re$\rightarrow$Cl, Re$\rightarrow$Pa, and Re$\rightarrow$Sk are from DomainNNet.

We also evaluate the sensitivity of hyper-parameters in our method on adaptation task, i.e., Re$\rightarrow$Cl, on DomainNet. Namely, the length of the round in Table 2, the ratio of selected labeled data in Table 3, the ratio of the selected reliable data in Table 4, and the number of top-k data for labeling in Table 5. We adopt the state-of-the-art source-free domain adaptation method, i.e., GPUE [1], as the baseline. More detials, please refer to Appendix A.6: Hyper-Parameter Analysis in our revised version.

Table 2: illustrates the impact of the length of round on adaptation task Re$\rightarrow$Cl on DomainNet dataset.

Table 3: illustrates the impact of the ratio of labeled data on adaptation task Re$\rightarrow$Cl on DomainNet dataset.

Table 4: illustrates the impact of the ratio of reliable data on adaptation task Re$\rightarrow$Cl on DomainNet dataset.

Table 5: illustrates the impact of the Top-K for neighbors on adaptation task Re$\rightarrow$Cl on DomainNet dataset.

[1] Mattia Litrico et al. (2023). "Guiding pseudo-labels with uncertainty estimation for test-time adaptation." In CVPR.

Q3: Details of the rounds for all datasets.

Response: Thanks for the valuable comments. In all adaptation tasks within OfficeHone, the number of rounds has been standardized to 5. For DomainNet, a consistent setting of 10 rounds has been applied across all adaptation tasks. Similarly, for VisDA, we have chosen to implement 10 rounds as well. Namely, the length of the round, the ratio of selected labeled data, the ratio of the selected reliable data, and the number of top-k data for labeling. We also provide the accuracy vs. rounds curve for adaptation task Re$\rightarrow$Cl on DomainNet in Table 6. More detials, please refer to Appendix A.5: Accuracy vs. Round Number Curve for DomainNet in our revised version.

Table 6: illustrates the accuracy vs. number of rounds on adaptation task Re$\rightarrow$Cl on DomainNet dataset.