Balanced Learning for Domain Adaptive Semantic Segmentation

We sincerely thank the reviewer for the positive and constructive feedback. We appreciate your recognition of motivation, theoretical intuition, and clear writing, as well as your kind comments on our visualizations and formula design.

---

Regarding your suggestion to include more experimental results for the Cityscapes → ACDC setting, we agree that evaluating BLDA under more diverse and challenging UDA scenarios is important and we conduct extended experiments on image classification/video semantic segmentation in Appendix G/H. We conduct additional experiments on the Cityscapes → ACDC benchmark and present the results below.

As shown in the following tables (∗ denotes the reproduced result), integrating BLDA consistently improves performance across multiple baselines, including DACS, DAFormer, and MIC, on both mIoU and mAcc metrics. Importantly, BLDA is designed as a plug-and-play module that is model-agnostic and can be seamlessly integrated into most existing UDA baselines without modifying their core architectures or training objectives. Since class bias is a pervasive issue across UDA settings, our method provides a general mechanism to mitigate this imbalance during training. As such, BLDA is able to consistently improve performance across a wide range of models and target domains, as demonstrated in our experiments on both standard benchmarks and the newly added Cityscapes → ACDC setting.

Cityscapes → ACDC (IoU, %)

Cityscapes → ACDC (Acc, %)

We will include these results in the revised version of the paper.

---

We hope our response can resolve your concern. Please do not hesitate to let us know if you have further questions.

We sincerely thank the reviewers for their valuable feedback and thoughtful comments. We appreciate the recognition of our clear writing, comprehensive experiments, and extensive qualitative analyses. We address each of your concerns point by point.

---

Q1: Clarification on Eq. (4) and the Definition of Positive Bias

A1: Sorry for any misunderstanding. In Eq. (4), the expression $\arg\max_{c' \in [C]} f_\theta(x)[c']$ refers to the predicted class for input $x$. $P(\arg\max_{c' \in [C]} f_\theta(x)[c'] = l \mid y = c)$ represents the probability that a sample from class $c$ is predicted as class $l$. There is no constraint that $c \ne l$ in Eq. (4). The summation in this equation is taken over the conditioning variable $c$, i.e., it averages the conditional probabilities $P(\arg\max_{c' \in [C]} f_\theta(x)[c'] = l \mid y = c)$ across all classes $c \in [C]$.

By summing over all $c$ and taking the average, we obtain the expected probability that a sample from any class (including $l$ itself) is predicted as class $l$. Under an unbiased model, this expectation should be approximately $1/C$. Therefore, a positive bias indicates that class $l$ is over-predicted, as its average prediction probability exceeds the uniform expectation.

---

Q2: Dependence on Baseline Models

A2: Our method is designed as a plug-and-play module that can be integrated into any self-training-based UDA framework. Naturally, its performance is influenced by the underlying baseline, especially when estimating the target-domain logits distribution, which relies on the pseudo-labels generated during self-training. Hence, the quality of pseudo-labels affects the accuracy of our distribution estimation and, consequently, the effectiveness of the class-aware adjustment.

However, we emphasize that BLDA consistently reduces class-level prediction variance across all baselines. As reported in the main paper, the standard deviation of per-class IoU and Acc drops significantly when BLDA is applied. This demonstrates that our method consistently yields more balanced predictions with reduced class bias, regardless of the baseline.

---

Q3: Fairness of Comparisons and Pseudo-Label Quality

A3: We would like to clarify that our method adheres to the standard UDA setting and does not modify the pseudo-label generation process of any baseline. As described in Section 3.2, we employ the self-training framework as implemented in existing works such as DACS, DAFormer, CDAC, HRDA, and MIC. All these methods adopt an online self-training protocol: the model is trained from scratch using labeled source data and unlabeled target data with pseudo-labels generated during training.

BLDA is inserted into this process as a lightweight module that adjusts the logits distributions based on estimated class-wise prediction behavior. The pseudo-label generation and training pipeline of each baseline remains unchanged. Therefore, the comparisons are conducted under fair and consistent settings, where each method follows the same online self-training paradigm (note that all baselines train the model from scratch, without using a source-only pretrained model).

---

Q4: Computation Overhead

A4: We provide a detailed analysis of the computational overhead in Appendix I, including actual resource usage and training time across various baselines. In summary, the additional cost introduced by our proposed components stems from three main operations. We have implemented them efficiently to minimize overhead:

1. GMM Implementation: Instead of using off-the-shelf libraries that update Gaussian parameters sequentially, we store the parameters of all $C \times C \times K$ components as tensors in PyTorch and update them in parallel using matrix operations.
2. CDF Computation: We approximate the cumulative distribution function using the Abramowitz-Stegun formula, which allows efficient polynomial evaluation.
3. Inverse CDF Computation: We use interpolation techniques within the estimated value range, which avoids costly numerical inversion.

All the above operations can be efficiently performed using simple matrix operations on tensors. Moreover, the storage of Gaussian component parameters and the additional regression head (a $1\times 1$ conv) introduced by our method are lightweight. Overall, our method demonstrates high efficiency in both training time and GPU memory, as reported in Table 13 of Appendix I. Moreover, our method introduces no additional overhead during inference.

---

We hope our response can resolve your concern. Please do not hesitate to let us know if you have further questions.

We sincerely thank the reviewer for the positive and constructive feedback. We appreciate your recognition of our method’s versatile design, the thoroughness of our experimental validation, and the clarity of our theoretical explanation. We address each of your concerns point by point.

---

Q1: Computational Cost and Scalability of GMM-based Modeling

A1: We provide a detailed analysis of the computational overhead in Appendix I, including actual resource usage and training time across various baselines. In summary, the additional cost introduced by our proposed components stems from three main operations. We have implemented them efficiently to minimize overhead:

1. GMM Implementation: Instead of using off-the-shelf libraries that update Gaussian parameters sequentially, we store the parameters of all $C \times C \times K$ components as tensors in PyTorch and update them in parallel using matrix operations.
2. CDF Computation: We approximate the cumulative distribution function using the Abramowitz-Stegun formula, which allows efficient polynomial evaluation.
3. Inverse CDF Computation: We use interpolation techniques within the estimated value range, which avoids costly numerical inversion.

All the above operations can be efficiently performed using simple matrix operations on tensors. Moreover, the storage of Gaussian component parameters and the additional regression head (a $1\times 1$ conv) introduced by our method are lightweight. Overall, our method demonstrates high efficiency in both training time and GPU memory, as reported in Table 13 of Appendix I. Moreover, our method introduces no additional overhead during inference.

For larger datasets, i.e., with more classes, the only change is that the GMM module needs to store a proportionally larger number of Gaussian components, which remains tractable in practice.

---

Q2: Generalization of Pre-defined Anchor Distributions

A2: We provide a detailed discussion of the anchor distribution design and its alternatives in Appendix J. A summary is provided below:

1. Definition and Role of Anchor Distributions: We use the global positive and negative logits distributions from the source domain to estimate a shared anchor distribution for both the source and target domains. This shared anchor allows the target logits distribution to gradually align with the source, serving two purposes:

   • It provides a reference to balance learning progress across classes within each domain.
   • It acts as a bridge to align the source and target domains in terms of class-wise logits behavior.

2. Different Selection Criteria for Anchor Distributions: While the anchor distribution is estimated from the source domain, we acknowledge that some discrepancy may exist between this estimation and the true distribution of the target domain. However, our analysis in Appendix B shows that the relative pos/neg bias is more important than the absolute value of the logits, and such relative structure tends to be preserved across domains due to the observation that the biases of positive and negative logits are coupled in Fig.1 (c). This explains why the impact of distributional mismatch is limited in practice.

   We also conduct ablation studies (Table 14) on different anchor selection strategies, including: Global source distribution (default); Global target distribution (oracle); Two biased classes (building and fence in Fig. 1(d)). The results show that different anchor choices lead to only marginal differences in performance, further demonstrating the robustness of our method to anchor selection and its ability to generalize across domain shifts.

3. Generalization Across Datasets and Domain Shifts: Our logits adjustment mechanism can be interpreted as a cross-entropy loss with an adaptive margin. As discussed in Appendix B, this margin is implicitly determined by the relative difference between the positive and negative logits distributions. There are two key cases:

   • If both positive and negative logits in the anchor increase or decrease simultaneously, the margin remains stable.
   • If the relative gap between pos/neg distributions in the target differs from that in the anchor, the margin adapts accordingly to guide alignment.

   This behavior enables the method to generalize across domain shifts. As demonstrated in Table 15, the adaptive margin mechanism drives the target logits distribution to progressively align with the anchor distribution over training, confirming the intended behavior of our method.

---

We hope our response can resolve your concern. Please do not hesitate to let us know if you have further questions.

We thank the reviewer for the positive and constructive feedback, as well as for acknowledging our contributions, including the clear problem formulation, the clarity of the writing, and the effectiveness of empirical validation. We address each of your concerns point by point.

---

Q1: Computation Overhead

A1: We provide a detailed analysis of the computational overhead in Appendix I, including actual resource usage and training time across various baselines. In summary, the additional cost introduced by our proposed components stems from three main operations. We have implemented them efficiently to minimize overhead:

1. GMM Implementation: Instead of using off-the-shelf libraries that update Gaussian parameters sequentially, we store the parameters of all $C \times C \times K$ components as tensors in PyTorch and update them in parallel using matrix operations.
2. CDF Computation: We approximate the cumulative distribution function using the Abramowitz-Stegun formula, which allows efficient polynomial evaluation.
3. Inverse CDF Computation: We use interpolation techniques within the estimated value range, which avoids costly numerical inversion.

All the above operations can be efficiently performed using simple matrix operations on tensors. Moreover, the storage of Gaussian component parameters and the additional regression head (a $1\times 1$ conv) introduced by our method are lightweight. Overall, our method demonstrates high efficiency in both training time and GPU memory, as reported in Table 13 of Appendix I. Moreover, our method introduces no additional overhead during inference.

---

Q2: Why use logits distributions

A2: In UDA, explicit class distribution priors are often unavailable, especially in the target domain, where distribution shift is severe. Therefore, we propose leveraging the logits distributions as an online proxy to assess a model’s class-wise prediction bias.

We justify this design choice from both theoretical and empirical perspectives:

1. Theoretical Justification: As shown in Definition 2, the prediction bias $\mathrm{Bias}(l)$ is directly related to the probability of predicting class $l$ across all ground-truth classes, which in turn depends on the relative distributions of logits. We further show in Eq. (5) that under the assumption of independent logits distribution, this probability can be estimated by comparing the positive and negative class logits. A sufficient condition for unbiased prediction is that these distributions are aligned.

2. Empirical Evidence: In Figure 1(d), we demonstrate a clear linear correlation between the prediction bias and the differences in logits distributions across classes. This supports our hypothesis that logits distributions serve as an effective proxy to capture class imbalance in the network’s behavior.

---

Q3: Alternative Distribution Estimation Methods

A3: Thank you for this valuable suggestion. We chose GMMs primarily for their balance between modeling capacity and computational efficiency, which is critical for our online training setup. Specifically:

• GMMs can approximate a wide range of 1D distributions with a small number of parameters.
• They allow closed-form CDF and inverse CDF computation (using polynomial approximations), enabling efficient matrix-based implementation for large-scale training.
• The distributions we model are scalar-valued logits for each class pair $(c, l)$, making GMMs a sufficiently expressive and computationally tractable choice.

Additionally, as shown in Appendix M, GMMs empirically fit the logits distributions well, and as discussed in Appendix E, they converge quickly during training. While more complex estimators (e.g., kernel density estimation or deep density models) could be considered, they often introduce significant overhead and do not offer closed-form CDFs, which are essential for our framework.

---

Q4: References

A4: Thank you for pointing this out. These references are indeed important and representative works in UDA for semantic segmentation. We will include them in the Related Work section in the revised version and compare with them in our experiments.

---

We hope our response can resolve your concern. Please do not hesitate to let us know if you have further questions.