Towards Robust Pseudo-Label Learning in Semantic Segmentation: An Encoding Perspective

Thanks for taking the time to share your comments in the review assessment, as well as for acknowledging the Clear Presentation, Novel Approach, Consistent Improvements and Comprehensive Validation.

Below, we address each concern in detail.

Q1: Class-wise IoU Variance and Codebook Design

A1:

We appreciate the reviewer’s insightful question regarding the observed per-class IoU variations, especially the degradation in classes such as rider and motorcycle, and the notable improvements in classes like sidewalk. We provide a multi-faceted analysis below:

1. Attribute Decoupling and Class-Specific Difficulty

ECOCSeg introduces a bit-wise classification paradigm that decouples classes into attributes. This is particularly beneficial for visually confusing classes (e.g., road vs. sidewalk, car vs. bus). However, for fine-grained or underrepresented classes (e.g., rider, motorcycle), this decoupling introduces new challenges:

• These classes are often underrepresented in the source domain and have sparse or low-confidence pseudo-labels.
• Their bit-wise predictions tend to have lower confidence, which negatively affects ECOC decoding and final prediction accuracy.

2. Reliable Bit Mining and Class Imbalance

Our Reliable Bit Mining module selects only high-confidence bits during training. For rare classes with low-confidence predictions across most bits, fewer bits may be selected, leading to weaker supervision and potential underfitting. This phenomenon is orthogonal to the codebook design, and primarily stems from data imbalance and pseudo-label sparsity.

We note that this issue is particularly pronounced under the DACS framework, which does not include rare-class sampling or class-balancing strategies. As a result, the quality and quantity of pseudo-labels for rare classes (e.g., rider, motorcycle) are inherently unstable, even before applying ECOCSeg. Our method may amplify this effect slightly due to its reliance on bit-level confidence, but it is not the root cause.

3. Codebook Design Is Not the Root Cause

We emphasize that the observed per-class IoU variations are not caused by our specific codebook design, as clarified in Appendix C. By default, we use random dense binary codes, which are class-agnostic, in line with the spirit of traditional one-hot encoding. Importantly, our theoretical analysis shows that under ideal learning conditions, ECOC is functionally equivalent to one-hot encoding, ensuring that the code structure itself does not inherently bias performance toward or against specific classes.

To further validate this, we conduct experiments using multiple random codebooks (denoted as different $M_{\text{mmd}}$ matrices). As shown below, mIoU remains stable across codebooks, and class-wise variations fall within expected stochastic ranges:

These results confirm that ECOCSeg is robust to different codebook instantiations, and the codeword assignment is not responsible for class-specific performance drops.

4. Optional Use of Semantics-Aware Codebooks

In Appendix C, we further explore semantics-aware encoding using pretrained text embeddings (denoted as $M_{\text{text}}$) to reflect inter-class similarity. However, this is not required for the method to work, and serves only as an optional enhancement to facilitate smoother optimization. Analogous to one-hot vectors, our default ECOC design is semantically agnostic by construction, ensuring fair and unbiased treatment of all classes.

Q2: Typos.

A2: We thank the reviewer for pointing this out. We will carefully fix all typographical errors in the revised version.

We hope our response can resolve your concern. Please do not hesitate to let us know if you have further questions :)

Thanks for taking the time to share your comments in the review assessment, as well as for acknowledging the well-written paper, clear motivation, interesting idea, well-supported claims.

Below, we address each concern in detail.

Q1: Comparison with pseudo-label refinement methods

A1:

Thank you for the suggestion. As partially discussed in Appendix L, we now explicitly compare representative pseudo-label refinement methods with ours in the table below.

Most existing works primarily focus on feature-space refinement, such as filtering noisy samples or fusing multi-view predictions. In contrast, our approach views pseudo-label learning from the angle of label representation, offering an orthogonal and complementary strategy.

Unlike prior methods, our method improves pseudo-label robustness by decoupling classes into fine-grained binary attributes and performing bit-level denoising. This perspective is rarely explored and can potentially be combined with the above methods.

Q2: Codebook design

A2:

We appreciate the reviewer’s interest in the design of the ECOC codebook. Our framework addresses this issue from three complementary perspectives:

1. Codebook Design

our framework is designed with two complementary codebook strategies, supported by both theoretical properties and empirical robustness:

• Two Codebook Strategies with Complementary Design Principles

  • $M\_{\text{mmd}}$: Ensures large inter-class Hamming distance, maximizing error-correction capacity.
  • $M\_{\text{text}}$: Leverages semantic relationships via pre-trained word embeddings to create more learnable and semantically consistent codewords.

• Robustness to Codebook Choice

  Thanks to the inherent noise-tolerant property of ECOC and our Reliable Bit Mining mechanism, ECOCSeg is robust to suboptimal codebooks. Even when some bits are noisy or non-discriminative, our method learns selectively from reliable bits, ensuring stable performance.

• Empirical Validation

  As shown in Tables 5, 6 and Figure 6, both codebooks consistently yield improvements across UDA and SSL settings, demonstrating robustness to codebook variations. We also provide an ablative study in Appendix C showing how each strategy contributes differently under different supervision settings.

To clarify further, we also considered two widely used ECOC strategies:

• Random codes offer flexibility in codeword length and class count, but often lack structure. Without filtering, they may generate redundant or poorly-separated codes, leading to inconsistent performance.
• Hadamard codes provide orthogonal codewords with strong theoretical error correction properties. However, they are only defined for certain code lengths (e.g., 8, 16, 32, ...), and thus cannot flexibly adapt to arbitrary class counts (e.g., 19 classes in Cityscapes). This introduces unnecessary code length, and increases training complexity.

2. Choice of $K$

We provide a detailed theoretical and empirical analysis in Appendix E and G, summarized as follows:

• Theoretical & Empirical Motivation

  • A minimum of $K ≥ \log_{2}{N}$ is required to uniquely represent $N$ classes.
  • Prior literature on ECOC [1] suggests a code length of $K = 10 \times \log_{2}{N}$ as a practical guideline for ensuring sufficient Hamming distance and error-correction.

  In our case, we follow $K = 10 \times \log_{2}{N}$ for scalability and effectiveness.:

  • For Cityscapes ($N = 19$) and Pascal ($N = 21$), this yields $K ≈ 40–45$
  • For COCO ($N = 81$), we use $K = 60$, aligning with this principle

• Ablation Study on $K$

  We conduct an extensive study in Appendix E, Table 7:

  • Performance improves as $K$ increases, due to increased error tolerance.
  • However, the gain saturates beyond $K = 40$, and overly long codes introduce slight redundancy.
  • This validates $K = 30–50$ as a sweet spot, balancing robustness and efficiency.

3. Manual Definition and Class Semantics

Our method is fully automatic and does not require manual class-wise design of codewords. Considering semantic or visual relationships is optional, not required:

• In $M_{\text{mmd}}$, we adopt a class-agnostic strategy that maximizes codeword separability without relying on class semantics.
• In $M_{\text{text}}$, semantic similarity is implicitly captured from text embeddings, which can reflect visual relationships as well.

As analyzed in Appendix C, both strategies function effectively within our framework. Importantly, like one-hot encoding, it is not necessary for each codeword to carry a specific meaning—the codewords only need to be separable in label space. This design allows flexibility and general applicability across datasets.

Q3: Sensitivity Analysis

A3: We thank the reviewer for the suggestion. We would like to emphasize that we have already provided detailed sensitivity analyses in the main paper and appendices.

Threshold $T$

We analyze the influence of $T$ in Section 5.4 and Table 4, and further extend the results across datasets in Appendix D. Our findings show:

1. Both code-wise and bit-wise pseudo-labels independently contribute to performance improvements, enabled by the attribute-level decoupling introduced by ECOC.
2. In Reliable Bit Mining, $T=0.5$ and $T=1$ correspond to pure code-wise and bit-wise forms, respectively. Intermediate values (e.g., $T=0.95$) produce more robust hybrid labels, making $T$ a non-sensitive hyperparameter.
3. The effect of $T$ is consistent across datasets. We adopt a fixed setting of $T=0.95$ in all experiments and observe stable gains.

Loss Weights $\lambda_1$, $\lambda_2$ and Temperature $\tau$

As analyzed in Appendix I (Table 14):

• $\lambda_1$ and $\lambda_2$ weigh auxiliary loss terms that enhance feature structure. These only need to ensure comparable gradient scales across losses. We find they are insensitive over a wide range and easy to tune.
• $\tau$, which controls the softness of predicted probabilities, influences pseudo-label confidence. We fix $\tau = 0.5$ in all experiments and observe stable, high performance.

These analyses confirm that our method is robust to all key hyperparameters ($T$, $\lambda_1$, $\lambda_2$, and $\tau$), and does not require extensive tuning to perform effectively.

[1] Reducing multiclass to binary: A unifying approach for margin classifiers. JMLR2000

Q4: Computational Overhead for Large $N$ and $K$

A4: We acknowledge the reviewer’s concern regarding scalability. As shown in Appendix J, we provide a detailed efficiency analysis (table 15).

• The FLOPs increase is negligible, as only the segmentation head is modified—replacing a single $N$-way classifier with $K$ binary classifiers.
• The increase in GPU memory and training time is due to:
  1. The $K$-bit prediction, which replaces the original output.
  2. A small ECOC codebook, represented as a binary matrix of size $N \times K$
  3. The Reliable Bit Mining algorithm, which involves simple operations (e.g., confidence ranking and bit masking) and is lightweight relative to the overall training cost.

Importantly, inference-time overhead is minimal, since Reliable Bit Mining is only used during training, and the binary classifiers are highly parallelizable. Moreover, for large-class datasets like COCO (N = 81), our design uses $K = 60 < N$, meaning that the number of classifiers is actually reduced compared to standard N-way classification.

Overall, ECOCSeg introduces only minor computational overhead while delivering significant performance gains, making it a practical and scalable solution even for large-scale semantic segmentation tasks.

Q5: Further comparison with related works:

A5: We appreciate the reviewer for pointing out the connection to [a] and [b]. We have already discussed partial works in Appendix L, and now restate and clarify the key differences here for completeness. While all three methods aim to improve learning under limited supervision, our approach differs significantly in both motivation and methodology.

• [a] introduces a correction network to refine pseudo-labels based on confidence, aiming to reduce overfitting in semi-supervised segmentation.
• [b] focuses on feature-space alignment by using a small amount of labeled target data to guide adaptation via anchor-based soft alignment.
• In contrast, our method takes a novel label-space perspective, leveraging ECOC-based class encoding to model shared attributes between confusing classes. We further propose Reliable Bit Mining and hybrid pseudo-labels to perform fine-grained, bit-level denoising—an orthogonal strategy not explored in [a] or [b].

Our method introduces a structured, fine-grained label encoding that enables attribute-level correction, offering an orthogonal perspective to the confidence-based and feature-alignment strategies in [a] and [b]. By addressing pseudo-label noise at the encoding level, it captures inter-class relationships—such as shared visual attributes—often overlooked by prior work. This complementary view enhances robustness and can be integrated with existing methods for further gains.

We hope our response can resolve your concern. Please do not hesitate to let us know if you have further questions :)

Thanks for taking the time to share your comments in the review assessment, as well as for acknowledging the well-organized structure, general and modular framework, technically sound method, clear motivation and strong evidence results.

Below, we address each concern in detail.

Q1: Codebook Design Sensitivity

A1:

Thank you for raising this important point. We agree that codebook quality is a key factor for ECOC-based methods.

To mitigate this sensitivity, our framework is designed with two complementary codebook strategies, supported by both theoretical properties and empirical robustness:

1. Two Codebook Strategies with Complementary Design Principles

• $M\_{\text{mmd}}$: Ensures large inter-class Hamming distance, maximizing error-correction capacity.
• $M\_{\text{text}}$: Leverages semantic relationships via pre-trained word embeddings to create more learnable and semantically consistent codewords.

2. Robustness to Codebook Choice

   Thanks to the inherent noise-tolerant property of ECOC and our Reliable Bit Mining mechanism, ECOCSeg is robust to suboptimal codebooks. Even when some bits are noisy or non-discriminative, our method learns selectively from reliable bits, ensuring stable performance.

3. Empirical Validation

   As shown in Tables 5, 6 and Figure 6, both codebooks consistently yield improvements across UDA and SSL settings, demonstrating robustness to codebook variations. We also provide an ablative study in Appendix C showing how each strategy contributes differently under different supervision settings.

Q2: Lack of Reported Statistical Significance

A2: We appreciate the reviewer’s suggestion regarding statistical significance.

We appreciate this suggestion and agree that demonstrating statistical significance is essential. We have now conducted 5 repeated runs on two representative tasks (UDA and SSL) and report the mean and standard deviation:

1. UDA: GTA5 → Cityscapes (DAFormer + ECOCSeg)

Compared to the baseline DAFormer (68.3), we achieve a $+2.0$ mIoU gain with low variance (0.24 std), confirming the improvement is statistically consistent.

2. SSL: Pascal 1/4 Partion (UniMatch + ECOCSeg)

Again, our method shows stable improvements over the UniMatch baseline (77.2), with $+1.7$ mIoU gain and low standard deviation (0.17 std).

These results confirm that our gains are statistically consistent and robust. We will include these results in the revised version of the paper.

Q3: Limited Novelty – ECOC is a Known Concept

A3:

We understand the concern and would like to clarify that while ECOC itself is a classical technique, our work contributes a novel perspective and practical framework tailored for pseudo-label learning in dense prediction tasks, which poses unique challenges not previously addressed by ECOC literature.

Specifically, our contributions go beyond a direct application of ECOC, and include several task-oriented innovations:

• Hybrid Pseudo-Labeling: We propose a new paradigm that combines bit-wise soft supervision and code-wise hard supervision, enabling partial but reliable supervision, especially beneficial in noisy contexts like UDA/SSL.
• Bit-Level Denoising via Reliable Bit Mining: Unlike traditional ECOC, we introduce a mechanism to selectively retain trusted bits from top-k nearest classes, adapting to pixel-wise uncertainty — a key innovation for semantic segmentation.
• Task-Specific Optimization Objectives: We design new loss functions (pixel-code distance and pixel-code contrast) to enforce intra-class compactness and inter-class separation in code space, which is absent in classical ECOC applications.
• Theoretical Justification of ECOC for Noisy Pseudo-Labels: We provide a formal analysis showing that ECOC offers tighter classification error bounds than one-hot encoding under label noise, supported by Neural Tangent Kernel (NTK)-based generalization theory. Our theorems establish conditions under which ECOC is provably more robust, offering a theoretical foundation for its use in pseudo-label learning.
• Modularity and Generality: ECOCSeg is plug-and-play and compatible with a wide range of architectures and learning paradigms, making it practical and impactful.

Importantly, the proposal of ECOCSeg is not an arbitrary or isolated tweak, but a task-specific strategy that is analytically motivated by the inherent challenges of pseudo-label learning in pixel-level segmentation. We believe that simplicity in implementation does not imply lack of novelty. On the contrary, we view the ability to integrate a classical idea into a modern context effectively and practically as an important and underexplored direction in representation learning.

Q4: Codeword Length K and Scalability

A4:

Thank you for this insightful question. We agree that the choice of codeword length K is important for both robustness and scalability. We provide a detailed analysis in Appendix E and G, and summarize our theoretical insights, empirical findings, and practical guidelines below.

1. Theoretical & Empirical Motivation

• A minimum of $K ≥ \log_{2}{N}$ is required to uniquely represent $N$ classes.

• Prior literature on ECOC [1] suggests a code length of $K = 10 \times \log_{2}{N}$ as a practical guideline for ensuring sufficient Hamming distance and error-correction.

In our case:

• For Cityscapes ($N = 19$) and Pascal ($N = 21$), this yields $K ≈ 40–45$
• For COCO ($N = 81$), we use $K = 60$, aligning with this principle

2. Ablation Study on $K$

We conduct an extensive study on Cityscapes under the fully supervised setting with DAFormer. Results are shown in Appendix E, Table 7:

• Performance improves as $K$ increases, due to increased error tolerance.
• However, the gain saturates beyond $K = 40$, and overly long codes introduce slight redundancy.
• This validates $K = 30–50$ as a sweet spot, balancing robustness and efficiency.

3. Practical Recommendation

• We adopt $K = 40$ as a default setting for all experiments on Cityscapes and Pascal.
• For larger class spaces (e.g., COCO), we follow $K = 10 \times \log_{2}{N}$ for scalability and effectiveness.

4. Scalability to Large-Class Datasets

We also address the scalability of ECOCSeg when applied to datasets with hundreds or thousands of classes, such as ImageNet ($N = 1000$):

• $K$ grows logarithmically, not linearly.

  Using the $K = 10 \times \log_2 N$ rule, we would use $K \approx 100$ for ImageNet — still much smaller than N.

• Binary classifiers are lightweight and parallelizable.

  Each of the $K$ binary heads is a single-output sigmoid classifier, which can be trained in parallel. Compared to a 1000-way softmax, this design is computationally efficient per-head.

• FLOPs and memory overhead grow modestly.

  As discussed in Appendix J, the increase in FLOPs and memory is negligible, and ECOCSeg remains efficient.

[1] Reducing multiclass to binary: A unifying approach for margin classifiers. JMLR2000

We hope our response can resolve your concern. Please do not hesitate to let us know if you have further questions :)

Thank you for your detailed and thoughtful rebuttal. Your responses have clarified several key points and addressed my primary concerns! Before updating my final score, I would like to see your rebuttal to the reviewer Tj3N’s comments.

Thanks for taking the time to share your comments in the review assessment, as well as for acknowledging the simple concept, accessible theoretical analysis and well-supported motivation. Below, we address each concern in detail.

Q1: Comparison with Other Encoding Schemes / Why Binary Encoding?

A1:
We appreciate the reviewer’s insightful observation. While there is conceptual overlap between our binary encoding and structured representation learning (e.g., concept learning), our formulation is specifically motivated by the pseudo-label learning paradigm, which is fundamentally classification-oriented. Using continuous codewords would transform the problem into multi-dimensional regression, which introduces several drawbacks in the context of pseudo-label learning:

1. Learning Difficulty: For each individual bit, binary classification is generally easier to optimize than regression over continuous targets. This is especially critical in low-data or noisy-label settings such as UDA and SSL.
2. Mismatch with Pseudo-labeling: Pseudo-label learning methods are inherently designed for classification tasks. Applying them to regression targets (e.g., continuous codes) violates the underlying assumption of discrete label prediction.
3. Bit-level Interpretability and Denoising: In binary encoding, each bit can be interpreted as a shared visual or semantic attribute. This allows us to isolate and refine reliable supervision at the bit level via our Reliable Bit Mining strategy. Such fine-grained denoising is not directly applicable in continuous encoding, where the semantics of each dimension are not well-defined or disentangled.

To validate this empirically, we implement a version of ECOCSeg using continuous-valued codewords (C) and compare it against our binary encoding on two UDA benchmarks. As shown below, the performance degrades significantly:

These results confirm that binary encoding not only aligns better with the pseudo-labeling framework but also leads to superior empirical robustness.

Q2: Theoretical Justification

A2: We thank the reviewer for this valuable point.

Our theoretical analysis (Theorems 4.1 and 4.2) is intended as a supporting justification for the feasibility and robustness of using ECOC in pseudo-label learning. Specifically, it shows that ECOC encoding is more tolerant to label noise than one-hot encoding under reasonable assumptions (e.g., sufficient code distance).

That said, our work is primarily application-oriented. The overall design of ECOCSeg—including the code length, hybrid pseudo-label strategy, and codebook construction—is guided by practical needs in real-world UDA and SSL settings. These components are systematically validated through extensive experiments (see Tables 4–7 and Figures 6–7), which we believe offer strong empirical support.

We will clarify in the revised manuscript that our theoretical results are meant to complement empirical findings—not to serve as formal guarantees for each design choice.

Q3: Typos, Captions, and References.

A3: We thank the reviewer for pointing this out. We will carefully fix all typographical errors (e.g., Lines 26, 81, 189, Algorithm 2) and revise the figure captions to provide clearer and more informative descriptions.

Regarding Line 28: DACS is indeed a classic self-training-based method, and while it does not explicitly frame itself as a teacher-student model, its implementation relies on an EMA-updated model to generate pseudo-labels, which effectively plays the role of a teacher. We will revise the description to accurately reflect this and cite a more appropriate reference if needed to clarify the context.

Q4: Computation In Algorithm 1

A4: We clarify that the iterations in Algorithm 1 refer to the process of class querying for each pixel, not the training iterations of the model. Specifically, for each unlabeled pixel, we perform the following steps:

1. We iterate over all $N$ classes, ranked by their codeword distance to the predicted bit vector.
2. The first iteration initializes the candidate class set $S_c$ with the top-1 nearest class.
3. The shared bit positions $P_s{(S_c)}$ are then computed as the set of bit indices where all classes in $S_c$ agree (i.e., bits that are identical across the selected codewords).
4. The confidence score $q^i_m$ is calculated as the mean of the predicted bit-wise confidence values over the shared bits $P_s$.
5. If $q^i_m<T$ (the confidence threshold), we expand $S_c$ by including the next closest class and recompute the shared bits and confidence. This process continues until either $q^i_m>T$ or the shared bit set becomes empty.
6. The resulting reliable mask $M_i$ is a binary mask indicating which bits are in the shared set $P_s$ and used as reliable bits for that pixel.

Figure 4(b) shows an simple example. We will revise Algorithm 1 to make this procedure clearer and include a concrete step-by-step example to illustrate how the confidence grows as more classes are queried.

Q5: Analysis of Hybrid Pseudo-Label Strategy

A5:

• Selection of top-$C$ classes: In our hybrid pseudo-label strategy, for each unlabeled pixel, we directly select the common bits among the top-$C$ most similar classes based on Hamming distance to the predicted bit vector. These shared bits define the partial supervision signal. Importantly, the mean confidence score is not used to select classes, but is instead used in the Reliable Bit Mining algorithm to guide the choice of $C$—i.e., to determine how many top classes should be included such that the shared bits remain sufficiently confident. We provide a detailed analysis of this process in Figure 4.
• Sharp performance changes near confidence threshold 1.0: As shown in Figure 6, we observe a sharp performance drop when the confidence threshold approaches 1.0. This is due to a well-known phenomenon: modern neural networks are typically overconfident and poorly calibrated, as discussed in [1]. Therefore, we empirically select a relatively high but not extreme threshold (e.g., 0.9), which balances reliability and coverage. As shown in Table 4, and further analyzed in Appendix D, our method is robust to a wide range of confidence thresholds between 0.5 and 1.0, consistently improving performance.

[1] On calibration of modern neural networks. ICML2017

Q6: Comparison Method

A6: We selected DACS, DAFormer, FixMatch, and UniMatch due to the following reasons:

• They are representative and widely-recognized benchmarks in UDA and SSL.
• All selected methods offer open-source implementations, enabling consistent and controlled integration of our ECOCSeg module.
• These methods adopt modular architectures, making them suitable for plug-and-play evaluation of new components like ours.

Regarding other methods such as CPSL and CDAC in UDA:

• Their report performance does not surpass our strongest baseline (MIC), which we already include.
• We intentionally select three baselines of increasing strength (DACS → DAFormer → MIC) to verify our method's generality across different backbones.

For methods like RankMatch and DAW in SSL:

• These methods introduce specialized supervision signals for pseudo-labels, often designed specifically for one-hot class predictions (e.g., class-wise consistency regularization or weighting functions).
• Their frameworks are tightly coupled with their pseudo-labeling strategies, making it non-trivial to integrate our bit-level supervision.
• Most importantly, our method still outperforms their reported results, even without such targeted designs.

To further support the generality of ECOCSeg, we implemented it on CDAC using its official codebase. As shown below, ECOCSeg still provides noticeable improvements:

This result confirms that even for methods not originally selected due to coupling or complexity, ECOCSeg is still compatible and beneficial. We will clarify these rationale and include the additional CDAC result in the revised version.

Q7: Codebook Using Image Features

A7:
We thank the reviewer for the constructive suggestion. Following your advice, we replace the Word2Vec (w2v) in Algorithm 3 with CLIP and conduct additional experiments in both UDA and SSL settings. The results are shown below:

UDA with DAFormer:

SSL with UniMatch:

These results show that while CLIP provides a slight improvement, the gains are not significant compared to w2v. We attribute this to the robustness of our overall framework, where the ECOC-based encoding and Reliable Bit Mining mechanism are both effective without strong prior knowledge from pretrained models.

Moreover, introducing CLIP embeddings brings in additional data-driven priors, which may complicate fair comparisons in low-label regimes like UDA and SSL. To keep the comparison fair and controlled, we chose to use w2v as a lightweight and open semantic prior, which still achieves strong performance.

We will include these new results and analysis in the final version.

We hope our response can resolve your concern. Please do not hesitate to let us know if you have further questions :)

Thank you for your thoughtful follow-up questions. We greatly appreciate your continued engagement and the opportunity to clarify our work further. Below, we respond to each point in detail.

Q1: Clarification on Continuous-Valued Codebook Setup

A1:
Thank you for requesting further clarification.

We explore two kinds of continuous-valued codebook.

1. Predefined Codebooks:

In our ablation study comparing binary and continuous-valued codebooks, we considered two variants:

1. Mmd-based Continuous Codebook: We use Alg. 2 and each class is assigned a codeword sampled from a standard Gaussian distribution $\mathcal{N}(0, 1)^K$

2. Text-Based Continuous Codebook: We use Alg. 3 and skip the quantization step, preserving the continuous values as the codebook.

Both of them are then L2-normalized to ensure all codewords lie on the unit hypersphere, similar to common practice in contrastive learning. The Text-Based Continuous Codebook performs better than the Mmd-based Continuous Codebook. So we reported the Text-Based one in our ablation study in previous A1.

Supervision:

During training, we apply an L2 regression loss between the predicted vector and the target codeword:

$\mathcal{L}_{\text{reg}} = \| \hat{z}_i - c_n \|^2$,

where $\hat{z}_i \in \mathbb{R}^K$ is the predicted vector for pixel $i$ , and $c_n$ is the normalized continuous codeword for class $n$ .

Inference:

At test time, we compute the predicted class via nearest neighbor matching:

$\hat{y}_i = \arg\min_n \| \hat{z}_i - c_n \|^2$.

This setup mirrors the structure of ECOCSeg but replaces binary classification with vector regression. As shown in our previous response, this results in a notable drop in performance, due to the increased regression complexity and lack of bit-level denoising.

2. Learnable Codebook (Failed Variant):

We also experimented with an online-learned codebook, initialized from $M_{text}$ and alternately:

1. Use our predefined codebook for initialization (with continuous-valued parameters).
2. During training, alternately perform thresholding the continuous values to binary for label assignment and updating the continuous-valued encoding using predictions (only updating with labeled data).

However, this implementation did not achieve good convergence and exhibited inferior performance. We believe this is due to

• ECOC encoding being a supervised label, and updating it during training can make network training unstable, which is amplified in pseudo-label learning;
• the update rate of the codebook has a significant impact on training, requiring additional hyperparameter design and compromising the method's generalizability. As a supervised label, it is more natural to use a predefined encoding to represent categories, just like one-hot encoding.

Given these challenges, we advocate for predefined codebooks, similar to one-hot encoding, which offer better stability, interpretability, and generalizability.

Q3: Clarification on EMA vs. Teacher-Student Terminology

A3:
Thank you for pointing this out.

You are absolutely right—not all EMA-based models qualify as teacher-student architectures in the strict sense. In our original description, we referred to the EMA-updated model as a "teacher-like" model following conventions in works like Mean Teacher and FixMatch.

To avoid confusion, we will revise the manuscript to clarify: “...a temporally smoothed EMA model is used to generate pseudo-labels, serving a teacher-like role without forming a separate teacher-student architecture.”

Q4: Clarification on Notation and Figure 4(b)

A4:
Thank you for the opportunity to clarify.

• Notation $q^i_m$:
  This denotes the mean bit-wise confidence for pixel $i$ , computed over the shared bit positions $P_s$:

  $q^i_m = \frac{1}{|P_s|} \sum_{k \in P_s} \max(p(k|z_i), 1 - p(k|z_i))$

  where $p(k|z_i) \in [0, 1]$ is the predicted probability for the $k$ -th bit being 1.

• Confidence Map vs. Area:

  • The confidence map visualizes this per-pixel $q^i_m$ across the spatial image.
  • The confidence area refers to regions where $q^i_m < T$ , indicating low-confidence predictions.

• Accurate classification in low-confidence regions:
  When $q^i_m$ is low, we query more top-$C$ nearest codewords. Even if Top-1 is incorrect, the shared bits among Top-$C$ classes can still capture ground-truth attributes. These bits often have higher consensus and thus higher confidence, leading to better supervision.

We will revise the figure and caption to reflect this explanation more clearly.