Thank you for your thoughtful feedback and for highlighting important aspects of our work. We appreciate the opportunity to address your concerns and refine our presentation accordingly.

1. Uncertainty Mechanism Analysis

We carried out an in-depth empirical study of our mutual information term (DALD) to understand how it reshapes pixel selection beyond simple entropy. Working on the ADE-Bed dataset, we computed three uncertainty maps for each image at AL rounds 1, 5, and 10: (i) Entropy ($E$), (ii) DALD ($D$), and (iii) eDALD ($H = E + D$, computed as a simple unweighted sum without normalization).

For each round, we compared the top-$b$ pixels chosen by $E$ versus those chosen by $H$. The following table reports the Jaccard overlap and Spearman’s rank correlation between the two rankings:

By round 10, only 34 % of the entropy-selected pixels remain among the eDALD picks, and the low Spearman $\rho$ (mean 0.26, sometimes negative) shows that DALD substantially reshuffles the uncertainty ranking rather than merely amplifying entropy. DALD alone contributes very few of the same pixels (Jaccard ≈ 0.01), indicating that it brings new, highly uncertain areas into focus.

Next, we examined the DALD magnitudes of the “eDALD-exclusive” pixels (those selected by $H$ but not by $E$). The table below shows how extreme their DALD values are relative to the global distribution:

On average, eDALD-exclusive pixels have a median DALD nearly 8× the dataset median, and even their 1 % percentile is 1.8× the global 99 %-tile. This confirms that DALD targets the heavy tail of the disagreement distribution, surfacing pixels of true epistemic uncertainty that entropy alone misses.

Finally, we evaluated how these shifts impact segmentation performance. The table below compares one-stage (pure uncertainty) versus two-stage (MaxHerding → uncertainty) mIoU on ADE-Bed:

While DALD alone matches entropy in a single stage (23.06 vs. 23.02 mIoU), integrating disagreement into our two-stage pipeline yields a +7.12 pp boost — much higher than the +5.25 pp from entropy. This demonstrates that the combination of coverage (MaxHerding) and disagreement (DALD) in two stages drives the largest gains and validates our uncertainty mechanism as both theoretically sound and practically impactful.

In summary, our empirical analysis shows that:

• DALD identifies high-uncertainty “needle-in-the-haystack” pixels that entropy misses.
• eDALD reshapes the selection ranking, not merely scaling entropy.
• The combination of coverage (MaxHerding) and disagreement (DALD) in two stages drives the largest gains, validating our uncertainty mechanism as both theoretically grounded and practically impactful.

2. Weighting Factor in eDALD

We chose equal weights ($\alpha=1$, $\beta=1$) to avoid introducing extra hyperparameters and to keep the balance between the mutual‐information term $I(Y;Z\mid x)$ and the confidence term $H(Y\mid z^{(0)},x)$. To validate this choice, we ran an ablation on CamVid (all other settings fixed) with several $(\alpha,\beta)$ pairs:

The equal‐weights setting ($1,1$) achieves the highest final‐round mIoU (36.14), confirming that our default choice is both simple and effective without requiring dataset‐specific tuning.

3. Overfitting Mitigation

We thank the reviewer for raising the important point of overfitting under extreme label scarcity. Several factors in our design help mitigate this risk:

1. Frozen backbone. We only train the lightweight segmentation head (0.79 M parameters for CamVid/Cityscapes; 1.61 M for ADE-Bed/Pascal-Context), while the diffusion model that produces the $6{,}144$-dim features remains fully frozen. This dramatically reduces the number of trainable weights and prevents the model from “memorizing” individual labeled pixels.
2. Fixed, high-level representations. Because each pixel’s input is a semantically rich feature vector extracted once by the backbone, the classifier sees only stable, high-level patterns rather than raw pixel noise. This further constrains the head’s capacity to overfit.
3. Regularization. We apply standard weight decay ($1\times10^{-5}$) and early stopping based on validation-set mIoU to halt training once performance plateaus.

Collectively, these measures keep our segmentation head's effective capacity in check, allowing it to generalize from as few as $N$ total labeled pixels (one per image) up to only $0.1\times N$ per round, without evidence of overfitting in our empirical learning curves.

4. Novelty of Our Method

We thank the reviewer for this observation. While kernel herding itself is a well‐studied coverage technique, our work advances it in three key ways that together constitute a novel contribution:

1. Scalable Local $\to$ Global Herding for Pixels. Classical MaxHerding over all $N \times H \times W$ pixels is intractable. We introduce a two‐stage pipeline — first selecting $K$ representatives within each image, then re‐herding those $N K$ candidates across images — to reduce complexity to minutes on tens of thousands of features. This adaptation makes herding practical for dense, per-pixel active learning in semantic segmentation.
2. Extreme Low‐Budget, Pixel‐Granular Setting. We tackle a novel regime — only one pixel per image per round (total budget $=1\times N$ pixels) — where neither whole-image AL nor conventional region-based methods suffice. Designing an AL strategy that yields strong mIoU under these constraints required rethinking both coverage and uncertainty at the pixel level.
3. Diffusion‐Native Acquisition (eDALD). Beyond herding, we propose a new uncertainty measure — Entropy‐Augmented DALD — that combines mutual information from stochastic multi‐scale diffusion features with a Shannon entropy term. Unlike BALD or MC‐Dropout, eDALD exploits the diffusion model’s intrinsic noise process, capturing epistemic uncertainty without Bayesian networks or multiple forward passes. To our knowledge, no prior work has introduced such an acquisition function for semantic segmentation.

Taken together, our two‐stage pipeline — scalable herding + diffusion‐based uncertainty — enables efficient, per‐pixel selection under extreme labeling constraints. Although each component draws inspiration from existing ideas, their integration in this context (pixel‐level AL, diffusion features, 1‐pixel/image budgets) and the demonstration of substantial segmentation gains represent a genuine methodological advance in active learning for dense prediction tasks.

I thank the authors for preparing the rebuttal. Most of my questions and concerns are addressed. Therefore, I intend to keep my original (positive) rating.

We thank the reviewer for thoughtful feedback and constructive comments. We will incorporate these suggestions in the revised manuscript to strengthen our presentation and clarify the practical benefits of our approach.

1. Practical Benefit of Pixel‐Level AL

Thank you for raising this important point. Region-level annotation can indeed offer click-efficiency — one click can label an entire superpixel or prompt‐generated region. However, it also introduces two key challenges:

1. Semantic Ambiguity: Precomputed regions (e.g., superpixels) can span multiple object classes or mix foreground and background. Fixing such errors often requires extra clicks, diminishing the “one-click” advantage.
2. Segmentation Overhead: Generating high-quality regions (e.g., via SAM or oversegmentation) adds compute and latency costs that may be nontrivial in practice.

Importantly, our two‐stage pixel‐level strategy is fully compatible with region‐based interfaces. Once informative pixels are selected:

• Superpixel Aggregation: Selected pixels can be grouped into regions and labeled with a single click.
• Prompt‐Based Propagation: Selected pixels can serve as prompts for models like SAM, automatically expanding to a full mask after one or two clicks.

This hybrid setup combines our method’s selection benefit (via diverse and uncertain pixels) with the labeling efficiency of region-based tools, enabling a highly click-efficient pipeline. We see this integration as a promising direction and will highlight it in the revised manuscript.

2. Dependence on Diffusion Pretraining Data

Thank you for this observation. We agree that closer domain alignment can further improve feature relevance, but it is not strictly required for our method to be effective:

• General‐Purpose Backbone: For CamVid, Cityscapes and Pascal-C, we employed a single DDPM model pretrained on ImageNet-256 (see Appendix B.1). Despite being street-scene datasets, the ImageNet-leveraged features still delivered consistent gains over all uncertainty and two‐stage baselines.
• Domain‐Specialized Backbone (ADE-Bed): ADE-Bed consists of indoor bedroom scenes, which differ substantially from ImageNet. Here, we used a DDPM pretrained on LSUN-Bedroom, achieving the best performance — suggesting that in highly specialized domains, a closer-aligned model can help.

Overall, our results show that loosely aligned diffusion features (ImageNet-256) suffice to outperform prior pixel‐level AL methods across benchmarks. In cases of major domain shift, switching to a domain-specific model (e.g., LSUN-Bedroom) is a simple and effective plug-in with no changes to our two-stage pipeline.

3. MaxHerding‐Only Performance

We agree that showing MaxHerding as a standalone 1-stage baseline is important. Below we present two tables:

1. Round-by-Round Progression
Mean mIoU over 10 AL rounds using MaxHerding only (no uncertainty):

Even without any uncertainty measure, MaxHerding’s coverage yields steady mIoU gains as more pixels are labeled.

2. Final mIoU Comparison
Final (round 10) mIoU of three approaches:

On ADE-Bed, CamVid and Cityscapes, our 2-stage eDALD substantially outperforms both 1-stage baselines, confirming the benefit of combining diversity and uncertainty. On Pascal-Context — where scenes are extremely diverse and there are 33 classes — MaxHerding alone slightly surpasses our full pipeline. We believe that in such settings, pure feature-space coverage can better capture rare or widely distributed classes without the noise introduced by uncertainty estimates.

These tables demonstrate that while MaxHerding alone is a strong representation-based baseline, adding eDALD uncertainty consistently boosts performance in most practical scenarios.

4. Method Comparison: Diversity-Based Strategies and Backbones

1 Diversity-based Algorithm Comparison
To validate that MaxHerding is not merely “reinventing” CoreSet in diffusion feature space, we plugged both diversity modules into our two‐stage pipeline (diversity $\to$ eDALD) under a fixed low‐budget ($b=0.1N$ pixels per round) and measured round‐by‐round mIoU on four benchmarks (we provided results on only two datasets due to space limit).

CamVid (2‐stage, $b=0.1N$)

ADE-Bed (2‐stage, $b=0.1N$)

Across all four datasets, MaxHerding yields substantially higher mIoU — especially in later rounds (R5–R10) — while CoreSet often saturates or even degrades. This confirms that MaxHerding provides stronger structural coverage in the diffusion‐feature space and better primes the second‐stage uncertainty selection. These results demonstrate that our diversity module (MaxHerding) is a competitive and effective alternative to existing diversity-based methods like CoreSet, particularly in the context of diffusion feature space and low-budget active learning.

2 Backbone‐Agnostic Two‐Stage Sampling
Our two-stage sampling strategy — first covering the data distribution, then selecting uncertain pixels — is backbone-agnostic and applicable to various architectures. However, the eDALD uncertainty measure is specific to diffusion models. Its DALD component estimates mutual information via the stochastic denoising process, leveraging diffusion’s unique properties to better capture epistemic uncertainty.

To test generality, we replaced the DDPM backbone with DeepLabV3 and substituted eDALD with eBALD (MC-Dropout BALD + entropy). Results on CamVid under a 10% pixel budget are shown below:

• On both backbones, 2‐stage variants improve by $\sim4$–$5$ pp over 1‐stage baselines, confirming the general benefit of separating diversity and uncertainty.
• Diffusion + eDALD (35.73 %) remains the best, showcasing diffusion’s added representational power.
• DeepLabV3 + 2‐stage Entropy (31.54 %) outperforms its 1‐stage counterpart by over 5 pp and approaches the diffusion model’s 2‐stage Entropy result — demonstrating broad applicability of our pipeline.

This demonstrates that our two‑stage approach also works effectively on a standard CNN backbone, while diffusion representations combined with eDALD yield the largest gains.

5. DALD Performance

We agree that the substantial performance gain from adding an entropy term to DALD is somewhat surprising. However, as discussed at the end of Section 3.3 (lines 215–228), this difference stems from the distinct roles of DALD and entropy.

DALD captures disagreement between predictions on noised versions of the same input. This disagreement serves as a proxy for informativeness, as high disagreement typically indicates areas where the model is less confident or more sensitive to input perturbations.

However, some informative pixels may not exhibit high disagreement, especially when the model remains consistently uncertain across all noise levels. In such cases, the entropy term captures uncertainty that DALD may miss. By combining both signals, eDALD better identifies a broader range of informative pixels, leading to improved performance.

We appreciate the reviewer bringing this up, as it highlights the complementary nature of disagreement and uncertainty in our framework.

6. Novelty of Our Method

In response to the reviewer’s question, we identified two recent diffusion-based active learning (AL) works:

• Barba et al. [1] apply a diffusion model for AL in CT reconstruction, selecting measurements via posterior variance on image reconstructions.
• Du et al. [2] use latent diffusion for active domain adaptation, sampling unlabeled domains via diffusion‐driven uncertainty tests.

Both focus on image-level selection in specialized settings (inverse problems or domain shift), whereas our method is the first to tackle pixel-level AL for semantic segmentation under extremely low budgets, combining:

1. Diffusion‐based multi-scale feature herding to form a diverse candidate pool,
2. DALD/eDALD, a diffusion-specific mutual‐information plus entropy criterion,
3. Per-pixel annotation with only 0.1 click per image per round.

These aspects — pixel granularity, two-stage design, and practical segmentation gains — distinguish our work from prior diffusion-driven AL methods. If the reviewer is aware of any diffusion-based AL methods directly addressing supervised segmentation, we would greatly appreciate the reference and will include a comparison in a future revision.

[1] Barba et al., "Diffusion Active Learning: Towards Data-Driven Experimental Design in Computed Tomography," arXiv:2504.03491.
[2] Du et al., "Diffusion-Based Probabilistic Uncertainty Estimation for Active Domain Adaptation," NeurIPS 2023.

Thanks for your rebuttal. There are still a few questions I have about your work.:

Practical Benefit of Pixel‐Level AL: One issue I still have is that it's not clear how impactful your work will be in the field of active learning. For example, if you were able to show (quantitatively) that pixel-level AL was more effective than region-level AL, or that your method could also be applied to region-level AL, and you could demonstrate use cases in which a particular type of AL would be more beneficial, then this would make your work much more impactful. While you do propose an interesting method, it's unclear what the real-world impact will be.

Coreset Results: These results look very weird to me. If I'm understanding the method correctly, every round you are adding 10% of the overall points? Then, how is the model performing so poorly in comparison to the other model with 90% of the data? Additionally, it would be good to evaluate Coreset using different embeddings (diffusion-based embeddings, DeepLabV3, etc.) alongside the entropy-based methods, in addition to comparing it to MaxHerding. It would also be helpful to add MaxHerding as one of your baseline methods. You could group these methods as your "diversity-based" methods when comparing against other baselines.

Additional backbone results: I think if you had at least one more backbone, this would be more substantial evidence that your diffusion-based backbone was better. Honestly, two more popular backbones would be great.

DALD Performance: Is there another way to explain the performance boost? It's not immediately clear to me why there is such an increase in performance. Perhaps an empirical or theoretical argument would be helpful.

Thank you for your thoughtful review and insightful comments. We appreciate the time you took to evaluate our work and the constructive suggestions you provided. We have addressed each of your concerns in detail below and will incorporate all accepted changes into the final version.

1. Click‐Cost and Compatibility with Region-Based Labeling

Thank you for the insightful suggestion on click-cost comparisons. While region-level AL can annotate a large area with a single click, it introduces two key challenges:

1. Annotation ambiguity. Automatically-generated regions (e.g., superpixels or SAM masks) may span multiple objects or miss thin structures, making class labels ambiguous and reducing annotation quality.
2. Inference overhead. Generating accurate regions — whether via SAM prompts or superpixel algorithms — requires additional model inference or preprocessing steps, which can outweigh the savings from fewer clicks.

Our two-stage sampler (MaxHerding → eDALD) selects $B$ pixel seeds per round that are both diverse and epistemically uncertain. When these seeds are fed into any region-proposal tool, they act as high-value “anchor clicks,” guiding the tool to produce masks that align more precisely to object boundaries and small classes. In practice, this hybrid workflow can:

• Reduce ambiguity by seeding proposals across the entire scene manifold — avoiding over-concentration on a few high-uncertainty patches.
• Improve mask fidelity for thin structures and adjacent objects, since seed diversity and uncertainty refinement correct common region-proposal failures.
• Leverage future advances in region-proposal quality — our pixel-level acquisition serves as a drop-in enhancement for any click-based model, yielding greater synergy as region-segmentation backbones improve.

We view this integration as a promising direction: it maintains the low click-cost of region-based AL while delivering more precise, unambiguous annotations under extreme budgets.

2. Low Absolute mIoU

Thank you for raising this important point. To the best of our knowledge, we are the first to study active learning for semantic segmentation in an extreme low-budget regime — labeling only $0.1\times N$ pixels per round for 10 rounds (total budget $=1\times N$) — a setup that brings annotation cost down to truly practical levels. Under these constraints, even basic uncertainty sampling (e.g., BALD, Entropy) yields unusable models, whereas our two-stage pipeline (MaxHerding → eDALD) recovers a substantial fraction of full-data performance.

For example, on ADE-Bed the fully supervised mIoU is $45.58\%$ (Appendix C.4), while our two-stage eDALD achieves $35.68\%$, closing nearly a $10\%$ gap with only one labeled pixel per image. This demonstrates that — in contrast to prior AL work using per-round budgets of $5\times N$ or more — we can achieve high-quality segmentation with an order-of-magnitude fewer annotations.

Moreover, our framework naturally supports an adaptive querying transition:

• Early rounds: enforce diversity + uncertainty to build a broad, informative seed set.
• Later rounds: shift focus to rare classes or boundary refinement to push toward higher mIoU.

Following the spirit of SelectAL [1], one could define a budget threshold at which to switch from low- to high-budget acquisition strategies, ultimately approaching the 95% recovery mark as labeling resources grow. We will clarify this pathway in the revision.

[1] O. Hacohen and D. Weinshall, “How to select which active learning strategy is best suited for your specific problem and budget,” NeurIPS 36, 2023.

3. Theoretical Assumption on $Y \perp X \mid Z$

Thank you for the constructive feedback. As noted in Shah and Peters [1], however, conditional independence testing is known to be extremely challenging to perform reliably. More importantly, our assumption of $X \rightarrow Z \rightarrow Y$ aligns with a common modeling choice in probabilistic graphical models, where $X$ is an input variable, $Z$ represents latent or feature variables, and $Y$ is a label variable. Here, the stochasticity in $Z$ arises either from noise in the input or from the model itself e.g. dropout. A wide range of neural networks with inherent stochastic behavior is based on this assumption.

While it is possible to test for the additional direct effect of $X$ on $𝑌$ given $Z$, for instance, by incorporating a parallel deterministic model to represent $X \rightarrow Y$, such an ensemble structure tends to yield only marginal improvements in practice. This is especially true when the base model (in our case, UNet) is already large, as the benefits of ensembling tend to diminish with increasing model capacity.

Therefore, given the capacity of modern large models including the one used in our work, it is reasonable to assume the causal relation $X \rightarrow Z \rightarrow Y$. However, we acknowledge that rigorously validating this assumption is extremely challenging in practice. If the reviewer is aware of a principled method for doing so, we would greatly appreciate the suggestion and are willing to incorporate it into our analysis.

[1] Rajen D Shah, Jonas Peters, The Hardness of Conditional Independence Testing and the Generalised Covariance Measure.
[2] Taiga Abe, E. Kelly Buchanan, Geoff Pleiss, Richard Zemel, Deep Ensembles Work, But Are They Necessary?, NeurIPS 2022.

4. Pretraining Data for Diffusion Models

These pretraining details are provided in Appendix B.1. In summary, the diffusion backbone for ADE-Bed was trained on the LSUN-Bedroom dataset, and for CamVid, Cityscapes, and Pascal-Context we used a class-unconditional ImageNet-256 diffusion model. Crucially, none of these pretraining datasets overlap with our active-learning benchmarks, so all reported gains come from true out-of-domain feature transfer.

5. Weighting Between $I$ and $H$

We chose equal weights ($\alpha=1$, $\beta=1$) to avoid introducing extra hyperparameters and to keep the balance between the mutual‐information term $I(Y;Z\mid x)$ and the confidence term $H(Y\mid z^{(0)},x)$. To validate this choice, we ran an ablation on CamVid (all other settings fixed) with several $(\alpha,\beta)$ pairs:

The equal‐weights setting ($1,1$) achieves the highest final‐round mIoU (36.14), confirming that our default choice is both simple and effective without requiring dataset‐specific tuning.

6. Minor Remarks

Thank you for these helpful suggestions. We will incorporate the following clarifications in the revision:

• Line 68 (“outliers”) We will replace “outliers” with a precise description: pixels whose predicted class probabilities are extreme or erratic — e.g., isolated noise artifacts or very rare patterns — resulting in high uncertainty yet lying far from the true decision boundary and contributing little to model improvement.

• Line 158 (set notation) We agree that the notation can be tightened. We will clarify that

  “Let $I_{w,h}$ denote the pixel at coordinates $(w,h)$ in image $I$. Then $\lbrace I_{w,h} \mid 1\le w\le W, 1\le h\le H\rbrace$ enumerates all pixels of $I$, and writing $x \in \lbrace I_{w,h}\mid\cdots\rbrace$ simply means that $x$ is one such pixel.”

• Line 185 ($M$ vs. $b$) We will explicitly note that

  “$M$ is the size of the global candidate pool produced by MaxHerding in Stage 1, whereas $b$ is the per-round labeling budget in Stage 2; they serve different roles and generally take different values.”

• Table 1 (interpretation of numbers) We will amend the caption to read:

  “Table 1: Mean Intersection-over-Union (mIoU, %) of single-stage vs.\ two-stage selection on CamVid. ‘UC Only’ is one-stage uncertainty; ‘Herding → UC’ adds representation filtering.”

Dear Reviewer xSso,

Thank you for the time and effort you have put into reviewing our paper. We greatly appreciate your constructive feedback, which helped us strengthen the work. In our rebuttal, we aimed to address all of your main questions (not limited to):

1. Click-Cost and Compatibility with Region-Based Labeling: We provided additional explanation describing the limitations of region-based methods and potential workflow of integration of the region-proposal tools with our two-stage sampler.
2. Theoretical Assumption (X conditionally independent of Y given Z): We clarified the commonality of this assumption, with hardship of empirical demonstration.
3. Ablation on Weighting Between I and H: We included new experiments quantifying the impact of varying the weighting parameters $\alpha, \beta$, to provide further insight for the impact of different weights.

Demonstration of Compatibility with Region-Based Labeling

During the discussion phase, we conducted an additional experiment to directly demonstrate the compatibility of our method with region-based labeling, as part of our response to Reviewer V27y.

We integrated SAM [1] to generate regional labels: our method first selects a pixel for annotation, and the label is then expanded to the entire region identified by SAM as matching that pixel. The following results compare performance with and without SAM on three datasets, all using our two-stage setting with $b=0.1N$.

CamVid (2‐stage, $b=0.1N$)

ADE-Bed (2‐stage, $b=0.1N$)

Cityscapes (2‐stage, $b=0.1N$)

Our experiments across three diverse datasets reveal a consistent and notable advantage of the hybrid eDALD + SAM approach: accelerated learning in the early-to-mid stages. This demonstrates its utility in substantially reducing the annotation time and cost required to reach a strong performance baseline. For instance, across all datasets, the SAM-integrated method consistently establishes a clear performance lead by the middle rounds (R4–R6), achieving levels of mIoU that the baseline method requires several additional rounds to reach.

Regarding the final performance, the benefits of this synergy are most pronounced on CamVid, where the integration of SAM led to a +4.16 pp gain, reaching a final mIoU of 39.81%. We attribute this significant gain to the high-quality masks generated by SAM for CamVid's well-defined objects, which provides a superior supervision signal. In contrast, for the more complex scenes in ADE-Bed and Cityscapes, the primary benefit remains the accelerated convergence, with the final performance being comparable to or presenting a trade-off with the baseline. In summary, this analysis validates our hybrid workflow as a highly cost-effective and powerful strategy, directly addressing the practical impact of our work.

[1] Kirillov, Alexander, et al. "Segment anything." Proceedings of the IEEE/CVF international conference on computer vision. 2023.

We would be grateful if you could let us know whether these clarifications resolve your earlier concerns, and whether there are remaining points you feel still need further discussion. We are happy to elaborate on any part that could be clearer. We also note that other reviewers (gmyd, V27y) have considered our rebuttal and raised their scores following the additional clarifications and experiments. We would be grateful if you might similarly re-evaluate our work based on these responses and the new evidence presented here.

Thank you again for your thoughtful review and valuable input.

Thank you for your thorough review and insightful feedback. We appreciate the time and effort you invested in evaluating our work and your constructive suggestions. If our paper is accepted, we will gladly incorporate all of your recommendations to improve clarity, rigor, and impact.

1. Statiscal Significance

As suggested, we report mean $\pm$ standard deviation over three seeds to reflect the statistical significance of our mIoU improvements. The results show MaxHerding yields consistent gains across uncertainty measures.

The table below shows two key effects on CamVid: (1) how our DALD objective improves performance over BALD, and (2) how MaxHerding further boosts performance over uncertainty-based (UC) acquisition alone. Gains are reported in absolute points and percentages. Similar improvements are observed from DALD and MaxHerding as in single-seed results.

The next table reports mIoU (%) under the same low-budget regime (10 rounds). It shows similar trends to those reported in Table 2 of the manuscript: the two-stage eDALD consistently outperforms all baselines across multiple datasets.

2. Computational Cost

Below is a detailed breakdown of per-round wall-clock time and peak GPU memory for each method on Cityscapes (N=2,975, K=15 ⇒ M=44,625), measured on a single NVIDIA A100 40 GB.

On diffusion backbones, the two-stage pipeline takes ~80 minutes per round, with feature extraction and training each taking about half that time, and herding ~25%. MaxHerding adds 12–14 minutes (≈20–25%), which remains manageable within the overall cycle.

Memory usage is dominated by global herding over 6,144-dim diffusion features, peaking at 33.5 GB; feature extraction also demands over 25 GB. In contrast, DeepLab-based methods finish in under 10 minutes per round and stay below 2 GB, illustrating a clear efficiency–accuracy trade-off.

Overall, while diffusion backbones are more demanding, the herding overhead is modest, making the method practical for large-scale data given adequate GPU resources.

3. Implementation Details and Hyperparameter Sensitivity

1. RBF bandwidth $\sigma$
We set the Gaussian‐kernel bandwidth $\sigma$ via the median heuristic: subsample 1% of candidate features, compute all pairwise distances, and take their median. To test robustness, we also evaluated the 30th, 40th, 60th, and 70th percentiles of the same distance distribution. Across all five settings, final‐round mIoU varied by $\le0.7$ pp, with the median choice giving the best performance. This confirms that our method is insensitive to the exact $\sigma$ and that the median heuristic is a reliable, data‐driven default.

2. MaxHerding hyperparameters ($K$, $M$)
In Section C.2 of the supplementary, we report an ablation on per‐image candidates $K \in \lbrace 20,50,100 \rbrace$ and global pool size $M \in \lbrace 0.25M_0, 0.4M_0, 0.5M_0 \rbrace$, where $M_0 = N \times K$. On CamVid, final mIoU fluctuates by at most 2.8 pp across all nine configurations, and our default ($K=100$, $M=0.5M_0$) lies near the top. This demonstrates that our two‐stage pipeline is not overly sensitive to these choices, and that smaller $K$ or $M$ can even improve performance in some cases — helping trade off compute versus accuracy.

3. Number of diffusion timesteps $T$
Following LEDM [15], we use $T=${50,150,250} by default. To isolate the effect of $T$, we ran eDALD on CamVid with $T=${50}, {50,150}, and {50,150,250}.

The final‐round mIoUs were:

• {50}: 30.5
• {50,150}: 36.7
• {50,150,250}: 37.1

Adding each extra timestep increases mIoU by 6—7 pp, confirming that our three‐step combination best balances feature richness versus dimensionality cost. While each additional step linearly raises feature dimension, the resulting gains in query diversity and classifier learning justify the modest extra memory and compute.

4. Applicability Beyond Diffusion Backbones

We acknowledge that parts of our framework — particularly eDALD uncertainty measure — are specific to diffusion models. The DALD term estimates mutual information from the stochastic denoising process unique to diffusion backbones. We introduced it to better exploit their inherent stochasticity and capture epistemic uncertainty. However, the two‐stage sampling pipeline (representation‐first $\to$ uncertainty‐second) is entirely backbone‐agnostic.

To validate this, we applied the same two‐stage flow to DeepLabV3 by replacing eDALD with eBALD, where mutual information is estimated via MC‐dropout. We then compared 1‐stage vs. 2‐stage eBALD on both DeepLabV3 and our diffusion model:

• Representation stage (MaxHerding) improves performance on both backbones (gains of 1.7–6.1 pp), confirming its generality.
• While diffusion yields the best results due to richer self-supervised features, the DeepLabV3 + 2-stage pipeline still outperforms its 1-stage variant by ~1.8–2.1 pp, especially on Cityscapes.

In conclusion, any feature extractor (ViT, CNN, etc.) can slot into our two‐stage framework: swap in an uncertainty estimator suited to that backbone (e.g., MC‐dropout BALD for CNNs) for Stage 2, and retain MaxHerding for Stage 1.

5. Novelty of Our Method

Our focus has been on building an effective yet efficient framework that incorporates both coverage and uncertainty through diffusion-based features, under realistic resource limits.

Thank you for the thoughtful comment. We are aware of hybrid active learning methods such as Weighted $k$-means [1], ALFA-Mix [2], and BADGE [3], which combine coverage and uncertainty. However, to our knowledge, none adopt a two-stage coverage-then-uncertainty strategy tailored to neural network-based active learning. If the reviewer knows of such a method, we would greatly appreciate the reference and would be happy to compare.

We excluded the above methods due to computational constraints. Specifically, [1] and [3] perform $k$-means clustering over all dataset pixels, which is impractical even with multiple GPUs. ALFA-Mix [2] requires pixel-wise similarity computations, which are also costly at scale. In general, many hybrid methods are hard to scale to pixel-level tasks like semantic segmentation.

Our focus, therefore, has been on building an effective yet efficient framework that incorporates both coverage and uncertainty through diffusion-based features, under realistic resource limits.

[1] Zhdanov et al., "Diverse mini-batch active learning", arXiv:1901.05954, 2019.
[2] Parvaneh et al., "Active Learning by Feature Mixing", CVPR 2022.
[3] Ash et al., "Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds", ICLR 2020.

6. Pixel Selection

Thank you for pointing this out. We qualitatively analyzed the spatial distribution of selected pixels across multiple benchmarks; see Fig. 3 and 8.

The baseline method (Margin) often focuses on object boundaries and selects redundant pixels within the same class region, limiting structural and semantic diversity.

In contrast, our two-stage method first uses MaxHerding to ensure diverse coverage in feature space, then selects epistemically uncertain pixels via eDALD. This results in broader spatial and semantic coverage, providing richer supervision signals. This improved diversity aligns with our quantitative results, where our method consistently achieves more stable and higher mIoU across datasets.