Enhancing Deep Batch Active Learning for Regression with Imperfect Data Guided Selection

We sincerely appreciate your positive feedback highlighting our innovative yet intuitive approach to leveraging auxiliary data in active learning, along with your recognition of our rigorous theoretical analysis, clear presentation, and thorough experimental validation including detailed ablation studies. Below we address all comments to further strengthen these contributions.

W1&Q1: We sincerely appreciate the reviewer's insightful comment regarding the comparison with transfer active learning methods. Our initial submission lacked explicit discussion of this important aspect, and we thank the reviewer for bringing this to our attention.

• Our work differs from transfer learning approaches like [1] in the definition of auxiliary data:
  • transfer learning and domain adaptation assume covariate shift with different $X$ distribution but consistent distribution of $P(Y\mid X)$ between source and target domains [1];
  • our setting considers imperfect auxiliary data where both $P(X)$ and $P(Y\mid X)$ may differ.
• This distinction is crucial because standard transfer methods could suffer from negative transfer under such joint distribution shift. To validate this empirically, we compared directly combining target and auxiliary data versus our approach, with results of our approach demonstrating superior performance (reported in Table 3 of our original submission).
• We agree that exploring connections to transfer active learning methods would be valuable, and we will include this discussion in the revised version, along with potential extensions combining our method with transfer techniques when the $P(Y\mid X)$ consistency holds. This clarification will better position our contribution relative to prior work.

W2/Q3: We sincerely appreciate the reviewer's thoughtful question regarding the practical implications of our NTK-based analysis.

• While our current theoretical analysis employs the infinite-width neural network approximation for tractability, we note that prior work [2] has established rigorous connections between finite-width networks and their neural tangent kernel behavior, suggesting our theoretical insights could be extended to practical settings. However, such extension would require substantial additional technical details.
• The infinite-width analysis serves to provide theoretical justification for our approach, while empirical results demonstrate the method's performance with finite-width networks across multiple real-world datasets. We agree that further investigation of finite-width cases would be valuable and will consider this direction in future work.

W3/Q2: We greatly appreciate the reviewer's important question regarding the computational efficiency of our method.

• Our approach introduces computational overhead through density ratio estimation and auxiliary loss computation, but we mitigate this through careful design choices:
  • employing efficient machine learning methods for density ratio and loss estimation;
  • allowing flexible control of auxiliary dataset sizes.
• The selection period maintains the same computational complexity as BMDAL.
• Empirical results demonstrate that our method achieves meaningful performance improvements even with modest auxiliary data sizes (100, 500, 1000 samples), where the additional computational overhead remains negligible. With only 100 auxiliary data, the maxdet selection strategy, which performs best among the compared approaches, still outperforms BMDAL.
• All experiments were conducted on a standard server (dual Intel Xeon Gold 6330 CPUs, 125GB RAM), with detailed time/memory comparisons provided in the table below. The performance gains consistently justify the modest computational cost, particularly in active learning settings where annotation costs typically dominate computation costs.

We will include the computational analysis in the revised version.

We sincerely appreciate your thoughtful and constructive feedback. Your observations have helped pinpoint several significant areas for future study. We are more than happy to answer any further questions during the rebuttal period.

Reference

[1] Wang, Xuezhi, Tzu-Kuo Huang, and Jeff Schneider. "Active transfer learning under model shift." International Conference on Machine Learning. PMLR, 2014.

[2] Novak, Roman, Jascha Sohl-Dickstein, and Samuel S. Schoenholz. "Fast finite width neural tangent kernel." International Conference on Machine Learning. PMLR, 2022.

We are grateful for your valuable assessment of our work's significance and mathematical soundness, along with your appreciation of the core idea's originality and the encouraging experimental results. Below we provide detailed responses to all comments to further strengthen these aspects.

W1: Thank you for your thoughtful question regarding the theoretical guarantees of our method.

• Our theoretical result builds upon NTK theory to establish that in infinitely wide networks, the estimation error $\phi_2^{(2)}$ at each point $x$ stems from the variance $\sigma^2(x)$ of NTK kernel regression on the target sample set. The key insight is that when we have two low-correlation estimates $\phi_2^{(2)}$ and $\widetilde{\phi}_2^{(2)}$ derived from this variance, the variance of their difference becomes approximately proportional to $\sigma^2(x)$. This allows us to use the difference between two estimates as a proxy for estimation error.
• The theoretical guarantee holds under practical conditions where (i) the auxiliary data provides reasonably accurate estimates (ensured by prior knowledge about information relevance), and (ii) the auxiliary data is either independent or exhibits low correlation with the target data. These conditions are satisfied by our definition of auxiliary data.

W2: Thank you for your insightful comment regarding the applicability of our theoretical results. While our current analysis is presented in the infinite-width regime, we note that theoretical treatments of finite-width neural tangent kernels already exist in prior work [1], which contains extensive technical derivations. Our analysis could be extended to the finite-width case following [1], but this would introduce substantial additional complexity. Since our primary contribution focuses on the active learning framework rather than NTK theory itself, we have chosen to present the cleaner infinite-width case. We will consider expanding this theoretical aspect in future work.

W3: We sincerely appreciate the reviewer's observation regarding the loss function used in our analysis.

• While our theoretical development employs the $L_2$ loss, we emphasize that our core methodology is fundamentally applicable to any differentiable loss function.
• The $L_2$ loss provides superior mathematical convenience due to its smooth, everywhere-differentiable nature and its analytical tractability.
• In contrast, non-smooth losses like $L_1$ (whose gradient is discontinuous) introduce significant analytical challenges that would obscure our primary contributions. However, the proposed active learning framework itself remains valid for general loss functions in practice.

Q1: We are grateful for the reviewer's careful attention to this technical detail.

• The equality in lines 154-155 requires $\hat{r}$ to be independent of $X_{i}^{\prime},Y_{i}^{\prime}\sim Q$ to ensure the objective of argmin can be expressed as a sum of independent terms.
• The assumption is practically reasonable because we assume the availability of abundant auxiliary data. In practice, we can split the auxiliary data into two parts: one subset for estimating the density ratio $\hat{r}$ and another independent subset for loss estimation. This data-splitting strategy satisfies the independence requirement.

Q2: Thanks for raising this detail.

• The training set $L_t$ contains a finite number of samples $\mathbb{N}$, but algorithm $\mathcal{A}$ can incorporate arbitrarily many samples. The current expression was chosen for simplicity. Also, to maintain notational consistency, we require an initialization parameter for the model.
• The initial parameters can be either 0 or randomly generated from a given distribution. This initialization has two purposes: (1) providing a starting point for the NTK parameters, and (2) following standard machine learning practice where parameter initialization is required for training procedures.

Q3: We are not certain which specific expectation the reviewer is referring to. However, if the question concerns the expectation in Equation (1), we note that this operation is formally defined in Line 82 of our paper. The expectation appears because we aim to optimize the model parameters with respect to the true risk $R$ over distribution $P$, whose gradient direction requires this expectation. In practice, since we cannot compute this expectation directly, we resort to empirical estimation. This approach aligns with standard practice in statistical learning theory, where population objectives are approximated through finite samples.

Q4: The assumption that $\rho$ should be small stems from our core methodological design where we use the difference between two estimates as a proxy for estimation error (as explained in W1). For this approach to be valid, the two estimates must exhibit sufficiently low correlation, hence we require $\rho$ to be small. When $\rho=1$, the estimates become identical, making their difference vanish . This degenerate case provides no meaningful information.

Q5: $\hat{\phi}_2(\hat{\theta},x_0)$ is the estimated loss gradient using auxiliary data. As long as the estimator $\hat{\theta}$ converges to the optimal parameter and the density ratio estimator converges to the true density ratio, the limit of $\mathbb{E}[\hat{\phi}_2(\hat{\theta},x_0)]$ converges to $0$.

Q6: In Table 1, Area Under the Curve (AUC) metric is employed to evaluate the overall convergence efficiency of different active learning methods. As specified in line 201 of our paper, the AUC metric is calculated by $\text{AUC}=\sum_{i=1}^T(\xi_{i-1}+\xi_i)/2$, where $\xi_0,\ldots,\xi_T$ represents the sequence of average MSE values obtained by each method. In this context, a lower AUC value indicates superior performance, corresponding to smaller prediction errors and more effective sample selection. Conversely, when evaluating based on accuracy curves (where higher values are desirable), the interpretation of AUC values would be inverted.

We sincerely appreciate your thoughtful and constructive feedback. Your questions have opened up a broader space for discussion. We are more than happy to answer any further questions during the rebuttal period.

Reference

[1] Novak, Roman, Jascha Sohl-Dickstein, and Samuel S. Schoenholz. "Fast finite width neural tangent kernel." International Conference on Machine Learning. PMLR, 2022.

Thank you for your positive assessment of our intuitive approach to leveraging auxiliary data for uncertainty estimation, the clear presentation of motivations and key concepts, the well-founded theoretical analysis, and the comprehensive literature review. Below we address all comments and provide additional results.

W1: We sincerely appreciate your valuable feedback regarding the computational complexity analysis of our method.

• While our approach requires processing auxiliary data through density ratio weighting, we balance computational efficiency through two design choices:
  • using lightweight machine learning methods (e.g., random forests);
  • controlling the size of auxiliary datasets.
• The core selection mechanism preserves the same complexity as gradient-only methods. The additional overhead comes from the density ratio estimation and auxiliary loss computation.
• Our experiments shown in table below demonstrate that both the required resource and the performance improvement scale with the auxiliary data size. We conducted experiments across 5 real-world datasets with varying auxiliary dataset sizes (100, 500, 1000, 10,000 samples) on a server with dual Intel Xeon Gold 6330 CPUs (112 threads total) and 125GB RAM.
• Even with only 100 auxiliary data, the maxdet selection strategy, which performs best among the compared approaches, still achieves meaningful performance improvements. For auxiliary datasets below 1,000 samples, the additional time and memory overhead remain negligible.
• This allows us to flexibly balance performance and computational cost by adjusting the auxiliary data size according to their specific requirements. The consistent improvements over gradient-only baselines justify the additional computation, particularly since active learning typically operates in label-efficient regimes where query costs dominate computational costs.

We'll add these discussions in the revised version.

W2: We sincerely appreciate the reviewer's suggestion to compare with uncertainty-based active learning methods.

• As shown in the table below, both deep ensemble and dropout methods underperform random sampling in our experiments, while AGBAL demonstrates consistent improvements (in Table 1 of our origin manuscript).
• This observation aligns with findings in [1], which show that pure uncertainty batch sampling without diversity considerations can perform worse than random selection. The limitations of batch uncertainty-based approaches stem from several factors:
  1. when selecting batches based solely on uncertainty, the chosen samples may cluster in localized high-uncertainty regions while neglecting other informative areas of the input space [1];
  2. the fundamental assumption that high uncertainty necessarily correlates with prediction errors does not always hold in practice, particularly when model uncertainty estimates are miscalibrated [2];
  3. the quality of uncertainty estimates is highly sensitive to architectural choices and hyperparameters (e.g., dropout rates, ensemble size) [2,3], requiring careful tuning that may not generalize across different datasets.
• In contrast, our gradient-based approach directly optimizes the risk minimization objective, making it more robust to dataset characteristics.

Q1: We sincerely appreciate this insightful question regarding the key properties of effective auxiliary datasets for our method.

• Currently, the selection of suitable auxiliary data relies on prior knowledge, and our experimental setup in Appendix A.6.1 provides concrete examples of how we partition target and auxiliary datasets.
• Through empirical observations, we find that higher-quality auxiliary datasets exhibit smaller variance in the estimated density ratios $\hat{r}(x',y')$. This can be understood through the lens of distributional shift: when target and auxiliary distributions are identical (0% shift), $\hat{r}(x',y')=1$ with zero variance. As the distributional discrepancy increases, the density ratios deviate more substantially from 1, leading to greater variance. However, this is only an empirical observation, and developing more rigorous quantitative measures for assessing auxiliary data utility is an important direction for future work.

Q2: We sincerely thank the reviewer for highlighting this important question.

• The size of the auxiliary dataset plays a crucial role in balancing performance and computing efficiency in our approach. The results of the experiment in the W1 table demonstrate that a larger auxiliary data set leads to both better performance and more resource consumption.
• Notably, even minimal auxiliary dataset (100 samples) improves performance under our maxdet selection strategy, which performs best among the compared approaches. The additional time/memory incurred by auxiliary datasets below 1,000 samples is negligible.

We sincerely appreciate your thoughtful and constructive feedback. The issues you raised have brought to light important questions. We are more than happy to answer any further questions during the rebuttal period.

Reference

[1] Settles, Burr. "Active learning literature survey." (2009).

[2] Tsymbalov, Evgenii, Maxim Panov, and Alexander Shapeev. "Dropout-based active learning for regression." International conference on analysis of images, social networks and texts. Cham: Springer International Publishing, 2018.

[3] Beluch, William H., et al. "The power of ensembles for active learning in image classification." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.

We sincerely thank the reviewer for their positive assessment recognizing the novelty of our density ratio-weighted gradient method for regression uncertainty, the theoretical rigor of our NTK analysis, and the empirical effectiveness across diverse datasets. Below we address all comments and provide additional results.

W1&Q1&Q4: Thank you for your thoughtful comment regarding the dependency on auxiliary data quality. We appreciate your attention to this important aspect of our method.

• As shown in Figure 2, the performance degrades only linearly even when the parameter $\zeta$ controlling the distribution shift increases exponentially. This suggests that our approach is not overly sensitive to moderate mismatches in auxiliary data quality.
• We acknowledge that when auxiliary data is poorly aligned with the target distribution (e.g., having completely disjoint support), the estimated loss may become uncorrelated with the true loss. In such extreme cases, if the estimated loss is small where the true loss is large, this could indeed misguide the acquisition function.
• Currently, we determine whether the auxiliary data is suitable based on prior domain knowledge. We fully agree that developing quantitative measures to evaluate auxiliary data quality would be valuable to prevent such adversarial scenarios. This is an important direction we plan to explore in future work including both auxiliary data quality measurement and high-quality auxiliary data selection.

W2&Q2: Thank you for this thoughtful question regarding the relationship between our method and importance sampling or domain adaptation weighting schemes.

• Our method differs from standard domain adaptation in two key aspects:
  1. most domain adaptation methods assume a different distribution of $X$ but consistent distribution of $Y\mid X$ across domains (covariate shift setting)
  2. in the setting of our paper, we assume arbitrary distribution shifts between auxiliary and target data (including both covariate and label shifts), due to the corrupted nature of the auxiliary data. This makes traditional covariate weighting schemes unsuitable for loss estimation in our setting.
• On the other hand, while the estimation of $\phi_2$ indeed incorporates importance sampling principles, our methodology extends beyond conventional implementations. Specifically, we utilize the auxiliary data's joint distribution over both covariates and labels as our proposal distribution for target loss estimation, rather than relying solely on marginal covariate distributions.

W3&Q3: Thank you for raising this important concern about potential bias in the labeled target set $L_t$. This is an insightful observation about our density ratio estimation procedure.

• First, your point is particularly relevant, and we would like to clarify that our method's design naturally accommodates this form of bias. While active learning does indeed induce a shift in the covariate distribution of $L_t$ (as certain examples are preferentially selected), the conditional distribution of $Y\mid X$ remains unaffected since we always observe the true labels for selected points.
• Crucially, as shown in Eq. (1) of our origin paper and the derivation between lines 127-128, our estimation of $\phi_2$ depends only on the ratio $r(Y\mid x)$ of conditional distributions rather than the marginal covariate distributions. This means that while the composition of $L_t$ may be biased in terms of $x$, this bias does not compromise the validity of our density ratio estimates for the purpose of loss reweighting, since the key quantity $r(y\mid x)$ can still be accurately estimated from the available data.

W4: Thank you for raising this important point about model dependency in our auxiliary uncertainty estimation.

• We acknowledge that our method is model-dependent and that errors in the trained auxiliary model will propagate into the selection process. However, we emphasize that these estimation errors are correlated with the true loss, which provides meaningful signal for sample selection.
• Our empirical results demonstrate that simple models like random forests can provide meaningful uncertainty estimates for effective sample selection. Crucially, when using appropriate auxiliary data, the selection performance with these imperfect estimates still outperforms the BMDAL baseline that ignores this term (by setting it to 1).

W5: Thank you for raising these important concerns regarding computational complexity and scalability.

• Our method introduces additional components (density ratio estimation and auxiliary loss estimation) compared to gradient-only approaches BMDAL. However the selection kernel itself remains unchanged; we only modify the features used for selection. The primary computational overhead stems from processing auxiliary data. For efficiency, lightweight machine learning methods (e.g., random forests) can be adopted for density ratio and loss estimation without compromising performance.
• To illustrate scalability, we conducted experiments across 5 real-world datasets with varying auxiliary dataset sizes (100, 500, 1000, 10,000 samples) on a server with dual Intel Xeon Gold 6330 CPUs (112 threads total) and 125GB RAM.
• The results are shown in the table below. Even with only 100 auxiliary data, the maxdet selection strategy, which performs best among the compared approaches, still achieves meaningful performance improvements. The gain grows with larger auxiliary datasets. Crucially, for auxiliary datasets below 1,000 samples, the additional time and memory overhead remains negligible.
• In practical applications, we can choose the auxiliary data size based on their specific needs for either better performance (larger auxiliary sets) or faster computation (smaller auxiliary sets).

We'll add these discussions in the revised version.

W6: Thank you for your valuable feedback regarding the readability of Section 2.3. We apologize for the current mathematical density in this section and fully agree that additional intuitive explanations would enhance understanding. In our revision we will add more intuitive struture about the NTK-based decomposition.

Q5: Our approach can be extended to classification tasks by using appropriate loss functions such as cross-entropy. However, we note an important technical distinction: since class labels are discrete rather than continuous, the joint density ratio estimation would decompose into estimating separate covariate density ratios within each class-specific subset of the auxiliary data.

We sincerely appreciate your thoughtful and constructive feedback. Many of the points you raised have helped us identify important directions for future research. We are more than happy to answer any further questions during the rebuttal period.