Direct Acquisition Optimization for Low-Budget Active Learning

Contribution: This paper focus on active learning with low budge setting and the algorithm presented solves three issues: 1) how to obtain pseudo-labels in unlabeled data; 2) how to approximate new model parameter without retraining; 3) how provide low bias estimation on true expected loss on unlabled data.

The issue for 1) is that the obtaining a good proposed surrogate function is difficult.This model plays a central role for the success of this algorithm both theoretically and practically showing in experiments. It is always possible to use the old model from last iteration for surrogate, but it is an poor estimate of the oracle given low budget setup. Thus although this framework claims that it is superior to the heuristic it still inject the heuristic within the surrogate model. More issue with surrogate model is the update of the model. Would you retrain the surrogate model and the main model using the same acquired labels? how to make sure that surrogate model guides us towards the true target better than current production? if not, the cross entropy correction would be a misleading term.

• BTW, the deep learning score is also used as confidence when computing the parameter update. It should be aware that the possible overconfidence issue for deep learning would be a factor to watch esp. when the data is noisy.

The issue for 2) is that to approximate model parameter using influence function, we have to approximate the influence function itself with some iterative method. Both CG and Quasi-Newton like solution has its own challenge, e.g. in CG, in order to compute the influence of one unlabeled sample. we have to do two back-propagation, which is more time-consuming than retraining the model with early stopping. Another issue for this approximation method is additional memory requirement for large model, like LLM. Finally, when the Hessian is indefinite ( e.g. saddle points), this is not clear that if this second-order estimate would result in severe underwhelming result in parameter estimation. Since the new model parameter is used in cross-entropy correction term in importance sampling, it is likely that the error rate will affect the sample stage.

The issue for 3) is that since cross-entropy acquisition function is the key for bias-variance reduction, it is critical to have some guarantee that the impact of error in both surrogate function selection, and parameter update is bounded. It is expected that when the cross-entropy is low but both proposal and target function are limited, this active learning would be worse than random selection.

All in all, a major concern for this model is that it is a very complicated system with multiple approximation steps, which has its own errors in each step. The good performance is thus not guaranteed. The uninformative choice of surrogate function and the approximation of parameter update would damage the performance of this model in larger and more complicated dataset. As for the computation cost as compared to model retraiin, it is not clear parameter update with second-order gradient like estimation would reduce the cost of computation as compared to partial retrain (like using Contrastive Divergence).

The main concern lies in the limited technical contributions of the methodology, as the primary techniques appear to have been adapted from existing studies. Besides, although influence functions offer an efficient approximation for parameter updates, they can be both memory and computationally intensive, especially with deep models. This may limit the practicality of applying the proposed method to deep models. A minor point is that the authors claim to use a surrogate model to estimate labels for unlabeled data. However, the implementation details suggest that the current model is merely assigning pseudo-labels rather than employing a distinct surrogate.

1. The paper does not compare against TypiClust [1], which is known to outperform all the baseline strategies here in low budget settings.
2. In the high budget experiments, the total budget is not sufficiently large, since each class only has 50 annotation budget on average. It would be interesting to see the algorithm's performance with even larger annotation budget.
3. The notation $\ell(x, f)$ is very confusing. The loss function should be a function of the label $y$ as well. This seems to have been omitted in 3.3 and 3.4. In section 3.3 equation (4), it seems this is approximated using the model's prediction. However, there is no mention of this in section 3.4. How is the loss computed for equations (6) and (9)?
4. If the author is already using the predicted label as ground truth y, instead of the method in section 3.4, the authors can use this to approximate $L_{\text{true}}$, and an ablation study should be conducted. In addition, the authors mention Mussmann et. al. as an expected error reduction algorithm, but never compares against it in the experiments.
5. There has also been quite a few works utilizing NTK to approximate the model's behavior when training on a new data point. The authors should distinguish from these methods, potentially by comparing against them.
6. The authors could consider rearranging the order between sections 3.2 and 3.3. This is because the current 3.2 naturally flows into 3.4, but is interrupted by 3.3 at the moment.

[1] Hacohen, Guy, Avihu Dekel, and Daphna Weinshall. "Active learning on a budget: Opposite strategies suit high and low budgets." arXiv preprint arXiv:2202.02794 (2022).

[2] Mohamadi, M. A., Bae, W., & Sutherland, D. J. (2022). Making look-ahead active learning strategies feasible with neural tangent kernels. Advances in Neural Information Processing Systems, 35, 12542-12553.

[3] Wang, H., Huang, W., Wu, Z., Tong, H., Margenot, A. J., & He, J. (2022). Deep active learning by leveraging training dynamics. Advances in Neural Information Processing Systems, 35, 25171-25184.

[4] Wen, Z., Pizarro, O., & Williams, S. (2023). NTKCPL: Active Learning on Top of Self-Supervised Model by Estimating True Coverage. arXiv preprint arXiv:2306.04099.