AutoAL: Automated Active Learning with Differentiable Query Strategy Search

We thank for your feedback and the confirmation of our proposed AutoAL! We agree that AutoAL is an end-to-end framework could effectively select the best strategy for a given dataset or task. We also thank for your valuable question, so we want to clarify the followings:

Q1: Is there any optimization strategy used to compute the scores fast, as shown in Table A.1 (Line 565)?

A1: This is due to our proposed differential bi-level optimization strategy. Actually all the time cost for AutoAL-Search only contains the training time of the two neural networks. We do find AutoAL may need extra time compared with other baselines, however, the main cost is due to the sample query process of the candidate ALs.

Q2: Is FitNet truly necessary? Why do the ablation studies not evaluate its impact?

A2: Thanks for pointing out this. Because FitNet is the same net as the final classification net, and is used in AutoAL to yield the informativeness of each unlabeled sample, so we thought it's unnecessary to do ablation on it in our original version. However, we added this experiment on CIFAR100 for you to refer to the results:

Q3: Why is FitNet both trained and fine-tuned twice? What is the influence of loss Ls and Lf on the final selection?

A3: In our settings, FitNet is first trained on the labeled dataset to yield the informativeness of unlabeled sample, then co-optimized with the SearchNet. For $L_s$, it guides the update of SearchNet, to better decide which AL candidate performs the best in the settings now. For $L_f$, it will guide the FitNet to better draw the distribution of unlabeled dataset.

Q4: The experiments focus on classification problems—can the framework also be applied to regression tasks?

A4: Yes, this is definitely possible. Such as in object detection tasks, as long as the candidate AL strategy can output the most possible coordinates of the target object, AutoAL can learn from it and decide which one is the best coordinate. However, in this paper, we just demonstrate the superior of AutoAL in classification task. We plan to do more tasks as a future work.

Essential References Not Discussed: Thanks fo providing the related works. In our initial version, we mainly consider the algorithm selection works with many candidate ALs, but not focused on providing a new AL strategy. However, we believe the works you provided is great and we plan to adds them into our final version.

Thank you for your valuable feedback. We appreciate that you confirm our contribution to the community on solving the strategy selection problem and our method: using differentiable bi-level framework.

For your concerns, we add more experiments to show the results as below: Q1: The author should compare against (Hacohen & Weinshall) and (Zhang et al.), and in their own games. (Hacohen & Weinshall) is proposed for different computation budget settings, and (Zhang et al.) is proposed for imbalance.

A1: We thank for your insightful comments. We do find we should compare with their works so that it can further show the effectiveness of our proposed AutoAL. To compare with [1], we conducted the experiements on two datasets, CIFAR10 and CIFAR100, which are used in both of our experiments and their experiements. In their settings, the CIFAR10 dataset only has two classes and CIFAR100 with 10 classes respectively. The results are shown as follows:

For CIFAR-10:

For CIFAR-100:

We found that AutoAL can outperform TAILOR in nearly all AL iterations, both in CIFAR-10 and CIFAR-100, this result is consistent with the results shown before because for SVHN and other medical datsets, although they are imbalanced, AutoAL can still outperform the baselines.

For [2], unfortunately they didn't open-source their code, but we found a toolbox from [3] which officially implemented Probcover [4] and TypiClust [3], two baselines used in [2]. From the experiment results in [2], we observed that Probcover consistently performs best in low budget settings while Badge [5] excels in high budget settings. Therefore, we selected Probcover, TypiClust, and Badge as our baseline methods, focusing our tests on CIFAR-10. As shown in our results, AutoAL performs well in medium and high budget settings but underperforms in low budget scenarios. We attribute this to deep neural networks typically requiring larger datasets for proper training—when the budget is low, SearchNet and FitNet cannot be fully optimized. However, traditional AL settings rarely use extremely low budgets, and AutoAL demonstrates its effectiveness when the budget is adequate. We acknowledge that AutoAL is designed for general active learning scenarios, not specifically for low-budget settings.

Essential References Not Discussed: Thanks for providing the related works. In our initial version, we mainly consider the algorithm selection works with many candidate ALs, but not focused on providing a new AL strategy. However, we believe the works you provided is great and we plan to adds them into our final version.

We hope these results can address your problems, and reconsider our contributions. We appreciate if you can consider raising your score.

References: [1] Zhang, Jifan, et al. "Algorithm selection for deep active learning with imbalanced datasets." Advances in Neural Information Processing Systems 36 (2023): 9614-9647.

[2] Hacohen, Guy, and Daphna Weinshall. "How to select which active learning strategy is best suited for your specific problem and budget." Advances in Neural Information Processing Systems 36 (2023): 13395-13407.

[3] Hacohen, Guy, Avihu Dekel, and Daphna Weinshall. "Active learning on a budget: Opposite strategies suit high and low budgets." arXiv preprint arXiv:2202.02794 (2022).

[4] Yehuda, Ofer, et al. "Active learning through a covering lens." Advances in Neural Information Processing Systems 35 (2022): 22354-22367.

[5] Ash, Jordan T., et al. "Deep batch active learning by diverse, uncertain gradient lower bounds." arXiv preprint arXiv:1906.03671 (2019).

Thank you for your feedback. For your questions, we make the following comments to clarify our points:

Q1: Sampling Bias in Actively Selected Data

A1: For our initial seed dataset setting, it is i.i.d., ensuring an unbiased starting point. Traditional single-criterion AL strategies, especially those based solely on uncertainty [1], are more susceptible to accumulating such bias over iterations. In contrast, AutoAL integrates multiple AL strategies, each contributing different perspectives (uncertainty, diversity, etc.). This fusion, combining multiple AL candidates while maintaining a differentiable framework between search and fit nets, inherently reduces bias, prevents overfitting to any individual AL sampling strategy, and adapts dynamically as the AL process evolves.

Q2: Baseline implementation Issue

A2: Thanks for the question. We observed this problem too. To clarify, all baseline implementations come from public GitHub repositories, not from our own. Our framework builds upon the open-source deepAL [1]. The accuracy degradation appears in many works, including Figure 1 in [3,4], Figure 3(b) in [5], Figure 3 and 4 in [6], and deepAL [1]. We attribute this to two factors:

1. The amount of data causing overfitting in the classification model.
2. Medical datasets often contain redundancy and confusing information. These medical images have many-to-one relationships, as one patient may correspond to several pathology images. Some patients may have multiple conditions (e.g., X-rays showing posterior spinal fixators used for spine repair). These features can influence predictions. Proper diagnosis requires both local and global features, making bias problems more severe in these cases.

Q3: Writing Issues Q3.1: Notion Issues.

A3.1: We are sorry that there are many notations in our method parts. We will make a table in the appendix to define the important notions.

We tried to describe the input for SearchNet and Fitnet in Figure 1. The initialed labeled pool is divided to two queues. The first queue is used to train FitNet. The second queue is used to train both FitNet and SearchNet. Thanks for pointing out, we will polish the writing in our final version.

Q3.2: Notions used before defined.

A3.2: Our writing logic was to first interpret the whole framework of AutoAL, then the detailed parts such as the loss functions in Equation (1).

Q3.3: Figure 1.

A3.3: AutoAL's FitNet is first trained on the first queue of labeled dataset. After the parameter of FitNet is adapted to the second training step, FitNet and SearchNet are jointly trained with the second data queue. We will add more explanation to the description in Figure 1.

Q4: Runtime Complexity

A4: Please first refer to the rebuttal to Reviewer khE5, A2. Follow the part 5.1 of pulished work [7], we made a similar complexity analysis. As described in our appendix A.1. let $N_{search}$ denotes the updating time of SearchNet and FitNet in each AL round per batch, $N_{train}$ denotes the sample querying and classification model training in each AL round per batch. We can define the AL score querying time for each candidate AL per batch as $Q_{i}$. When there are $M$ AL algorithms, in each batch, the computation complexity is O(T$\sum_{b=1}^{B_{max}}(N_{search}+N_{train}+\sum_{i=1}^{M}Q_{i})$) in the worst case, where T is the total number of rounds and $B_{max}$ is the number of batches depending on the stop criterion of AL. In the future, we plan to do parallel processing for the candidate AL score calculation, which will further improve the upper bound to O(T$\sum_{b=1}^{B_{max}}(N_{search}+N_{train}+Q_{imax}$))

Q5: Essential References Not Discussed

A5: Thanks for finding these. About the low budget and high budget setting, we have added new experiments to show the results. Please refer to the reply to Reviewer bNSN.

We hope you can reconsider the final rating.

Reference: [1] Sharma, Manali, and Mustafa Bilgic. "Evidence-based uncertainty sampling for active learning." Data Mining and Knowledge Discovery 31 (2017): 164-202.

[2] Zhan, Xueying, et al. "A comparative survey of deep active learning." arXiv preprint arXiv:2203.13450 (2022).

[3] Gal, Yarin, Riashat Islam, and Zoubin Ghahramani. "Deep bayesian active learning with image data." International conference on machine learning. PMLR, 2017.

[4] Geifman, Yonatan, and Ran El-Yaniv. "Deep active learning over the long tail." arXiv preprint arXiv:1711.00941 (2017).

[5] Kim, Seong Tae, Farrukh Mushtaq, and Nassir Navab. "Confident coreset for active learning in medical image analysis." arXiv preprint arXiv:2004.02200 (2020).

[6] Mishal, Inbal, and Daphna Weinshall. "DCoM: Active Learning for All Learners." arXiv preprint arXiv:2407.01804 (2024).

[7] Zhang, Jifan, et al. "Algorithm selection for deep active learning with imbalanced datasets." Advances in Neural Information Processing Systems 36 (2023): 9614-9647.

hank you for your valuable feedback. We appreciate that you confirm our method development, promising performance, and fluent paper writing.

For your concerns, we make the following comments to clarify our points:

Essential References Not Discussed: We thank for your efforts in finding these related materials for us. We plan to add them to the related works. Especially for [1], it contains a lot of datasets, which worth trying.

We plan to add one sentence to related works part, in adaptive sample selection in AL section, [1] frame the algorithm selection task as "learning how to actively learn" and use deep Q-learning algorithm to select which strategy is best suited.

Q1: The proposed AutoAL does not seem novel. It utilizes the same uncertainty and diversity-based criterion to select batches of samples.

A1: Our contribution creates a differentiable bridge connecting "search" and "fit" models in Active Learning, outperforming manual and non-differentiable approaches. While AutoAL builds on existing uncertainty and diversity-based methods, it solves two key problems:

1. Single-criterion AL methods accumulate bias over iterations. AutoAL incorporates multiple methods to reduce this bias.
2. Criteria that work well in current settings may perform bad in future rounds. For example, BADGE [3] performs poorly with low budgets but excels with increased resources. AutoAL selects the optimal criterion each round to address this issue.

This integration reduces cumulative bias, with experiments confirming AutoAL outperforms two-stage selection methods like BADGE [3].

Q2: The average run time of the total AutoAL is significantly higher than most of the baselines used as part of the experiments, perhaps because of the involvement of deep neural nets, SearchNet and FitNet co-optimization in a bi-level optimization structure, whereas the accuracy is relatively 1%-3% better than most of the baselines.

A2: Thanks for pointing out the average runtime issue. Please refer to Appendix A.1, where we divided the total runtime into three main parts. The co-optimization runtime is relatively small compared to the time that candidate ALs spend querying scores for images. We believe this cost is acceptable. Our tests used 7 candidate ALs, which isn't always necessary (as shown in Figure 4 for OrganCMNIST). Using fewer candidates would significantly reduce runtime. We plan to integrate AL methods with lower computational complexity to further reduce AutoAL's total runtime.

Q3: It is not clear how AutoAL could be used to add more AL frameworks to extend its applications to other domains such as image segmentation, object detection etc.

A3: Thanks for your feedback. We plan to test AutoAL on various applications, as our method is task-agnostic. For tasks like image segmentation, whenever candidate AL methods [4,5] can score instances, SearchNet and FitNet will train on these scores. Since most published works [2,6] also only focus on image classification datasets, we believe our results demonstrate AutoAL's superiority in most scenarios.

Q4: It would be interesting to see how vision transformer models would work based on this framework.

A4: Thanks for your feedback. We want to clarify that using a basic learning is a normal setting which has been used in published works [1,4], however, we plan to add the experiments for transformers in the future to see the performance of AutoAL [6,7].

Reference: [1] Desreumaux, Louis, and Vincent Lemaire. "Learning active learning at the crossroads? evaluation and discussion." arXiv preprint arXiv:2012.09631 (2020).

[2] Hacohen, Guy, and Daphna Weinshall. "How to select which active learning strategy is best suited for your specific problem and budget." Advances in Neural Information Processing Systems 36 (2023): 13395-13407.

[3] Ash, Jordan T., et al. "Deep batch active learning by diverse, uncertain gradient lower bounds." arXiv preprint arXiv:1906.03671 (2019).

[4] Li, Jun, José M. Bioucas-Dias, and Antonio Plaza. "Hyperspectral image segmentation using a new Bayesian approach with active learning." IEEE Transactions on Geoscience and Remote Sensing 49.10 (2011): 3947-3960.

[5] Yang, Lin, et al. "Suggestive annotation: A deep active learning framework for biomedical image segmentation." Medical Image Computing and Computer Assisted Intervention− MICCAI 2017: 20th International Conference, Quebec City, QC, Canada, September 11-13, 2017, Proceedings, Part III 20. Springer International Publishing, 2017.

[6] Sener, Ozan, and Silvio Savarese. "Active learning for convolutional neural networks: A core-set approach." arXiv preprint arXiv:1708.00489 (2017).

[7] Yoo, Donggeun, and In So Kweon. "Learning loss for active learning." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019.

Thank you for your valuable comments and time. We appreciate that you confirm our contribution and appreciate our work.

For your questions, we add more experiment results for your reference: Q1: Experiments are only done on image data and with resnets. I wonder how contingent the performance of the approach is on the convolution inductive bias—it would be useful to see some experiments with more naive models, like MLPs. Along these lines, I’ve seen other papers show results for different acquisition batch sizes, which I feel would also give more clarity to how the approach performs. Q2: Experiments only show the earliest parts of learning curves. The results look very promising, but it would be nice to see asymptotic performance.

A1 and A2: To answer your questions, we conduct new experiments to show the results, especially on CIFAR10 dataset, as it's commonly used by other related works [1,2]. Speicially, we changed the acquisition batch size from 1000 (original setting of our paper) to 5000 to show the difference. Then we compare the results between our proposed AutoAL with two baselines, KMeansSampling [3] and LPL [4], which are the worst and best performed baseline in our original CIFAR 10 experiments. The results show that our method can still outperform the baselines. Also using a basic learning is a normal setting which has been used in published works [1,4], but we plan to add the experiments for MLPs in the future.

Q3: This is a weak point, but calling the method AutoAL seems very general, as other techniques, such as ALBL, also do what I think I’d classify superficially as “automated active learning” in the same way. I'd consider switching to something more descriptive of this technique in particular.

A3: Thanks for your suggestions. We plan to change the name AutoAL to DDAL, representing Differentiable Deep Active Learning in the final version.

References: [1] Sener, Ozan, and Silvio Savarese. "Active learning for convolutional neural networks: A core-set approach." arXiv preprint arXiv:1708.00489 (2017).

[2] Hacohen, Guy, and Daphna Weinshall. "How to select which active learning strategy is best suited for your specific problem and budget." Advances in Neural Information Processing Systems 36 (2023): 13395-13407.

[3] Ahmed, Mohiuddin, Raihan Seraj, and Syed Mohammed Shamsul Islam. "The k-means algorithm: A comprehensive survey and performance evaluation." Electronics 9.8 (2020): 1295.

[4] Yoo, Donggeun, and In So Kweon. "Learning loss for active learning." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019.