Boundary Matters: A Bi-Level Active Finetuning Method

Q1: Fine-grained classification?

R1: Following your suggestion, we utilized the CUB-200-2011 dataset, which includes $200$ bird species with a total of $11,788$ images. According to the default configuration of the dataset, $5,994$ samples are used for training, with the remainder used for testing. Given the large number of classes, we used 10% of the data as Core Samples to select Boundary Samples, while all other parameters were set according to the default values reported in the paper. The following table clearly demonstrates that our approach continues to hold a leading position in fine-grained classification datasets.

Q2: What is the gap between active fine-tuning learning and the upper performance of fine-tuning using all samples?

R2: The table presents the upper bound performance achieved by training with all available data. Despite a significant gap between training with a subset of samples and full selection, our method consistently outperforms random sampling at equivalent annotation rates.

Q3:Does using a pre-trained model to select fine-tuned samples amplify errors in the pre-trained model? Significant performance differences in Table 7.

R3: In Table 7 and the principal experiments of our paper, ViT-S models pretrained with iBOT and DINO frameworks exhibit comparable performance, whereas ResNet50 differs due to its unique architecture and model size. Objectively speaking, variations in the samples selected can arise based on the quality of features and the upper limits of the pretrained models. Nonetheless, Table 7 underscores the universality of our approach, showcasing its capability to effectively select superior samples across various pretrained models.

Q4: Is it reasonable to consider samples that deviate from the center of the pseudo-class as noise?

R4: 1. Similar to typical denoising scenarios, points significantly distant from their cluster are often deemed as noise. In our approach, we implement localized denoising, where each category has a center, and noise specific to that locality is removed. 2. In practice, due to inadequate feature learning, it's common for samples potentially belonging to two categories to intermingle at the boundaries. Therefore, our denoising aims to clear these mixed areas to avoid erroneous sample selection. This is a trade-off: denoising allows us to conservatively select boundary samples to address poorly learned features and classifications, while opting not to denoise would constitute a bolder choice.

Q5: On different datasets, when the labeled sample size is less than, which approximate range is boundary sample selection not required?

R5: We employed the Core Sample Selection method to obtain different numbers of central points and then analyzed the benefits these central points brought. Specifically, we define Distance as the mean Euclidean distance from each sample to the nearest selected point in the feature space and investigate the Rate of Return (Incremental Benefit per Core Sample) in different range, , where Rate of Return = Distance Difference / Core Number Difference between two adjacent columns.

• CIFAR10

• CIFAR100

We found that the rate of return diminishes gradually, indicating that core samples are crucial in the early stages, while the benefits decrease significantly later on. A clear demarcation point can serve as a guide for when to begin Boundary Sample Selection, such as the range of 250-375 for CIFAR-10 and 375-500 for CIFAR-100. This provides a simple yet effective guideline. Additionally, in practical applications, we discovered that introducing boundary points earlier may yield better results, such as CIFAR100 with 1% (500 samples) annotation samples.

Q6: The notation for intra-class and inter-class distances should be changed from Xi to Ci.

R6: Thank you for your suggestion. Given that our distance metric applies to each individual sample, I understand your recommendation to express $d_{intra}(x_j)$ as $d_{intra}(x_j, C_i)$ for greater clarity. We will adopt your suggestion and implement this change in the revised version.

Q7: Recommend Tasks with higher annotation costs, such as detection segmentation, remote sensing and medicine.

R7: There are two aspects to consider: which data should be annotated and how to increase the speed of annotators. Due to time constraints and our lack of exposure to remote sensing and medical data, we plan to conduct further research in the future. As for detection and segmentation tasks, our experiments have also covered these, specifically in PASCAL VOC and ADE20k datasets.

We hope this response could help address your concerns, and wish to receive your further feedback soon.

Thank you for your insightful suggestions. We believe we have comprehensively addressed your questions regarding the applicability of different pre-trained models, scenarios involving fine-grained classification, the rationale for denoising, and the threshold for selecting boundary samples.

It is worth noting that our method can maintain a leading performance across different pre-trained models and effectively enhance data quality in fine-grained classification scenarios to achieve optimal performance. Additionally, we have proposed a simple yet effective guideline as a threshold for selecting boundary samples.

We are wondering whether you have any additional questions or comments regarding our response to your review comments. We will do our best to address them.

We appreciate your time and effort reviewing our manuscript. Thanks for your consideration!

Q1: Figure 2 should be modified.

R1: Thank you for your suggestion. In the revised version, we will enhance the figure to improve its detail and clarity.

Q2: Is there consistency between the decision boundaries of the K pseudo-classes and the decision boundaries of the true task categories? Are the decision boundaries of the unsupervised method consistent with the decision boundaries when the pretrained model is fine-tuned on downstream tasks?

R2: Yes, these conditions are consistent! We conducted linear probing on all the samples with true labels using features from both the pre-trained model and the oracle model (fine-tuned on all samples). We analyzed whether samples selected using different methods—Random, ActiveFT, BiLAF (ours)—tend to be near the decision boundaries. We used two metrics for this analysis: 1) Entropy, where a higher value indicates greater uncertainty and a propensity towards boundary samples. 2) Prob_Diff (probability difference between the top two classes) calculated as the difference between the highest and second-highest probabilities. A smaller value indicates that the sample is closer to the boundary between these two classes.

We computed the mean values for all samples and the top 50 samples. We found that samples selected based on pre-trained features exhibit higher Entropy and smaller Probability Differences in both scenarios, sufficiently demonstrating that our method aligns with the true decision boundaries and the samples selected in unsupervised pretrained features aligns with the pretrained model is finetuned on downstream tasks.

Q3: The theoretical foundation is weak. Is it necessary to introduce data on class decision boundaries? Is there a theoretical basis for applying "diversity and uncertainty" sampling to active fine-tuning methods? Is there a theoretical basis for the design and implementation methods?

R3:

1. Theoretical Assurance: Assuming a binary classification problem that is linearly separable. Our solid theoretical foundation lies in Support Vector Machines (SVM). The objective of SVM is to find a hyperplane that maximizes the margin, enhancing the classifier's generalization ability. Only the points on the boundary (support vectors) contribute to the margin, while internal points do not affect its size. This classical design emphasizes the importance of boundary samples.

2. Related Work: Previous works in Active Learning invariably involve diversity and uncertainty. Diversity aims to fit the original data distribution better with fewer samples. As the labeled data increases, allocating the remaining budget to uncertainty becomes common.

3. Design Philosophy: Faced with multiple challenges in a single iteration scenario for data selection, we must determine class centers and choose boundary samples. We also need to consider the potential noise, where different classes mix at the boundaries and each class may have multiple feature centers. Thus, we select more pseudo-classes than actual classes to ensure coverage of different classes, and we employ denoising to reduce sample mixing at boundaries and avoid noise as much as possible. Denoising process mainly targets noise in non-ideal scenarios. Opponent Penalty and Iterative Selection encourage the boundary samples with the regional diversity. Overall, based on solid boundary theory, we have designed each part for real-world noisy scenarios, using a set default of parameters throughout except for the number of pseudo-classes, which varies with different sample sizes, further demonstrating our method's effectiveness.

4. Oracle Verification: We apply our method an feature Oracle model(finetuned on all samples). At 5% budget of CIFAR100, we achieved 65.7% and at 10% budget, we reached 75.3%—both over 1.5% higher than selecting central points alone, proving the practical upper bound. We also confirmed our method's consistency with actual conditions in Q2. Thus, this also validates the effectiveness of our approach from this perspective.

Q4: Exploration of the number of uncertain sample points: How is K determined? Having too many such sample points presents two problems: incorrectly labeled samples and disruption of learning the general features of the categories.

R4:

1. Core Number K: We conduct experiments to estimate an approximate threshold for the Core Number needed.
2. Boundary Sample Selection: Our method's components tackle the problem:
   • Denoising: Removes highly uncertain boundary points to prevent class mixing, conservatively choosing more accurate points.
   • Opponent Penalty: Encourages diverse explorations across boundaries, allowing a more varied distribution.
   • Iterative Selection: Removes nearby points of selected samples, maintaining diversity by choosing boundary-nearer points instead of regional centers.
3. Practical Experiments: Since exact sample centers are unknown, we use numerous pseudo-classes, K, with boundaries between same-label pseudo-classes acting as internal points.
4. Experiment Consistency: This flexible framework can achieve SOTA results using the same default hyperparameters. The ablation study showed that a suitable K enhances sample quality which represent much room for improvement.

Q5: Should it be "but neglects the contributions of local boundary samples" in abstract?

R5: We will adopt your suggestion in the revised version, as it appears more reasonable. Thank you for helping to further enhance the readability of our paper.

Thank you for valuable time and insightful comments. We hope our responses have addressed your concerns. We welcome further discussions and sincerely appreciate it if you could reconsider your rating.

[Finally, experiments for Q2 and Q4 will be detailed in official comments due to the characters limitation.]

Experiments For Q2.

Experiments are as follows:

Therefore, there is consistency between the decision boundaries of the K pseudo-classes and the decision boundaries of the true task categories. The decision boundaries of the unsupervised method is consistent with the decision boundaries when the pretrained model is finetuned on downstream tasks.

Experiments For Q4.

We employed the Core Sample Selection method to obtain different numbers of central points and then analyzed the benefits these central points brought. Specifically, we define Distance as the mean Euclidean distance from each sample to the nearest selected point in the feature space and investigate the Rate of Return (Incremental Benefit per Core Sample) in different range, where Rate of Return = Distance Difference / Core Number Difference between two adjacent columns.

• CIFAR10

• CIFAR100

We found that the rate of return diminishes gradually, indicating that core samples are crucial in the early stages, while the benefits decrease significantly later on. A clear demarcation point can serve as a guide for when to begin Boundary Sample Selection, such as the range of 250-375 for CIFAR-10 and 375-500 for CIFAR-100. This provides a simple yet effective guideline. Additionally, in practical applications, we discovered that introducing boundary points earlier may yield better results, such as CIFAR100 with 1% (500 samples) annotation samples.

Thank you for your insightful suggestions. We believe we have comprehensively addressed your questions regarding the consistency of decision boundaries, the basis for designing boundary samples, and the threshold for the number of uncertain samples.

It is worth noting that our method aligns with the true decision boundaries, and the samples selected from unsupervised pre-trained features are consistent with those used when the pre-trained model is fine-tuned on downstream tasks. We have validated the effectiveness of boundary samples from three perspectives: theoretical guarantees, related work, and Oracle experiments. Ultimately, we have proposed a simple yet effective guideline as a threshold for selecting boundary samples.

We are wondering whether you have any additional questions or comments regarding our response to your review comments. We will do our best to address them.

We appreciate your time and effort reviewing our manuscript. Thanks for your consideration!

Thank you very much for your recognition of our paper and the increased score! We also appreciate the valuable suggestions you've provided. We will make further improvements in the revised version to meet your expectations.

Q1: Threshold: Whether BiLAF requires a certain threshold of fine-tuning data to be effective?.

R1: This is an excellent question. The performance gap with just 0.5% of CIFAR-10 data indeed triggers considerations about scale. In traditional active learning, the applicability of different methods varies with scale. The classic Coreset[1] method aims to cover all samples with the selected points, whereas ProbCover[2] notes that Coreset performs poorly when the data scale is very small, leading to the proposal of a coverage radius to ensure each selected point controls a fixed area. In our scenario, it is evident that using central points to fit different dense distributions appears to be a more rational choice when the budget is extremely limited, without using additional samples for boundary selection. We believe that this situation is both normal and objectively present.

Following this, we explored what such a threshold might look like. We employed the Core Sample Selection method to obtain different numbers of central points and then analyzed the benefits these central points brought. Specifically, we define Distance as the mean Euclidean distance from each sample to the nearest selected point in the feature space and investigate the Rate of Return (Incremental Benefit per Core Sample) in different range, where Rate of Return = Distance Difference / Core Number Difference between two adjacent columns.

• CIFAR10

• CIFAR100

We found that the rate of return diminishes gradually, indicating that core samples are crucial in the early stages, while the benefits decrease significantly later on. A clear demarcation point can serve as a guide for when to begin Boundary Sample Selection, such as the range of 250-375 for CIFAR-10 and 375-500 for CIFAR-100. This provides a simple yet effective guideline. Additionally, in practical applications, we discovered that introducing boundary points earlier may yield better results, such as CIFAR100 with 1% (500 samples) annotation samples.

Similarly, how to allocate proportions remains a question. Our ablation study shows that for different budgets, there are varying optimal core numbers, which remains an open question. To some extent, we can determine this based on the rate of return and the number of sample categories. In the paper, we adhered to a uniform hyperparameter setting, which actually demonstrates that our framework offers more flexible parameter choices with significant potential for performance improvement.

[1] Sener, Ozan, and Silvio Savarese. "Active Learning for Convolutional Neural Networks: A Core-Set Approach." ICLR18

[2] Yehuda, Ofer, et al. "Active learning through a covering lens." NIPS22

Q2: Limited Data Condition: ActiveDC reports on the performance of CIFAR10 under 0.1% and 0.2% settings. Active Finetuning would be more meaningful in scenarios with extremely limited data?

R2: ActiveDC employs pseudo-labeling from semi-supervised learning, which pertains to how to fine-tune after sample selection, rather than the selection of the samples themselves. Therefore, the appropriate comparisons should be with other semi-supervised training or fine-tuning methods. Following Reviewer V2Vc's suggestion in Q2, we experimented with KNN and Linear Probing in scenarios with very small data volumes and achieved results that significantly surpassed those of ActiveDC.

We also found that these basic methods become ineffective as the data volume increases. Consequently, we believe that active selection for full-scale fine-tuning should focus on scenarios where the data volume is not extremely small. Our method selects higher-quality samples and achieves superior results across different fine-tuning paradigms. We will reference the ActiveDC paper in the revised version and discuss the differences between our work and ActiveDC.

Q3: Larger Data Condition: How about larger amounts of finetuning data, such as 20% of ImageNet.

R3: We have supplemented our results with 10% and 20% budget scenarios on ImageNet, and our method continues to maintain a strong advantage. Theoretically, as data volume increases, the selection method ActiveFT tends to lean towards Random, thus tending towards a uniform distribution. However, our method effectively identifies boundaries, providing better separability. Empirically, our method has shown impressive results. The Improvement Ratio has increased, where the Improvement Ratio is defined as Improvement Ratio=(BiLAF - Random) / (ActiveFT - Random). In active learning, it is normal for accuracy differences to decrease as data volumes increase, making the improvement ratio a reasonable metric to assess performance.

Overall, we have effectively analyzed our method across different scales, proposed effective boundary metrics for smaller data volumes, and empirically demonstrated the superior results and feasibility of our method with larger data volumes. We hope our responses have addressed your concerns. We welcome further discussions and sincerely appreciate it if you could reconsider your rating.

Thank you for your insightful suggestions. We believe we have comprehensively addressed your questions regarding the data scale used for fine-tuning the model and the scenarios where our method is applicable.

It is worth noting that we have proposed a simple yet effective guideline as a threshold for selecting boundary samples, which enhances the flexibility of our method. Additionally, we have conducted supplementary experiments that demonstrate our method maintains its advantages and is even more effective when applied to larger datasets.

We are wondering whether you have any additional questions or comments regarding our response to your review comments. We will do our best to address them.

We appreciate your time and effort reviewing our manuscript. Thanks for your consideration!

Thank you for your recognition of our paper and the increased score!

Important Notes. Reviewer’s recognition of "numerous hyperparameters can cause instability in different scenarios" in Limitations is not justified. Except Core Number, we consistently employs the same hyperparameters across all six different tasks, such as nearest neighbors number, denoising rate, clustering fraction and opponent penalty. The adjustment in Core Number is tailored to the varying class numbers across tasks, and we ensure uniformity in Core Number for different budgets within the same task. This uniformity underscores the stability and scalability of our framework. Changes in these parameters could potentially elevate the performance ceiling of our method. In our Ablation Study, alterations in the Core Number and Core Selection Method indicated further enhancements.

Q1: There is ambiguity in the use of symbols, such as $c_i$ in equation (1), $a_i$ of $X_i$ in Sec.3.3.2.

R1: The sample set is $X=\{x_1,\ldots,x_N\}$, where the subscript $i$ represents the sample $x_i$ and its corresponding feature $f_i$. In Section 3.2, Core Sample Selection identifies $|C|$ center points, represented by the index set $C=\{c_1,\ldots,c_K\}, c_i\in[N]$. In Section 3.3.1, Equation (1), for each sample $x_j, j\in[N]$, we find its nearest center point $f_{c_i}$ in the feature space, which is closest to its feature $f_j$. We then assign this sample $x_j$ to the cluster controlled by the center $c_i$, denoted as $U_i$.

In Sec.3.3.2, we introduce $a_{i,j}$, where $j \in [|U_i|]$, to represent the index of the $j$-th element in the set $U_i$ controlled by the $i$-th center $c_i$. Since subsequent operations are performed on each set $U_i$ individually, $a_{i,j}$ serves as the index of the sample $x_{a_{i,j}}$ and its feature $f_{a_{i,j}}$. Actually, $U_i$ records all the indices of samples $X_i$ belong to the center $c_i$.

The variable $j$ serves as an enumeration variable in different contexts, which might have led to some ambiguity. We will address these ambiguities and provide clearer explanations in the revised version.

Q2: Why is the method effective on imbalanced datasets? Improvement on the CIFAR100LT dataset with 15% ratio is only 0.3% with the claim "the default denoising removal ratio parameters might remove minority samples".

R2: Firstly, these methods are not specifically tailored for long-tailed distributions. Our method, which focuses on boundary-based approaches, exhibits strong representational capabilities in general scenarios, providing us with a natural advantage. This supports our claim of the method's universality based on its performance in long tailed dataset.

1. Denoising involves a trade-off; excessive denoising can eliminate minority classes at the boundary. However, totally without any denoising, selecting boundary samples based on minority class centers could mistakenly include majority class samples because minority and majority classes are often close in boundaries.
2. Objectively, as the annotation rate increases, ActiveFT tends to select long-tailed samples as well, which may reduce the gap as the data volume grows.
3. The default core number and denoising ratio were selected in the paper, which could potentially exclude long-tailed samples due to their isolated distribution or outlier status. Upon adjusting the Core Number and Denoising Ratio in experiments, we discovered that the model has considerable improvement.

Q3: The necessity and design of the opponent penalty need to be reconsidered with little contribution to the performance improvement.

R3: In Ablation Study, we observed an intriguing trend: the negative impact of the components examined in IDs 1-4 generally diminishes as the number of selected samples increases, while the benefits of the opponent penalty grow with a larger sample size. The design intention behind this metric is to prevent the selection of too many similar boundary points, which can degrade model performance. At sample sizes of 5% and 10%, the opponent penalty yields an accuracy improvement of over 0.5%, which we find significant.

As the important notes explained, we used the same hyperparameters in all experiments. In practice, parameters such as Core Number, Denoising Rate, and Opponent Penalty can be further tailored to specific scenarios to enhance the model's training capacity. This metric can also be applied flexibly depending on the amount of data selected.

Q4: Learned center points are still biased towards the boundaries? Does this not risk the model focusing too much on the boundaries and neglecting the intra-class distribution characteristics?

R4: The possible reasons are 1)Each class have multiple centers 2)TSNE visualizes data in two dimensions, which can be the centers in high-dimensional space. Actually, in our method, Opponent Penalty encourages diverse explorations across boundaries, allowing a more varied distribution. Iterative Selection removes nearby points of selected samples, maintaining diversity by choosing boundary-nearer points instead of regional centers. What's more, we might sample multiple points within a class as pseudo-centers. Boundary points between pseudo-classes of the same label can act as internal points of the class to better fit the distribution. Therefore, our method not only focuses on the classification boundaries but can also enhance the intra-class distribution.

Thank you for valuable time and insightful comments. We hope our responses have addressed your concerns. We welcome further discussions and sincerely appreciate it if you could reconsider your rating.

Thank you for your insightful suggestions. We believe we have comprehensively addressed your questions regarding the effectiveness of our method in long-tail problems, the necessity of the opponent penalty design, and the selection of sample distributions.

It is worth noting that our method maintained default parameters across all experiments, demonstrating the universality of the model. The design of different components not only preserves this universality but also enhances the flexible architecture and performance ceiling of our method. Additional experiments have confirmed that our approach further improves performance on long-tail tasks.

We are wondering whether you have any additional questions or comments regarding our response to your review comments. We will do our best to address them.

We appreciate your time and effort reviewing our manuscript. Thanks for your consideration!

Q1: Concern about pre-trained features. Would there be a consistent improvement if we first apply unsupervised pre-training on the unlabeled fine-tuning set?

R1:

1. Our main experiment utilized a checkpoint pre-trained on ImageNet using DINO. We conducted studies both on consistent tasks (ImageNet) and inconsistent ones (CIFAR & Others), demonstrating that our method is effective regardless of whether the pre-trained features come from the same task.
2. Furthermore, we explored features trained using different paradigms and architectures, such as iBOT and DINO, as well as ResNet50 and DeiT-S, detailed in Appendix F’s Ablation Study. This highlights the generalizability of our approach across different feature sets.
3. It is widely acknowledged and applied that many pre-trained models, such as DINOv2 and CLIP, claim robust transferability across diverse downstream tasks after extensive pre-training on large datasets.
4. Due to constraints of time and resources, we were unable to perform additional unsupervised training. However, we conduct experiments to show that the boundary points selected based on pseudo-classes are consistent with the actual boundaries, and the boundaries selected based on pre-trained features are consistent with using fully-trained Oracle models. The experiments results will be detailed in the following offical comments due to the characters limitations.

Q2: Full fine-tuning may not always be the optimal approach? Linear probing or even nearest-neighbor classifiers can yield better results. Specifically, does BiLAF remain necessary for selecting high-quality data points with more suitable fine-tuning methods?

R2: Thank you for your insightful comments and suggestions. Following your ideas, we experimented with Linear Probing and KNN on the CIFAR10 dataset and compared the results across Random, ActiveFT, and BiLAF(ours).

Based on these results, we have supplemented our experiments with 5% budget. From the table, the following conclusions can be drawn:

• At extremely low data volumes, Linear Probing and KNN outperform full fine-tuning.
• As the data volume increases, the performance improvements of Linear Probing and KNN start to slow down, which gradually necessitates the use of full fine-tuning.
• Interestingly, the quality of data selected by different methods shows a consistent trend across KNN, Linear, and full fine-tuning. Our method, compared to competitors, is able to select more suitable data, which is effective across different fine-tuning paradigms.
• Following prior work, our original focus was on full fine-tuning. We appreciate your comments, which have helped enhance the comprehensiveness of our work.

Q3: Why BiLAF is faster than ActiveFT if it needs to run ActiveFT first?

R3: The primary reason is that ActiveFT in our approach is utilized solely for selecting a core sample set, not the entire dudget $B$ as 2%, 5%, 10%. In our CIFAR100 experiments, we fixed the core number as 0.5%. Therefore, the time consumption for BiLAF mainly comprises the time ActiveFT takes to select 0.5% of the samples and the time for further boundary sample selection, which can be faster than directly selecting all samples using ActiveFT. For example, when B=2%, the denoising process takes $5.98$ seconds and iterative boundary sample selection takes $4.12$ seconds, with additional time allocated to ActiveFT tasks selecting core points and other operations. A significant portion of the time, including the denoising, is independent of $B$. When $B$ is small, the impact of boundary sample selection based on $B$ is minimal, resulting in a negligible increase in runtime for our method across different $B$ values. For a comprehensive time complexity analysis, please see Appendix D.

Q4: Specifically, does it help more on a pseudo class with more neighboring classes while helping less for those on the border?

R4: In an ideal scenario, if a central representative is identified for each class, the class on the border would select samples closer to other classes rather than samples farther away, as demonstrated in Figure 1. For instance, as shown in Figure 3, the green class would select adjacent samples from the blue, pink, orange, or even light blue categories, thereby promoting diversity at these boundaries. Such selections can significantly enhance the distinguishability of boundary classes from others, demonstrating that the 'opponent penalty to encourage diversity across boundaries' is a universally applicable strategy. However, in light of your concerns and upon further reflection, we recognize that as the number of boundary selections increases, the advantages for border classes may not be as pronounced as for central classes, which may require more samples to depict their more complex and varied boundaries accurately. Optimizing the distribution of boundary points based on the positions of the centers presents a compelling and valuable new direction for future research.

Thank you once again for your valuable time and insightful comments, which have greatly enhanced our work. We hope our responses have addressed your concerns and demonstrated the versatility of our method. We look forward to further discussions and sincerely appreciate it if you could reconsider your rating.

Experiments For Q1.

We conducted linear probing on all the samples with true labels using features from both the pre-trained model and the oracle model (fine-tuned on all samples). We analyzed whether samples selected using different methods—Random, ActiveFT, BiLAF (ours)—tend to be near the decision boundaries. We used two metrics for this analysis: 1) Entropy, where a higher value indicates greater uncertainty and a propensity towards boundary samples. 2) Prob_Diff (probability difference between the top two classes) calculated as the difference between the highest and second-highest probabilities. A smaller value indicates that the sample is closer to the boundary between these two classes.

Therefore, the conclusion is that our method effectively selects boundary samples, maintaining consistency across models with various capabilities. Consequently, as the quality of model features improves, training with our selected high-quality data can also yield consistent improvements.

Thank you for your insightful suggestions. We believe we have comprehensively addressed your questions concerning the impacts of various pre-training features, fine-tuning paradigms, and the opponent penalty design.

It is worth noting that our method effectively identifies high-quality samples that significantly enhance model training across various pre-training features and tasks. These samples facilitate best accuracy across different fine-tuning paradigms. Our approach is fast, efficient, and generalizable.

We are wondering whether you have any additional questions or comments regarding our response to your review comments. We will do our best to address them.

We appreciate your time and effort reviewing our manuscript. Thanks for your consideration!