Info-Coevolution: An Efficient Framework for Data Model Coevolution

We sincerely thank the reviewer i2qw for the suggestions as well as the potential improvements. We make responses as follows.

Q1: The paper only compares with one AL method, DQ (coreset-based), in Fig. 6. More AL methods could be compared, from the early entropy-based methods to recent AL methods. I understand the proposed method is both data-agnostic and model-agnostic, but it'd be better to include more AL methods to comprehensively evaluate its effectiveness. More experiments on different semantic understanding tasks could be considered.

Thanks for the suggestions. We attach more baselines here and extend our algorithm to the semantic segmentation task.

More baselines

ImageNet-1k with ViT supervised training from 1% data to 10% data,

Analysis:

• Previous active learning methods require training for each step. At the same step number, Info-Coevolution has the best performance with better time cost. Given the full budget, other baseline methods have much higher costs ($O(n^2)$ for these step-wise methods) and still have lower performance than Info-Coevolution.

• What's more, these methods are also hard to further scale (due to $O(n^2)$ training complexity) while Info-Coevolution can directly scale from 12w samples to 87w samples without a in-middle training step.

• Additionaly, Info-Coevolution doesn't introduce any data distribution problems, so we are able to do continual training. We have additionally verified that using continual training on previous checkpoints has no performance loss. (1% data with 50 epoch, continual training with selected10% data with 45% epoch, then continual training on 68% data with 40 epoch) The training cost of our algorithm could be O(n) in total for selected data.

Conclusion:

• Info-Coevolution has better performance and much better scalability than previous baselines.

Semantic segmentation

We here extend our algorithm to semantic segmentation and attach results on ADE20k with UperNet(BEiT-v2 backbone). The UperNet has a backbone (BEiT-v2-large) and a UperHead. We use the backbone feature from the last layer (dim 1024*16*16) and mean pooling it to dim 1024. We adapt Info-Coevolution accordingly, using pixel-wise confidence avg as model confidence, class-wise similarity weighted confidence as knn confidence, and $gain=1-avg(confidence_{model},confidence_{knn})$. The result for using 1% data trained model to select 10% data is as follows:

ADE20K 10% random

ADE20K 10% our selection (using BEiT feature)

Analysis:

• It can be seen that the mAcc, mIoU, and mAcc are improved by 0.65, 1.50, and 2.09 respectively, which is significant at this data amount. (For larger data ratio we are still running experiments and will update the results in the later updates. We will add thorough experiments in the revision. The code of this part will also be published.)

Conclusion:

• Info-Coevolution is generalizable to more tasks as analyzed.

We sincerely thank the reviewer 9pQu for the suggestions as well as the potential improvements. We make responses as follows.

Q1: Adding references and baselines

A1:Thanks for the suggestions. We attach more baselines here, and will add the corresponding references.

ImageNet-1k with ViT supervised training from 1% data to 10% data,

Analysis:

• Previous active learning methods require training for each step. At the same step number, Info-Coevolution has the best performance with better time cost. Given the full budget, other baseline methods have much higher costs ($O(n^2)$ for these step-wise methods) and still have lower performance than Info-Coevolution.

• What's more, these methods are also hard to further scale (due to $O(n^2)$ training complexity) while Info-Coevolution can directly scale from 12w samples to 87w samples without a in-middle training step.

• Additionally, Info-Coevolution doesn't introduce any data distribution problems, so we are able to do continual training. We have additionally verified that using continual training on previous checkpoints has no performance loss. (1% data with 50 epoch, continual training with selected10% data with 45% epoch, then continual training on 68% data with 40 epoch) The training cost of our algorithm could be O(n) in total for selected data.

Conclusion:

• Info-Coevolution has better performance and much better scalability than previous baselines.

Q2: Ablation study to show the effectiveness of (1) Bayesian fusion, (2) kNN approximation

A2: Thanks for the question. We make a clarification here, and ask for the reviewers' clarification on some question points to further clarify/discuss in the next response.

We have conducted part of the ablation in sec 4.3 ablation, including kNN only, model-only, fused, and their combination of dynamic rechecking. The Data column is referring to kNN-only. It can be seen dynamic rechecking is the one guaranteed non-biased distribution when incorporating model-based gain estimation (model estimation without dynamic rechecking will lower the performance), while kNN-only without dynamic rechecking could improve performance already.

For the calibration, which calibration (e.g., temperature scaling) is referred to? Could you give a reference so we can be more clear and add a comparison?

Monte Carlo Dropout-based uncertainty estimation introduces random noise in the feature space, which is somehow similar to our aggregating information from nearby samples to estimate. But the differences are:

1. our method does not have to do inference multiple times.
2. As a model-based method, Monte Carlo Dropout-based uncertainty estimation is estimating the model optimization space. It could still suffer distribution problems, as the dropout is not considering data distribution (it is more about the local curvature of the model).

For kNN approximation vs. true model entropy estimation, our weighted KNN-based method doesn't introduce much distribution bias. Incorporating model-based estimation usually causes this (as observed with lower performance than randomly selected data), and needs to be fixed by dynamic rechecking or other distribution trimming methods.

Q3: For the Bayesian information gain estimation, no proof shows how kNN-based entropy estimation approximates the true situations or the model-based entropy estimates (approximation error).

A3: We would like to first clarify on the point: is the reviewer referring to the problem that kNN-based entropy estimation or model-based entropy estimation could differ to real entropy?

We add some preliminary discussions here and will update it in the next reply based on the reviewer's feedback. Empirically, the model confidence (defined in section 3) is linear correlated (more than 0.95) to sample prediction accuracy on both training data and validation data. We are using this probability proxy instead of real entropy (mentioned in sec 3), because it keeps this observed linearity instead of introducing log function with an unstable bound, and thus when linearity does hold, the knn prediction can help to decide the sample priority.

We also added some discussion in A1 to Reviewer aJ9B about our Lipschitz constant assumption, which is somehow related.

We are open to further discussions.

We sincerely thank the reviewer aJ9B for the recognition and appreciation of our work, and for the valuable questions as well as comments. For the comments and questions, here are our responses:

Q1: The Lipschitz constant may be difficult to compute rigorously and large in practice.

A1:Thanks for the good question. Previously we use the theorem 3.1 to support our introduction of locality, but it is true that the Lipschitz constant is not tight enough to be used in many real settings. We here discuss a tighter bound and the overall reasonability.

For a single linear layer ($g$) with W ($\mathbb{R}^{d\times C}$) and b, its Lipschitz constant is bounded by the spectral norm of W, which is $||W||_2$. However, for a Gaussian weight with $N(0,\sigma^2)$, it could still have a bound of $O(\sigma(\sqrt d + \sqrt C))$. In practice, this is still quite big, and it only serves as a worst-case bound.

There is a tighter bound using the local gradient assuming smoothness (which is true for most type of $g$): $\lVert g(z_1)-g(z_2)\rVert_2 \leq sup_{\lambda \in [0,1]} \lVert\nabla g(z_\lambda)\rVert_2 \leq sup_{\lambda \in [0,1]} \lVert \nabla g(z_\lambda)\rVert_2 \cdot\epsilon ,$ Where $z_\lambda = (1-\lambda)z_1 + \lambda z_2$ .

Empirically, the linearity mainly takes effect when the predictions are reasonably good. It is because

• On one hand, when the prediction is poor, the neighbour could be more random and the predicted knn-prediction will give a high entropy itself as well as the model, which will not contradict our purpose of the algorithm (the corresponding sample would very likely have a high entropy and gain, and the theoretical guarantee of knn pred is from the k value);
• On the other hand, when the prediction is good (e.g. p>0.7 or higher) in the region, the local gradient will be smaller, and the softmax backward propagation further suppresses the bound with $||p-y_t||<=1$.

So in the cases where the local gradient might fail to give a reasonable bound, the result itself would very likely have high entropy and be correctly estimated as high entropy by both model and knn, and the knn's estimation error doesn't matter too much. The actual algorithm reliance on this linearity is relaxed in bad cases due to the algorithm design. And for regions with good predictions we can empirically evaluate the value.

Q2: The method claims to be able to address data distribution shifts however it was not shown in the benchmarks

A2: The data distribution shifts in the article refer to the problem of model-based active learning: it only emphasizes on samples learned poorly by the model, and the selection causes a distribution problem and worse performance. This is shown in the ablation as dynamic-rechecking/KNN-only are all without this performance drop. We will revise the corresponding parts to make this clear.

Q3: The set of experiments does not demonstrate the generalizability of the method to other data modalities or models, and expanding that evaluation would make the analysis stronger.

A3: Thanks for the advice. We further add semantic segmentation experiment on ADE20k with UperNet(BEiT-V2 backbone). See A2 to Reviewer X845. For modalities other than vision, if there is a sample-level feature, it is also possible to extend the framework to them. Will add more discussion in the revision of future work.

Q4: Typo in the abstract

A4: Thanks, we have fixed it in local revision. Should be "… Info-Coevolution reduces annotation and training costs by 32% without performance loss".

Q5: Hyperparameters used in the framework (such as the distance and similarity thresholds). Could the authors give more insight into finding the optimal values, and how adjusting them might affect the method’s performance.

A5: The hyperparameters should depend on the feature space itself (how dense is the data in the space), and can use empirical methods to evaluate(retrieve some samples to estimate the knn distance distribution and knn-prediction correlation, and an adequate k).

Generally, k = 8, cosine similarity 0.9 is good enough, and cosine similarity 0.85 also works in the setting; 0.95 will be too big as there will be very few near neighbour pairs.

Too small distance (too large similarity threshold) may fail to reduce cost and become a model-based method (for dynamic rechecking. Actually using 0.9 for knn and 0.85 for dynamic rechecking is also a good choice), while too large distance threshold may affect sparse-region samples' knn-prediction (their k-near-nighbour could be further and less linear correlated, and should be kept for the sake of generalization).

In our actual use cases, the threshold value is not sensitive, cosine similarity 0.9 directly works without tuning, and cosine similarity 0.85 doesn't make a statistically significant difference.

We sincerely thank the reviewer X845 for pointing out the missing references as well as the potential improvements. We make responses as follows.

Q1: Dataset Distillation references not discussed.

A1: Thanks for the advice. In general, dataset distillation and active learning have a main overlapping research area which is coreset selection (if not requiring full annotation in advance, then the coreset selection can also serve as active learning), and is introduced in related works. Dataset distillation methods also include synthetic data methods, which is another setting not comparable in this work. We will add a discussion of dataset distillation to the related works to make it more clear.

Q2: Task Scope beyond image classification such as object detection and segmentation, is not investigated.

A2: Thanks. We here extend our algorithm to semantic segmentation and attach results on ADE20k with UperNet(BEiT-v2 backbone). The UperNet has a backbone (BEiT-v2-large) and a UperHead. We use the backbone feature from the last layer (dim 1024*16*16) and mean pooling it to dim 1024. We adapt Info-Coevolution accordingly, using pixel-wise confidence avg as model confidence, class-wise similarity weighted confidence as knn confidence, and $gain=1-avg(confidence_{model},confidence_{knn})$. The result for using 1% data trained model to select 10% data is as follows:

ADE20K 10% random

ADE20K 10% our selection (using BEiT feature)

Analysis:

• It can be seen that the mAcc, mIoU, and mAcc are improved by 0.65, 1.50, and 2.09 respectively, which is significant at this data amount. (For larger data ratio we are still running experiments and will update the results in the later updates. We will add thorough experiments in the revision. The code of this part will also be published.)

Conclusion:

• Info-Coevolution is generalizable to more tasks such as segmentation as analyzed.

Q3: Theoretical Depth: requires further expansion, particularly regarding limitations and assumptions.

A3: Thanks for the advice. The current theoretical part supports us to introduce locality into the algorithm, which could greatly improve the efficiency of distribution-based entropy estimation methods, and benefit efficient data re-balance (dynamic data rechecking).

We agree an expansion should be in the assumption of Lipschitz-continuous, please refer to the discussion in the reply A1 to reviewer aJ9B Q1 (due to char limit this year). We are open to further discussion.

On the other hand, the current assumption is that the target distribution should be between IID and uniform, so any sample predicted as redundant by the framework doesn't affect the final classification result. We will attach a theoretical discussion in the next update for this argument. So a limitation is that it currently doesn't handle a target distribution beyond the range of IID to uniform. A solution could be adjusting the sampling frequency based on target distribution.

All the discussed parts will be added to the revision, and we are open to further discussion for the theoretical part.

Q4: Include a broader discussion on Limitations and Future Work (applicability to weakly supervised and non-classification tasks)

A4: Thanks. The semantic segmentation experiment is now added in A2. We will add a more detailed discussion in the revision.

In general, the framework could be applied to models with an $f$, which produces a feature for each sample (and could be used for a retrieval task). If mutual information could be approximated as in classification, it could be a good usage case; even if not a direct approximation, the ANN could also provide distribution-based estimation (with a weighted mean of uncertainty).

Classification can be interpreted as a kind of coarse-grained retrieval.

For weakly supervised training, as for CLIP/BLIP schema, they are retrieval tasks directly training the $f$ with contrastive training. The locality is also taking effect, and mutual information could be estimated in the feature space. So theoretically our framework is also applicable to weakly supervised training, and the reannotation gain could be estimated for human/ChatGPT annotation to improve data quality.

Q5: Real-World Validation

A5: We evaluate the method with some in-house data pipeline, and it achieves promising results.