Semantic-aligned Query Synthesis for Active Learning

1. We cannot guarantee that $v_x^* (= t_x^*)$ is the same with ImgEncode(StableDiffusion(DeCap($t_x^*$))). It is necessary to evaluate the cosine similarity or distance between these embeddings.

2. It is difficult to confirm whether the experimental results are caused by the most important components $x^*$ in the proposed method. GenMQD outperforms RandomText, which uses only class names, but it should also be compared to using sentences generated by LLMs that simply describe the class.

3. The experiments were conducted solely on datasets with a limited number of classes.

[Baselines] I think this method should be compared with not only query / data synthesizing methods but also sample selection methods that is traditional AL techniques. This is because sample selection is currently the major stream in active learning research domain.

[Prove the effectiveness] Tab. 1 shows that the proposed method has lower performance than the other baselines, which seem to be a different setting. However, in the same setting, there is no other methods to compare with proposed method. I think several baselines should be added in the same setting.

[Unfamiliar datasets for CLIP] The authors adopted the pre-trained CLIP for extracting image features and generating the text from the features. However, there are several datasets that are not familiar with CLIP such as Flowers102, EuroSAT, etc. I'm wondering that This method is applicable for all the datasets? Or is it only applicable for specific datasets that are familiar with CLIP?. In former one, I think the PEFT method (e.g, LoRA) can be applied into CLIP.

[Related Works] It seems to be that Related Work section needs to be enhanced. I suggest the recent AL papers as below:

[1] Active Prompt Learning in Vision Language Models, CVPR 2024

[2] Active Generalized Category Discovery, CVPR 2024

[3] Entropic Open-Set Active Learning, AAAI 2024

Overall, it seems to be good approach, but there are several concerns in this work. I am going to raise my score if the concerns are solved.

1. The literature review lacks comprehensive coverage of recent developments in synthetic data generation for training, particularly from the past three years.
2. The comparative analysis relies on outdated baselines, with ActiveGAN (published 5 years ago) being the most recent comparison. More recent approaches are suggested to add.
3. The improvement is mild. The proposed method uses the knowledge from the CLIP model, which is pretrained on a large-scale data. However, the zero-shot of the CLIP model on CIFAR-10 can achieve 91.3 Acc, whereas the proposed method has similar performance but using additional resources.
4. The computational complexity analysis is insufficient, particularly regarding data generation time. Critical computational bottlenecks include: 1) Hessian matrix calculation; 2) Image generation via stable diffusion. These components likely incur significant computational overhead as sample size, image resolution, and hyperparameter optimization epochs T increase.

1. I have concerns on the influence of perturbation, as we can find from the Eq. 5 to Eq. 3, it seems that you are looking for the perturbation of $x^*$ which can most attach to the validation set, in other word, you are looking for the data which should mostly fit into the training and validation distribution. However, since both training and validation set you claimed is very small, therefore, I have little concern on whether this would lead the training to direct involve with the validation set (loss computation of Eq. 5) instead of active querying on the validation set as previous works.
2. It seems the generation process of the $\delta$ would requires a lot iteration steps, I have concerns on the computation overhead.
3. Some benchmark missed, e.g. CIFAR-100
4. The fundamental implementation of this work replies heavily on the pretrained networks, which introduce the prior knowledge of the data and is one thing I concern, is other models your used are also pretrained, e.g. GAN.
5. Some notations and demonstrations are ambiguous, e.g. $p(\cdot)$ process in Algorithm 1, line 11, is the $\bold{t}$ is the matched text results for the $\bold{x}^*$? So is $\bold{t}$ are generated from the images (the previous text embeddings from train)or just calculated from Eq.6, if so, why the $\bold{t}$ is not equal to the $\bold{v}_{\bold{x}^*}$? Why it is necessary to mention Eq.6?

1. Although the authors acknowledge that the performance difference between GenMQD and GenMQD-Fea across datasets is an open question, this issue must be addressed.

2. Given the marginal performance differences across entities, a random-seed analysis should be conducted. Without this, it is challenging to interpret the impact of "the number of query examples" and $n_l$ or $n_v$ in Table 1.