Contrastive Visual Data Augmentation

We are pleased that reviewer DV9x finds:

• our idea original and sensible
• our technique novel
• our references sufficient
• and the modularity of our approach beneficial

We value your constructive comments and address them in detail:

---

Related Data Augmentation Methods

We want to highlight a key distinction between our generation-based and existing editing-based augmentation (ARMADA and all methods in [1]): While our method generates T2I augmented images from scratch, editing-based methods create augmented images by perturbing attributes of real images of other concepts (L94-105), leading to two drawbacks:

• they require large amounts real images, which are rare for novel concepts.
• augmented images often lack background and view variation.

Experiments in Table 2 show that fine-tuning with such data yield very limited gains, on par with traditional editing-based augmentation methods like Flipping.

Furthermore, while we aim to thoroughly compare with published related works such as ARMADA, at the time of submission, both TTIDA and [1] were not published in any known conference or journal. Nevertheless, we are happy to cite and thoroughly discuss both works in our camera ready version.

---

Can VLMs Describe Features of Truly Novel Concepts

While editing-based methods can only create augmented images by perturbing known attributes present in existing concepts, our generation-based approach can design any novel feature to describe novel concepts. Therefore, this limitation only exists for editing-based methods, while CoDA is able to create any novel feature that can be described with human language.

Furthermore, Chiquier et al. [2] showed modern VLMs can generate high-quality novel descriptive features for rare species and even otherworldly concepts such as those in the KikiBouba Dataset.

---

Clarification on Compute Efficiency

We'd like to clarify: By "compute efficiency", we refer to efficiency during fine-tuning with augmented / real images. As described in L631-654, CoDA is inference-only and does not involve any expensive back-propagation. Thus, CoDA's compute cost is an order of magnitude less than the fine-tuning process.

Our efficiency claim refers to: when comparing models fine-tuned with CoDA augmented images vs models fine-tuned with all real images or ARMADA images, models fine-tuned with CoDA images can achieve higher performance with lower fine-tuning cost. Table 2 results justify this: training on 6 images per concept of mixed Real+CoDA data outperforms training with 20 real images per concept (using over 2x the compute).

We acknowledge the word 'compute efficiency' was ambiguous in the paper and will clarify it in the camera-ready version.

---

Can T2I Models Generate Multiple Attributes

This is an issue we encountered many times. Consistent with Conceptmix, we found older T2I models like SD-2.1 tend to fail at generating >3 attributes in a single image. However, newer T2I models not included in the benchmark, like SD-3.5, can reliably generate >5 concepts.

Furthermore, our two-layer feature filtering (L171-209) and image filtering (L236-251) strategy is specifically designed to filter out any remaining generation failures. Our human evaluation in (Table 1) demonstrates that the vast majority of features (83.97%) were successfully generated:

---

CLIP Discriminability/Generability scores

Our intentional math formulation of Discriminability/Generability scores ensure that the proposed association bias of CLIP would actually aid in filtering out non-discriminative features, while having no impact on the relative ranking of generability scores. Due to word limits, please respond for a detailed breakdown.

---

Dataset Size and Generality of Results

We want to clarify: while the NovelSpecies dataset has 64 different species, it actually contains 2240 annotated images. Results on NovelSpecies is reported on val and test sets each containing 960 unique images (L275-277). We tested three models on this dataset to show CoDA's generality (Table 3).

To further prove the statistical significance of our NovelSpecies results, we ran 5 independent sets of experiments on 10 different settings of NovelSpecies with different random seeds:

Results show remarkable consistency: average standard deviation of scores is 0.0133 across 10 settings, with a max of 0.017. The average coefficient of variation is 1.82% across all settings, far below the general 5% threshold for statistical stability.

(Respond to see full score table)

[1] Advances in diffusion models for image data augmentation. AI Review (2025)

[2] Evolving Interpretable Visual Classifiers with Large Language Models. ECCV (2024)

We sincerely thank Reviewer bmru for their insightful and encouraging comments. We are pleased that you found:

• our paper well-written to follow,
• our proposed data augmentation strategy effective,
• our method's performances competitive,
• and our new NovelSpecies dataset interesting and novel.

We are also encouraged that you recognize that our solid experimental results demonstrate the effectiveness of CoDA.

Please find your suggesion and additional feedback carefully addressed below:

---

Spacing Between Figure and Table Notation

Thank you very much for catching this formatting issue, your detailed efforts are greatly appreciated. We fully agree that correcting this notation throughout our work would improve the readibility and clarity of our work. Therefore, we will make sure to correct all such usage instances in our camera ready version.

---

More Visualizations like Figure 3 in Paper

We fully agree with the reviewer that visualizations like Figure 3 can help illustrate our method and provide straightforward comparisons with other related methods. Moreover, we currently plan to include an additional figure in our camera ready version to provide example of success and (rare) failure cases in our visual data generation process. We believe this will help readers better understand the abilities and potential limitations of our work and facilitate better method adaptation to more domains.

---

Whether Online Large Models can Recognize Novel Concepts

This is a very interesting question raised by the reviewer. We believe that indeed online models with access to up-to-date internet information may be better suited for recognizing novel concepts compared to static models. However, when qualitatively testing current online VLMs such as GPT4o and Claude 3.7 Sonnet, we find that they still fail to recognize most of the concepts in our NovelSpecies Dataset. We believe this is likely due to their reliance on textual retrieval and inability to retrieve the relevant information based on novel images.

We sincerely thank Reviewer pV1R for their detailed review. We are delighted you recognize that:

• our method is novel and provides concrete contributions to the state of the art,
• our claims are extensively demonstrated,
• our datasets are solid choices to provide extensive validation

We are also encouraged that you find our paper well-placed in the current literature, addressing an important problem.

We have carefully considered your valuable feedback and suggestions for improvement, which we address in detail below:

---

Comparison to Fine-tuning Methods

We very much appreciate the reviewer bringing up this important point and would like to take this opportunity to clarify: While a main contribution of our work is providing a state-of-the-art visual data augmentation strategy, we leave downstream users the freedom to decide how to use our augmented synthetic visual data, whether it is model adaptation, few-shot learning, fine-tuning, or even pre-training. In our experiments, we focus extensively on the fine-tuning use case as it is the most general and intuitive way to utilize our augmented visual data and demonstrate its general usefulness for improving different models.

Specifically, we compare fine-tuning existing models with all real data, mixed real and synthetic data from baseline data augmentation strategies, and mixed real and synthetic data from our CoDA method. Results in Table 2 and Table 3 show that basic fine-tuning with all real data or including synthetic data from baseline data augmentation strategies produces much lower performance compared to fine-tuning with mixed real and synthetic data from our CoDA method. (L330-420)

For better reference, we paste the following results subset from Table 2 of our main paper to illustrate the performance disparity of basic fine-tuning with all real data vs including one synthetic image per concept from baseline data augmentation strategy ARMADA and our CoDA method:

---

Relation to Test-time Augmentation Methods

We thank the reviewer for pointing out the relations between our work and existing test-time augmentation techniques, a topic we extensively considered and explored during our project. As elaborated in our response to Issue #1 above, we position our work as providing a state-of-the-art visual data augmentation strategy, while allowing downstream users to decide whether to use our augmented data for pre-training, fine-tuning, or test-time augmentation.

To validate the usefulness of our augmented data, we currently focus on the fine-tuning scenario because:

• General applicability, as fine-tuning is generally applicable to LMMs as well as traditional classifiers.
• Empirical effectiveness, as the difficulty of the visual novel/confusing concept recognition task limits the effectiveness of test-time augmentation methods.

In fact, during our exploration stage, we ran multiple test-time few-shot generalization experiments on VLMs. Here we attach our earlier results for 1-shot and 3-shot generalization of LLaVA 1.6 34b, under the extremely simplified binary classification setting:

The model achieves no better than random chance performance on the three datasets with few-shot learning. Further qualitative experiments on proprietary VLMs also showed little promise, which prompted us to move towards fine-tuning methods for better performance.

However, we recognize that the reviewer's comment regarding test-time augmentation methods goes beyond basic few-shot learning. We have identified several areas within multimodal test-time augmentation methods that may be promising given our CoDA method or relevant to the novel/confusing concept recognition task:

• Multimodal meta learning and fast adaptation methods such as Flamingo
• Code-based multimodal reasoning methods including VisProg and ViperGPT
• Multimodal reflection and self-critique methods including LLaVA-Critic, Critic-V, and VDebugger
• RL-based inference time scaling methods such as Visual-RFT and VAGEN
• Multimodal RAG methods including Wiki-LLaVA, UDKAG and UniRAG

We will include discussions relating our method to these test-time augmentation strategies in the camera ready version.

---

Highlighting the Effectiveness/Efficiency Gains

We thank the reviewer for recognizing the substantial effectiveness / efficiency gains of CoDA demonstrated in the experimental section. To highlight these gains, we will include an additional bar chart comparing performances of CoDA against other baseline methods. We believe this visual comparison will help readers more easily grasp the effectiveness / efficiency gains of CoDA.

We sincerely appreciate Reviewer pV1R for the insightful review. We are encouraged that you find:

• our claims well-supported by evidence,
• our evaluation criteria and benchmarks appropriate,
• our experiments sound and well-executed.

We are also glad that you enjoyed reading our submission, while finding it well-written, well-structured, with good discussion.

Please find your suggestions carefully addressed below:

---

Discussion on Failure Modes of Synthetic Image Generation

We appreciate the reviewer for this constructive suggestion. Given more space in the camera ready version, we will add the following section in the main paper to explain failure modes in synthetic image generation, along with image examples.

As with any neural generative approach, our method is inherently constrained by the underlying T2I model’s capacity to represent the real world. While we expect this phenomenon to be gradually diminished with the advent of newer and more powerful T2I generative models, as of current publication we can still observe occasional generation results of nonsensical or biologically implausible artifacts (e.g., two-headed snakes) in rare cases. Additionally, while CoDA emphasizes class-discriminative features, it does not explicitly control for other attributes in the generated images. As a result, there can be unintended biases—for example, the backgrounds in generated images for some classes can be quite similar. While we did not observe any significant impact of such biases to our novel and confusing concept recognition tasks, users adapting our method to different use cases should be aware of potential unintentional impacts.

---

Dedicated Future Works Section

We fully agree with the reviewer that problems are never solved and would be useful to provide more insights on how this work could be improved upon or carried out further. Given additional space in the camera ready version, we will add the following future works section to our main paper:

While a main contribution of our work is providing a state-of-the-art visual data augmentation strategy, we leave the downstream innovation on how best to use our augmented visual data to improve models for future work. In our experiments, we focus on the fine-tuning use case as it is the most general and intuitive way to utilize our augmented visual data. Besides this, other conceivable potential use cases for our augmented data include model adaptation, test-time augmentation, visual RAG or even pre-training. The modularity of our method also invites other researchers to replace components of CoDA with superior models to achieve better performance. The NovelSpecies dataset, which will continue to be updated with new species every year, may also be used to evaluate future VLMs' novel concept recognition abilities. Finally, we also expect improved versions of T2I generation-based visual data augmentation techniques to eventually surpass CoDA in effectiveness and efficiency, potential improvements may include more robust image / feature filtering and more controllable text-conditioned image generation like multi-view synthesis.

---

Computational Overhead Comparison

We thank the reviewer for bringing up this point: While we have thoroughly discussed the computational cost of CoDA in our appendix A.3.(L629-654), it would be very helpful to additionally compare this cost with that of existing visual data augmentation baselines such as ARMADA, so practitioners may reference it when considering adoption. We plan to provide this information in the camera ready version via an additional figure.

---

Explanation of "Confusable Concept"

We are glad that the reviewer brings up this point, which we would like to clarify: While CoDA chooses a confusable concept for each target concept, this process is very general and simply based on model misrecognition.

For example, when LMMs are tasked to classify a novel concept it has no previous knowledge of, the VLM will simply hallucinate and provide the closest confusable concept. CoDA then teaches the VLM the target concept by illustrating the visual differences between the target concept and its corresponding confusable concept. This effectively reduces the difficult and costly task of learning this novel concept from scratch to the much simpler task of learning the difference between the target concept and its confusable concept.

The effective learning cost reduction will depend on the level of similarity between the target concept and its closest confusable concept. However, we believe that as VLMs become more knowledgeable, they will also become better at finding higher quality similar confusable concepts, and thus dramatically reducing the cost of learning to recognize novel concepts.