How Many Van Goghs Does It Take to Van Gogh? Finding the Imitation Threshold

We thank the reviewer for their time and effort to review our work. We are glad that the reviewer found our work to have a clear logic, assumptions clearly stated, and have a rigorous experimental verification. We address the limitations as follows:

1. Is it a generalized result in a wide range of scenarios, such as other datasets Laion-400m [1], so as to truly provide valuable guidance for judging whether the infringement claim is established and guiding developers to avoid infringement?

We agree with the reviewer that our claims about the imitation thresholds of around 200-600 images should generalize to other datasets in order for them to be valuable to the community. While our main contribution is a generalizable methodology that will be applicable to all model/data pairs in the future, we rigorously test whether our thresholds are generalizable. For this, we experimented with all the models whose training data was open-source (SD1.1, SD1.2, SD1.3, SD1.4, SD1.5, and SD2.1), with four different domains and found the threshold to be within this range (Table 3 in Section 6 and Table 6 in Appendix G). To answer the reviewer’s question, we also ran experiments with LAION-400M dataset on which a latent diffusion model was trained.

We report the imitation thresholds in the table above. We find that all the thresholds are around the 200-600 range we found for LAION-2B and LAION-5B, and we therefore hope that our threshold will generalize to other datasets as well. We have added this experiment to the paper in Appendix A (L. 864-880).

2. If the imitation threshold is not reached, can it be considered that there is no infringement?

Based on our results showing the relation between a concept’s frequency and its imitation score (Figure 5 in the main paper and Figures 14-28 in the Appendix), we conclude that if the frequency of a concept is below the imitation threshold, it is unlikely for a model to be able to imitate it. Therefore, if the imitation threshold is not reached, infringement is unlikely.

3. Can the quantitative relationship between Concept Similarity and Concept Frequency be given?

We believe that the reviewer is interested in understanding the general mathematical relationship between concept frequency and similarity (which we empirically plot in Figure 5 in the main paper and Figures 14-28 in the Appendix). Our work does not focus on determining the precise mathematical relationship, and instead focuses on identifying the threshold that leads to imitation. However, previous works have found a “log-linear relationship” between frequency and similarity: https://arxiv.org/abs/2404.04125, https://arxiv.org/abs/2211.08411.

Kindly let us know if you meant something different by 'quantitative relationship,' and we would be happy to provide clarification.

4. Add optimal approach to find the imitation threshold (Pearl (2009)) as a baseline comparison to verify the correctness of the MIMETIC2 method

Even though we would like to verify the correctness of MIMETIC^2 using the optimal approach as a baseline, this is an infeasible experiment as we note in Section 3 of the paper. For the optimal experiment, we would need to train O(log m) models, where m is the total number of available images of a concept Z^j. So if a concept has 100,000 images, then we need to train log_2(100, 000) = 16 models to optimally estimate the imitation threshold, which would cost about $10 million. This is way beyond the computational capability of our academic lab (and even industry in most cases).

Therefore, we verify the correctness of the imitation thresholds in a tractable manner using an approximate method. Concretely, we calibrate MIMETIC^2 using one of the sets in each domain (Celebrities for Human Faces and Classical art styles for Art Style) and test the calibrated approach on the other set (Politicians for Human Faces and Modern art styles for Art Style). In essence, the sets that we use to calibrate MIMETIC^2 on, serve as the “train datasets” and the other sets serve as the “test dataset” (these sets are mutually exclusive). Table 3 in Section 6 shows that the imitation thresholds MIMETIC^2 finds for “train” and “test” sets in each domain are very close, indicating that the imitation thresholds MIMETIC^2 finds are accurate.

We note this on Lines 456-467 in Section 6 of the paper.

However, to answer the reviewer’s question, we are conducting a finetuning experiment that is the closest to the optimal experiment that is tractable. We will update the rebuttal with the result in a few days.

We thank the reviewer for their time and effort in reviewing our paper. We are glad that the reviewer agrees that our paper marks the first effort to provide an estimate of the imitation threshold, which is both a significant and underexplored problem. We address the limitations as follows:

1. While this paper represents a novel attempt to determine the imitation threshold of concepts in text-to-image models, its technical implications are unclear. Apart from conceptual policy-making related implications (i.e., informing text-to-image model developers what concepts are in risk of being imitated, and serving as a basis for copyright and privacy complaints), how can this metric possibly be adopted to advance imitation mitigation strategies, such as those outlined in Appendix A?

Our results on estimating the imitation threshold (Figure 5 in the main paper and Figures 14-28 in the Appendix), indicate that if the frequency of a concept is below the imitation threshold, it is unlikely for a model to imitate it. Therefore, the implications of our results are relevant to the data curation practices. of text-to-image models like Stable Diffusion or Dalle: We hypothesize that if developers restrict the frequency of concepts they want to avoid imitating by keeping their frequency below the threshold, trained models are unlikely to imitate those concepts. In addition, our work has implications for lawsuits and copyright compensation for artists whose training data was used to train models. The damage can be estimated by auditing companies that train such models to determine whether the frequency of their training images falls above or below the threshold.

2. what is the primary driver of these threshold differences between SD2.1 and SD1.5 — dataset size or text encoder architecture?

Our results indicate that the difference in the thresholds between SD2.1 and SD1.5 is due to the text encoders. This is because even though all models in SD series 1 are trained on different datasets (https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-v1-5), they have the same imitation threshold ((Line 353 in Section 6), while SD2.1 has a different threshold. Notably all models in SD series 1 have the same text encoder, while the text encoder of SD2.1 differs. We provide the full experimental details in Appendix G of the paper and our conclusion has also been shared by other practitioners: https://www.assemblyai.com/blog/stable-diffusion-1-vs-2-what-you-need-to-know/

Moreover, the paper does not explore other potentially influential factors, such as model size, which limits the comprehensiveness of the analysis.

Our approach requires access to open-source models whose training data is available (we conducted experiments with all such models in the paper). The models that satisfy this criterion vary in factors like model sizes, training data, and text encoders. Therefore it is not possible to ablate the effect of each factor separately. The paper’s methodology is applicable to any model/data pairs in the future that can help to separate out the effects of all factors affecting the threshold once such models become available.

To test whether model size could affect the imitation threshold, we conducted experiments with the smaller Latent Diffusion model trained on LAION-400M dataset and found the thresholds to be similar to the ones found for SD1.1, SD1.5, SD2.1. We have added this experiment in Appendix A in the updated paper.

We thank the reviewer for taking the time and effort to review our paper and we are glad that the reviewer found our proposed approach efficient to estimate the imitation threshold. We address the limitations as follows:

1. The paper does not define "concepts," assuming they are either self-explanatory or domain-specific. How distinct should concepts be from one another to qualify as separate concepts? How is the distance between concepts computed? The paper does not provide details on defining a concept or its boundaries

We define a concept as a specific entity like a person or a specific person’s art style (L46 in Section 1). For the case of faces, each person’s face is considered a distinct concept. For the case of art styles, each artist’s art style is considered a distinct concept and we measure imitation specifically with respect to the artworks that a particular artist has produced (instead of measuring imitation with respect to a group of art styles like Cubism or Futurism). We have added this clarification in the paper (L. 213-214).

The only requirement for an entity to be a valid concept is if the embedding models are able to distinguish it from other concepts (which is important for accurately measuring a concept’s frequency and imitation score). We use domain-specific embedding models (one for faces and another for art style) that are able to distinguish between concepts accurately. We conduct experiments to empirically verify this and present the results in Figure 12 (a) and (b) in Appendix F: we observe that the embedding similarity between artworks of the different artists is much lower than the similarity between the artworks of the same artist -- indicating that our embedding model can separate the art styles of different artists accurately, and these art styles are a valid set of concepts. In Figure 7 in Appendix E, we present the results for the embedding similarity between the faces of the same and different individuals. We find that the similarity between faces of different individuals is much lower than the similarity between faces of the same individual -- indicating that the embedding model distinguishes between faces of different individuals accurately.

We hope that these experimental results allay the reviewer’s concern about how a valid concept is selected. We note that using a different set of embeddings models can change the set of concepts one can use for the experiment.

In cases of composite and overlapping concepts, learning one concept may transfer to another, leading to erroneous imitation thresholds.

As our experimental results show in Figure 7 and Figure 12, the embedding models can accurately distinguish between the different concepts we experiment with, the individual faces and art styles. Therefore even if there is an overlap between certain concepts (e.g. multiple art styles of Post-impressionism kind), we can accurately measure the frequency and imitation score for each concept, alleviating concerns about the overlap between different concepts.

2. How do the authors make sure that the increasing number of samples of a particular concept is causing the models to learn to imitate?

To establish causal relationship there are two major approaches: RCTs and using observational data to estimate the effect. While RCTs in our case would be extremely expensive (L 150-151), we are able to estimate such effects using the observational data approach. As such, we assume there are no confounding factors that affect both the frequency of a concept and its imitation score. Under this assumption, from our experimental plots, the frequency of a concept causally affects its imitation score (an increase in frequency increases the imitation.) This intuition has also been verified by previous works which argue about a concept’s frequency and a model’s ability to perform tasks related to those concepts (such as image classification, image generation, or text retrieval). All these works find that as the number of samples of a particular concept increases, it increases the ability of the model to perform these tasks. No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performance [https://arxiv.org/abs/2404.04125] show this relation for image classification and generation tasks, and Large Language Models Struggle to Learn Long-Tail Knowledge [https://arxiv.org/abs/2211.08411] show this for question answering tasks. Large Language Models Struggle to Learn Long-Tail Knowledge even performs causal experiments to establish that frequency indeed affects model’s performance for that concept. Therefore based on these past works, we can claim that increasing the number of samples of a concept would increase the ability of a text-to-image model to imitate it.

For example, how do we discount the case that if we train a diffusion model on 1 million concepts with each concept represented by 100 images the model may still be able to imitate a particular concept due to an overall increase in general image generation understanding?

We would like to address this misunderstanding. We designed our setup to find the imitation threshold for text-to-image models that are trained with real world datasets. In this setup, we find that concepts lower than the threshold are unlikely to be imitated. Our setup cannot be used to deduce the imitation threshold for this scenario of each concept having the same number of images, as it is significantly different from our setup. Therefore, we do not claim our imitation thresholds would transfer to all model/data pairs (L 520-522 in Section 8).

3. The assumptions of the paper are too strict, which makes the method to be of much practical relevance. Can the authors justify these assumptions? Further, can the authors show experiments upon loosening these assumptions?

The assumptions we make are standard when answering causal questions using observational data and have been made in several previous works: Pearl (2009); Lesci et al. (2024); Lyu et al. (2024). We have also empirically verified the validity of the assumptions, specifically the distribution invariance assumption between the concepts. If this holds true, then the imitation scores of two concepts (from the same domain) whose image counts are similar, should also be similar. To test whether this is empirically true for the domains we experiment with, we measure the difference in the imitation scores for concepts whose image counts differ by less than 10 images and report the difference averaged over all such pairs. The table below shows that the average difference in the imitation scores for all the pairs, for all the datasets we experiment with, is very close to 0. This provides empirical support for the validity of our assumptions. We also present this in Appendix B (Appendix C in the updated paper).

Can we introduce error bounds to account for the strict assumptions?

Thank you for this suggestion! In order to introduce error bounds on the imitation thresholds, we conduct an experiment in which we randomly sample 300 concepts from the set of 400 concepts for each domain we experimented with, and estimate the imitation threshold for each sampled set. We repeat this sampling process 1000 times for each domain and present the mean and variance of the imitation thresholds for each model. This experiment simulates a scenario where the invariance assumption holds only for a subset of the 400 concepts.

We find that for almost all domains and models the imitation thresholds we report in the paper are within the one standard deviation of the mean threshold this experiment finds. Our original claim about 200-600 images being required for concept imitation holds with this experiment as well. We updated the paper with this new experiment in Appendix C (L. 921-937)

4. Another assumption not listed in the assumption section is "prompts are distinct from the captions used in the pretraining dataset to minimize reproduction of training images". Interestingly, the famous New York Times vs. OpenAI lawsuit mentions many examples where exact reproduction is observed.

We would like to highlight that this is not an assumption of our work, but a core property of our research question. In this work, we are not tackling the question of memorization (which was well studied in previous works, e.g., L 812-814), but instead, are interested in the question of imitation, which is under-studied. The NYT vs. OpenAI lawsuit would correspond to an exact replication of a training image by a text-to-image model (if the lawsuit was related to text-to-image models). Therefore using “prompts distinct from the captions in the pretraining dataset” is an intentional choice rather than an assumption of the paper (because we want to measure imitation and not memorization). We choose to use different prompts as this was suggested by Somepalli et al. [https://arxiv.org/abs/2305.20086] to prevent generation of memorized training images, and therefore if a concept is generated by a model when using the different prompts, we can be confident about the imitation of the concept and not memorization of a training image.