Rethinking Artistic Copyright Infringements In the Era Of Text-to-Image Generative Models

Thank you for your valuable feedback. We first address your concern the level of legal discussion.

We completely agree that legal context is very important, as ICLR’s audience may have less exposure to relevant legal discourse. To provide greater legal context in a more pedagogical manner, we add a new appendix section (B; after limitations) that reviews the history of copyright (from concerns with printing presses in 1710 to chromolithography in 1870 to motion pictures in 1912 and sound recordings in 1971), the current legal landscape, and how our work may fit into it. Further, we clarify that we do not seek to ‘advance a new legal theory’ nor to simplify (via automation) the nuanced issue of copyright. We highlight the following paragraph from our new appendix section:

Our purpose in this paper is not to take a position on the issue of whether copyright law should be extended to protect individual artistic styles... Neither is our purpose to automate the analysis of copyright infringement. Rather, we are interested in investigating whether there is any scientific support for the idea that there are identifiable individual artistic styles, and that those styles could be correlated with a group of human-understandable stylistic terms. …when we try to train a model that can correctly identify authors of previously unseen works, we may get closer to understanding whether and when individual artistic styles exist. If we can link that classification to a selection of terms describing characteristics of those works, then we can explain to human beings what the components of such individual artistic styles might be. If some kind of protection of individual style ever became part of copyright law, judges or juries would still have to decide whether a particular output of a generative AI tool too closely mimicked the individual style of an artist. The degree to which an AI model could or could not correctly identify the author of a work, and the stylistic terms that that model could or could not correlate with that classification, would simply provide additional information to those human decision makers.

In other words, our tool aims to assist human decision making, not replace it. We identify how deep learning techniques can be used to quantitatively study relevant questions to the copyright debate, like if unique styles exist, how they can be articulated, and how much they reappear in AI-generated work.

We specifically designed our methods so that jurors can engage with their outputs, even as our tool processes tens of thousands of artworks. Namely, our tool can aid the ‘intrinsic’ decisions by:

• Surfacing the most similar artists (and their works) to a plaintiff artist, by seeing whose work is most frequently misclassified to the plaintiff or vice versa. The humans can then decide if the plaintiff’s style differs from the most similar artists sufficiently to have a 'unique' style. We explore this Appendix C.1 for instances where artists whose styles do not meet our threshold for uniqueness.
• Presenting side-by-side sets of art that match a tag signature, with the elements comprising that style articulated via TagMatch’s interpretability and data attribution. The humans can compare the similarity of the two sets (do they both match the listed tags?) and compare these sets to artwork matching a subset of the tags (does adding one more atomic tag actually make the style more unique than other existing work?)
• Engaging questions like ‘unprecedented similarity’, where humans can inspect a plaintiff’s work, generated art, and art from the most similar existing artist. Then, humans can decide if the generated art meets an unprecedented level of similarity to the plaintiff’s style, exceeding the similarity seen by any other human artist. We explore this deeper in Appendix C.2. Our tool uses the scalability of machines to gain insight and retrieve the most relevant artworks from a vast corpus, so that human decision makers can be more informed in their judgments. Also, we present these insights in an easy-to-digest manner: instead of saying Artist A and generated art A’ have a similarity of X (via some black-box metric), our tool says generated art A’ is confused as having been authored by artist A (over hundreds of other artists) Y% of the time, while also surfacing examples and articulating stylistic terms that contribute to the similarity.

Intent of the AI user: we exclude this from our framework, as determining a user’s intent may be infeasible – we instead focus on questions with greater potential for quantitative approaches. We mention the Adobe work to show the emerging importance of artistic style copyright to legal and technical stakeholders alike.

We'd be happy to discuss this topic further, as we agree that it is very important. In fact, a legal expert has been an integral part of our team from the start, so to appropriately ground our work in existing legal scholarship.

Thanks to the authors for the addition of Appendix B. This discussion is much clearer than the framing in the main text!

The appendix now clarifies (1) the articulation of the goal of "investigating whether there is any scientific support for the idea that there are identifiable individual artistic styles," and the (2) explicit clarification that "neither is our purpose to automate the analysis of copyright infringement."

Given the clarified goals of the paper in the appendix, I suggest that the authors consider revising the paper to clarify (and pursue) this these points in the main text rather than just the appendix.

Specifically: On lines ~87 the argument that the "existence and uniqueness of artistic styles can be reduced to classification" sticks out as an undefended assertion (empirically). This equivalence of style to artist classification is not obvious to a computer vision expert - classifiers can have many inductive biases different from style - and this equivalence it is not investigated empirically in this paper, even though the paper asserts that the system is useful for understanding style.

For example: an artist classifier might achieve high accuracy by looking for an artist's signature in the corner of the image (or some other confounding feature) rather than making a judgment based on the holistic style. In light of the clarified goals, the lack of any check of this assertion is one of the paper's main weaknesses, and should be fixed before publication.

The problem could be addressed if the authors were able to do two things:

(1) Change the wording around line 87 to clarify that, although legal arguments would suggest that styles can be reduced to artist identification, one of your goals in this paper is to, as the authors say, "investigate whether there is any scientific support" for this view, i.e.,g to empirically test whether an artist classifier can successfully identify an artists' style.

(2) Then such an investigation should be done, i.e., conduct an evaluation of your classifier(s) to compare whether their classification judgments match what people perceive as "style" in an image. I can see a few ways to do it. One way to do this would be to conduct a human evaluation. Another way is to compare your classifier to the prior state-of-the-art in style identification, the CSD method from Somepalli 2024, i.e., to see if the judgements of your classifier would match their style metric. The Somepalli work is open-source and should be able to be done with an automatic evaluation. Alternately, you could adopt the Somepalli method as the basis for your classifier, and then you'd be able to argue that the empirical evidence they collected would support your point.

Look forward to the authors' perspectives on the suggestions above.

Thank you for your insightful comments! We apologize for extenuating circumstances leading to delay in our response. We address comments and discuss a new experiment below.

Ambiguity of style. You're right! Style is qualitative, and this precisely is what makes questions about copyright protection for styles very challenging. Our work helps clear up some ambiguity around style by taking a quantitative approach based on legal scholarship and an intuitive logical argument (i.e. an artist has a unique style if their work can be consistently recognized). With this framework and central idea, we study key relevant questions, leading to new evidence suggesting that many artists have unique (recognizable) styles, and that generative AI may copy some of these styles. Thus, despite the ambiguity of style infringement, it requires study, as some artists may already be at risk.

We also find evidence of some artists with very similar styles – exactly as you mention! – using our method (Appendix B.1), demonstrating the utility of our framework. Other specific ways in which our tool can help human decision makers include (a) surfacing the most similar artists to a plaintiff artist (based on whose work is most frequently misclassified to the plaintiff or vice versa), along with generated art, so that a jury can have more evidence to decide if the AI generations pose an unprecedented level of style similarity; (b) presenting side-by-side sets of art that match a tag signature, with elements comprising that style articulated via TagMatch. In summary, our tool provides quantitative insight to key questions around style and style infringement, toward helping the human decision makers (judges, juries, lawyers) navigate the nuance/ambiguity you mention.

Deeper comparison to image-similarity methods

We now present a new experiment comparing our method to baselines like CLIP and CSD (a version of CLIP fine-tuned to better measure style similarity; Somepalli et al ECCV '24). We test how well each method can retrieve images from a fixed reference set of artworks based on similarity to a query artwork. We take the train/test split of our dataset serve as reference/query sets for retrieval. We count each retrieved artwork created by the same artist as the query as a true positive, and if created by a different artist as a false positive. As DeepMatch is not intended for retrieval (unlike the baselines), we repurpose it's classifier for retrieval by using the output softmax probabilities for an image as an encoding of that image (i.e. each image is represented by a 372-dimensional vector, where the $i^{th}$ element corresponds to the likelihood that the image is authored by artist i).

In Fig. 11 of App. D.3 (new), we plot the distribution of AUROCs over the artists in our dataset for the baseline methods and DeepMatch. We find that CSD outperforms CLIP in this task (mean AUROC of 0.89 vs. 0.86), and DeepMatch performs best of all (mean AUROC 0.98).

Intuitively, our improved performance can be linked to the original objective of the three methods considered. Image similarity methods are not equipped or intended to measure stylistic similarity, but rather only a general sense of similarity between two images. Even methods such as CSD which are specifically trained using a contrastive loss to be invariant to style preserving transformations are not aware of the components that constitute a unique artistic style. That is, \emph{while CSD is trained to contrast general art styles (e.g. impressionism from cubism), it is not trained to contrast between two (potentially very similar) artists (e.g. Manet's impressionism vs. Monet's impressionism). From a copyright perspective, we are precisely interested in only those stylistic components that set an artist’s work apart from their peers - that is, a unique artistic signature. This naturally suggests a framework which analyzes style from the lens of image classification. Our method, DeepMatch, is trained to classify artworks and thus upweights stylistic features which are most unique and thus useful for classification.

Composing tags makes them more specific, better capturing the nuance required to distinguish artists. Precisely as the you note, each individual tag is insufficient to define a unique style, as artworks sharing one tag can be vastly different from one another. This is why tag composition is necessary: we observe that while each individual tag does not define any artist’s unique style, combinations of tags can form signatures, where only one artist frequently uses all the tags in the combination together (see Fig. 6). While TagMatch employs a zero-shot multi-label tagging scheme with CLIP, the true novelty of it comes in this tag composition step and how classification is ultimately performed (via look-up of tag signatures), which makes it interpretable and attributable by design.

We'd love to answer any follow up Qs! Thank you.

Thank you very much for your time and valuable feedback! We answer questions below.

Performance for lesser known artists. As detailed in our response to reviewer gWF2, we find very little difference in performance for TagMatch over artists grouped by popularity (measured based on web traffic to their Wikipedia page); over four quartiles, TagMatch accuracies fall within a 7% range. While there is little to no gap based on popularity of the artist, we nonetheless note that TagMatch is flexible in that both its vocabulary and tagger can be swapped out with analogs that better cover more niche styles as needed, giving the user greater ability to engage with and modify TagMatch as needed.

Spurious correlations. This is a great point and further emphasizes the need for an interpretable component of ArtSavant. Importantly, it is rather non-trivial to decide what constitutes a ‘spurious’ feature in this case, as any sort of pattern can arguably be seen as a stylistic motif. With the interpretability of our approach, ideally humans can decide for themselves if articulated stylistic elements are valid or spurious. Having qualitatively inspected many cases, including in a recent human study for reviewer yjz4, we can say that there have not been any obvious spurious features being relied upon. Nonetheless, one could leverage input-attribution techniques to better understand how DeepMatch makes predictions.

Classifier vs. embedding based approach. We have added a number of explorations of using/comparing to style-finetuned embeddings. In short, (i) these embeddings can serve as a drop in replacement to the CLIP embeddings in our framework, without much of a change in outputs (ii) in a retrieval task (retrieving art from the same artist given a query image), softmax probabilities from our classifier turn out to be more effective than both vanilla and style-finetuned embeddings, likely because our classifier explicitly upweights information related to distinguishing artists (relevant to our objective of understanding when and why artistic styles are ‘unique’), while other embeddings capture much more general information about the image (this is the key pro / con or distinction between the two approaches, in addition to the enhanced interpretability of our approach).

Retraining capacity. An earlier iteration of our dataset contained about 10 more artists, and we saw accuracy of our classifier then was only slightly less (~1pp less) due to the higher total count. Thus, for our proposed usage of just adding art for the one artist at a time, we believe there is minimal risk of forgetting.

Answers to questions.

• Thanks for pointing out the typo - it is now fixed.
• The lowest number of works per artist in our training set is 80, and even at this count, there is an artist whose work is recognized with 82% accuracy.
• DeepMatch is more accurate, so TagMatch sometimes confuses stylistically similar artists when DeepMatch is correct, though generally, the predicted artist using DeepMatch is within the top 5 predicted artists using TagMatch. We can still inspect matched tags in these cases, allowing us to make use of TagMatch’s inherent interpretability and attributions.

Thank you for your kind words (especially on the topic of our legal discussions and the accessibility of our approach to broader audiences) and valuable feedback! We answer questions below. Sorry for the delay.

Checking for potential biases. We appreciate the reviewer’s attention to this important issue. To check for biases more closely, we collect two new forms of metadata: we obtain artist nationality from WikiArt when available, and we proxy ‘popularity’ by counting the number of visits to each artist’s wikipedia page using PageMetrics (inspired by [1] who also used web traffic to proxy popularity). Then, we inspect performance of DeepMatch and TagMatch on real held-out art over different continents and popularity levels. Both are stable: for regions, DeepMatch average accuracies and TagMatch accuracies fall within a 6% and 3% range respectively. For popularity, over four equally sized quartiles, DeepMatch average accuracies and TagMatch accuracies fall within a 3% and 7% range. Since we have an accuracy per artist for DeepMatch (whereas TagMatch either matches or does not match the artist), we can also inspect the correlation between our popularity measure and the DeepMatch confidence per artist: we find there to be virtually none (Pearson r of 0.04).

Diversity of dataset. With our new nationality data, along with previously collected metadata, we were able to see that our dataset has artists from all 6 continents (aside from Antarctica) and at least 40 nations, while also spanning 81 distinct styles (according to WikiArt’s categorizations). Importantly, our code (to be released) allows users to easily expand our dataset to artists with less than 100 artworks on wikiart – this threshold is somewhat arbitrary, and related more to our goal of measuring copying by generative models of ‘prolific’ artists.

As for artists that fall under styles (based on WikiArt) that are more niche, we observe them to actually be recognized at higher rates, with a small to moderate positive correlation of r=0.4 between DeepMatch accuracy on held out art for an artist and the number of other artists who fall under the same style category as that artist. The intuition here is that if only one artist practices some style, their work will be more recognizable than an artist that practices a style that is very common, like Italian renaissance art (see Figs. 4 and 10).

TagMatch’s components (vocabulary, tagger) are flexible. We note that TagMatch is sufficiently modular to where an improved tagger can easily be swapped in. More readily, one can modify the underlying vocabulary to incorporate more descriptors relevant to some broader style of interest. Thus, the reviewer is correct in that TagMatch can easily be improved as underlying technology evolves.

[1] Sun et al, "Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs?", NAACL 2024