Comprehensive Artistic Style Representation for Quantitative Evaluation

1. Limited comparison with existing methods:

The comparisons with state-of-the-art methods in artistic style representation are not sufficient. To strengthen this aspect, the authors are suggested to:

(1) Conduct a comparison with methods including ArtFID [A], the attribution model by Wang et al. [B], and the style descriptor framework by Somepalli et al. [C].

[A] Matthias Wright and Bjo ̈rn Ommer. Artfid: Quantitative evaluation of neural style transfer. DAGM German Conference on Pattern Recognition 2022.

[B] Wang et al. Evaluating data attribution for text-to-image models. ICCV 2023.

[C] Somepalli et al. Measuring style similarity in diffusion models. Arxiv 2024.

(2) Provide quantitative results comparing the proposed method's performance against these baselines on standard datasets like WikiART, as well as the new WeART dataset.

2. Lack of ablation studies:

There're no ablation studies to validate the contribution of each component in the proposed method. The authors are suggested to:

(1) Conduct ablation experiments to isolate the impact of the decoupling technique VS other aspects of the proposed approach.

(2) Analyze how different components affect performance on various artistic styles or genres.

3. Potential bias in the WeART dataset:

Although introducing a new dataset is valuable, there's no discussion of potential biases. The authors are suggested to:

(1) Provide a detailed breakdown of the dataset composition, including artist demographics, art styles, and historical periods represented.

(2) Discuss any potential biases in the dataset selection process, and how these biases might impact model performance or generalizability.

(3) Compare the characteristics of WeART with existing datasets like WikiART to highlight its unique contributions of WeART dataset.

4. Limited exploration of failure cases:

There lacks an analysis of cases where the proposed method might underperform. The authors are suggested to:

(1) Discuss failure cases or limitations of the proposed approach.

(2) Provide examples of artistic styles or scenarios where the proposed method struggles and analyze the reasons behind these challenges.

5. Lack of human perceptual evaluation:

The paper seems to focus on quantitative metrics for style retrieval, but doesn't mention any human perceptual studies. The authors are suggested to:

(1) Conduct a user study with art experts to evaluate the quality and accuracy of the style representations.

(2) Compare the proposed method's style classifications with human judgments, to validate its alignment with human perception of artistic style.

6. Limited discussion on computational efficiency:

The paper did not mention the computational efficiency of the proposed method. The authors are suggested to compare the computational cost of the decoupling technique to the existing methods.

7. Insufficient exploration of cross-domain generalization:

Although the method is tested on multiple datasets, the paper did not explicitly mention its performance across different art forms. The authors are suggested to evaluate the proposed method's performance on diverse art forms (e.g., digital art, sculptures, architecture) to test its cross-domain generalization ability.

The paper contains major flaws in several respects.

First, the paper is not well contextualized within prior work. A new style dataset (WeART) is proposed, but there is no review of prior style datasets beyond WikiArt (2015) to argue for its uniqueness or value. For example, the authors state that WikiArt contains only 80K artworks (it has 81K) and ~200 artists vs. the proposed WeART 197K artworks and 600 artistis. But there is OminArt (432K artworks), BA M (2M artworks), BAM-FG (2.6M artworks in fine-grained artistic variations), SemArt, AVA, ArtEmis, the list goes on. The authors should not only cite and argue for the benefit of their new dataset against prior datasets, but also evaluate their proporsed method against them. In addition to the lack of comprehensive study of prior datasets, the paper also does not cite or compare the proposed new style representation learning approach to relevant baselines. There are several techniques in common use e.g. ALADIN, ALADIN-NST, StyleBabel, MUNIT based approaches, etc. all that seek to learn disentangled representations.

Second, there is unnecessary formalism around information theory introduced within the paper (Section 2) that is then unused. Specifically the paper makes an opening argument about the need to consider information theory in the design of a disentangled style representation. Notation is introduced to motivate this argument, but is not actually used in the method itself nor is any information theory used in the training of the method. Rather, Section 2 culminates in a short quantitative study which appears to show retrieval results using various CLIP (multimodal VLM) and DINO (self-supervised VLM) derivatives with no reference to the formalisms introduced in Section 2. Overall it is very unclear where a main thrust of the paper in its information theory based motivation delivers in the method. More generally, the definition of style itself is very vague in this paper (Section 1) relying upon quotation and phrases such as ‘essence of an artists personalized expression’. It would be better to make a clear definition of style (appearance) vs. content (subject matter) and align the work with the body of prior work exploring disentangled content/style learning.

Third, the technical method itself seems to lack novelty and is poorly justified. The method is essentially extracting visual features from an image using CLIP and DINOv2 and then captioning the image with ‘a VLM such as GPT-4’ (it is never stated which LLM is actually used in the experiments) and using CLIP to encode the captioned text. It is not justified why the LLM would produce a style caption that leads to learning of a disentangled representation. Rather there is a reliance on the ‘orthogonal projection technique’ which seems to imply that vector math in the CLIP/DINO hybrid space (which appears to be some concatenated, normalized vector of the two) can somehow disentangle style and content – this seems a circular argument since developing a space with such properties is infact the goal of the learning. Also, as a matter of clarity - The paper introduces the name ‘DS-CLIP’ (Decoupled Style CLIP) in Section 6 as the name of the proposed method, which would have been much clearer to state in the introduction in terms of readability and explaining the core motivation of the paper at the outset. The discussion of large VLMs throughout the methodology seems to obfuscate this end goal, whereas the LLM aspect of the work is simply using LLM (such as GPT) to caption the images.

Fourth, the paper targets ‘comprehensive’ style, implying that prior style representation learning work is not able to generalize to a comprehensive gamut of styles. However the paper does not evaluate or study most available style datasets, and does not baseline against common prior style representation learning techniques. It is unclear how well the paper therefore delivers on this core claim of improved generality.

The author did not provide rigorous proof for the hypothesis he proposed, and subsequent experiments were unable to validate his theory. While the author believes that the adjustments he made can enhance the model's ability to extract style features from images, the experimental results suggest that the model may only be learning a single correspondence, such as image-text matching, rather than genuinely identifying the style features.

The experimental results do not support the formula proposed by the author, and the author has not provided evidence for their theory, which makes it unconvincing. If the author's hypothesis is valid, more intuitive experiments, such as style transfer, should be conducted to demonstrate this. Alternatively, a logical explanation for the proposed formula should be provided.

1. Although the authors discuss style transfer in the related work section, they do not experiment with style transfer tasks, which are crucial for the proposed method's aim of decoupling content from style representations.
2. The WeART dataset comprises only seven genres, which may not provide compelling evidence for the representation of fine-grained styles frequently employed in practical applications.
3. The analysis of mutual information between style and content in multimodal data lacks sufficient theoretical support or deductive proof. Specifically, Equations 1 and 2 appear to be based on the assumption that image information can be directly divided into content and style components, yet this division is not adequately substantiated.
4. There are some formatting issues and problems with image references that slightly hinder the reading experience.