StyleAdapter: A Unified Stylized Image Generation Model without Test-Time Fine-Tuning

1. The qualitative results in Figure 6 are confusing. The stylized image from the proposed approach does not follow the style well.
2. The evaluation is insufficient and confusing. The results in Table 1 shows there is no significant improvement between the proposed approach and the baselines. The proposed approach is trying to achieve good alignment for both text prompt and the style. However, the paper evaluates the text alignment, style alignment, and image quality separately. It is hard to see how much improvement the proposed approach achieved compared to the other approaches.
3. More ablation studies are needed to justify the contribution. For example, how much does image shuffling improve the performance? How does the combined attention impact the attention weight in Figure 3(e). How would the design of the transformer model in Figure 2 impact the performance?
4. Would mixing various styles impact the performance. How to keep the extracted style consistent between multiple images?
5. The paper writing style can be improved. For example, the vanilla style adapter is not well defined in section 3.2. Figure 2 is not aligned with the text description. There are also a lot of typos in the paper, e.g. TOW, etc.

1. It's not novel to use an extra cross-attention to inject image condition into the T2I model, as the similar idea has been proposed before. This paper seems like a straightforward application of this idea to stylization.

2. The decoupling strategies are also simple and like some engineering tricks, which may limit the quality of generated image. This paper lacks an in-depth discussion of how to disentangle the style information and inject it into the T2I model.

3. The method lacks a lot of details, especially in embedding suppressing and multiple style references. More implementation details should be included to ensure reproducibility.

4. limited styles are provided for test to fully evaluate the method.

1. I am not sure if the vanilla style adapter is a proper starting point. Is it better than simply using CLIP image encoder by itself?

2. Test-time finetuning is a misleading word choice because existing methods do not require finetuning everytime it generates images with different prompts but require finetuning for different styles. Perhaps per-style finetuning?

3. DreamBooth and LoRA takes only minutes for finetuning for given small number of images as in the configuration of this paper. It can be both practical and impractical regarding the application. It would be clearer to specify the target application.

4. Figure 3 should include below to support the motivation.

• Attention weight distribution of TPCA
• Diverse examples with different prompts or [same prompts, different noises], maybe in the appendix

5. Configuration for the comparison to existing methods should be clear.

• Results from the existing methods are there. However, inputs for them are not clearly described. Hence, currently the comparisons are invalid.
• I am not sure how style transfer methods can receive content images equivalent to the text prompts.
• Prompts for textual inversion also should be described. Probably it is "A palm tree in style of S*".

6. Two-path cross-attention is not novel.

• But I cannot use this for the reason for rejection because the references are not accepted anywhere. It will be a reason if someone knows an accepted paper.
• Mou et al, T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models
• Ye et al., IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models

(minor)

7. Unnecessary abbreviations harm readability, e.g., VM for vision model.

8. The target dimension of the concatenation operation in $3.2 is not specified. I suppose it is concatenation of the tokens.

9. Caption of Figure 4 lacks clarity. I suppose (c,d,e) all use TPCA, implied from the text.

10. AIGC is not defined.

11. Using boldface for different categories (single-ref and multi-ref) may improve readability.

12. Reference image set for FID is not defined.

• Overall technical contribution is rather marginal.

• While the proposed method seems to outperform other diffusion-based style consistent image generation, it is unclear how it compares against results based on the StyleDrop. Results in Figure 12 suggests that, while generated images are consistent styles to each other, it is not as consistent to the style reference images to StyleDrop.

• First two rows of Figure 12 suggest that the style is not really well transferred to different text prompt. For example, baby penguin, monkey are not really "rusty" for the second row, and monkey or bench are not covered with grass in the first row.

• Some styles considered in the paper may be too generic and may be generated via text prompt engineering. For example, Figure 11 third row. Also, the generation results only captures the texture, but not the color scheme or the layout. It would be great if authors can provide more results on the diverse set of style reference images (e.g., those considered in StyleDrop) and discuss the limitation more comprehensively.

• It would be great to include the interface used for user study and more details about how the question was asked, which options raters are provided to answer, etc.

• This begs a question on the definition of "style" the paper has considered. Are color or texture schemes style? How about the layout or composition? It seems the shuffling technique of SSVM seems to make style to be limited to more local properties (e.g., color, texture) but not really the global properties. It would be great if authors can provide textual description of the style that the model is trying to extract. Also, it would be great to have an ablation study with different shuffling strategy (e.g., block shuffling to retain more global properties).