RetriBooru: Leakage-Free Retrieval of Conditions from Reference Images for Subject-Driven Generation

• Limited Dataset Application: After validating the conclusions, it seems that the dataset can only be applied to customization or conditional encoder training within the anime domain.
• Insufficient Innovation in Concept Combination Task: The proposed concept combination task, which simply combines anime faces and clothing into a single image, lacks significant innovation or meaning. The visual results, such as in Figure 5 (prompt 1, second row), show relatively ordinary combinations, with low similarity between faces or clothing and the reference images.
• Vague Qualitative Results: The qualitative comparison results are somewhat ambiguous. It would be better to specify exactly what aspects are improved or to annotate the specific areas of improvement in the images.
• Need for More Experimental Results in Domain Generalization: Simple qualitative comparison images between two groups are insufficient. Therefore, conducting generalization tests for different conclusions, and even providing quantitative metrics, would be more convincing.
• Clarity in Explanation and Writing: Some parts of the explanation and writing could be clearer. For instance, the proposed balancing metric (Equation 2), which appears to be missing a left parenthesis, lacks clear and understandable explanations. Illustrating how this metric addresses ‘a worse trade-off’ mentioned in your paper would clarify its effectiveness.
• Domain Difference In Section 3.1: Your evaluation set includes 40 in-domain characters and 10 out-of-domain characters, but this distinction is not further explained in the results. I am curious whether the domain difference has any impact on the metrics. Are there any qualitative results you can show?

1. I question whether the motivation is reasonable. The authors suggest that the similarity between the reference image and the target image can easily lead to spatial information leakage. Although they propose using partial images with different attributes to address this issue, these partial images can also contain spatial information. Furthermore, collecting such partial images for general domains is challenging, and ensuring their accurate and comprehensive annotation may not be fully achievable.

2. The comparisons presented in Table 1 are not clearly delineated. Based solely on the information in the table, readers may struggle to identify any significant differences between this dataset and others.

3. The authors have exclusively utilized anime images to construct the dataset, which may limit its generalizability. Anime images often feature simplistic backgrounds and restricted attributes. I wonder if this could contribute to the poor performance observed in Figure 7 (prompt 2).

4. In Figure 4, I have concerns regarding the comprehensiveness of the annotation for the partial images. Would providing attributes from the entire area enhance generation performance? Additionally, how can we effectively control the details of the entire face? Should the dataset be updated to incorporate these insights? Lastly, how should the granularity of the annotations be defined?

1. In the comparison of qualitative results Figure3, apart from prompt2 with scale=0.75 where the advantage of method B is evident, I cannot see clear advantages of IP-Adapter-b over IP-Adapter-a in other case.
2. In Figure7, I have the same opinion for the result generalizing to human face. ie, I don't think training pipeline of this paper keeps a better identity-diversity balance, and the human ID is worse than other methods. Although the author claims the result is limited by V100, it is hard to distinguish whether the training pipeline or GPU limit the performance. And the effectiveness of this method on real datasets is not convincing.

The paper title and several parts in the main text of paper mention “leakage-free” and “reduce leakage”. However, there is no supported related experiments conducted by authors. Authors propose -b training scheme that uses a same-identity but different image from target as the reference image. However, the definition target is not clear and it is not unique for each identity. Training scheme -a could also be one case of -b training setting.

-b training scheme doesn’t ensure a general good performance in CLIP-I and CLIP-T scores. For example, -b training scheme decrease CLIP-I on ControlNet and IP-Adapter and decrease CLIP-T on Kandinsky-0.1, Kandinsky-0.2.