Response to Reviewer 58qS

We thank the reviewer for the constructive comments and feedback on our paper. Regarding the concerns of the reviewer 58qS, we provide the following responses.

Q1. As mentioned in "Strengths And Weaknesses", the generation speed might be an issue. It would be good to list the break down of time usage for different modules so that the readers can better understand the bottleneck of the proposed method.

A1. Thanks for your valuable feedback. We employ the null-text inversion technique via DDPM-Inversion [a], which constructs the latent noise sequence through a forward process, computes and retains the true noise values, and directly utilizes these values during sampling to estimate the target noise—enabling perfect reconstruction. In other words, it can be considered an optimized version of DDIM-Inversion [b]. Crucially, this is not an optimization-based approach, thus avoiding the multi-minute optimization time required by methods like [c]. For single-style generation tasks, our method's average inference time per image is reported below:

• M3S (SD v1.5)​: 41 seconds in total, where 6 seconds for DDPM-Inversion trajectory collection and 35 seconds for denoising synthesis.

• M3S (SDXL)​: 63 seconds in total, where 10 seconds for DDPM-Inversion trajectory collection and 53 seconds for denoising synthesis.

The table below records the inference time per sketch generation across different models. Our M3S (SD v1.5) demonstrates slower generation than InstantStyle but comparable speed to CSGO, StyleStudio, and RB-Modulation. While M3S (SDXL) requires additional processing time, it outperforms StyleAligned and AttentionDistillation in efficiency. Notably, InstantStyle, CSGO, RB-Modulation, and StyleStudio rely on external pre-trained models beyond their diffusion backbones, incurring non-inference overheads (e.g., RB-Modulation's SAM [d] loading takes minutes). Although InstantStyle offers speed advantages, its dependency on IP-Adapter [e] necessitates model-specific pre-training. In contrast, M3S, StyleAligned, and AttentionDistillation—despite longer inference—achieve superior style consistency with training-free, plug-and-play flexibility.

Q2. The paper compares with many general baseline works that are designed for general styles. It would be good if the author can clearly mark which methods are designed for general styles, while which methods are designed for sketches only. This will help readers understand the scope of the baseline approaches.

A2. Thanks for your suggestions. The baseline methods, including InstantStyle, CSGO, StyleStudio, RB-Modulation, AttentionDistillation, and StyleAligned are designed for general scenarios. Other methods, like SketchRNN, CLIPasso, and DiffSketcher are specifically built for sketch generation. We will clarify this categorization more explicitly in the revised version. Extensive comparisons with general-purpose methods are conducted because the scarcity of same-style sketch data, e.g., the 4skst [f] dataset contains only 25 sketches per style across 4 styles, makes it impractical to train dedicated style-specific text-to-sketch models. Addressing this data scarcity to achieve zero-shot style-specific generation is a core challenge our work tackles.

Q3. It is usually subjective to evaluate the quality of a sketch. It would be more convincing if user study results are included in the paper.

A3. We appreciate your constructive comments. We conducted a ​human preference assessment​ via structured questionnaires to evaluate text-to-sketch synthesis performance. Each questionnaire contained:

1. Six randomly selected sets of generated sketches.
2. Eight anonymized outputs per set from different models under identical prompts and reference styles.
3. Evaluation criteria: Comprehensive assessment across three dimensions—text alignment, style consistency, and generation quality.

Participants ranked results on a scale of 1-8 (8=optimal). From 58 submissions (avg. completion: 4m16s), we excluded 14 invalid responses (<60s completion/missing rankings), retaining ​44 validated questionnaires. The results are shown on the table below.

Quantitative analysis of user preferences revealed:

1. ​M3S Dominance:
   • M3S (SDXL) achieved the highest average score (6.19) and M3S (SD v1.5) secured strong performance (5.44).
   • Both variants significantly outperformed baseline methods, demonstrating excellent balance across evaluation dimensions.
   • We evaluate the statistical significance by rank tests, revealing that M3S (SD v1.5) significantly outperforms all baseline methods except InstantStyle (p-value=0.26). When we align the backbone with InstantStyle (i.e., SDXL), our M3S (SDXL) demonstrates superiority over it (p-value=1.06e-5).
2. Baseline Comparisons:
   • ​InstantStyle​ attained a competitive score (5.08) due to high aesthetic quality despite style consistency limitations.
   • ​StyleAligned​'s style advantages were compromised by content leakage issues.
   • AttentionDistillation​ achieved marginal improvements over StyleStudio/RB-Modulation through style-focused optimization, notwithstanding quality deficiencies.

In conclusion, the study affirms M3S's superior alignment with human evaluator preferences. We will include the results and analysis in the revision version.

Q4. The main weakness is that text-to-sketch generation might be a sub-domain of text-to-image generation. It is questionable whether it is necessary to design a sophisticated method for the text-to-sketch task. The author mentioned in the paper that existing text-to-image models are not good at sketch generation, e.g., they are designed for color and natural images. However, this problem might just come from training data. If we replace training data, or finetune the existing model with additional sketch images, or train a sketch lora, this problem might have been solved. Different trigger words can be added to training captions to incorporate different sketch styles.

A4. Thank you for your suggestion for style-specific sketch generation solutions. For specific-style sketch generation, training-based methods face three major challenges in practical applications:

1. Data Scarcity and Annotation Difficulties Professionally styled sketch datasets are scarce and typically small-scale (e.g., 25 sketches per style in 4skst [f], 140 in APDrawing [g]). Effective fine-tuning requires dozens of samples per style, necessitating thousands of sketches for diverse styles like our Style 5 dataset's 20 distinct categories. Although web-sourced sketches are plentiful, style annotation poses significant hurdles: multimodal models (e.g., BLIP-2 [h]) trained on real image-language pairs will struggle with sketch style categorization. Manual annotation is costly and inconsistent due to sketches' abstract nature—style descriptors vary across annotators' age, cultural background, and regional context. Such inaccuracies impede models from learning precise token representations for style triggering.

2. ​Limited Flexibility in Trained Approaches Training-based methods require model-specific adaptations, whereas our training-free technique transfers seamlessly across architectures. Furthermore, generating novel styles absent from training data necessitates re-training, while M3S operates zero-shot without this constraint. Critically, our framework inherently supports multi-style fusion, whereas trained methods lack this capability without extensive additional research.

3. ​The "An Image is Worth One Word" Principle [i] Given the linguistic ambiguity of sketch styles, users must memorize dozens to hundreds of trigger tokens to match target styles accurately. M3S eliminates this burden by using a single reference image for style guidance, significantly enhancing usability.

Overall, our approach circumvents the data acquisition constraints and flexibility limitations inherent in training-based methods while enabling novel multi-style fused sketch generation. We appreciate your valuable suggestion regarding training-based alternatives and will investigate these directions in future work.

[a] Huberman-Spiegelglas, et al. "An edit friendly ddpm noise space: Inversion and manipulations." CVPR 2024.

[b] Song, Jiaming, et al. "Denoising diffusion implicit models." arXiv preprint arXiv 2020.

[c] Mokady, Ron, et al. "Null-text inversion for editing real images using guided diffusion models." CVPR 2023.

[d] Kirillov, Alexander, et al. "Segment anything." ICCV 2023.

[e] Ye, Hu, et al. "Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models." arXiv 2023.

[f] Seo, Chang Wook, Amirsaman Ashtari, and Junyong Noh. "Semi-supervised reference-based sketch extraction using a contrastive learning framework." TOG 2023: 1-12.

[g] Yi, Ran, et al. "Apdrawinggan: Generating artistic portrait drawings from face photos with hierarchical gans." CVPR 2019.

[h] Li, Junnan, et al. "Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models." ICML 2023.

[i] Gal, Rinon, et al. "An image is worth one word: Personalizing text-to-image generation using textual inversion." arxiv 2022.

Response to Reviewer cQmr

We thank the reviewer for the constructive comments and positive feedback on our paper. Regarding the concerns of the reviewer cQmr, we provide the following responses.

Q1. This method has certain bias when processing certain objects. As shown in the clown example in Figure 1, even with $\eta$ set to 0, the generated clown image doesn't have a simple drawing style, though clowns are suitable for simple drawing. This echoes the fact that the method doesn't achieve optimal results in all cases in Table 2. What's the deep reason for this?

A1. We thank the reviewer for the valuable feedback. When $η=0$, M3S synthesizes sketches that ​blend both reference styles​ rather than adopting a single style in this case, while strategically biasing the output toward the aesthetic attributes of Reference Style 1. We explain this phenomenon through M3S' three core components:

• K/V Injection: The clown example involves multi-style generation where K/V features from both reference images are injected. The diffusion model automatically assigns contour and texture to different objects. At $\eta=0$, the unicycle exhibits pure freehand sketch style, and the clown maintains freehand-style contours but develops body textures. This occurs because Stable Diffusion's natural image training data contains more detailed clown depictions (colorful costumes, facial makeup) than unicycles. Consequently, the model prioritizes texture-rich styles for clown details.

• Joint AdaIN Module: AdaIN modulates pixel/feature distributions statistically rather than enforcing explicit structural constraints. Thus, at $\eta=0$, the result shows global distribution aligns with freehand sketch style and local discrepancies persist in structural details.

• Style-Content Guidance: The phenomenon arises from the interplay between reference style characteristics and guidance scaling. When using sparse (cup) and dense (wave) references, the wave’s rich textures enhance content generation due to their structural relevance. For the clown example, however, the default setting of $\omega_1 = 15, \omega_2 = 15$ amplifies texture density from the non-freehand reference style. This occurs because higher $\omega_2$ prioritizes style alignment, favoring denser patterns. When we reduce $\omega_2$ to 7.5, the textures for the clown are more line-dominant (closer to freehand). Since OpenReview does not allow to upload images during the discussion period, we recommend that the reviewer verify the results via the code and dataset in the Supplementary materials.

In essence, this phenomenon reflects the model's automatic selection of multi-style features for category-specific synthesis. Since we inject features from multiple references, the generated image inherently blends diverse styles even at $\eta=0$. To achieve pure freehand sketch style, we recommend using sparse references (e.g., the cup) as the sole style source.

Q2. The paper only shows results of referring to multiple styles, but the used reference sketches are all of high - quality. So when encountering cases with big differences in style and drawing quality, the method's performance is unclear. For example, what's the effect when using amateur sketches (such as those in QuickDraw) and professional sketches as style references?

A2. We appreciate the reviewer's constructive comments. To verify generation quality when using two visually distinct reference styles, we experimented with the following design. We first constructed a style dataset by randomly selecting 50 images from the low quality dataset QuickDraw (QD), then generated 50 multi-style images using prompts identical to our main quantitative experiments. Specifically, we select one random reference from QD and one from high quality dataset Style 5 (S5) per generation. To contrast results against cases where both references exhibit high quality, we generated additional outputs using exclusively S5 references. Quantitative results are provided in the table below.

The results indicate that QD-S5 and S5-S5 achieve comparable CLIP-T scores with only marginal degradation, demonstrating our method's robust text alignment even with significantly divergent reference styles. For SD v1.5, DINO metrics show only minor deterioration when using QD references. However, QD-S5 SDXL exhibits substantially reduced DINO-ref1, indicating the model's stronger preference for high-quality S5-aligned outputs over QD. This occurs because SDXL is explicitly optimized for high-fidelity generation and significantly outperforms SD v1.5 in user evaluations [a], making it more susceptible to quality discrepancies in reference images. Consequently, M3S (SD v1.5) is preferable for multi-style generation involving references with major quality differences.

Q3. However, the need to adjust many hyperparameters restricts its practical application convenience.

A3. Thanks for your thoughtful comments. Since human evaluation of text-conditioned sketch generation typically requires multi-factor considerations, style-specific generation frameworks commonly offer multiple parameter values for user customization [b,c]. Similar to the literature, our method also has tunable hyperparameters. We provide default parameter values for both professional and abstract sketches to satisfy user requirements, as evidenced by the user study (See Response to W249, Q1).

[a] Podell, Dustin, et al. "Sdxl: Improving latent diffusion models for high-resolution image synthesis." arXiv 2023.

[b] Lei, Mingkun, et al. "StyleStudio: Text-Driven Style Transfer with Selective Control of Style Elements." CVPR 2025.

[c] Zhou, Yang, et al. "Attention distillation: A unified approach to visual characteristics transfer." CVPR 2025.

Response to Reviewer W249

We thank the reviewer for the constructive comments and positive feedback on our paper. Regarding the concerns of the reviewer W249, we provide the following responses.

Q1. Conducting a user study.

A1. We appreciate the reviewer's constructive comments. We have conducted a user study employing a questionnaire-based methodology to assess human preference. Each questionnaire included six randomly selected sets of generated results. Every set contained eight outputs produced by different models using identical prompts and reference styles. Participants were instructed to rank all eight anonymized results per set based on text alignment, style consistency, and generation quality. The user rankings for each of the eight models are converted to scores from 1 to 8 (8=best) correspondingly. We collected 58 questionnaires with an average completion time of 4 minutes and 16 seconds. After excluding 14 invalid submissions with completion time under 60 seconds and missing rankings, 44 valid responses remained. Average scores per question for each method are tabulated below.

In the user study, M3S (SDXL) achieved an average score of 6.19 while M3S (SD v1.5) scored 5.44. We evaluate the statistical significance by rank tests, revealing that M3S (SD v1.5) significantly outperforms all baseline methods except InstantStyle (p-value=0.26). When we align the backbone with InstantStyle (i.e., SDXL), our M3S (SDXL) demonstrates superiority over it (p-value=1.06e-5). This demonstrates M3S's superior balance across style alignment, text fidelity, and generation quality. Despite InstantStyle's relatively weaker style consistency, its high aesthetic quality yielded a competitive score of 5.08. Conversely, StyleAligned's advantages in style consistency were undermined by content leakage issues. AttentionDistillation achieved marginal score improvements despite inferior generation quality compared to StyleStudio and RB-Modulation, primarily due to its style consistency. The results affirm M3S's alignment with human preferences.

Q2. Handling diverse reference styles and quantitative evaluation of multi-style interpolation.

A2. Thanks for your valuable feedback. Following your suggestion, we have added supplementary experiments for multi-style generation. Using the same 50 prompts as the manuscript, we generated outputs for each prompt by randomly selecting two reference sketches from the Style 5 (S5) dataset. To specifically validate generation performance with significantly distinct reference styles, we conducted an additional experiment set pairing one randomly selected S5 image with one randomly chosen image from the QuickDraw (QD) dataset per prompt. It is noted that the styles of the QD sketches are very different from those in S5. The quantitative results are provided in the table below.

When the references exclusively originate from S5, M3S maintains text alignment comparable to single-style generation (CLIP-T: M3S (SDXL) 0.3467, M3S (SD v1.5) 0.3494). For style consistency, DINO-ref1 exhibits a positive correlation with $\eta$ (i.e., increasing $\eta$ enhances alignment with reference 1), while DINO-ref2 shows a negative correlation. This monotonic relationship aligns with qualitative results in Figs. 1 and 5. At boundary conditions ($\eta$=0 or 1) of multi-style sketch generation, only one style participates in AdaIN modulation for image generation. Compared to single-style synthesis using that style alone, the DINO metric remains relatively ​lower​ due to persistent multi-style feature injection (see Response to Reviewer cQmr Q1 for more analysis).

For the two reference styles, replacing the pair of S5-S5 with QD-S5 reduces style consistency for both implementations (i.e., SD v1.5 and SDXL), though M3S (SD v1.5) demonstrates superior robustness. Crucially, M3S (SDXL) struggles to effectively utilize QD's abstract features, evidenced by significantly lower scores of DINO-ref1 than DINO-ref2 in QD-S5 pairs. This limitation stems from SDXL's high-fidelity optimization [a] - its user-tested superiority over SD v1.5 creates inherent incompatibility with low-quality, abstract datasets like QD. Due to OpenReview's restrictions on image uploads during the discussion phase, we cannot directly provide corresponding visual results. However, the overall generation quality is visually analogous to the butterfly reference outputs in Fig. 5 of the main paper.

Q3. Eq. 5 might be better represented.

A3. We appreciate your suggestion. We will revise Eq. (5) to explicitly represent the $K/V$ injection mechanism as follows. Our formulation follows the notation established in [b]. We will also add explanatory text regarding the equation and cite [b] to enhance readability.

$\tilde{\epsilon}\_t=\epsilon\_{\theta}(**z**\_t^{tar},t,\emptyset)+\omega\_1\underbrace{(\epsilon\_\theta^\times(**z**\_t^{tar},t,text,K^{ref},V^{ref}) -\epsilon\_\theta(**z**\_t^{tar},t,\emptyset ))}\_{content\ guidance\ direction} + \omega\_2 \underbrace{(\epsilon\_\theta^\times (**z**\_t^{tar},t,\emptyset,K^{ref},V^{ref}) -\epsilon\_\theta(**z**\_t^{tar},t,\emptyset ))}\_{style\ guidance \ direction}$

Q4. Failure cases.

A4. We apologize for placing the failure cases in an insufficiently prominent location (specifically, Appendix H and Fig. 15 of manuscript). In the upcoming revision, we will incorporate these directly into the main body. Our method's primary failure mode occurs when reference images are excessively sparse with content concentrated in small regions, potentially yielding near-blank outputs. To address this, we introduce a solution in Appendix H as follows, i.e., enhancing reference images via zooming and replication-based padding to increase pixel density.

Following your suggestion, we tested our method using Google-collected pointillism sketches (depicting flowers, balls, tigers, etc.). Our method succeeded in most examples. However, we observed one failure case where SD's VAE failed to accurately reconstruct the reference image, indicating inaccurate latent space representations. This example will be incorporated into the failure cases section in the revised manuscript.

Q5. Few missing references.

A5. We appreciate your feedback on the missing literature. In the revised manuscript, we will incorporate the referenced computer graphics papers relevant to our work and conduct an additional literature review to provide readers with a more comprehensive domain context.

Q6. Limited technical novelty.

A6. Thanks for your constructive feedback. While building upon prior concepts, M3S extends existing techniques in novel directions to address unique challenges in ​multi-style sketch synthesis:

1. Multi-reference feature fusion Prior K/V injection methods (e.g., [b,c,d]) focused on single-style transfer. M3S introduces: Concurrent injection​ of features from multiple references (Section 3.1) and blending them through linear smoothing.

2. Adaptive noise modulation Traditional AdaIN [e] achieves style transfer by modulating image features, but its outputs remain constrained by the content image. Recent diffusion-based methods [b] primarily focus on artifact removal without exploring AdaIN's style modulation capabilities. In contrast, M3S demonstrates that multi-style fusion can be achieved by applying AdaIN to denoising trajectories in noise space, free from encoded-space limitations. Furthermore, M3S leverages diffusion models' generative capacity to automatically produce creative content under varying $\eta$ conditions, such as the evolving beard details in Messi's portrait (Fig. 5).

3. Performance Validation M3S demonstrates consistent advantages in human evaluations, which achieves highest user preference score (6.19​ for SDXL).

In summary, our core innovation centers on both the novel task of multi-style sketch synthesis and its corresponding framework design. Building on this framework, we will further explore the feasibility of multi-style fusion generation in RGB space.

[a] Podell, Dustin, et al. "SDXL ...." arXiv 2023.

[b] Alaluf, Yuval, et al. "Cross-image attention ..." SIGGRAPH 2024.

[c] Hertz, Amir, et al. "Style aligned ..." CVPR 2024.

[d] Cao, Mingdeng, et al. "Masactrl ..." ICCV 2023.

[e] Huang, Xun, et al. "Arbitrary style transfer ..." ICCV 2017.

Response to Reviewer 58qS

We sincerely thank the Reviewer for your thoughtful feedback and positive assessment of our work. We answer specific questions below.

Q1. While the general motivation for introducing the content and style guidance losses (eq 5 is clear), the motivation to linearly increase the weight of the style guidance through the generation process is not clear. Why should this be done and how sensitive is the method to the schedule?

A1. We thank the reviewer for the valuable feedback. We linearly increase the style guidance weight during denoising to account for diffusion models' temporal dynamics in generation [a]. Early denoising primarily establishes image structure, while later stages focus on texture refinement. Applying strong style guidance $\omega_2$ prematurely could disrupt structural integrity due to the absence of textual alignment, i.e., $(\epsilon\_\theta^\times ({z}\_t^{tar},t,\emptyset ) -\epsilon\_\theta({z}\_t^{tar},t,\emptyset ))$. For example, when style guidance is not properly scheduled, the prompt "a sketch of a monkey eating a peach" might activate the generation of connecting the arm directly to the peach. Quantitative results for Style 1-5 generations using M3S (SDXL) are provided in the Table below. Without linear scheduling, CLIP-T and aesthetic scores decrease while the DINO score increases. This occurs because without linear scheduling, the generation process over-prioritizes style consistency. However, higher style consistency doesn't guarantee better visual quality and may cause severe unnatural artifacts in local content. We therefore employ linear scheduling to balance generation quality, text alignment, and style consistency. This technique also implemented in [b].

Q2. For the SDXL Implementation, the authors mention that style injection is important for the last layers, but injection into all layers compromises text alignment. Is there a principled way for deciding which layers, or some motivating ablations around it?

A2. We appreciate the reviewer's constructive comment. Our layer selection strategy prioritizes deeper near-output blocks. It is motivated by the ablation studies in [c], which has demonstrated that diffusion models generate outlines primarily in lower-resolution blocks and textures in higher-resolution blocks, respectively. Injecting features across all layers causes noticeable content leakage. To balance style transfer with generation quality, we reduce the injection frequency into lower-resolution blocks. We compare the results between M3S (SDXL) and full-layer injection under identical settings in the Table below. Full injection significantly reduces CLIP-T and aesthetic scores, while slightly increasing the DINO score due to content leakage – a similar observation to that in Q1.

Q3. The proposed technique is novel and well-motivated for the application domain (which is somewhat narrow, as some of the assumptions made by the authors rely, for example, on images being grayscale).

A3. We sincerely thank you for recognizing our work's contributions and noting its limitations. In future work, we will validate our multi-style interpolation framework in the complex RGB space and refine the Joint AdaIN module.

[a] Patashnik, Or, et al. "Localizing object-level shape variations with text-to-image diffusion models." ICCV 2023.

[b] Alaluf, Yuval, et al. "Cross-image attention for zero-shot appearance transfer." SIGGRAPH 2024.

[c] Cao, Mingdeng, et al. "Masactrl: Tuning-free mutual self-attention control for consistent image synthesis and editing." ICCV 2023.