Thank you for your thoughtful and constructive feedback on our paper. We appreciate the time you invested in reviewing our work and the specific suggestions you provided to improve the clarity, organization, and significance of our contributions. We have carefully considered each of your concerns and will address them individually below. Your feedback has been instrumental in helping us strengthen both the presentation and the technical content of our work.

Shared response

We recommend having a look at our shared response (contained in our responses to reviewer SVh8 and siKW) for our new quantitative results on FLUX.

Individual concerns

Q.1. / W.3. How do the authors decide on which transformer blocks to investigate?

R. We provide detailed analysis in Appendix C "Finding causally influential transformer blocks." We performed ablation studies on each individual transformer block and found that only these 4 blocks have significant effects on generated images when modified. This selection is also supported by prior work (particularly the P+ study) showing that blocks from the center of the U-Net architecture are responsible for high-level concepts like objects, composition, and colors. Also, in the scope of our rebuttal for fSpm (their W.2.) we provided additional results for resnet blocks.

Q.2. Are the discovered representations labeled with text instructions, or are they more ambiguous?

R. We used a comprehensive annotation pipeline to label features of one of the blocks (down.2.1), as detailed in Appendix H.1 with examples in Appendix I.3. To annotate features, we prompted GPT-4o with series of images including: (1) images where the feature is inactive, (2) progression of images with increasing average activation values, and (3) images with the highest average activation values. This approach allows us to generate meaningful text labels for discovered representations, making them interpretable and usable for targeted editing applications.

Q.3 In the set of representations discovered, are there any overlaps, or do all of the representations discovered represent distinct concepts?

R. Learning features with top-k SAEs involves inherent trade-offs in choosing expansion factors and numbers of nonzero coefficients, which affect feature overlap properties. We observe some degree of feature splitting (where single concepts get divided into multiple features, e.g., separate features for cat ears, eyes, mouth, fur), but this rarely occurs with our chosen hyperparameters for our demo application (k=10, expansion factor=4). We also observe features for similar concepts across different blocks, but these are rarely redundant functionally. Instead, different blocks specialize in different aspects: down.2.1 may place a "cat" in the scene, up.0.0 may modulate details like facial expressions, and up.0.1 may determine color and texture details. Additionally, we find significant feature universality between FLUX and SDXL models, as demonstrated in Figure 1 and our demo applications (Appendix A).

W.1. The organization and presentation of the paper should be improved.

R. Since we found this concern highly important, we provide an overview of the changes to our manuscript that we have performed since the submission in the shared response.

W.2. The readability of the figures should be improved significantly.

R. We acknowledge this important concern and will significantly improve the figure readability for the camera-ready version. For Figure 5 specifically, we will implement a cleaner visualization approach that conveys the same information about the Pareto dominance of SAEs versus other methods without overwhelming the reader. Our planned improvements include: color-coding the number of features transported using a clear legend, using different symbols for different steering strengths, and potentially splitting complex plots into multiple subplots for better clarity. We are committed to ensuring all figures are easily interpretable and support the paper's narrative effectively.

W.4. The quality of the editing results via the discovered features is uncertain.

R. We provide comprehensive case studies in our appendices that address these concerns. Specifically, Appendix A contains an installation free demo application to try out our brush tool. Appendix B contains additional results for FLUX. Appendix D contains qualitative results for applying interventions during different ranges of the multi-step setting. Appendix E contains qualitative examples for each edit category in RIEBench. Appendix F presents "Case study: Most active features on a prompt," Appendix G covers "Case study: random features," and Appendix H provides "Quantitative evaluation of the roles of the blocks." These studies demonstrate the quality, uniqueness and diversity of discovered representations. Our analysis shows that while some degree of feature overlap exists (particularly feature splitting with higher expansion factors), the features we discovered with our chosen hyperparameters for the demo application (k=10, expansion factor=4) are largely distinct and functionally non-redundant across different blocks.

W.5. To demonstrate if the discovered representations are beneficial for image editing tasks, a comparison with existing editing methods is essential.

R. To the best of our knowledge, the suggested related works like TurboEdit don't fall within the same category of method. Our interventions currently can only modify images while they are generated, whereas, TurboEdit is meant for editing real images. We are not aware of related works in the same category. Our SAE features and brush tools enable customization of text-to-image generation in ways that are not achievable through prompting-based techniques. While existing methods could potentially be adapted through masking restrictions, the diversity of SAE features learned in an unsupervised manner makes it challenging to match their effects with competing approaches using similar resources. For example, we can locally apply animal skin/fur textures onto human faces without perturbing the underlying facial structure—our brush tools can apply 5 different textures as demonstrated in our supplementary materials. Additionally, our multi-timestep approach with timed-hooks that act only on subsets of timesteps provides novel means of trading off similarity to the original image with edit strength. Real image editing is also possible when combining our SAE features with diffusion inversion techniques, though this area requires further exploration beyond the current scope.

W.6. As we have no explicit information on which types of representations are discovered, the limitations of the method is ambiguous. An analysis on the discovered representations will improve the significance of the methodology.

R. We provide detailed analysis of discovered representations throughout our work (Section 4.2., Appendix F, G, and H). Our key findings include: (1) different transformer blocks specialize in different aspects of image generation (e.g., down.2.1 places objects in scenes, up.0.0 modulates details like facial expressions, up.0.1 determines colors and textures); (2) features remain stable across timesteps, enabling one-step learning to transfer to multi-step generation; (3) we observe feature universality across different model architectures (SDXL and FLUX), as demonstrated in Figure 1 and our demo applications. The discovered representations span various semantic levels from high-level concepts to fine-grained textural details, providing comprehensive control over generation aspects.

W.7. The novelty of the paper is questionable.

R. Our work presents several significant novel contributions beyond simply applying existing methods. Key innovations include: (1) the crucial finding that one-step activation patterns can be applied to multi-step generation without retraining, which is essential for both understanding representation spaces and practical applications; (2) addressing unique challenges in the denoising process that differ substantially from LLM applications, including how activations evolve across different forward passes; (3) developing empty prompt interventions and specialized techniques for classifier-free guidance models; (4) creating image editing methods with semantic segmentation masks and brush tools that are unique to the visual modality and address spatial localization challenges not present in text applications. Furthermore, when applying SAEs to DiTs like FLUX, we found it of key importance to distinguish our "feature extraction" phase from "feature injection", which is in stark contrast to how things are done in LLMs. We find that those two are not strictly connected, and our injection methods represent novel contributions to visual generation control.

Regarding missing limitations

R. We acknowledge this important point and will add a comprehensive limitations section to our revised paper. Key limitations include: (1) our current method focuses on individual block analysis and may miss features that require combinations of multiple transformer blocks; (2) the success rate and disentanglement quality of edits varies depending on the feature type and target image content; (3) while we demonstrate local changes effectively (like skin color modifications), the identification and control of more global compositional features remains challenging; (4) our approach is currently limited to the specific architectures we studied (SDXL and FLUX), and generalization to other diffusion models requires further validation; (5) the hyperparameter choices for SAE training (expansion factor, sparsity level) significantly impact the types and granularity of discovered features, requiring careful tuning for optimal results.

Dear reviewer uRaJ,

As you have seen already, the authors have responded to the questions in your initial review. Can you please share your thoughts post rebuttal, once you've had a chance to see the author responses and also the other reviews?

Best, AC

We thank the reviewer for their thoughtful evaluation of our work and for recognizing the interesting nature of our proposed method and the comprehensive experimental design. We appreciate the constructive feedback! The rebuttal is structured by first responding to common concerns raised by multiple reviewers and subsequently will address each of the raised concerns individually below.

Shared response

Improved writing We significantly improved their manuscript's organization and clarity through several key updates:

• Enhanced Structure & Contributions

  • Added clear paragraph headings for their four main contributions: SDLens (caching library), SAEPaint (visualization demo), RIEBench (editing benchmark), and findings on generalization across steps/architectures
  • Updated abstract and section headers to match the new organization

• Content Consolidation & Clarification

  • Section 3: Merged spatial activation and sparse feature maps into a single "feature interventions" paragraph; added details on SDXL base interventions and classifier-free guidance handling
  • Section 4: Renamed subsection 4.1 to better reflect the RIEBench benchmark; fixed a bug in equation (9); expanded details on benchmark construction and edit categories

• New Content & References

  • Added a new Appendix B specifically covering SDXL Base implementation details, including optimal strategies for feature addition during conditional/unconditional forward passes
  • Systematically added appendix references throughout the paper to provide readers with additional technical details and examples
  • Enhanced cross-referencing between sections and appendices (with updated lettering after inserting the new appendix)

Quantitative results for FLUX In addition, to address the common concern of overly focusing our paper on SDXL Turbo, we performed an evaluation of FLUX Schnell on our representation-based editing benchmark (RIEBench). We will add these results to the camera-ready version of our paper in case of acceptance. For ease of presentation in the rebuttal format, we averaged across tasks here. We provide mean and Gaussian 95% confidence intervals of LPIPS between our edited-image and original-image CLIP similarity of our edited-image and the edit-prompt averaged over our 9 edit categories of which each contains about 30-50 examples.

As can be seen, our FLUX SAE (hyperparameters: k=20, expansion factor=4) interventions, in which we transport 5 features from the mask within the edit-image into the mask of the original-image with varying strength, dominate the activation steering baseline (higher CLIP similarity with edit-prompt, while achieving lower perturbation of the original-image in terms of LPIPS distance). For reference we also provide a baseline in which we compute LPIPS between original image and itself (= 0) and CLIP similarity between the unmodified original-image and the edit-image.

Qualitative results and demo It seems like reviewers did not find many of our provided qualitative results in our supplementary material as well as our installation-free demo in Appendix A. We highly recommend checking out the demo application if time permits it.

Individual concerns

Below, we address each of the raised concerns (prefixed with “W” when raised in weaknesses and with “Q” when raised in questions) individually. Responses are prefixed with “R”.

W.1. The generalizability of the proposed method in this paper needs further analysis. What conclusions would arise if the original SDXL model or other U-net-based diffusion models, such as SD 1.5, were used? Currently, many diffusion models are based on transformer architectures, so how can this type of analysis be applied to transformer-based models? Analyzing only the SDXL-Turbo model may not provide enough of a technical contribution to be widely applicable.

R. We address the generalizability concern through multiple demonstrations in our work. First, in the paper and supplementary materials, we provide qualitative analysis for Flux-schnell (Figure 1 and Appendix B for more details), showing that our approach extends beyond SDXL-Turbo to transformer-based diffusion models. Additionally, we demonstrate that SAEs trained on SDXL-Turbo can be successfully applied to SDXL - the full multistep model - indicating cross-model transferability within the same architecture family. For our SDXL base results consider (Fig. 1 and Fig. 3 in the main paper, as well as Fig. 11, 17, 18, 19, 20, 21, 22, and 23 in the appendix provided in our supplementary material) or check out our installation-free demo application that we provide in Appendix A. While we acknowledge that broader evaluation across different architectures (SD 1.5, various transformer-based models) would strengthen our claims, our current results provide strong evidence for the method's generalizability across both distilled and full models, as well as different architectural paradigms (U-Net vs. transformer-based). Based on the fact that our SAE features transfer from SDXL Turbo to SDXL base without retraining, we are confident that they also transfer to other SDXL finetunes, of which there are about 10.000 on huggingface today 06/31/25.

W.2. The relationship with other existing works. The existing research [P+: Extended Textual Conditioning in Text-to-Image Generation] also draws similar conclusions, and the author needs to further explain the connections and differences between their work and the existing literature.

R. We discuss this work comprehensively in our "Related Work" section. Here is the key excerpt: "In contrast to these prior works, our approach takes a fundamentally different perspective and does not rely on either handcrafted attributes or prompts. Instead, we introduce a new lens for analyzing SDXL Turbo's transformer blocks, which reveals specialization among the blocks as well. Interestingly, our findings on SDXL Turbo, which is a distilled, few-step diffusion model parallel [56]'s observations, which identify that composition precedes material and style. In SDXL Turbo, this progression occurs across the layers instead of the denoising timestamps." The key distinction is that our method provides interpretation without requiring predefined concepts or prompts, offering a more general and unbiased approach to understanding model behavior.

W.3. Can the conclusions drawn from these observations further enhance the generation capabilities of diffusion models or expand their potential applications?

R. Our approach significantly enhances image editing capabilities through interpretable feature manipulation. We envision a system where users have access to thousands of conceptual "brushes," each corresponding to a specific SAE feature (color, texture, object, concept, detail, expression, illumination, shadow). Users can selectively apply these concepts to specific areas of an image, making targeted modifications with minimal impact on surrounding areas - a level of control not achievable through prompting alone. We provide a demonstration of this tool in Appendix A with an accompanying demo application. Furthermore, our method enables safety-oriented applications, such as modifying text-to-image diffusion models to adhere to safety standards by suppressing features associated with undesirable generations (NSFW, gore, etc.). The related work by Cywinski and Deja ([11] in our paper) demonstrates this possibility for SD1.5, and our approach extends this capability with more interpretable and controllable mechanisms.

W.4. Can these properties be leveraged to further enhance these capabilities during the training of diffusion models to improve the generative performance of the diffusion model?

R. While our work primarily focuses on model inference and interpretation, these insights hold potential for constructing inherently interpretable architectures. We envision that our SAEs, particularly after labeling specific features, could enable novel distillation techniques that selectively transfer desirable features to student models while excluding undesirable ones. This could lead to more controllable and safer diffusion models from the training stage. However, we acknowledge that such applications represent a significant extension beyond our current scope and would require substantial additional research to validate these possibilities with certainty.

W.5. The authors should check the paper carefully before submission to avoid the typo error.

R. Thank you for bringing this to our attention and for helping maintain high scientific standards. We have thoroughly revised our submission (as mentioned in the shared response) and corrected all identified typographical errors. We greatly appreciate your diligence in pointing this out. If you have identified any specific typos, we would be grateful for your feedback so we can cross-reference them with our updated version to ensure complete accuracy.

[11] Bartosz Cywi´ nski and Kamil Deja. Saeuron: Interpretable concept unlearning in diffusion models with sparse autoencoders, 2025

Dear reviewer siKW,

As you have seen already, the authors have responded to the questions in your initial review. Can you please share your thoughts post rebuttal, once you've had a chance to see the author responses and also the other reviews?

Best, AC

We thank the reviewer for their thoughtful evaluation of our work and for recognizing the motivation behind applying sparse autoencoders to text-to-image diffusion models. We appreciate the acknowledgment of our experimental findings showing interesting specialization among blocks and the demonstrated ability to use learned features for steering generation. We proceed by a shared response followed by addressing the individual concerns of the reviewer.

Shared response

Improved writing We significantly improved their manuscript's organization and clarity through several key updates:

• Enhanced Structure & Contributions

  • Added clear paragraph headings for their four main contributions: SDLens (caching library), SAEPaint (visualization demo), RIEBench (editing benchmark), and findings on generalization across steps/architectures
  • Updated abstract and section headers to match the new organization

• Content Consolidation & Clarification

  • Section 3: Merged spatial activation and sparse feature maps into a single "feature interventions" paragraph; added details on SDXL base interventions and classifier-free guidance handling
  • Section 4: Renamed subsection 4.1 to better reflect the RIEBench benchmark; fixed a bug in equation (9); expanded details on benchmark construction and edit categories

• New Content & References

  • Added a new Appendix B specifically covering SDXL Base implementation details, including optimal strategies for feature addition during conditional/unconditional forward passes
  • Systematically added appendix references throughout the paper to provide readers with additional technical details and examples
  • Enhanced cross-referencing between sections and appendices (with updated lettering after inserting the new appendix)

Quantitative results for FLUX In addition, to address the common concern of overly focusing our paper on SDXL Turbo, we performed an evaluation of FLUX Schnell on our representation-based editing benchmark (RIEBench). We will add these results to the camera-ready version of our paper in case of acceptance. For ease of presentation in the rebuttal format, we averaged across tasks here. We provide mean and Gaussian 95% confidence intervals of LPIPS between our edited-image and original-image CLIP similarity of our edited-image and the edit-prompt averaged over our 9 edit categories of which each contains about 30-50 examples.

As can be seen, our FLUX SAE (hyperparameters: k=20, expansion factor=4) interventions, in which we transport 5 features from the mask within the edit-image into the mask of the original-image with varying strength, dominate the activation steering baseline (higher CLIP similarity with edit-prompt, while achieving lower perturbation of the original-image in terms of LPIPS distance). For reference we also provide a baseline in which we compute LPIPS between original image and itself (= 0) and CLIP similarity between the unmodified original-image and the edit-image.

Qualitative results and demo It seems like reviewers did not find many of our provided qualitative results in our supplementary material as well as our installation-free demo in Appendix A. We highly recommend checking out the demo application if time permits it.

Individual concerns

Below, we address each of the raised concerns (prefixed with “W” when raised in weaknesses and with “Q” when raised in questions) individually. Responses are prefixed with “R”.

Q.1 & W.1 It's not clear how stable the steering effect is. For example, if multiple samples were taken, would all show the same steering effect? Also, if the steering coefficient changes smoothly (from a large negative value to a large positive value), would the steering effect also change smoothly?

R. We demonstrate that the steering effect is both stable and smooth across multiple dimensions. Regarding smoothness, we provide comprehensive evidence in Figures 7 and 8 (App. B) for Flux and Figure 35 (App. G) for SDXL, which show smooth transitions as steering coefficients vary continuously from large negative to large positive values. Additionally, our demo application (App. A) allows interactive exploration of this smoothness property.

For stability across multiple samples, Figure 36 (App. G) provides clear evidence that the steering effects are consistent across different generations. Furthermore, Appendix F presents a detailed case study demonstrating that empty-prompt interventions produce consistent image changes, reinforcing the reliability of our approach. We encourage the reviewer to explore our demo application, which provides additional interactive evidence of both the stability and smoothness of steering effects.

Q.2 What's the benefit of training the sparse autoencoder on 1-step updates? If for most experiments you used 4-step updates, why not directly train it on 4-step updates? Is it an efficiency argument? (but I assume you can use 1/4 of the data to train on 4-step updates with the same cost?)

R. Training on one-step updates provides multiple advantages beyond simple efficiency considerations. While computational efficiency is indeed a primary benefit, the deeper insight lies in the nature of feature activation patterns across timesteps. As demonstrated in Figure 3, features activated across different timesteps exhibit remarkable consistency, with the quality of this consistency reflected in the explained variance metrics. Using 4-step generations would oversample correlated activations and repeat similar features/samples, effectively reducing the diversity of training samples within a fixed computational budget. This redundancy would not improve feature quality while increasing computational cost.

More fundamentally, our key insight is the successful decoupling of feature extraction methodology from feature injection during denoising. The surprising and practically valuable finding is that features extracted from 1-step activations effectively generalize to multi-step generation (4-steps for SDXL-Turbo, and even 20-25 steps for SDXL base). This demonstrates that our simpler training scheme is sufficient for extracting interpretable features, significantly simplifying the workflow for practitioners who need to train SAEs on diffusion models.

This finding challenges the intuitive assumption that training and inference should use matching step counts, instead revealing that the underlying feature representations are robust across different sampling strategies.

W.2 It's not clear how well this method generalizes as experiments are mostly done on SDXL-Turbo.

R. Our work demonstrates generalizability across multiple model architectures and configurations. Specifically, we show that SDXL-Turbo features, without any additional training, successfully generalize to SDXL base models (as evidenced in Figures 1, 2, 3, and 36). This cross-model transferability is particularly noteworthy as it demonstrates the robustness of our learned representations. More importantly, we replicate our SAE training methodology on FLUX, a state-of-the-art diffusion transformer-based text-to-image model, achieving successful results (Figures 1, 7, 8). This represents a significant architectural leap from the U-Net based SDXL models to transformer-based architectures. At the time of our initial submission and arXiv publication, we were the first work to train SAEs on intermediate results of the visual backbone in diffusion models. Furthermore, at the time of this NeurIPS submission, we were the first to achieve successful FLUX SAEs with effective feature steering capabilities.

W.3 The results on Flux-schnell are not clear. Why does Figure 1 show "the generalizability of our approach to recent Diffusion-Transformers" (Line 146)? Why didn't you apply the same steering interventions as those applied to SDXL in each column (or did I misunderstand something here)?

R. We applied the same SAE training methodology to Flux-schnell activations, but the learned features naturally differ between architectures, and one-to-one feature matching is not always present. In some cases, achieving specific concepts requires combining multiple features (for example, creating a Santa Claus hat involves two distinct learned features working in conjunction). Figure 1 serves as a qualitative showcase of representative features learned by our SAEs on Flux-schnell, demonstrating successful causal interventions through feature steering. While the specific features differ from those learned on SDXL-Turbo (as expected given architectural differences), we present features with similar semantic content (blue color, "evil" characteristics, cartoon style) to illustrate conceptual correspondence. In contrast, features extracted from SDXL-Turbo one-step training successfully transfer to both SDXL-Turbo 4-step and SDXL base 25-step generation.

The generalizability of our approach refers to the effectiveness of training SAEs to extract interpretable features that enable causal interventions for activating high-level concepts across different architectures. This generalizability is non-trivial, particularly for Diffusion Transformer (DiT) architectures where the role of each layer and the causal relationship between directions in the representation space differ significantly from U-Net architectures.

Dear reviewer SVh8,

As you have seen already, the authors have responded to the questions in your initial review. Can you please share your thoughts post rebuttal, once you've had a chance to see the author responses and also the other reviews?

Best, AC

We sincerely thank the reviewer for their thorough evaluation and insightful feedback on our work. We are encouraged that the reviewer found our work to be the "first application of SAEs to diffusion models," filling a "significant gap," and appreciated the discovery of "causal and interpretable features" and the analysis of "block-level functional specialization."

Shared response

Please find our response addressing shared concerns in our response to reviewer SVh8.

Individual concerns

Below, we address each of the raised concerns (prefixed with “W” when raised in weaknesses and with “Q” when raised in questions) individually. Responses are prefixed with “R”.

W.1. Limited baseline & Insufficient evaluation on Diffusion Transformers

R. We agree that a more thorough evaluation of our method on DiT architectures would significantly strengthen the paper. To this end we will include an evaluation of FLUX on our RIEBench benchmark into the camera-ready version of our paper (see shared response).

W.2. Missing analysis of convolutional components in SDXL-Turbo

R. The reviewer raises an excellent point regarding the role of other architectural components like ResNet blocks. Our initial focus was on the transformer blocks, given their established role in handling high-level semantic and compositional reasoning in generative models.

However, to directly address the reviewer's comment, we ran an additional ablation study where we measured the causal impact of each ResNet block on the final image, similar to the analysis we performed for the attention blocks in our paper. The impact is quantified using the LPIPS distance between the original image and the image generated with a specific block ablated. The results are presented below (R stands for resnet and A for attention blocks):

This new analysis confirms that several ResNet blocks indeed have a substantial causal effect on the final output. Specifically, blocks such as down_blocks.0.resnets.1, down_blocks.1.resnets.0, and down_blocks.2.resnets.0. exhibit an impact comparable to or even greater than the most significant attention blocks we studied. There are resnet layers at the beginning and end of the unet that have an even larger LPIPS score but those destroy the entire image (ablating them results in pure noise), we think they encode/decode specifically to the noise and won’t yield interpretable features.

While we find it highly interesting and agree that decomposing features from these ResNet blocks is a valuable research direction, we believe it is out of the scope of the current work.

W.3. Minor point The expression “One Step is Enough” in the title may not accurately reflect the core contributions of the paper.

R. The idea behind our title is to highlight a key and, we believe, surprising finding of our work: SAE features learned exclusively from the intermediate states of a one-step distilled model generalize remarkably well. They retain their semantic meaning and causal influence when applied in a multi-step generation process with the larger, non-distilled base models. We consider this a crucial result because it demonstrates that meaningful, interpretable features can be extracted with significantly lower computational resources (by training on the distilled model) and then effectively deployed to understand and edit more powerful, multi-step models. We will clarify this motivation in the introduction to better align reader expectations from the outset.

Q.1. What about diffusion transformers?

R. This is an important question that touches upon the generality of our findings. As mentioned in our response R.1, we have taken steps to explore the applicability of our method to other architectures. We have successfully trained SAEs on FLUX-Schnell, which is based on the Diffusion Transformer (DiT) architecture. Our results show that our method effectively learns interpretable and causally effective features in this different architectural paradigm as well. In our revised manuscript, we will present both qualitative examples and new quantitative results from our RIEBench benchmark on FLUX-Schnell to demonstrate that the approach is not limited to U-Net-based models.

Q.2. What semantic and structural properties are captured by the features and what insights were gained?

R. We thank the reviewer for this question and take this opportunity to clarify two central insights of our work. First, as detailed in Section 4.2 of our paper, our analysis reveals a strong functional specialization among different transformer blocks in SDXL. For example:

• The down.2.1 block handles high-level composition and object creation. Interventions on this layer can generate entire objects corresponding to prompt concepts and have a global effect on the image.
• The up.0.1 block specializes in style and texture. Its features can apply localized effects like animal textures, specific colors, or illumination without altering the underlying object structure.
• The up.0.0 block acts as a detail-oriented layer. Its features are highly context-dependent and can be used to add or remove specific semantic properties of existing objects, such as removing a bow tie from a cat (as shown in our figures).
• For FLUX-Schnell, we observed that the learned features tend to be more spatially localized, which is likely attributable to the higher spatial resolution of its latent space. We invite the reviewer to explore these behaviors interactively in our supplementary demo application. Second, a key finding is the remarkable stability of these semantic features across different noise levels. Once a feature is activated during the denoising process, it tends to remain active. This stability is precisely what enables our SAEs, trained on one-step SDXL Turbo, to be transferred directly to the multi-step SDXL Base model without retraining, suggesting a shared semantic subspace between the distilled and base models.

Q.3. The caption for Figure 1 is somewhat minimal.

R. Thank you for pointing this out. We will revise the caption for Figure 1 and the corresponding methodology section to provide a more comprehensive explanation. The numbers below the images (e.g., #2301) are the indices of the specific features being activated from our learned SAE dictionary, which contains thousands of feature vectors. For the FLUX-Schnell example, the SAE was indeed trained on the intermediate features from its one-step generation process. The intervention was then performed during a multi-step generation to demonstrate the generalization of the feature's causal effect. We will make this process explicit in the revised manuscript.

Q.4. Could the authors clarify what specifically is being generalized?

R. The reviewer is correct in their interpretation. When we refer to "generalization," we are referring to the preservation of all three aspects when moving from the one-step distilled model (where the SAEs were trained) to the multi-step base model:

• Semantic Meaning: The features correspond to the same visual concepts (e.g., a specific style, object part, or texture) in both settings.
• Causal Influence: Activating a specific feature produces a consistent and predictable change in the final image in both the one-step and multi-step generation processes.
• Reconstruction Quality: For SAEs that achieve good reconstruction of the hidden state in the one-step model, they continue to be effective in the multi-step context. This finding is intuitive, as the distilled model is initialized from and trained to mimic the base model, suggesting they share a significant portion of their representational structure. We will clarify this definition of "generalization" in the paper.

Q.5. Missing analysis of convolutional components.

R. See the response to W.2.

Q.6. The description of the generalization study lacks sufficient detail.

R. We apologize for the lack of clarity and thank the reviewer for highlighting it. As discussed in our responses R.6 and R.7, the process is as follows: Training: An SAE is trained on a specific layer's hidden state activations, collected from numerous prompts using the one-step distilled model (e.g., SDXL Turbo). Intervention: During a multi-step generation process with the base model (e.g., SDXL Base over 20 steps), we intervene at a chosen denoising step. The intervention consists of adding a scaled version of a specific SAE feature vector (e.g., the 2301st feature from the dictionary) to the model's hidden state at the corresponding layer before it is passed to the next block. We recognize this process is central to our claims and will revise the methodology section to include this detailed, step-by-step description to ensure it is perfectly clear and reproducible.

Regarding missing limitations

R. We thank the reviewer for this comment. We wish to clarify that a limitations section was indeed included in our original submission. As per NeurIPS guidelines, it is located within the checklist section, which appears after the references. We acknowledge that this placement can be easily overlooked. The key limitations discussed there include the points raised by the reviewer, namely the focus on transformer blocks (addressed in R.2) and the need for more quantitative evaluation on DiT models (addressed in R.1). We will ensure the limitations are more prominently signposted in the camera-ready version.

Dear reviewer fSpm,

As you have seen already, the authors have responded to the questions in your initial review. Can you please share your thoughts post rebuttal, once you've had a chance to see the author responses and also the other reviews?

Best, AC