SceneDecorator: Towards Scene-Oriented Story Generation with Scene Planning and Scene Consistency

We sincerely appreciate the time you dedicated to reviewing our paper and the insightful feedback you provided. In the following, we would like to present our point‑by‑point responses to your comments and concerns.

Q1: The main concern is that, the paper claims as a scene decorator, but the generation results are so different in terms of the background. It somehow totally change the background sturcture and does a re-creation from the input patches. My point is that, the generation consistency is not that good and it's caused by the method design.

We appreciate the reviewer’s insightful concern and understand the concern regarding the changes in background observed in the generated results. Below, we would like to provide a clarification regarding our task setting.

The task we study, scene-oriented story generation, aims to maintain high-level scene consistency, which involves preserving the environment, visual style, and overall atmosphere of a scene, rather than reproducing scenes at the pixel level. Such a task definition aligns with the widely recognized objective of mainstream story generation, allowing reasonable scene changes for contextual adjustments and adaptations to better fit the storylines, since scenes are not expected to remain completely static as the story changes.

In other words, the essence of scene-oriented story generation lies in generating coherent and engaging story images, where variations in the scene are inevitably introduced to adapt to storytelling content. In this sense, our SceneDecorator is designed to function not as a low-level visual detail preserver but as a high-level visual semantic transmitter.

If strict low-level visual preservation were enforced, the task would devolve into image editing or inpainting, which diverges from the objective of mainstream story generation. In contrast, scene-oriented story generation preserves the core identity of the scene while allowing for controlled creative flexibility, which we believe is more aligned with real-world applications.

Furthermore, we have conducted extensive experiments demonstrating that our approach achieves state-of-the-art performance, as validated across three aspects of evaluation: quantitative metrics, qualitative analysis, and user studies. These results showcase the effectiveness of our method in balancing scene coherence and adaptability in the context of story generation.

We sincerely thank the reviewer for raising this critical point, and we hope this clarification addresses the concern.

We sincerely appreciate the time you dedicated to reviewing our paper and the insightful feedback you provided. In the following, we would like to present our point‑by‑point responses to your comments and concerns.

Q1: Performance is bounded by the capabilities of FLUX.1-dev and Qwen2-VL.

Thank you for raising this important point. While our method is indeed built upon existing backbones, we would like to emphasize the flexibility of the designed paradigm.

The core contribution of our work lies in the structured VLM-based scene planning framework and the long-term scene-sharing attention mechanism. These components are inherently plug-and-play and highly flexible, enabling seamless integration with both current and future foundation models without requiring architectural modifications. As demonstrated in Sec. 5 of the main paper, our approach is also compatible with a wide range of existing techniques, including PhotoMaker, ControlNet, and stylized LoRAs.

Currently, we adopt FLUX.1-dev and Qwen2-VL as our base models, as they represent some of the state-of-the-art open-source models in their respective domains. We are confident that as advanced foundation models emerge, integrating them into our framework will further enhance performance and unlock greater potential.

We hope this clarifies your concern, and we sincerely appreciate the opportunity to address this point in our final version.

Q2: The evaluation relies on 32 users which may not capture diverse cultural interpretations of scene coherence.

We appreciate your valuable feedback and agree that broader and more diverse participants are important for evaluating cultural interpretations of generated results.

To address this concern, we have expanded our user study to include an extended participant pool. Specifically, we conduct an extended evaluation involving participants from a range of countries, cultural backgrounds, genders, and age groups. This updated user study yields 61 valid responses, which we believe offer a more representative and comprehensive view of how Text Alignment, Scene Alignment, and Image Quality are perceived across different demographics. The results are summarized below:

These results show that our method, SceneDecorator, consistently outperforms existing baselines such as CustomDiffusion, ConsiStory, and StoryDiffusion, even when evaluated by participants from diverse cultural backgrounds. This superior performance further highlights the robustness and generalizability of our framework across cross-cultural users.

We hope this expanded analysis helps address your concern, and the extended user study and its findings will be included in our final version.

Thank you for your detailed response. Your rebuttal address my concerns. I think this is a good paper and consider to update my confidence about my judgment.

We sincerely appreciate the time you dedicated to reviewing our paper and the insightful feedback you provided. In the following, we would like to present our point‑by‑point responses to your comments and concerns.

Q1: What do the authors think about extending from scene-oriented story generation to scene-oriented movie generation? What would be the challenges and technical problems to solve?

We appreciate the reviewer’s forward-looking insight. Extending from scene-oriented story generation to scene-oriented movie generation is a valuable and exciting direction with significant potential to push the boundaries of AI-driven visual content creation. In the following, we discuss potential challenges and technical problems, respectively.

Challenges

Extending from scene-oriented story generation to movie generation is a natural and promising direction, but it also introduces new technical demands. Current methods rely on image generation models to handle isolated scenes. Moving toward movie generation requires video generation models that can account for temporal dynamics and maintain consistency over time. Based on our observations, we outline three challenges worth exploring:

1. Multi-Scene Shot Transitions: In a movie, transitions between shots and scenes also carry narrative expression. How a scene begins and ends, how camera angles shift, and how characters move across shots all affect storytelling. One key challenge is making these transitions smooth and logically consistent so that the final result doesn’t feel fragmented. This will likely require finer control over visual continuity and temporal alignment.
2. Scene-Expandable Movie Generation: Generating movies where scenes can dynamically expand or evolve depending on the narrative context presents another challenge. For instance, enabling the system to transition from focused, character-driven moments to broader, environment-focused shots requires the ability to generate multi-scale content while maintaining narrative cohesion. This capability can support richer storytelling and more immersive visual experiences.
3. Multi-Modal Integration with Scenes: Combining scene information with multi-modal conditions, such as synchronized audio (e.g., dialogue, ambient sounds) or textual descriptions, is key to enriching storytelling. This requires aligning visual semantics with auditory and textual elements while ensuring temporal consistency. Effective integration across modalities can greatly enhance the emotional depth and immersion of generated movies.

Technical Problems

Naturally, scene-oriented movie generation still faces several technical limitations due to the increased complexity of generating temporally consistent, high-quality video content compared to static images. Below, we highlight two critical technical problems that require further exploration:

1. Computational Cost: Movie generation is inherently more resource-intensive than image generation, as it involves producing high-resolution video frames with consistent temporal dynamics. This leads to significantly higher computation, memory, and storage requirements. To address these issues, methods such as model compression and sparse computation could be explored to reduce computational overhead while ensuring scalability during training and inference.
2. Generation Quality: Ensuring generation quality is crucial, as it directly impacts the audience's engagement and the overall storytelling experience. High-quality movie generation requires maintaining narrative coherence, stylistic consistency, and a balance between creativity and realism. The system must ensure that visual elements align with the storyline, follow a cohesive artistic direction, and produce compelling yet believable content.

We believe that scene-oriented movie generation will emerge as an important research area in the near future, and we hope our work can serve as a foundation for further exploration in this domain. Once again, we sincerely thank the reviewer for the insightful comments, which have inspired us to think deeply about the future directions of our research.

We sincerely appreciate the time you dedicated to reviewing our paper and the insightful feedback you provided. In the following, we would like to present our point‑by‑point responses to your comments and concerns.

Q1: The “VLM-Guided Scene Planning” module is described as the core novelty of the pipeline. However, it appears to be primarily a well-crafted prompting mechanism applied to a powerful pretrained VLM (Qwen2-VL), with little algorithmic innovation. The formulation (e.g., Q = Fₜ(T)) may overstate the underlying complexity.

We appreciate the reviewer’s attention to the innovation of VLM-Guided Scene Planning. Our method indeed builds upon the pretrained Qwen2-VL model, but we would like to emphasize that our approach equips VLMs with the capability of scene planning, which they do not inherently possess.

Specifically, our innovation does not lie in modifying the model architecture itself, but in the deliberate design of a set of strategies for achieving scene planning via VLMs, which includes:

1. conceptualizing the global scene, where the VLM generates a comprehensive and cohesive scene description based on a user-provided theme;
2. visualizing the global scene, using an excellent T2I generator to transform these descriptions into high-quality visual representations; and
3. crafting local scenes and stories, where the global scene is decomposed into multiple local sub-scenes and narratives to ensure both semantic coherence and narrative continuity.

These strategies collectively empower VLMs to undertake scene planning, transforming them from general-purpose models into specialized ones capable of bridging the gap between abstract themes and semantically coherent visual storytelling, which is a capability not previously explored in existing methods.

As mentioned above, our method adopts a three-stage process for scene planning. To precisely explain the functionality of each stage, we represent each stage as a series of formalized functions in our paper. These formulations are not intended to overstate the complexity but to explicitly clarify the distinct inputs and outputs at each stage, thereby providing a more systematic and informative illustration of the overall workflow.

In summary, our method introduces a novel way of utilizing VLMs for scene planning by designing a three-stage process that includes conceptualizing the global scene, visualizing the global scene, and crafting local scenes and stories. This combination of strategies represents a significant step forward in achieving scene-oriented story generation, exploring a technical solution that has not been studied by existing methods.

Q2: Could authors please provide a precise description of the modifications made to each baseline (CustomDiffusion, ConsiStory, and StoryDiffusion)?

Thank you for the question regarding the implementation details of the baselines. Below, we clarify how each method is adapted to our task setting:

1. CustomDiffusion is a tuning-based method that requires fine-tuning to inject given visual concepts. Specifically, we fine-tune the CustomDiffusion using the local scene images obtained from our VLM planning module as training data. During inference, we adopt the corresponding story sub-prompts together with the fine-tuned model to enable a comparison under our task setting.
2. StoryDiffusion is a training-free method designed for subject-consistent story generation and naturally supports image conditioning. We directly use the local scene images and the associated story sub-prompts from the VLM planning module as input for comparison.
3. ConsiStory is also a training-free method for subject-consistent story generation. However, the official implementation does not support images as input. For a meaningful comparison, we make minimal modifications to adapt it to our task, prepending the local scene images to the input batch and leveraging it as a reference through self-attention and feature infusion mechanisms. Additionally, the same story sub-prompts are also adopted accordingly.

It is also worth noting that all experiments are conducted on the same scene dataset processed by the VLM planning module to ensure a fair comparison.

We hope this resolves your concern, and we sincerely appreciate the opportunity to include these descriptions in our final version.

Q3: How robust is the VLM planning stage? What is the approximate failure rate (e.g., instances where the VLM provided unusable coordinates, nonsensical storylines, or failed to adhere to the theme)?

We sincerely thank the reviewer for this insightful comment. Accordingly, we would like to discuss the robustness of VLM-Guided Scene Planning in two aspects:

1. Coordinate Rationality: Failures where the VLM generates unusable coordinates are exceptionally rare. To seamlessly handle such cases, we have implemented a simple yet effective correction strategy: invalid coordinates are automatically adjusted by snapping them to the nearest valid bounding box, which ensures proper cropping.
2. Semantic Rationality: To evaluate the success rate of producing semantically reasonable scenes, we conducted a quantitative assessment using GPT-4o. Specifically, for each sample, we input the story theme, the global scene, the obtained local scenes, and the story sub-prompts into GPT-4o, which is guided to conduct evaluation across three key criteria: Narrative Coherence, Theme Adherence, and Layout Reasonableness. Each criterion was scored on a 10-level scale, with scores ranging from 0 to 100. The evaluation was performed on 2,920 samples, showing that our method can also achieve satisfactory performance in the semantic aspect, with the results summarized below:

In summary, our VLM-Guided Scene Planning shows strong robustness in both coordinate and semantic rationality, highlighting its effectiveness in producing coherent and reasonable scenes for visual storytelling.

We will incorporate these discussions and quantitative results into our final version to provide a more comprehensive analysis.

Q4: The entire "global-to-local" pipeline is built on a fatal assumption: that a story's scenes are just different crops of a single, static master image. This is a profound limitation. This framework is incapable of depicting scenes that evolve or change over time, which is a core element of countless narratives. / Can "SceneDecorator" be called as Scene-Oriented Story Generation in light of its inability to handle stories with scenes that evolve or change over time (e.g., a landscape changing from day to night, a location being altered by character actions)?

We thank the reviewer for raising this important question regarding the temporal dynamics of scenes in our framework. In fact, our SceneDecorator is capable of handling stories with scenes that evolve or change over time, through two distinct capabilities integrated within our design:

1. Creating time-evolving story sub-prompts: Given a local scene image, our VLM-guided scene planning module can generate a series of story sub-prompts that unfold over time within that scene (e.g., a local detail being altered by character actions), rather than producing a static story description. In this way, the VLM can infer a sequence of events based on its commonsense knowledge, such as a character arriving, interacting with the environment, and leaving, all within the same scene context. These story sub-prompts can then be used by the subsequent image generator to produce story images that change over time, achieving visual storytelling with temporal evolution.
2. Creating time-evolving story scenes: Alternatively, we can also generate multiple global scene images that evolve over time (e.g., a landscape shifting from day to night) in the VLM-guided scene planing module. The VLM can then collaboratively plan and crop local scenes simultaneously from multiple global scenes to create a time-evolving storyline. The resultant local scenes naturally inherit the time-evolving visual semantics from their respective global scenes, enabling the generation of visual stories that reflect changes over time.

This flexibility allows SceneDecorator to support dynamic, time-evolving narratives, thereby addressing the mentioned limitations. Due to the policy prohibiting the presentation of images during the rebuttal phase, we will present the relevant experiments and visual results, along with these discussions, in our final version.

Q5: The authors have not adequately addressed the potential negative societal impact of their work.

We sincerely appreciate your valuable feedback and apologize for initially overlooking the potential societal impacts of our work.

Our work is designed for scene-oriented story generation in the domains of visual content creation. However, we recognize that any technology capable of generating multi-image narratives may carry societal risks if misused. Specifically:

1. Disinformation and propaganda: The ability to generate visual narratives could be exploited to create persuasive but false stories, amplifying the spread of disinformation.
2. Bias and stereotypes: Unintended biases in input story theme could lead to the reinforcement of harmful stereotypes or discriminatory representations.
3. Inappropriate or harmful content: Without proper safeguards, generated content might unintentionally include sensitive, violent, or inappropriate material, potentially causing emotional harm to audiences.

As you suggested, we will carefully re-check and revise our checklist response and include a thorough discussion of societal impacts in our final version.