Thank you for your thoughtful comments. We’re thrilled that you found the paper well-presented, clear, and with impressive results. We put significant effort into making our evaluations comprehensive and included detailed implementation information to support the reproducibility of our work.

We also want to thank you for your constructive review on limitations. It is important for us to not gloss over the weaknesses of our method, and we appreciate you raising this concern. While we have included our method’s limitations in the main text, and they are visible in the provided gameplay, we have added videos with explicit qualitative samples of failure cases to the supplementary including context issues and behavior in unexplored areas as you requested. In addition, we added a note in the eval section directing the readers to the longer videos provided in the supplementary for an example of the quality on longer time horizons (which is distinguishable from the original game but still playable and enjoyable).

Regarding the phrasing of the introduction – we feel that the wording we used (“While not an exact simulation…”) is accurate; to be sure this is read as “yes, but” we’ve added a reference to the limitation section. Importantly, the simulation is playable and enjoyable, and the simple cpu-based agent we introduced is able to fully explore significant portions of the game (for example, the first level is playable until the last room).

Regarding motivation, we do believe that neural game engines will enable new capabilities that were previously harder to achieve, and acknowledge that this is only partially explored in this work. For example, Appendix A4 provides a motivating example to how game behaviors can be developed from example images. Furthermore, it shows that a neural model is able to combine behaviors from different levels – making the argument that combining different behaviors from different games in order to create new ones may be possible as well. We are happy to continue working with you on the phrasing here as needed – our goal is to introduce possible research directions while acknowledging that are not fully explored in this work.

Below we answer additional questions you’ve raised:

Can a level be played end-to-end? The provided gameplay videos show the first level being played until the last room, taking ~1.5 minutes.

Short samples in the user-study: It is true that longer examples are distinguishable from actual gameplay, but they are still playable and enjoyable. We added a note about this in the relevant section.

Random agent ablation: The random agent repeats its actions for four time steps but it does not repeat last actions. There is indeed a significant difference between human actions and the random ones, however this is also the case for the agent-based data – when we tried increasing the bias for the last action too much, we saw that it prevents the agent from training well as it relies on stochastically exploring new behaviors. Training the agent for longer, or not including the initial random exploration is interesting to try – we hadn’t made these experiments.

Different levels of noise: Adding different noise levels during training increases the robustness of the generative model. Higher noise levels encourage the network’s predictions to stay close to the training distribution, even under highly corrupted conditions, while low noise levels ensure that the model will cosider smaller details in the input frame. We ran training experiments where we only added low noise levels but found it performed worse than training on randomly corrupted conditions (both high and low corruption levels). During testing, we set the noise level condition to zero, conditioning on non-corrupted frames generated from previous game steps. We did not fully ablate this particular choice, and adding some artificial noise to the history may prove useful.

How long did training take: 700k steps took 4 days.

50M env steps vs. 70M examples: The number of environment steps the agent was trained on does not fully determine the amount of output as we also captured gameplay from large eval runs throughout the agent training.

Human eval: The second set of clips is generated by rolling both the game and the simulation on 10 minute worth of actions. We compare clips between 5-10 minutes. It is true that after 5 minutes the game and the simulation diverge, so the clips look different.

Stable diffusion 1.4: We haven’t ablated other architectures, nor training from scratch. We found Stable Diffusion 1.4 to be expressive and efficient for our task. We hope further development and better architecture design will be suggested in future works.

More than 700k steps: We continued running for more steps (up to 1M) after the paper was written but did not run full evals. Indeed the PSNR continued to improve, and test loss went down.

Thank you for your thoughtful review! We appreciate your insightful questions, positive feedback on the work’s relevance to the community, and help identifying key related works we missed. Below, we address issues you’ve raised.

More details on speed: We agree and have moved the results into the main paper.

Related work: We would love to work with you on improving this section and appreciate your feedback.

"Diffusion for world modeling: Visual details matter in Atari" (Diamond): Thank you for highlighting the OpenReview version (named "Diffusion world models"). The authors recently added a high-res Counter-Strike simulation comparable to our work, though without quantitative evaluations; earlier versions were low-res and hint at degeneration after a few frames [1]. We agree this work is highly relevant and have updated our paper to further highlight it.

GameNGen achieves long trajectory, high-resolution simulation thanks to three key features not used by Diamond:

• Agent-driven data collection: Counter-Strike in Diamond does not use an agent and therefore has limited training data. The Atari version aims at improving the agent and not the simulation. Therefore, the agent’s reward is not an exploration reward like the one we use. Moreover, even if the reward was geared towards exploration, training only in simulation prevents the agent from learning to explore elements that are not simulated yet (e.g. opening doors).
• Noising the condition variable: Input noising prevents compounding errors when using the model autoregressively. Ablating noise from our model results in errors like in this link).
• Latent diffusion and optimizing the decoder - allows our model to render small details like text and faces at 20fps.

We also added GAIA-1, VideoGPT, and Temporally Consistent Transformers for Video Generation and Diffusion Forcing to the paper. These are all important and relevant contributions and we thank you for highlighting them.

Concurrency: We aim to include all relevant papers (concurrent or not) in our related work and appreciate your comments. For novelty, the ICLR FAQ considers publication at a peer-reviewed venue. The Diamond paper will appear at NEURIPS 2024 (December). A recent ArXiv update (October 30, 2024) added a high-res Counter-Strike simulation and cited GameNGen as concurrent work. We are fans of this effort (partially inspired by GameNGen [2]) and see it as a fruitful interaction between teams.

Extracting gameplay: This phrasing highlights our method’s ability to use an RL agent to crawl an existing game and generate diverse, scalable training data.

Figure 2: We included the HUD to highlight the challenge it adds to visual simulation (a naive approach with HUD fails; see Appendix A2). The caption now notes that other methods use different training data.

definition of 'Interactive Environment’: We wanted to focus on the simulation aspect and our definition is similar to POMDP without rewards, as we don’t need them.

Pretrained vs. Starting from scratch: We fine-tuned Stable Diffusion to leverage its visual knowledge and speed up training. While we didn't ablate this, starting from scratch would likely work but take longer to achieve similar quality.

Game coverage: Agent exploration varies by starting point—for example, it fully explores the first level except the last room but struggles in areas requiring finding keys or killing many monsters. Sparse data can cause the model to hallucinate smooth transitions to well-explored areas (see unexplored_area.mp4 in the supplementary). Quantitative comparison is in Appendix A7 (e.g., frames 1500–1750).

Data: The model is trained on ~70M examples (64 input frames and 1 target frame), randomly sampled from agent training sequences. Over 700k steps, each frame is reused ~45 times as input and ~1.3 times as a target.

Context length: See limited_context.mp4 in the supplementary for a relevant example. As noted in the paper, context effects are typically short-range (e.g., objects, game physics, animation), with marginal improvement over longer ranges. This may be due to the rarity of cases like the one you describe in our training data and is an area for future work.

Releasing code: We will look into releasing the code soon. Meanwhile, we used open-source tools and provided detailed implementation steps to allow for full reproduction by the community (example).

Long generations: See the supplementary (figure_1 folder) and videos here. The videos demonstrate temporal consistency, accurate level geometry, and correct weapon and health tracking in gameplay. More can be generated if needed.

[1] from the paper: “Whilst in immediate frames these have the intended effect, for longer roll-outs the observations can degenerate.”

[2] in a tweet by the author omitted for anonymity.

Thank you for reviewing our paper and providing valuable feedback to help us improve it.

Regarding using Doom: The focus on DOOM was driven by the desire for reproducibility (as it is an open-source game engine), while still providing complex game logic and graphics. It’s a good point that some aspects of Doom may have helped, we tried to highlight these in the limitations section (we’re revising the text to make this clearer - thank you). However, the game is sufficiently complex to serve as a proof of our concept, including many ways to interact with the environment (open doors, pick up armor, explode barrels, attack enemies, walk on lava), as well as diverse geometry and visual details.

Regarding downstream tasks: The agent-related tasks that you mentioned would be an interesting follow up (we’re inspired by the many important works here like [1, 2, 3]). Our work focuses on the relatively less-explored area of using neural models as game engines. In this context, a critical downstream application is the ability to generate new levels and behaviors seamlessly. Appendix A4 presents preliminary results demonstrating this capability, where new behaviors are generated using input images created with a graphical editor such as Photoshop. These results are particularly exciting as they highlight a potential advantage of neural game engines over traditional ones: the ability to guide game behavior simply by generating a static scene with a graphical editor. Please also see the supplementary material (folder appendix_A4_edit_game) for relevant videos illustrating these results. Would this address your request for additional downstream tasks?

[1] Ha, David, and Jürgen Schmidhuber. "Recurrent world models facilitate policy evolution." Advances in neural information processing systems 31 (2018).

[2] Kaiser, Łukasz, et al. "Model Based Reinforcement Learning for Atari." 8th International Conference on Learning Representations, ICLR 2020.

[3] Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Mohammad Norouzi. “Dream to control: Learning behaviors by latent imagination”. 8th International Conference on Learning Representations, ICLR 2020.

Thank you for your thoughtful comments and positive feedback. We appreciate your recognition of the significance of demonstrating that a neural network can simulate a relatively complex video game in real time. We also felt that this seemed far-fetched not long ago—and appreciate your acknowledgment of the engineering effort involved. We're particularly glad you found value in the diverse metrics and ablations, as it was important for us to fully evaluate and validate our approach.

We changed the paper to include the related work that we have missed in the main text. Please see below answers to specific questions raised by you.

Model Generalization to unseen gameplay: The model indeed memorizes the structure of the areas it was trained on, but it is able to simulate new trajectories, viewpoints and situations in these areas. When the model is initialized to a completely different unseen area, it will gradually transition it to a familiar area (see unexplored_area.mp4 in the supplementary for an example). In other, less extreme, cases (as seen in appendix A4) the model is able to combine new elements that did not appear in the training data into the simulation.

Code and model release: Unfortunately we are unable to release the model weights at this time but we are hoping to be able to do so or at least release the code soon. Regardless, we’ve made the training recipe as detailed as possible and used open source base models hoping that the community creates an open source reproduction (example). We’ve also added gameplay examples, including failure cases (added in the latest version of the supplementary), to further help readers evaluate the model performance.

Importance of noise augmentation: Without noise augmentation the simulation deteriorates quickly, especially when the player does not keep moving (see Fig. 4 for an example).

Thank you for your reply. I will maintain my score trusting that the authors will release the code soon. While GameNGen's amazing performance is in part due to training data overfitting, I still believe that GameNGen is an impressive step forward from the prior neural game engine-related work.

Thank you for reviewing our paper and providing thoughtful feedback. Please find our replies to your feedback below.

To the point of novelty: we see the novelty and contribution of the work in introducing the task of real-time interactive game simulation, and in the successful adaptation of a widely available open-source text-to-image model (Stable Diffusion) to it. Apriori, the fact that it is possible to simulate a complex game in real-time with high-res details (like text) using a diffusion model, was not immediately obvious [1]. We provide quantitative evidence for simulation quality, detailed ablations of key components, and a reproducible training recipe, which we hope are valuable contributions to the ICLR community.

At the level of technical novelty, we would like to highlight that straightforward application of text-to-image models does not work, and three technical innovations were needed:

1. We found noise-augmentation to be critical to make auto-regressive sampling work over a long time horizon. To the best of our knowledge, this method was not previously applied in an auto-regressive context.
2. Collection of training data using an RL agent was required for scaling data beyond human-play recordings.
3. Decoder fine-tuning for achieving comparable visual quality as the source game.

Clarifying Section 2: Thank you for your questions and apologies if the notation in this part was not clear enough. The simulation model $q$ gets as inputs an history of observations $o_{<n}$ and actions $a_{ \le n}$, and produces the next observation $o_n$. Note that we have two streams of observations, one coming from the model ($o_q^i$) and another coming from the ground truth environment ($o_p^i$). The input to the simulation can either be observations that were obtained auto-regressively from the model itself, or the ground truth observations. While training we use the latter (we refer to this as “teacher forcing” in the paper). We define the task of “Interactive World Simulation” as minimizing some given distance metric D between the two sequences mentioned. We’re trying to see if we can amend the writing slightly to make this clearer and please let us know if you have further questions or suggestions on how to improve the notation here.

Scope of Application: The focus on DOOM was driven by the desire for reproducibility (as it is an open-source game engine), while still providing complex game logic and graphics. The game includes many ways to interact with the environment (open doors, pick up armor, explode barrels, attack enemies, walk on lava), as well as diverse geometry and visual detail. It’s a good point that the photorealism of DOOM is lower than modern games. In terms of game complexity we see it as very challenging for simulation, especially given that many prior work in this space focused on simpler environments such as Atari games.

[1] For example see this quote from JqCB: “I appreciate the amount of engineering that went into making this, which seemed far-fetched a year or two ago”.