GenEx: Generating an Explorable World

We value the reviewer’s constructive feedback and want to provide additional clarification.

---

W1) Actual impact of imagination on embodied QA

In the designated Genex-EQA, POMDP without imagination is impractical and unreliable.

There are three possible forms of exploration in Genex Embodied QA (Genex-EQA):

• Physical Exploration (POMDP without imagination)
• Imaginative Exploration (Genex)
• No Exploration

Physical vs. Imaginative Exploration: In our EQA benchmark and even in most real-life situations, physical exploration—or POMDP without imagination—is largely infeasible. When making emergent decisions in situations requiring immediate responses, such as navigating a sudden traffic obstruction, agents cannot change their physical positions. Moreover, physical exploration is time-consuming and resource-intensive, whereas imagination allows agents to reason instantaneously. For example, in the case of avoiding the ambulance, the ambulance has driven away when the agent physically explores the scene. As a result, physical POMDPs without imagination are impractical and cannot be achieved in our Genex-EQA benchmark, but Genex serves the same purpose as physical exploration.

Imaginative vs. No Exploration: Our results show that without any form of exploration, single-agent accuracy reaches only 43.50%, while multi-agent accuracy drops further to 21.88%. These figures demonstrate the inability of GPT agents to perform effectively without mental exploration, as they lack the capacity for abstract reasoning or scenario simulation, given only visual and contexture input. When imagination is introduced through Genex, single-agent accuracy dramatically increases to 95.44%, and multi-agent accuracy improves to 94.87%. This substantial improvement underscores the critical role of imagination in enabling agents to reason and make informed decisions under physical constraints.

These findings highlight the impact of imagination on agent performance in Genex-EQA, particularly when physical exploration is not an option.

---

W2) Generation models (including the SVD used in this paper) are poor in the reasoning ability,

We appreciate the reviewer’s insightful observations. We fully agree that most generation models, including the SVD used in our work, are indeed limited in reasoning ability and primarily simulate the probability of objects appearing. This aligns with our own settings, which we consider in our benchmark setting. Specifically, we ensured that in all cases, objects were at least partially observed to avoid scenarios where the model would need to make purely uninformed guesses.

For instance:

• In Figure 4 (top), the model receives the front view of a stop sign and predicts its back view. This serves as an anchor for the model to infer novel views within the scene.
• In Figure 4 (bottom), the model project from a fully observed perspective to what another agent might perceive under partial observation. Genex is used to approximate the other agent’s belief while giving its full understanding of the environment.

This careful benchmark setup reflects our agreement with the reviewer’s point and avoids scenarios where the model is required to reason without sufficient observational input.

We thank the reviewer for their insightful comments and for appreciating our work.

---

W1) There is a gap between the training data and test data.

We also agree with the reviewer that while there are differences in testing environments, this gap provides a valuable opportunity to evaluate the model's generalizability, which is critical for real-world applications.

In terms of cycle consistency, the diffuser achieves a consistency score (latent MSE) as low as 0.07 when trained and tested on different data within the same scene. Additionally, it maintains a consistency score of approximately 0.1 for cross-scene, demonstrating impressive zero-shot generalizability to real-world scenarios. This zero-shot capability highlights its potential for future real-world applications.

The columns are by testing environment.

---

W2) On the use of a POMDP agent – Saying that incomplete visual observation necessarily leads to a POMDP is also not very rigorous.

We acknowledge the reviewer's concern regarding the phrasing in our manuscript and appreciate them pointing it out. POMDP is indeed a modeling framework that describes scenarios with partial observability, with visual observations being only one modality of information. While our intention was to emphasize that the agent operates under partial observability, we agree that the phrasing could be more precise. We will revise this statement to better reflect the framework's scope and ensure clarity.

---

If there is anything else you'd like us to address, please let us know.

Q1) How to determine what’s the trajectory to explore if the world is unlimited? And how to make sure the information is enough to make a decision?

We use large multimodal models (LMMs) such as GPT-4 for navigation and path planning, where exploration trajectories are determined by prompting the LMM agent with a chain of thought reasoning process. This iterative prompting guides the agent to prioritize areas that maximize information gain while aligning with the task objective. A simplified system pipeline is provided in Figure 15, and an example of an exploration prompt is shown in Figure 14.

Decision-making based on the collected information is also handled by the LMM. After exploring, the model evaluates the gathered observations to update its belief state and make decisions. If the information is deemed insufficient, the LMM is prompted to continue exploring until it determines that enough information has been gathered to make a confident decision. This feedback-driven approach ensures adaptability in complex and potentially unlimited environments, allowing the system to dynamically balance exploration and decision-making based on the task requirements.

---

Q2) Is there better way to evaluate the imagination ability, like the 3D concept error with GT (there is hidden car or not, how much unobserved information is discovered)

As we are leveraging image-to-video transformation, determining whether a purely hidden car exists might not be a realistic evaluation metric. For instance, if a hidden car is completely unseeable from the input image, generating such a car would require the diffuser to make a purely speculative guess. For applications requiring safe decision-making, relying on such speculative imagination may introduce risks. Instead, our approach prioritizes evaluating how Genex reconstructs and extrapolates partially observed regions based on the input image, which aligns better with the model's strengths and intended use cases. Currently, we are comparing 3D reconstruction models to evaluate Genex’s ability to imagine the novel 3D concept (Section 5.2; Figure 9; Table 3), focusing on how it generates unseen parts of an object. This evaluation tests its ability to construct partially observed objects, confirming it successfully extrapolates and fills in missing information based on the given input.

---

We greatly value your suggestions. If any concerns remain, we would appreciate further clarifications.

We appreciate the reviewer’s constructive feedbacks and would like to provide additional justification, clarification, and experimental results.

---

W1) The task setting seems not challenging and common enough to demonstrate the usefulness of such imagination ability.

Not Challenging Enough

We believe the challenges in our task setting arise from two key aspects: the necessity of imagination and its effectiveness in making decisions.

In our human study, participants were asked whether they relied on their imagination to answer questions based on text and image inputs. More than two-thirds of the successful respondents confirmed that imagination was essential—something that cannot be achieved using a single image alone.

Additionally, Genex helps humans make decisions. Specifically, we observed improvements in human performance from 91.50% to 94.00% in single-agent scenarios and from 55.24% to 77.41% in multi-agent scenarios. This shows that imagination is not purely intuitive or vague. For example, humans may speculate about what others see but often lack the ability to fully reconstruct the occlusions, perspectives, and detailed views of other agents. The strength of Genex lies in its familiarity with the domain of exploration and hence better than humans in constructing different views. These results confirm the difficulty of solving EQAs without Genex, as the task demands reconstructing unseen details and forming actionable representations. Without Genex, both humans and AI agents face significant limitations in inferring occlusions and perspectives, making accurate decision-making challenging.

By combining these factors, our results show a significant improvement in GPT-4o's performance, rising from 44% to 95% in single-agent settings and from 22% to 95% in multi-agent scenarios. This highlights GPT-4o's initial limitations in the designed scenario and the substantial improvements brought by Genex.

Not Common Enough

This work establishes a foundation and a starting point for exploring imaginative reasoning in complex scenarios. We demonstrate that such a model is plausible, achieves zero-shot transferability, and provides clear benefits to humans and agents in outdoor scenarios. We are actively working to scale this approach to more complex indoor environments and broader applications for Embodied tasks. We welcome ideas to expand this work further.

---

W1) Potentially, the method can serve as a role to generate the bird-eye map from a single panorama image and can reveal the hidden cars not in the observation.

Thank you for your insightful comment. We appreciate your suggestion that our method could generate bird's-eye view (BEV) maps from a single panoramic image to reveal hidden cars. While we considered 360° free navigation at the beginning, we opted to fix the z-axis for egocentric planar exploration to align with our focus on grounded embodied navigation. However, we agree with the potential of extending this method to BEV map generation (which comes with the exact same pipeline) and have conducted preliminary explorations in this direction: Anonymous GitHub link. This capability enables the agent to imagine a third-person perspective through BEV maps, supporting more informed and objective decision-making. We value your suggestion and would be open to further collaboration on this exciting future research. We will also add the demo to the updated version.

---

W2) The paper mentions such imagination can be further updated based on new observations, however, in this work, there is no integration of the imagination and the new observations.

Since our imaginative exploration is controlled by a large multimodal model, integrating new observations is straightforward. New observations can be incorporated as additional inputs in a multi-hop conversational format for the LMM, enabling it to control Genex with this new information. The model can then adjust its belief and incorporate it back into the conversational loop. This capability leverages the inherent flexibility of large multimodal models to process and integrate diverse streams of information dynamically, and it is closely tied to the current LMM's long-visual-context processing ability. Our primary aim is to showcase the potential of imaginative exploration and how LMMs can control and reason with it, while incorporating new observations will become more seamless as LMM capacities continue to improve.

We appreciate the detailed feedback from the reviewer. We would like to add additional clarification.

W1) Q1) Q2) There are some errors in the mathematical formulations presented, particularly in equations (3) and (4), which are confusing.

$\text{(Equation 3)} \quad b^{t+M}(s^{t+M}) = \prod_{t}^M \bigg( \underbrace{O(o^{t+1} | s^{t+1}, a^t) \sum_{s^t} T(s^{t+1} | s^t, a^t)}_{\text{Physical Exploration}} \bigg) b^{t}(s^{t})$

Equation (3) follows the standard POMDP formulation Kaelbling et al., 1998 (page 107). In our variation, we introduce a multiplication of exploration steps to account for a sequence of physical exploration. The timestep $t$ in our model represents an exploration sequence. This modification allows us to encapsulate iterative exploration within a single belief update.

If there are specific aspects of our formulation that appear incorrect, we would appreciate further clarification to address them appropriately.

---

$\text{(Equation 4)} \quad \hat{b}^{t}(s^{t}) = \prod_{i}^I \bigg( \underbrace{ p_{\theta}(\hat{o}^{i+1} | o^i, \hat{a}^i ) }_{\text{Imaginative Exploration}} \bigg) b^{t}(s^{t})$

Equation (4) is derived by replacing the traditional physical exploration components of the POMDP belief update with an imaginative exploration mechanism driven by a diffusion-based generative model parameterized by $\theta$. In the standard belief update (Equation 3), the agent transitions between states $s^t$ using the transition model $T(s^{t+1} | s^t, a^t)$ and incorporates actual observations $O(o^{t+1} | s^{t+1}, a^t)$, which requires summing over all possible prior states to account for uncertainty. By contrast, in imagination-driven belief revision, the agent remains in the current state $s^t$ and employs the diffusion model $p_{\hat{\theta}}(\hat{o}^{i+1} | \hat{o}^i, \hat{a}^i)$ to generate a sequence of hypothetical observations based on imagined actions $\hat{a}^i$. This substitution eliminates the need for state transitions and the associated summations because the physical state does not change; instead, the belief is updated multiplicatively by the probabilities of the imagined observations across the imaginative steps $I$. As a result, the belief $\hat{b}^t(s^t)$ is directly refined by the product of these generated observation probabilities applied to the initial belief $b^t(s^t)$, leading to Equation (4). This approach leverages the generative capabilities of the diffusion model $\theta$ to simulate potential observations, enabling the agent to perform instantaneous and iterative belief revisions without altering the underlying state, thereby enhancing the efficiency and flexibility of the belief update process.

W2) Although SCL is highlighted as a contribution, its effectiveness is not demonstrated in the experimental results.

We compare the performance of the SVD diffuser trained on Genex-DB dataset, both with and without SCL, on the same dataset.

The results demonstrate that SCL enhances video quality, achieving a 15% improvement in the FVD metric.

Furthermore, for exploration cycle consistency, we observe greater improvements as the number of generation steps increases during generative exploration.

As generation step occurs, edge inconsistencies accumulate over multiple generations, causing the input image to become increasingly out-of-distribution for the diffuser. This ultimately leads to a decline in exploration generation, where training with SCL keeps the image generation in-domain of the diffuser over many inferences.

W3) The use of latent diffusion with temporal attention is not a novel architecture.

We would like to clarify that our work does not emphasize the use of temporal attention as a core contribution. Our model is grounded in SVD, which we found sufficient for our task. The discussion of latent diffusion with temporal attention is included solely to explain the referenced work. Temporal attention is a module from SVD, which we have appropriately cited. This aspect was not intended to highlight our own contributions. We will revise the text to ensure clarity and accuracy in this regard.

I appreciate the authors' detailed responses, which addressed most of my initial concerns.

For Q1, about the formula error, I've checked P107 of the book, the formula follows $b'(s')=$ $\frac {O(s',a,o)\sum _ {s\in s}\left(T(s,a,s')b(s)\right)}{Pr(o|a,b)}$ .

while in your paper, this function becomes, $b'(s')=$ $\frac {\left(O(s',a,o)\sum _ {s\in s}T(s,a,s')\right)b(s)}{Pr(o|a,b)}$ .

It is clearly mistaken and wrong. Eq(3) should contain a matrix multiplication term in order to expand multi-step exploration. Could the authors please explain this misalignment step by step？Another issue is that the normalizer is neglected in most of the equations.

For Eq(4), it should be a random sampling process in order to marginalize the $o^i$ and $\hat{a}^i$ variables (for example, monte carlo Tree Search). The original formula make me very confusing. The future video generation is a complex transition distribution instead of a deterministic process. How does this model handle complex future frame prediction if the dangerous vehicle is completely unobserved?

In my point of view, this paper seems to employ world model and LLM to provide pseudo labels for the final policy model, which is not a new idea to me.

Note that some previous papers already expressed similar ideas: world model for reasoning. Reasoning with Language Model is Planning with World Model. Hao et al. (abstract: RAP repurposes the LLM as both a world model and a reasoning agent, and incorporates a principled planning algorithm based on Monte Carlo Tree Search for strategic exploration in the vast reasoning space. During reasoning, the LLM (as agent) incrementally builds a reasoning tree under the guidance of the LLM (as world model) and rewards, and efficiently obtains a high-reward reasoning path with a proper balance between exploration vs. exploitation.)

Equation Misalignment

For equation (3), a key distinction between our formulation and that of Kaelbling et al., 1998 lies in the timing of belief updates. In the original POMDP formulation, the belief is updated at every step as the agent takes actions and receives observations, for continuous refinement of the belief state. However, in our design, the exploration path is preplanned, meaning that all actions and observations during the exploration phase are fixed ahead of time. This allows us to treat exploration as a separate module, isolating it from the belief update process. As a result, for our formulation, the belief is only updated after the entire exploration phase is completed, reflecting the cumulative insights gathered during exploration.

In POMDP, the belief is defined as:

$b'(s') = \frac{O(s', a, o) \sum_{s \in S} \left( T(s, a, s') b(s) \right)}{Pr(o \mid a, b)},$

where $b'(s')$ is the updated belief state, $O(s', a, o)$ represents the observation probability, $T(s, a, s')$ is the state transition probability, and $Pr(o \mid a, b)$ is the normalizing factor.

When exploration spans multiple steps ($M$ steps), the agent accumulates observations and transitions over these steps. Belief updates occur only after this phase is complete. Below, we present the belief updates for $M = 1$ and $M = 2$, leading to the general $M$-step formulation:

1. For $M = 1$:

$b^{t+1}(s^{t+1}) = b^t(s^t) \cdot \left( O(o^{t+1} \mid s^{t+1}, a^t) \sum_{s^t} T(s^t, a^t, s^{t+1}) \right).$

2. For $M = 2$:

$b^{t+2}(s^{t+2}) = b^t(s^t) \cdot \left( O(o^{t+1} \mid s^{t+1}, a^t) \sum_{s^{t+1}} T(s^t, a^t, s^{t+1}) \right) \cdot \left( O(o^{t+2} \mid s^{t+2}, a^{t+1}) \sum_{s^{t+1}} T(s^{t+1}, a^{t+1}, s^{t+2}) \right).$

3. General Form for $M$-Steps:

$b^{t+M}(s^{t+M}) = b^t(s^t) \cdot \prod_{k=1}^{M} \left( O(o^{t+k} \mid s^{t+k}, a^{t+k-1}) \sum_{s^{t+k-1}} T(s^{t+k-1}, a^{t+k-1}, s^{t+k}) \right).$

Here, $t$ represents the time step within the exploration phase, starting from the initial time $t$ and progressing sequentially up to $t+M$.

The Physical Exploration term, $\sum_{s^t} T(s^{t+1} \mid s^t, a^t)$, models transitions between states during the exploration process. This, combined with the observation term $O(o^{t+1} \mid s^{t+1}, a^t)$, ensures that all contributions to the belief are accounted for before the final update.

By structuring belief updates in this way, exploration is a reasoning phase that aggregates information over multiple steps. The preplanning of exploration paths enables this modularity, and belief updates only occur once the process is complete. This modular approach allows the agent to process the effects of exploration comprehensively, reflecting the accumulated transitions and observations in the belief state at the end of the exploration phase. Equation (3) formalizes this process.

---

With that being said, we fully understand your concern regarding the placement of the belief term. If you believe it is better to adhere strictly to the original POMDP setup, we would adjust the formulation to place the belief term appropriately within the process to align with standard conventions.

---

Regarding the normalization term, indeed, for simplicity, we removed it in the current formulation. We will ensure that it is reintroduced in the revision for preciseness and to maintain consistency with the standard POMDP formulation. We appreciate the reviewer for pointing this out.

---

Equation (4) The future video generation is a complex transition distribution instead of a deterministic process.

As discussed earlier, we do not handle future events. The world state is fixed, with all dynamics in the current world halted. Instead of simulating state transitions, we generate new observations to complete the agent’s partial view of the world. Therefore, there is no state transition in our approach.

$\text{(Equation 4)} \quad \hat{b}^{t}(s^{t}) = \prod_{i}^I \bigg( \underbrace{ p_{\theta}(\hat{o}^{i+1} | o^i, \hat{a}^i ) }_{\text{Imaginative Exploration}} \bigg) b^{t}(s^{t})$

In Equation 4, the action $\hat{a}^i$ is explicitly marked with a hat to indicate that it has no real-world consequences. The world, along with everything in it, remains unchanged. While the video generation model introduces randomness, as reflected in $p_{\theta}$’s probabilistic nature, the underlying world state remains frozen, making it neither distributional nor deterministic.

---

If this does not clarify your concerns, we would like to provide further explanation.

In contrast, our work addresses scenarios where the agent lacks a full understanding of the current world state (i.e., received partial observation). Here, the agent performs imaginative actions—mental simulations that do not result in real-world consequences—to explore and revise their belief about the current world state, all while keeping the present state unchanged (i.e. freeze the time).

If I synthesize a new perspective view for a partially-occluded object that I saw in the previous frames, can this be called unobserved? I don’t think so. This is only unobservable to the perception model, but it is fully-observable for people. For objects that are fully unobserved in the previous frames, such as objects in occluded areas, it is obviously difficult to synthesize a corresponding new perspective using the method proposed in the paper. For example, for dangerous objects in occluded areas, how to use the method mentioned in the paper to model them to improve the safety.

So I feel that this paper is a bit like novel-view enhancement to improve perception capbility for LLM, and it has little to do with the POMDP and partial observation model mentioned in the paper. It will be interesting if the authors can provide some examples for fully unobservable objects, and it will make the proposed method more applicable to realistic world.

However, in our design, the exploration path is preplanned, meaning that all actions and observations during the exploration phase are fixed ahead of time. This allows us to treat exploration as a separate module, isolating it from the belief update process. As a result, for our formulation, the belief is only updated after the entire exploration phase is completed, reflecting the cumulative insights gathered during exploration.

I fully understand the exploration phase is fixed. However, in reality, the exploration observation should give a complex high-dimensional distribution, instead of a fixed result, due to the novel-view uncertainty.

I believe a better way is like this: By modeling novel-view exploration as a distribution using SVD, we can sample from SVD with different seeds to model multiple possibilities. Each sampling process can be seen as a Monte-Carlo sampling process. The overall belief updates can be computed by multiple MC samples.

If the exploration results are fixed, I believe you cannot call the initial states unobservable, since all the outcomings can be observed using our SVD model.

The conversion from original POMDP to the Equation in $M=1$, $b^{t+1}(s^{t+1})=b^t(s^t)\cdot (O(o^{t+1}|s^{t+1},a^t) \sum_{s^t}T(s^t, a^t, s^{t+1}))$

Apparently, in POMDP formulation, $s^t$ has multiple possible values, for example $s^t=s_1$, $s^t=s_2$, $s^t=s_3$. Since $s^t$ is not fully observed, it can be a complex distribution.

$b^{t+1}(s^{t+1})= O(o^{t+1}|s^{t+1},a^t) \sum_{s^t}T(s^t, a^t, s^{t+1})b^t(s^t)$

$= O(o^{t+1}|s^{t+1},a^t) (T(s_1, a^t, s^{t+1})b^t(s^t=s_1) + T(s_2, a^t, s^{t+1})b^t(s^t=s_2) +T(s_2, a^t, s^{t+1})b^t(s^t=s_2) )$

Apparently, it cannot be simplified as,

$b^{t+1}(s^{t+1})=b^t(s^t=?)\cdot (O(o^{t+1}|s^{t+1},a^t) (T(s_1, a^t, s^{t+1}) + T(s_2, a^t, s^{t+1}) + T(s_2, a^t, s^{t+1}))$

The only case in which the formula in the paper is correct is that there is only one kind of state for $s^t \in \{{s_{fixed}}\}$. However, it becomes a fully observable problem, since there is only one state. So the formulation of Eq(3)&Eq(4) makes me very confusing.

As discussed earlier, we do not handle future events. The world state is fixed, with all dynamics in the current world halted. Instead of simulating state transitions, we generate new observations to complete the agent’s partial view of the world. Therefore, there is no state transition in our approach.

As discussed earlier, if the states are partial observed, there are unlimited possibilities for new observations. If there is only one possibility for new observations, then the initial state should be fully observed (at least with the helf of SVD). So I still think the formulation in the paper is a little bit confusing.

The results in this paper is interesting, but I still think the formula in the paper should be improved to make it more clear to understand.

Dear Reviewer,

Thank you for your reply!

---

So I feel that this paper is a bit like novel-view enhancement to improve perception capability for LLM.

We agree that our work shares similarities with novel-view enhancement. Different from standard novel-view techniques, our approach enables unlimited spatial exploration in all dimensions. This allows for possibilities beyond static viewpoint synthesis and further control. Moreover, since the exploration process—though facilitated by diffusion generation—is controlled by an agent model, it mirrors a POMDP structure akin to real-world physical exploration (we agree with your points about uncertainty and would clarity below).

If I synthesize a new perspective view for a partially-occluded object that I saw in the previous frames, can this be called unobserved?

In our work, we primarily focus on objects that are never fully observed from any angle or perspective. For example, an object might remain consistently occluded or partially visible throughout all frames, leaving certain parts completely unviewed and requiring synthesis based on incomplete information. We also acknowledge the reviewer’s point that completely unobserved objects or scenarios inherently involve uncertainty due to the nature of the diffusion process. Because of this, our work centers on safe decision-making scenarios, specifically focusing on partially observed scenes. We also see the potential of the cases involving the generation of entirely new objects or environments, we provide sample demonstrations in our anonymous video demo: www.youtube.com/watch?v=rFwuCTsrYVU (e.g., partially observed scenarios at 1:25 ,completely unobserved cases at 0:35, zero-shot generation at 1:00). While time constraints during the rebuttal period limit our ability to provide extended exploration examples for completely unobserved scenes, we plan to include such examples in future revision.