Dear Reviewer C4Si, we sincerely thank you for your recognition of the quality of our results and our proposed ViewPoint representation. We have carefully considered your insightful suggestions and questions, and our responses are as follows:

Q1: Further Challenges and Limitations.

As you pointed out, video duration and resolution remain the main challenges in panoramic video generation. There are two core issues: First, the large data volume of high-resolution panoramas (e.g., 8K) imposes significant computational demands that current video generation models struggle to meet. Second, the lack of high-quality panoramic video datasets further exacerbates the problem, fundamentally limiting a model’s ability to produce coherent and richly detailed panoramic content. In response to your suggestion, we have added the following section discussing limitations and future work in the final version.

Limitations and Future Work

A significant challenge in panoramic video generation is achieving high resolution. Immersive applications such as virtual reality (VR) necessitate ultra-high resolutions (e.g., 8K, 16K) to deliver a realistic and compelling user experience. However, current pre-trained diffusion models are computationally constrained and cannot directly synthesize content at such scales. Consequently, in line with established practices, our work employs a two-stage generate-then-upscale pipeline to mitigate this limitation.

Looking ahead, our future work will focus on two primary directions. First, we aim to explore end-to-end models for ultra-high-resolution panoramic generation. Second, we will investigate the extension to long-form video generation, for instance, by leveraging autoregressive frameworks.

Q2: What would happen if we directly enlarge the FoV of the CubeMap projection?

Following your suggestion, we enlarge the standard 90° FoV in our CubeMap-4DROPE setup and retrain the models with FoVs set to 92°, 95°, and 100°. The results below demonstrate that enlarging the FoV does not significantly improve the model's performance. As for the visual results, color discrepancies and seams still exist, and the transitions between adjacent faces appear unnatural. (We regret that the rebuttal policy prohibits external links, which prevents us from providing the generated videos.)

Regarding the reason for this phenomenon, our analysis is as follows: Methods based on CubeMap projection (or multi-view formation) aim to enable the model to implicitly learn the relationships between multiple perspectives. Increasing the FoV of the CubeMap projection to create overlap between faces is an intuitive and reasonable strategy for the panoramic image generation task, as it helps the model better recognize the adjacency between faces. However, when we extend the task from panoramic images to panoramic videos, this advantage does not directly translate. Our hypothesis is that the temporal dimension in video naturally contains spatial relationship information.

For instance, consider a person walking left on a road. At time t, this person appears on the Front (F) face. At time t+1, this person appears on the Left (L) face. Even with a 90-degree FoV (no overlap), the model can infer the adjacency of the F and L faces purely from the temporal sequence. The spatial inference capability we expected to gain from a larger FoV is likely already captured by, or redundant to, the information present in the temporal dimension.

Another drawback is that increasing the FoV leads to higher computational complexity. Here, we provide the formulas for calculating the FoV and the projected area. As panoramas are inherently a spherical representation, assuming the camera is located at the center of the sphere and the distance from the camera to the projection plane (i.e., the equivalent focal length) is f, in the standard CubeMap projection where the horizontal FoV equals the vertical FoV, the side length L of each face is given as follows: $L = 2 \cdot f \cdot \tan\left(\frac{\text{FoV}}{2}\right)$ Therefore, we can derive the area A of each face (i.e., the number of pixels). $A = (2 \cdot f \cdot \tan\left(\frac{\text{FoV}}{2}\right) )^2$ Representing the spherical view as a CubeMap projection requires six perspectives, so the total area S of the entire sphere (i.e., the total number of pixels) is given by: $S = 6 \cdot A = 24 \cdot f^2 \cdot \tan^2\left(\frac{\text{FoV}}{2}\right)$ To give a concrete example, in a standard setup where each cubeface is 256x256, the FoV is 90 degrees. At this point, the six faces do not overlap, and the total pixel count is 393,216 (approx. 390k pixels).

Now, let's assume we increase the FoV to 95° to create overlap between the faces. The total pixel count rises to 468,302 (approx. 470k pixels). This redundancy is nearly equivalent to adding an entire extra standard CubeMap projection. When we factor in the temporal dimension—for example, a 49-frame video—this requires even more resources. Although the VAE compresses the video in both spatial and temporal dimensions, the overhead from this redundancy is still substantial in a computationally dense task like video generation.

Q3: Supplementing Illustrations and Extending Applications.

1. Following your suggestion, we have added our visual results to Figure 4 in the final version to better compare the detailed differences between methods.

2. Our representation has the potential to be applied to other modalities, such as those captured by a fisheye camera. We demonstrate this capability in Fig. 1, where we show a ViewPoint Map being converted into a fisheye image. The key challenge when adapting ViewPoint to other omnidirectional view types lies in how to design the processing and padding for the "void" regions. For example, if a fisheye camera has an FoV of only 220°, the question is how to handle the remaining 140° region—whether to pad it with zeros or ones, or to use a form of circular padding. This remains an interesting problem worthy of future exploration.

Finally, we again sincerely thank you for your valuable feedback. If you have any further concerns, please do not hesitate to let us know!

Dear Reviewer C4Si,

As the discussion phase is drawing to a close, we would appreciate it if you could share any remaining concerns with us. We are more than happy to address them. Thank you once again for your time and valuable comments.

Best regards,

ViewPoint authors

Dear Reviewer dty6, we would like to express our sincere gratitude for your insightful and thoughtful feedback. We are also grateful for your positive comments on the quality of our results. We have carefully considered your questions, and our responses are as follows:

Q1: The Effect of Prompt Engineering.

Prompt engineering experiments on the base model (Wan2.1-1.3B) fail to produce ERP-like videos, confirming a significant modality gap with the ERP representation.

In these experiments, we use phrases like "a 360 video of ..." to denote panoramic formats. Due to rebuttal guidelines, we cannot include external links or update the anonymous link in the paper. While we regret being unable to showcase the video results, we will describe the observed phenomena in detail below.

Since Wan2.1 only supports a fixed aspect ratio (e.g., 832x480), which does not conform to the 2:1 aspect ratio of the ERP format, we divide our experiments into two settings:

1. Using the model's native, fixed aspect ratio.

Observation: The generated results do not exhibit ERP-like characteristics. Instead, they resemble a bird's-eye perspective. Furthermore, both the visual quality and the dynamism are significantly degraded, with the video being nearly static.

2. Adjusting the aspect ratio to 2:1 (e.g., 960x480) to accommodate the ERP format.

Observation: After adjusting the aspect ratio to 2:1, the visual quality improves slightly, but the dynamism remains unchanged, and the output still does not resemble ERP format. Instead, it resembles a wide FoV perspective view, like a landscape wallpaper. (Again, we regret being unable to directly showcase the results.)

Regarding the Veo3 360 videos on YouTube, we have reviewed the content and agree that the results are impressive. However, creators note that they require extensive post-processing using professional software like Premiere.

Our analysis suggests that Wan2.1's failure to generate ERP-like videos stems from a modality gap introduced during pre-training, due to data preprocessing steps like cropping or resizing, which damage the spatial structure ERP data depends on.

To further investigate the impact of the prompt on the generation results, we design ERP-specific prompts in our ablation study. The results show no significant performance improvement, supporting our claim of a modality gap between Wan2.1 and the ERP format.

Q2: Camera Stabilization and Experiments on the VidPanos Dataset.

During training, we simulate camera movement by randomly perturbing the Euler angles, which enables our method to generalize flexibly to various input scenarios. This endows our method with a built-in camera stabilization capability, making it applicable to real-world scenarios. Intuitively, this process is equivalent to rotating each frame of the spherical video and then sampling the perspective view from a fixed anchor. We provide the details of the process below:

Since all training data is in ERP format—the most common for panoramic data—a specific preprocessing step is required. We extract a perspective video from the ERP as the condition and convert the full ERP into our ViewPoint Map representation (as shown in Fig. 2). Specifically, the conditional video is obtained through the following steps:

1. We randomly generate a sequence of continuously varying Euler angles, where the length of the sequence is identical to that of the ERP video. The angles are constrained to the following ranges: roll is within (-10°, 10°), pitch is within (-30°, 30°), and yaw is within (0°, 360°). For instance, for a 5-frame video, we would obtain three distinct sequences for the Euler angles, such as: roll: [0°, 1°, 0°, -1°, -2°]; pitch: [4°, 2°, 1°, -1°, 2°]; yaw: [20°, 23°, 22°, 20°, 18°]
2. Each set of Euler angles is then applied to its corresponding frame in the ERP video to perform a rotation. That is, the first frame is rotated using the angles {roll: 0°, pitch: 4°, yaw: 20°}, the second frame is rotated using {roll: 1°, pitch: 2°, yaw: 23°}, and so on.
3. From this rotated ERP video, we project the conditional perspective video using a fixed anchor (e.g., F). Finally, the entire rotated ERP is converted into a ViewPoint Map for training.

This simple yet effective data augmentation strategy enables our model to perform camera stabilization without needing to explicitly estimate camera parameters from the input perspective video.

To validate our design, following your suggestion, we conduct qualitative and quantitative evaluations on VidPanos-real. The results confirm the superiority of our method. Although we cannot provide new visualizations in this rebuttal, relevant real-world examples are always available via the anonymous link in our paper. For instance, in the "red car" case, while the INPUT VIDEO shows part of the car moving out of frame, the PANO VIDEO retains the full body, supporting our qualitative claims. Quantitative results are shown in the table below.

The results demonstrate the effectiveness of our method. Unlike Imagine360, which uses external models [1,2] to estimate camera parameters and generate panoramic video, our approach directly embeds camera movement into the model, avoiding potential error accumulation.

[1]A. Veicht, et al. GeoCalib: Single-image calibration with geometric optimization. ECCV,2024.

[2]L. Jin, et al. Perspective fields for single image camera calibration. CVPR,2023.

Q3: Explanation of the Pano-Perspective attention mechanism.

Although using the ERP representation combined with a well-designed attention mechanism may also help alleviate the seam issue, our approach goes beyond by not only addressing the seam problem but also mitigating the polar distortion and more effectively leveraging the generative capabilities of pre-trained video models, thus enabling the generation of high-quality panoramic videos.

The method you mentioned can be referenced to Imagine360, which projects ERP into multiple perspectives and employs a cross-domain spherical attention mechanism in an attempt to bridge the modality gap between panoramic data and the pre-training data of video models. While this approach is able to address the seam issue, the polar distortion remains severe (as shown in Figure 4). Moreover, it fails to fully exploit the generative power of existing video models, resulting in lower-quality panoramic video outputs (e.g., limited motion, see Table 1 in the main paper).

In contrast, our representation, together with the proposed Pano-Pers attention mechanism, enables global structural modeling of panoramic data and further exploits the generative capacity of video models through the construction of pseudo-perspective sub-regions. This capability is not achievable by other representations or attention mechanisms.

Q4: Wording and Phrasing.

Thank you for pointing out our inappropriate wording. We have revised line 192-193, changing the sentence

"The generated frames exhibit terrible image quality and possess little motion."

to

"The generated frames show limited visual quality and exhibit minimal motion."

and updated the caption of Figure 5 from

"360DVD is a text-driven approach and exhibits terrible image quality"

to

"360DVD is a text-driven approach and produces results with limited visual quality"

Q5: How is a single image fed into the model?

Consistent with the prevailing training paradigm of video generation models (e.g., Wan2.1), we feed images into the model by treating them as single-frame videos, thanks to the image-video unified encoding VAE. Specifically, a video tensor with a shape of [batch, channels(RGB:3), frames, height, width] is first passed into the Temporal AutoEncoder of Wan 2.1. Given that the VAE has a temporal-spatial downsampling factor of (4, 8, 8), the input data must satisfy two conditions: the number of frames must be frames = 4t + 1 (where t is a non-negative integer), and both height and width must be multiples of 8. After being encoded by the VAE, the video is transformed into a latent code with the shape [batch, channels(latent:16), t+1, height//8, width//8], which is then fed into the DiT for modeling. For image data, we treat it as a video with the shape [batch, channels(RGB:3), 1, height, width]. After encoding, its shape becomes [batch, channels(latent:16), 1, height//8, width//8].

Q6: Details on Overlapping Fusion Operation.

Different from MultiDiffusion, which enforces overlapping consistency in the noise space, our proposed overlapping fusion operation is performed in the pixel space, thereby improving spatial continuity. Specifically, we first rotate the four sub-regions (i.e., L, F, R, B) derived from the ViewPoint Map and align them horizontally, as illustrated in Fig. 3. Subsequently, a weighted linear interpolation is performed in pixel space. As shown in Fig. 3, each gradient black diamond shape represents an interpolation weight matrix, where a darker color indicates a smaller weight, and a more transparent color (i.e., closer to the original pixel color of the sub-region) signifies a larger weight. Each subregion is interpolated with its two adjacent faces based on this weight matrix.

Dear Reviewer dty6,

As the discussion period is about to conclude, we would like to kindly ask if you have any remaining concerns. Please do not hesitate to let us know, and we will be more than delighted to address them. Thank you very much for your time and effort.

Best regards,

ViewPoint authors

We thank the reviewer's comments and questions. However, we are very confused by your factually inaccurate statements regarding our paper's quantitative results. You mentioned that

"Table 2 shows Pano-FID, Pano-Smoothness, and user preference scores, but ViewPoint tops 360DVD and Imagine360 by only 3 to 6 percent on each metric (e.g., Pano-FID 16.7 vs. 17.9; Smoothness 0.423 vs. 0.437)..."

In fact, in our Table 2, we use four metrics: subject consistency, imaging quality, motion smoothness, and dynamic degree. The metrics you referred to—Pano-FID, Pano-Smoothness, and user preference scores—are NOT present anywhere in our paper. Furthermore, the numerical values you cited (i.e., "Pano-FID 16.7 vs. 17.9; Smoothness 0.423 vs. 0.437") also do NOT exist in our manuscript.

Regarding the computational latency, you stated

"Considering that ViewPoint requires roughly 10 seconds per frame......".

However, this is inaccurate. As shown in Table 1 in our supplementary material, our method requires only 1 minute and 47 seconds to generate a 49-frame video, which translates to 2.18 seconds per frame. In fact, our method is the fastest among all compared approaches.

Also, we are confused by your comments regarding the lack of human evaluation, specifically when you stated

"There is no human evaluation or task-based assessment of flicker or potential VR experience quality..."

"Have you considered human studies or task-based assessments for motion artifacts and flickering..."

In fact, as detailed in Section 4.5, User Study, we conducted a comprehensive human evaluation across four distinct dimensions: Spatial Continuity, Temporal Quality, Aesthetic Preference, and Condition Alignment. We explicitly defined one of these dimensions, Temporal Quality, in the paper as follows (lines 250-251)

"Temporal Quality is used to evaluate motion effects; for example, being nearly motionless and flickering are considered poor performance."

The results of this user study, presented in the main paper, confirmed the effectiveness of our method in all evaluated aspects, including temporal quality.

For the remaining questions, we provide our responses below.

Q1: Replacing the Baseline.

We re-implement our method on top of Stable Diffusion v1.5 + AnimateDiff v2 and Stable Diffusion v2.1 + AnimateDiff v2. The evaluation results are presented in the table below:

Even with weaker base models, our method outperforms prior works across four metrics. We would love to demonstrate the re-implemented results. Regrettably, the rebuttal guidelines prevent us from sharing an external link for the videos.

Q2: Variance of the Random Seed.

In the quantitative evaluation (Table 1 and Table 2), we use random seeds ranging from 42 to 52 for all compared methods, and take the average as the final result. Here, we present the variance in qualitative evaluation.

Dear Reviewer WqAv, we thank you for recognizing the strengths of our work, particularly our novel approach for converting single-view videos into panoramic ones and the method's visually impressive results. We appreciate your valuable comments and questions, and we provide our detailed responses below.

Q1: Analysis of Results with Suboptimal Quality at the Poles.

As shown in Fig. 4 in the main paper, compared to previous methods, our proposed approach significantly mitigates distortion at the poles, both subjectively and objectively. Prior works often exhibit severe artifacts like "black holes" or "swirls" (Fig.4), which drastically undermine the sense of immersion and realism. By contrast, our method may exhibit spatial fluctuations in rare cases, a behavior analogous to that in perspective video generation. (It is worth noting that even in standard perspective scenarios, pre-trained video generation models can suffer from spatial distortion, as this is an issue inherited from the base models themselves).

To further demonstrate the effectiveness of our method, we re-evaluate the quality of the top and bottom poles of each panoramic video and conduct a user study. The results blow show that our method outperforms other methods. (user preference: we ask 30 participants to rate the video quality at the poles for each method on a score of 1 to 100. The average score is reported as the final result.) The results demonstrate that our method significantly reduces distortion at the poles compared to previous state-of-the-art methods.

Q2: Distinctions and Advantages over Multi-view Generation Methods.

Our proposed ViewPoint Map is a unified representation that projects a non-Euclidean panorama into a single, Euclidean 2D square, thereby providing explicit global continuity. This is fundamentally different from multi-view representations, which use multiple isolated perspectives to represent a panorama. In such approaches, the spatial-temporal correlations of the panorama are entirely learned implicitly through subsequent attention mechanisms, making it more challenging to generate high-quality panoramic videos.

Most of the multi-view generation methods (e.g., the methods you mentioned [1, 2]) first project the panorama into multiple overlapping perspective views. They then leverage multi-view attention to learn the relationships between these views, and finally stitch them back together into a panorama. However, these approaches struggle to grasp global continuity and plausibility. For instance, color discrepancies between the individual perspectives can lead to visible seams, or the stitched result might produce incoherent scenarios, such as a room appearing with two doors.

Moreover, multi-view generation methods face a trade-off between the amount of overlaps and computational resources. To ensure better consistency, larger overlapping regions are often required, which in turn necessitates generating more perspective images. For example, the method you cited in [2] projects an ERP panorama into eight perspective views. While the six horizontal views partially overlap with each other, the top and bottom views are completely isolated, having no overlap with their horizontal neighbors. To create overlap for the top and bottom views as well, even more perspective images would be needed, implying a significant increase in computational cost. This is particularly costly because panoramic video generation is an inherently compute-intensive task.

By contrast, our representation is inherently global and unified. This not only enables the model to enforce true global consistency—going beyond just the relationships between separate views—but also offers superior computational efficiency. As detailed in Table 1 in the supplementary material, our method outperforms all previous methods in latency and VRAM usage.

[1] Tang, S., Zhang, F., Chen, J., Wang, P., & Furukawa, Y. (2023). MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusion. ArXiv, abs/2307.01097.

[2] Xie, K., Sabour, A., Huang, J., Paschalidou, D., Klar, G., Iqbal, U., ... & Zeng, X. (2025). VideoPanda: Video panoramic diffusion with multi-view attention. arXiv preprint arXiv:2504.11389.

Q3: Re-implementing ViewPoint on AnimateDiff.

Following your suggestion, we re-implement our method based on Stable Diffusion v1.5 + AnimateDiff v2 and Stable Diffusion v2.1 + AnimateDiff v2, and the results confirm its superiority over other approaches across all metrics. The evaluation results are presented in the table below.

Although the superior generation capabilities of Wan2.1 significantly benefit many downstream tasks, our experiments demonstrate that our method remains effective even with weaker base models, outperforming prior works across four metrics. We have added the above table results to the main comparison in the latest manuscript, and included the visualization results of ViewPoint-AnimateDiff in the supplementary material. Unfortunately, due to the rebuttal policy, we are unable to display the generated videos in any links.

Q4: Intuitive Explanation of the Advantages of Per-view Attention Layers.

The modality gap between panoramic data and the pre-training perspective data of video generation models makes it difficult for the network to directly model the panorama, resulting in poor performance (as shown in Table 2 in the main paper). Therefore, we propose per-view attention to divide the panoramic representation into multiple pseudo-perspective views to align with the pre-training priors and thus fully exploit the generative capabilities of video generation models.

Specifically, our method cyclically divides the ViewPoint into four pseudo-perspective sub-regions, a data format that better aligns with the pre-trained priors of Wan2.1. Subsequently, these four sub-regions are merged back into the ViewPoint Map, and Wan2.1's 3D attention mechanism is utilized to model global spatio-temporal relationships. This divide-and-merge design enables us to fully leverage the model's generative power (e.g., enhancing dynamism and improving visual quality) while simultaneously strengthening the global continuity of the ViewPoint Map. It is important to note that we are not introducing new layers. Instead, we retain the original attention layers of Wan2.1 and partition them into two functional parts: one part focuses on per-view attention, while the other is dedicated to capturing global continuity.

Dear Reviewer WqAv,

The Reviewer-Author discussion session is coming to a close. We sincerely appreciate the opportunity to engage in this discussion and welcome any further feedback or questions you may have. If there are additional areas you believe we should consider to enhance this work, please don’t hesitate to let us know.

Best regards,

ViewPoint authors

Dear Reviewer WqAv,

As the discussion period is drawing to a close, we would like to gently inquire if you have any remaining concerns we can address. We would like to express our sincerest gratitude for the time and effort you have dedicated to our manuscript. Your valuable feedback and insightful comments are incredibly important to us. Thank you once again for your invaluable contribution and guidance.

Best regards,

ViewPoint authors