PanoWan: Lifting Diffusion Video Generation Models to 360$^\circ$ with Latitude/Longitude-aware Mechanisms

We thank the reviewers for their insightful and constructive feedback.

In accordance with the rebuttal policy, we are unable to include new qualitative results (e.g., generated videos) or external links. We assure the reviewers that, should the paper be accepted, the camera-ready version will be thoroughly updated to include all additional experiments and qualitative results discussed in this rebuttal.

We now address the specific points raised by the reviewer.

W1: Lack of novelty

We thank the reviewer for their insightful feedback. We would like clarify our novel contributions, addressing the concerns about Rotated Semantic Denoising (RSD) and Pixel-wise Padded Decoding (PPD), respectively.

• PanoFusion proposes Two-End Alignment (TEA) to apply a fixed 90° rotation to the latent code with the stated goal to "explicitly enforce the two ends of an image to be wraparound-consistent" for static image outpainting. In contrast, our RSD implements a step-dependent circular shift, ensuring the cumulative prediction error is evenly distributed across all longitudes during text-based panoramic video generation. To further address the reviewer's concern, we conduct an additional experiment using a fixed 90° angle within our framework. This results in four slightly visible seams.

• StitchDiffusion augments the denoising process by performing additional pre-denoising operations on a "stitch block" created from the image's leftmost and rightmost regions at each timestep. This method operates entirely within the latent space to influence the U-Net's predictions. In contrast, our proposed PPD targets the VAE decoding phase to prevent boundary artifacts that are re-introduced by standard VAEs (trained with zero-padding) when decoding the latent code into pixel space. Specifically, PPD pads the latent code with its own wrapped content, ensuring the decoder's convolutional kernels never process invalid boundary padding. Notably, the overhead of PPD is negligible, whereas StitchDiffusion's approach significantly increases computational cost.

Beyond these points, we also propose Latitude-aware Sampling (LAS) as a zero-parameter solution to address latitude distortion at the poles. LAS introduces distortion-aware noise initialization directly into the diffusion process, ensuring spherical awareness while fully preserving the powerful generative priors of the pre-trained backbone (Sec. 4.2).

We thank the reviewer for pointing out these relevant works and will follow the suggestion to discuss them in our final version.

W2: Unfair comparison

We would like to clarify that our primary objective with PanoWan is to demonstrate that generative priors of pre-trained conventional text-to-video generation models can be effectively lifted to the panorama with minimal equipped modules (Line 30-32). As the results in Fig. 1 and Sec. 5.5 show, PanoWan effectively preserves the generalization ability of the backbone, enabling high-quality panoramic video generation, effectively zero-shot achieving downstream applications (e.g., long-video generation, super-resolution, semantic editing, and video outpainting).

However, our framework is indeed not limited to Wan 2.1. The core modules (LAS, RSD, and PPD) are independent of the base model and they are designed to address fundamental challenges in panoramic generation, rather than being specific to any single architecture:

• LAS adapts noise initialization to spherical geometry.

• RSD and PPD restructure the denoising and decoding process to improve spatial consistency and perceptual fidelity.

To empirically validate this generalizability, we conduct an additional experiment as suggested. We integrate our modules into the T2V backbone used by 360DVD and retrain them under the same settings. The results in the table below demonstrate that our modules provide a clear improvement in edge continuity when applied to the 360DVD backbone. Notably, we focus here on objective metrics, while "motion pattern" and "scene richness" are subjective and require a separate user study.

W3: Missing citation (DiffPano)

We thank the reviewer for pointing out this relevant work.

However, we believe there is a fundamental distinction between the task of DiffPano and our PanoWan. DiffPano is designed for multi-view panoramic image generation, whereas our PanoWan focuses on temporally coherent panoramic video generation. DiffPano generates each view independently and lacks an explicit temporal modeling mechanism, which is essential for creating smooth video sequences. This limitation is acknowledged in the DiffPano paper itself, which states: "...as the number of frames increases during inference, the model tends to hallucinate content." Therefore, a direct comparison would not be meaningful.

Furthermore, adapting DiffPano into a video generation model would require significant architectural modifications (e.g., adding temporal modules) and retraining. This is beyond the scope of a fair baseline comparison for this work.

We appreciate the suggestion and will ensure our final version includes a thorough discussion with it.

Q1: Base model

Yes, our proposed method is generally applicable. To demonstrate this, we have integrated our modules into the 360DVD backbone and retrained it under the same settings. The detailed results of this experiment are presented in our response to W2.

Q2: Improving DynamicScaler

We thank the reviewer for this insightful question. While DynamicScaler is theoretically applicable to Wan2.1 backbone, we argue that such an adaptation may not effectively address two fundamental limitations of its approach:

1. Quality limitations. The core strategy of DynamicScaler is to generate the panorama in independent patches. This approach inherently lacks a global context, leading to repetitive patterns across different patches and a limited motion range (as movements rarely cross patch boundaries).

2. Computational cost. The patch-based generation is less efficient. The official DynamicScaler takes over 360 seconds to generate a 16-frame video (448×896) on an H100 GPU. Given that Wan 2.1 backbone is more computationally intensive, a naive adaptation to Wan2.1 would be very slow. In contrast, our PanoWan generates an 81-frame video in just 90 seconds.

We will add this discussion to our final paper.

We thank the reviewers for their insightful and constructive feedback.

In accordance with the rebuttal policy, we are unable to include new qualitative results (e.g., generated videos) or external links. We assure the reviewers that, should the paper be accepted, the camera-ready version will be thoroughly updated to include all additional experiments and qualitative results discussed in this rebuttal.

We now address the specific points raised by the reviewer.

W1: Technical novelty

We thank the reviewer for this critical feedback on our technical novelty. We would like to clarify our key contributions from both the latitude and longitude perspectives:

1. Latitude-related novelty

• PanFusion introduces EPP spherical positional encoding, which modifies core attention layers. While effective in feature representation, this alters the architecture of the pre-trained conventional model, which can compromise its valuable generative priors and limit its ability to generate diverse content.

• 360DVD uses a latitude-aware loss to emphasize low- and mid-latitude regions. However, this approach essentially de-emphasizes the highly distorted polar regions rather than fundamentally solving the representation issue, which results in persistent distortion artifacts at the poles.

• In contrast, our Latitude-aware Sampling (LAS) is a zero-parameter solution that requires no architectural changes. Instead, LAS introduces distortion-aware noise initialization directly into the diffusion process. This ensures spherical awareness while fully preserving the powerful generative priors of the pre-trained backbone.

2. Longitude-related novelty

• Although previous work (e.g., 360DVD) proposes the latent rotation mechanism to address boundary discontinuity during denoising, we empirically observe that persistent visible seams remain. Our key insight is that this issue stems from two distinct stages: (1) the denoising process and (2) the VAE decoding process. While latent rotation helps with the first stage, it overlooks the second. As a result, standard VAEs trained on conventional videos with zero-padding re-introduce artifacts when decoding the latent into pixel space.

• To resolve this, we first adapt the latent rotation mechanism and introduce it into the rectified flow of the Wan 2.1 backbone as Rotated Semantic Denoising (RSD). As a complement, we propose Padded Pixel-wise Decoding (PPD) to pad the latent with its own wrapped content before feeding it to the VAE, and then crop the final pixel output. This ensures the decoder never processes invalid boundary padding.

• Our proposed PPD is fundamentally different from previous "circular padding mechanism" (from 360DVD), which modifies convolutional layers within the U-Net denoiser, thus only addressing the denoising stage. In contrast, our PPD specifically targets the VAE decoding stage. Notably, the overhead of PPD is minimal, as the padding width only needs to exceed the VAE's receptive field.

Summary of novelty:

• Our framework introduces LAS, RSD, and PPD to adapt conventional text-to-video models for panoramic video generation. Our approach is guided by the core principles of minimal architectural change and full preservation of the base model's generative priors.

• We apologize for the confusion. We sincerely thank the reviewer for pushing us to clarify our work's novelty, and we will incorporate this detailed comparison into the final version.

W2: Downstream applications

We thank the reviewer for this observation, which in fact highlights a central goal of our work.

We would like to clarify that our primary objective with PanoWan is to demonstrate that generative priors of pre-trained conventional text-to-video generation models can be effectively lifted to the panorama with minimal equipped modules (Line 30-32). The downstream applications presented in our paper (e.g., long-video generation, super-resolution, semantic editing, and video outpainting) are therefore intended as a direct validation of this principle.

As the results in Fig. 1 and Sec. 5.5 show, PanoWan effectively preserves the generalization ability of the backbone while enabling high-quality panoramic video generation. Notably, all of these experiments are performed in a zero-shot manner.

Q1: Long video generation

To generate a video longer than the model's context length (e.g., 324 frames using a model trained for 81-frame segments), we adopt a sliding window approach during the inference process in latent space. The methodology is as follows:

1. A full-length latent code (324 frames) is initialized with standard Gaussian noise. Concurrently, the user-provided text description is expanded by an LLM into a sequence of prompts ([prompt_1, prompt_2, ...]), each corresponding to a specific temporal segment.

2. For each step during inference, we iterate through the full latent code using overlapping windows (e.g., an overlap of 40 frames). For each window, the corresponding latent segment is processed by the DiT backbone, conditioned on its corresponding prompt. To ensure temporal consistency, context frames from the previously processed adjacent window are used to guide the generation of the current segment. The output of each window is then fused back into the full-length latent code by applying a weighted average in the overlapping chunks, ensuring a seamless transition.

3. After the inference phase, the resulting clean latent code is decoded by the VAE to produce the final long video.

Q2: Annotation reliability

We agree that raw VLM outputs can be unstable in open-ended annotation tasks. In practice, we encounter occasional misclassification of non-panoramic videos and generated Points-of-Interest (POIs) that do not adhere to our predefined label set (e.g., case variations or synonyms).

To address this, we design a dedicated multi-step prompt strategy (detailed in the supplementary materials, Sec. 9) to first verify panoramic properties (Line 83-84) and then extract POI information (Line 85-93). Furthermore, we employ the same VLM in a post-processing step to reformat the extracted information (Line 97-107), ensuring strict consistency with our target label set (Line 114-129).

To further validate the sample quality of our dataset, we conduct an additional human evaluation. We randomly sample 1000 samples from the dataset and ask 10 volunteers to classify the alignment of each one into three categories: "Highly aligned", "Partially aligned", or "Misaligned".

As the table below shows, over 94% of text description annotations and 90% of POI annotations are rated as "Highly aligned", respectively. This result demonstrates that our generated annotations are reliable.

We thank the reviewers for their insightful and constructive feedback.

In accordance with the rebuttal policy, we are unable to include new qualitative results (e.g., generated videos) or external links. We assure the reviewers that, should the paper be accepted, the camera-ready version will be thoroughly updated to include all additional experiments and qualitative results discussed in this rebuttal.

We now address the specific points raised by the reviewer.

W1: Technical novelty

We thank the reviewer for this critical feedback on our technical novelty. We would like to clarify our key contributions from both the latitude and longitude perspectives:

1. Latitude-related novelty

• 360DVD uses a latitude-aware loss to emphasize low- and mid-latitude regions. However, this approach essentially de-emphasizes the highly distorted polar regions rather than fundamentally solving the representation issue, which results in persistent distortion artifacts at the poles.

• PanFusion introduces EPP spherical positional encoding, which modifies core attention layers. While effective in feature representation, this alters the architecture of the pre-trained conventional model, which can compromise its valuable generative priors and limit its ability to generate diverse content.

• In contrast, our Latitude-aware Sampling (LAS) is a zero-parameter solution that requires no architectural changes. Instead, LAS introduces distortion-aware noise initialization directly into the diffusion process. This ensures spherical awareness while fully preserving the powerful generative priors of the pre-trained backbone.

2. Longitude-related novelty

• Although 360DVD proposes the latent rotation mechanism to address boundary discontinuity during denoising, we empirically observe that persistent visible seams remain. Our key insight is that this issue stems from two distinct stages: (1) the denoising process and (2) the VAE decoding process. While latent rotation helps with the first stage, it overlooks the second. As a result, standard VAEs trained on conventional videos with zero-padding re-introduce artifacts when decoding the latent into pixel space.

• To resolve this, we first adapt the latent rotation mechanism and introduce it into the rectified flow of the Wan 2.1 backbone as Rotated Semantic Denoising (RSD). As a complement, we propose Padded Pixel-wise Decoding (PPD) to pad the latent with its own wrapped content before feeding it to the VAE, and then crop the final pixel output. This ensures the decoder never processes invalid boundary padding.

• Our proposed PPD is fundamentally different from 360DVD's "circular padding mechanism", which modifies convolutional layers within the U-Net denoiser, thus only addressing the denoising stage. In contrast, our PPD specifically targets the VAE decoding stage. Notably, the overhead of PPD is minimal, as the padding width only needs to exceed the VAE's receptive field.

Summary of novelty:

• Our framework introduces LAS, RSD, and PPD to adapt conventional text-to-video models for panoramic video generation. Our approach is guided by the core principles of minimal architectural change and full preservation of the base model's generative priors.

• We apologize for the confusion. We sincerely thank the reviewer for pushing us to clarify our work's novelty, and we will incorporate this detailed comparison into the final version.

W2: Re-training 360DVD on PanoVid

We follow the reviewer's suggestion to retrain 360DVD on our full PanoVid dataset. As shown in the results below, 360DVD's performance improves, which validates the quality of our contributed dataset. Despite this improvement, our PanoWan still consistently outperforms the retrained 360DVD. Since DynamicScaler is a training-free method, this experiment is not applicable to it. Notably, we focus here on objective metrics, while "motion pattern" and "scene richness" are subjective and require a separate user study.

We will include these results in the final version.

W3: Evaluation details

We follow the reviewer's suggestion to clarify our evaluation details.

All quantitative results are computed on the PanoVid test set, which consists of 67 real-world video clips collected from various online platforms and public datasets. These clips have no overlap with the training data to ensure a fair evaluation. For each clip, we use its corresponding text description as input to generate video samples, and then compute all metrics on these generated videos.

We will incorporate these details in the final version to ensure the reproducibility and transparency.

Q1: Evaluation setup

See W3 above.

Thank you for your detailed follow-up and for providing further insightful feedback on our work. We appreciate the opportunity to provide further clarification.

We address each of your remaining points below and welcome any further discussion.

[W1]-1. Longitude continuity

Regarding the concern about conventional operations, our core novelty lies in a two-stage approach for longitude continuity. This approach explicitly decouples the panoramic discontinuity problem into two sub-problems: (1) latent-level misalignment (addressed by RSD during the denoising process), and (2) decoder-level inconsistency (addressed by PPD during the VAE decoding stage).

[W1]-2. RSD

As discussed with Reviewer sK8Y, while RSD may appear similar to TEA in its use of latent rotation, their motivation and design are fundamentally different. TEA applies a fixed 90° rotation to enforce consistency, which is sufficient for static panoramic images. However, in video generation, even subtle discontinuities are amplified across temporal frames, making a static rotation inadequate. Therefore, we proposed RSD to perform a step-dependent and randomized shift during the denoising process. This dynamic approach is essential for preventing the accumulation of seam artifacts and preserving temporal continuity in panoramic videos.

While we have already conducted a qualitative comparison where the visual advantages of RSD are obvious, the rebuttal policy prevents us from including new qualitative results at this stage. We will add these visualizations to the final version to clearly demonstrate the advantage of our RSD method.

[W1]-3. PDD

While prior works (e.g., PanFusion) may incorporate rotation or circular operations, they are typically applied within the U-Net or attention modules. However, these approaches overlook a critical vulnerability: a standard VAE decoder (trained on conventional images with zero-padding) will re-introduce seam artifacts when decoding the panoramic latent code, even if that latent code is perfectly seamless.

In contrast, our proposed PPD is specifically designed to solve this problem at the VAE decoding stage. By padding the latent feature with its own wrapped content and subsequently cropping the final pixel output, PPD effectively avoid seam artifacts without modifying the model architecture or its convolutional layers.

Therefore, we believe our proposed approach is not a conventional reuse of prior techniques, but rather a systematic and targeted design, tailored to the unique challenges of panoramic video generation.

[W3] Test set

We thank the reviewer for the constructive suggestion regarding the test set size. Theoretically, a larger test set provides a more accurate estimation of the real-world video's statistical distribution, which can lead to more stable evaluation metrics.

However, as detailed in our supplementary material (Sec. 9), our original 67-video test set is meticulously curated to ensure a balanced and diverse POI (Point-of-Interest) distribution. We believe its diversity is already sufficient to provide a reliable statistical representation.

To empirically validate this, we follow the reviewer's suggestion and construct an expanded 300-video set. These video clips are sourced from our initial pool of high-quality candidates and are strictly excluded from the training set. Through a rigorous second round of filtering, we ensure these samples maintain high aesthetic scores and broad POI coverage, with quality comparable to our original test set.

As the table shows below, the evaluation results on this larger 300-video set are highly consistent with our original findings. The FVD score and other metrics remain stable with only minor fluctuations. This demonstrates that our original 67-video test set is sufficient for a reliable evaluation, effectively capturing the target data distribution.

Additionally, the scale of our test set is consistent with prior work in this area. For example, DynamicScaler "assesses quality by randomly selecting views from 100 generated videos". 360DVD does not specify a test set size, but notes that "31 participants are surveyed to evaluate... 8 sets of generated results". PanoDiT also does not specify its test size, and its corresponding PHQ360 dataset contains a total of only 649 clips. Therefore, our evaluation setup aligns with standard practice in this domain.

We will elaborate on this in the final version.

We thank the reviewers for their insightful and constructive feedback.

In accordance with the rebuttal policy, we are unable to include new qualitative results (e.g., generated videos) or external links. We assure the reviewers that, should the paper be accepted, the camera-ready version will be thoroughly updated to include all additional experiments and qualitative results discussed in this rebuttal.

We now address the specific points raised by the reviewer.

W1: Comparison with PanoDiT and VideoPanda

We thank the reviewer for the insightful suggestion regarding more recent baselines. However, at the time of our submission, both PanoDiT and VideoPanda are not open-sourced, making a direct and reproducible comparison infeasible. Therefore, we chose the publicly available 360DVD and DynamicScaler, which represent two distinct and representative technical paradigms: the classic ERP-based generation (360DVD) and training-free diffusion adaptation (DynamicScaler). We are confident this comparison framework is fair and effectively demonstrates the advantages of our method.

W2: Human evaluation

Following 360DVD, we have provided initial subjective evaluations in our main paper regarding motion pattern and scene richness (Lines 223-224). To further address the reviewer's concerns, we conduct an additional user study as requested.

In this study, we present videos generated by our method and all baselines to participants and ask them to choose the one they prefer. We randomly select 50 text descriptions from the test set, and the evaluations are conducted with 25 volunteers. The table below shows the percentage of times each method is chosen as the winner. As the results show, our method achieves the highest score.

W3: Annotation quality

As we illustrate in Sec. 3 (Lines 96-98) and supplementary materials (Sec. 9), we use Qwen-VL to generate text descriptions and Point-of-Interest (POI) categories for each video, and also apply a series of data filtering and post-processing steps.

To further validate the sample quality of our dataset, we conduct an additional human evaluation. We randomly sample 1000 samples from the dataset and ask 10 volunteers to classify the alignment of each one into three categories: "Highly aligned", "Partially aligned", or "Misaligned".

As the table below shows, over 94% of text description annotations and 90% of POI annotations are rated as "Highly Aligned", respectively. This result demonstrates that our data pipeline is robust and the generated annotations are reliable for both training and evaluation.

Q1: Base model

Our framework is indeed not limited to Wan 2.1. The core modules (LAS, RSD, and PPD) are independent of the base model and they are designed to address fundamental challenges in panoramic generation, rather than being specific to any single architecture:

• LAS adapts noise initialization to spherical geometry.

• RSD and PPD restructure the denoising and decoding process to improve spatial consistency and perceptual fidelity.

To empirically validate this generalizability, we conduct an additional experiment as suggested. We integrate our modules into the T2V backbone used by 360DVD and retrain them under the same settings. The results in the table below demonstrate that our modules provide a clear improvement in edge continuity when applied to the 360DVD backbone. Notably, we focus here on objective metrics, while "motion pattern" and "scene richness" are subjective and require a separate user study.

Q2: Temporal consistency of RSD

We clarify that RSD does not harm temporal consistency. The core principle is that a horizontal roll in the ERP format is a spatially invariant operation, which does not change the visual content. Prior works like 360DVD validate this by using random roll as a standard data augmentation technique.

Our RSD module leverages this property at each denoising step to solve the problem of persistent boundary artifacts. To demonstrate this, we conduct an additional study using fixed rotation angles during sampling. The results show that fixed angles of 0°, 180°, and 120° introduce 1, 2, and 3 visible seams, respectively.

In contrast, our RSD disperses these potential artifacts spatially. This prevents any single boundary from becoming a visible seam, thereby improving overall spatial and temporal coherence. We will include visualizations of this study in the final version for clarity.

Q3: Annotation quality

See W3 please.