Cavia: Camera-controllable Multi-view Video Diffusion with View-Integrated Attention

We thank the reviewer T9u5 for their detailed comments and constructive suggestions. However, we respectfully disagree with the reviewer's concerns regarding our contribution. In response, we have added additional comparisons against concurrent works ViewCrafter and CVD, which are available on our anonymous webpage: https://cavia2025.github.io/. These new results clearly demonstrate that our proposed method, Cavia, significantly outperforms both ViewCrafter (Fig. A) and CVD (Fig. B). We address the reviewer's questions in detail below.

Q1. Multi-view experiments are mainly conducted on 2 views with small camera angle change. 4-view experiments’camera angle change is still not evident.

A1. We emphasize that camera control, the factor the reviewer prioritizes, is precisely where Cavia excels. Cavia is the first to enable multi-video generation with precise camera motion control while simultaneously preserving object motion. Concurrent works cited by the reviewer fail to achieve comparable quality. Specifically, ViewCrafter only generates static scenes, SynCamMaster is restricted to fixed-viewpoint videos, and CVD suffers from poor pixel quality with severe morphing artifacts.

Q2. 2-view video generation with limited camera change is not convincing. Does the training data also contains such small camera change? MotionCtrl and CamCtrl are not designed for multi-view video generation, so it is unfair. It is suggested to compare with SynCamMaster.

A2. As we illustrate above, SynCamMaster only supports video generation with fixed viewpoints, allowing no camera movement. In contrast, Cavia enables precise camera control while preserving object motion, a challenging and underexplored capability. While MotionCtrl and CameraCtrl are monocular methods, they are the closest relevant baselines as they also target precise camera control. Additionally, Cavia builds upon SVD, which supports video generation up to 14 frames, leading to naturally limited camera ranges. Extending this to larger ranges with a more powerful long context base video generation model is an exciting direction for future work.

We provide a detailed comparison with SynCamMaster in our response to reviewer vi5f’s Q2 (Tab. A). However, we note that SynCamMaster's official GitHub repository contains only boilerplate code, and the absence of essential components such as the model checkpoint and architecture makes it impossible to compare with SynCamMaster fairly.

Q3. The joint training strategy for static video is confusing. I don't know if the authors refers to V cameras move separately during F-1 frames, or they simply repeat the first frame to F-1 frames.

A3. We do not perform single-frame training. Our "static" videos refer to sequences where static scenes lack dynamic objects. During training, our Cavia framework consistently accepts $F$ frames, each with a distinct camera matrix.

Q4. For 3D consistency of monocular video generation, it is suggested to compare with ViewCrafter.

A4. We have included comparisons against ViewCrafter in Fig. A on https://cavia2025.github.io/#FigA . ViewCrafter primarily generates videos of static 3D scenes, likely due to its reliance on 3D point clouds. We evaluate ViewCrafter using test images and trajectories from the RealEstate10K dataset and observe that it often produces frames with noticeable color and lighting artifacts, as well as geometric distortions. In contrast, Cavia achieves visually pleasing results with accurate geometry. Furthermore, ViewCrafter requires approximately 4 minutes to generate a video sequence, whereas Cavia produces a set of 2-view videos in just 14 seconds.

Q5. It is suggested to move the comparison with CVD to the main text and conduct quantitative experiments under its evaluation pipeline.

A5. Thank you for the suggestion. We initially placed CVD’s results in the appendix due to space constraints but have now moved them to the main text. We have also provided additional comparisons using CVD’s recently released official codebase. The results can be found in Fig. B on https://cavia2025.github.io/#FigB . Our comparisons show that CVD suffers from severe morphing artifacts and unnatural object motion. It also fails to follow text prompt instructions and overlooks important details. In contrast, Cavia produces outputs with greater geometric consistency and more natural object motion.

For quantitative evaluation, we not only adopt SuperGlue, following CameraCtrl and CVD, but also incorporate COLMAP, a widely used tool in the 3D reconstruction community for estimating camera poses. COLMAP shares the same purpose as SuperGlue by measuring camera pose accuracy but is more effective in identifying morphing artifacts, as it focuses on global geometry rather than just local feature matching. Our extensive quantitative results in Tab. 1 and 2 demonstrate that Cavia significantly outperforms existing camera-control video generation methods.

We thank the reviewer vi5f for their valuable effort and for recognizing the strength of our experimental results. However, we respectfully disagree with the concerns regarding "large viewpoint changes like SynCamMaster." We would like to clarify that the concurrent work SynCamMaster is limited to generating fixed-viewpoint videos, whereas our proposed method, Cavia, enables precise camera control for each individual frame under a multi-view video generation setting.

Q1. Why not include absolute pose error (APE) and relative pose error (RPE) metrics?

A1. APE and RPE combine translation and rotation errors into a single value. However, following common practices in camera-controllable video generation (e.g., CameraCtrl, Collaborative Video Diffusion), we evaluate translation and rotation errors separately. Our chosen metrics align with the spirit of RPE, as they also measure relative differences between camera matrices but allow for a more detailed assessment.

Q2. Should discuss and clarify the difference between syncammaster [ICLR2025]. It is still questionable whether Cavia can generate multi-view videos with large viewpoint changes like syncammaster.

A2. We appreciate the reviewer's suggestion and will cite and discuss SynCamMaster accordingly. However, SynCamMaster is designed for generating videos with fixed camera viewpoints and does not support dynamic viewpoint changes. In contrast, Cavia provides precise per-frame camera control in multi-video generation scenarios.

The table below (Tab. A) provides a detailed comparison:

Q3. Limitation of Cavia to address multi-view video generation with large viewpoint changes.

A3. The current performance of Cavia is constrained by the limited context length of SVD (14 frames). However, Cavia's framework is general and scalable, enabling the generation of longer videos with larger viewpoint variations when paired with a stronger foundation model. It is worth emphasizing that SynCamMaster entirely lacks the ability to handle viewpoint changes. In contrast, Cavia is explicitly designed to address this limitation, directly responding to the reviewer's primary concern.

We thank reviewer KtKL for the detailed comments and for recognizing our strong performance compared to existing works. Below are our detailed responses to the questions raised.

Q1. Baseline details are missing: 1) How did SVD achieve camera pose? 2) How are MotionCtrl and CameraCtrl implemented for 2-view video generation?

A1. Thank you for this insightful question. We clarify that the SVD results in our experiments are obtained from the vanilla SVD model without any modification; thus, SVD does not offer camera pose controllability. We include SVD's results solely to showcase the base model's motion generation capability, with additional visualizations provided in Fig. 6.

For 2-view comparisons involving MotionCtrl and CameraCtrl, we independently run each view with the corresponding camera poses, as these methods are originally designed for monocular video generation.

Q2. Speed and memory report on the 3D attention is needed.

A2. Indeed, the proposed 3D attention introduces additional computational overhead due to the extended sequence length. However, thanks to the 8$\times$ compression of SVD's latent space, our attention operates on latent features sized $14\times 32\times 32$ for a $256 \times 256$ video. As shown in the table below, the increased cost in speed and memory remains acceptable.

Q3. Performance is limited for this highly challenging task. Some visual issues are: a. The motion magnitude in the results is still very small. b. Camera trajectories are simple. Not sure if the model can support 360-degree views.

A3. We wish to emphasize that the factors you prioritize most, such as object motion strength and camera trajectory complexity, are precisely the areas in which Cavia significantly outperforms prior works.

• Object motion strength: SVD's results serve as a reference for motion generation capabilities, with additional visualizations in Fig. 6. More importantly, existing camera-controllable video generation methods, such as MotionCtrl and CameraCtrl, are limited to generating videos of static scenes. In contrast, Cavia retains the ability to generate object motion comparable to the base SVD model.
• Camera trajectory complexity: Extensive experiments in Fig. 3 and Tab. 1 demonstrate Cavia's superior camera control compared to state-of-the-art methods. However, the current version does not support 360-degree view generation due to the limited context length of SVD (14 frames). We plan to explore longer video generation with more capable base models in future work.

Q4. Some concurrent works can be discussed: DimensionX (arXiv:2411.04928), CAT4D (arXiv:2411.18613), SynCamMaster (arXiv:2412.07760), GenXD (arXiv:2411.02319).

A4. We will include and discuss these concurrent works in our revision.

• DimensionX: Utilizes separate S-director and T-director modules to generate time-frozen and viewpoint-frozen videos independently. Cavia, instead, generates multi-view videos simultaneously, ensuring better spatial-temporal consistency.
• CAT4D: Converts monocular videos into 4D scenes. In contrast, Cavia generates multi-view videos from a single image input.
• SynCamMaster: Generates multi-view videos but only supports fixed cameras, resulting in viewpoint-frozen videos. Cavia enables precise camera control for each frame of multiple generated videos.
• GenXD: Employs a masked video diffusion framework for camera-controllable monocular video generation. Cavia, however, targets multi-view video generation, which is inherently more challenging.

While these works are commendable, none address camera-controllable multi-view video generation. Cavia is the first framework to generate multiple videos of the same scene with precise per-frame camera control, while maintaining object motion quality comparable to SVD.

Q5. Will the authors release the model and the curated datasets?

A5. We are currently awaiting legal approval for the public release of the model and curated datasets. Meanwhile, we have provided sufficient details to facilitate the reproducibility of our model and dataset.

We thank the reviewer RKXa for their positive evaluation of the technical novelty and superior performance of our method. As suggested, we will enhance the presentation of our method and figures in the revised draft. Formal mathematical equations will be added to clarify the computation of the proposed attention modules, and the typos in the abstract will be corrected as recommended. Below, we provide detailed responses to the additional comments:

Q1. Additional quantitative ablation studies should be included in the main text.

A1. Due to space limitations, we initially placed the ablation studies in the supplementary material. However, we are happy to incorporate these studies into the main manuscript as suggested.

Q2. Figure 1(c) lacks textual annotations.

A2. Thank you for pointing this out. Your understanding is correct: V (View), F (Frame), H (Height), and W (Width) denote the respective dimensions. We used purple and orange to indicate corresponding blocks in Fig. 1(a). We apologize for any confusion and will update Fig. 1 with clear textual annotations and a legend to improve readability.

Q3. Figure 1(a) lacks sufficient detail. Specifically:(1) The inputs to the cross-attention mechanism are not clearly indicated. (2) The roles of the key, query, and value are not labeled or explained. (3) There are no formal equations provided to illustrate how the two types of cross-attention (cross-view and cross-frame) are calculated.

A3. Thank you for raising these important points. Our cross-view and cross-frame attention modules are both self-attention mechanisms, where the key, query, and value are derived from the same input features. We will clarify this in the revised draft and will also include formal equations to illustrate the computation of both cross-view and cross-frame attention modules.