MIRROR: Make Your Object-Level Multi-View Generation More Consistent with Training-Free Rectification

Thank you for your thorough analysis and constructive feedback on our paper. We will address the concerns you raised and hope our responses will clarify your doubts.

Q1. Scale of Depth

A1. Based on the camera parameters of the base model, we approximate relative-to-metric depth conversion and achieve depth alignment across multiple views.

As the object-camera distance is fixed during training, the model learns a consistent depth scale under circular camera poses. This scale factor allows transforming relative depth into absolute depth for cross-view alignment. We further apply grid search over scale factors and use dual-anchor fusion of block features to reduce potential scale shifts during inference. The transformed depth enables more accurate trajectory tracking in TTM via geometric warping.

Details on Eq. (5) can be found in Reviewer y23b, A3. And alignment details of depth estimation will be included in the appendix of the revised version.

Q2. More Quantitative Results

A2. Based on your suggestion, we have provided additional quantitative evaluation results. Please refer to the table in Reviewer BJCB, A1, for details.

Q3. Extension to Scene-level Tasks

A3. Importantly, the nature and manifestations of inconsistency differ between object-level and scene-level multi-view generation tasks. Object-level inconsistency mainly arises from the lack of 3D structural modeling and consistency supervision, often manifesting as the Janus problem and content drifting. In contrast, scene-level tasks typically suffer from layout disarray and semantic drift due to the absence of structural representations and layout supervision. Thus, the underlying challenges are fundamentally different.

Our task track focuses on object-level multi-view generation, a key branch in 3D generation, with the baseline models representing mainstream, state-of-the-art methods. Our motivation is to correct the inconsistencies in object-level base models via explicit consistency supervision in a lightweight, plug-and-play, training-free manner, thereby improving 3D reconstruction quality.

While scene-level generation is another important branch with fundamentally different inconsistency issues, we believe our method can offer insights for advancing this direction. To adapt MIRROR to this task, future work could incorporate layout-aware priors or scene maps to handle the broader spatial context. We consider MIRROR a core foundational step toward such extensions, and potential directions for scene-level adaptation will be discussed in the appendix.

Q4. Symbols in FRM

A4. We corrected the typo by redefining the 3×3 block as $B_{u _ \alpha}$ instead of $M(u _ \alpha)$ to align with the notation used in subsequent sections. Following your suggestion, we clarify key symbols in FRM, along with the updated pipeline (see Fig. 1 in https://anonymous.4open.science/r/mirror-A9B9/figs.pdf).

$u, u_\alpha$ denote the coordinates of a point in the current view and its corresponding tracked point in the neighboring view, respectively. And $Z(u), Z_\alpha(u_\alpha)$ represent the feature values indexed by point $u$ and $u_\alpha$. By traversing all features ${Z_\alpha(v), v \in B_{u_\alpha}}$, within block $B_{u_\alpha}$, and applying the dual-anchor weights $W$, we obtain the aggregated block feature $\mathcal{M}(Z_\alpha(u_\alpha))$. The L2 feature distance between $Z(u)$ and $\mathcal{M}(Z_\alpha(u_\alpha))$ is then computed by Eq. (10) to form the consistency correction loss.

Q5. Background Exclusion

A5. As mentioned in our response A1, irrelevant background information would not caused by depth warping. The design of negative anchors aims to exclude both the additional information from depth map downsampling into the latent space and the redundant or irrelevant signals arising from expanding point features into block-level form to preserve spatial continuity.

This design does not hinder extension to scene-level tasks. The dual-anchor feature fusion mechanism effectively suppresses irrelevant information while enhancing the contribution of relevant features. Moreover, dual anchors and their weights can be adapted to different application scenarios for better generalization.

We will incorporate the revisions into the paper and hope our responses could address your concerns.

We sincerely believe that our work is deserving of acceptance, and we would be grateful if you could recognize the contributions we’ve made. We kindly hope that, you might consider raising the score accordingly. Thank you again for your thoughtful feedback and consideration!

We greatly appreciate your thorough and detailed review, offering valuable insights on methodology, theory, experiments, and scalability, and thank you for recognizing our work!

Q1. Limitation of TTM

A1. (1) TTM is designed to ensuring uniform geometry coverage and is theoretically extendable to any pose. In fact, sampling multiple views around a fixed elevation outperforms arbitrary 6DoF poses for reconstruction by reducing occlusions and preserving overall geometry. Specifying a set of elevations and sampling around each can further enhance results.

(2) The number of views is determined by baselines, providing flexibility and decoupling from MIRROR.

(3) Optical flow is unsuitable for multi-view generation due to high computational overhead, while dedicated point trackers suffer from drift during viewpoint rotation, causing geometric and texture distortions.

In contrast, TTM enables fast tracking with block-level spatial fusion, reducing errors. 3D metrics in Reviewer BJCB, A1, also confirm our effectiveness.

Q2. Broader Applicability

A2. Our task focuses on object-level multi-view generation and reconstruction, a key branch of 3D generation, with baselines being powerful mainstream methods. Generation of real-world or dynamic scenes and deformable objects are other important branches, and we believe MIRROR could inspire advancements in these areas. More details refer to Reviewer ir3S, A3.

Q3. Definition of Eq.(5)

A3. Eq.(5) defines the tracking error upper bound as base model's sampling error, with $x_0$ and $\hat{x}_0(x_t, t)$ representing the true image and predicted image at state t, respectively.

Q4. Proof of Eq.(15)

A4. Using the first-order Taylor expansion, we obtain:

$$H(x_0)=H(\hat{x}_0(x_t,t))+\nabla H(x_0) (x_0-\hat{x}_0(x_t,t))+o(||x_0-\hat{x}_0(x_t,t)||).$$

Here, $H$ is a pretrained ViT network with a continuous, bounded gradient that outputs scale-consistent absolute depth, showing that the depth-level error is of the same order as the pixel-level error:

$$||H(\hat{x}_0(x_t,t))-H(x_0)||\approx||\nabla H(x_0)(x_0-\hat{x}_0(x_t, t))||\simeq O(||x_0-\hat{x}_0(x_t,t)||).$$

Q5. Derivation of Eq.(19)

A5. There was a typo in Eq.(18), now corrected as:

$$\nabla_{z_t}\log p_\theta(z_t)=-\frac{1}{\sqrt{1-\overline{\alpha}_t}}\varepsilon _ \theta(z_t,t).\tag{18}$$

Using Eq.(17) and (18), Eq.(19) is easily derived:

$$\varepsilon_\theta(z_t,t,c)=\varepsilon_\theta(z_t,t)-\sqrt{1-\overline{\alpha}_{t}}\nabla _ {z_t} \log p _ \theta(c|z_t).\tag{19}$$

Q6. Depth Estimation

A6. Fig.12 and Tab.3 shows improving depth estimation accuracy benefits MIRROR, but this does not necessarily mean that multi-view depth estimation methods are superior.

Our goal is a lightweight plugin to enhance consistency. While methods like Dust3R and Mast3R improve robustness, they require constructing multi-view cost volumes or 3D reconstruction, leading to high memory usage (2GB vs. 100MB for DA2) and slow inference, which compromises our advantages. Besides, in monocular tasks, DA2 significantly outperforms Dust3R, indicating that multi-view methods, despite improving depth consistency, may exacerbate Janus Problem due to the accumulation of estimation errors during denoising.

For depth scale consistency, refer to Reviewer ir3S, A1.

Q7. Selection of s(t)

A7. s(t) aims to match the consistency gradient magnitude with $\varepsilon_\theta$. With negligible differences across models, we use a general parameter, though adjustments per model are feasible.

Q8. Failure Cases and Limitation of Baselines

A8. Failure cases (see Fig.2 in https://anonymous.4open.science/r/mirror-A9B9/figs.pdf) show that when the baseline produces unreasonable, severely flawed geometry (an inherent limitation), we struggle to correct these fundamental issues.

Q9. Essential References

A9. Without training, ConsiStory enhances subject consistency in text-to-image generation by modifying the network with Subject-Driven Shared Attention and Feature Injection. However, it lacks plug-and-play compatibility with other models, limiting generalizability.

Multi-view diffusion models generate consistent images from noise, while inverse diffusion infers the initial state from known outcomes, with distinct objectives.

Q10. DreamFusion

A10. You're right. We'll clarify it.

Q11. Text-based Results

A11. $x_0$ is the theoretical true image in tracking loss (5). As the text-based method lacks a reference image, VideoMV uses half of the views for reconstruction and rendering as pseudo-ground truth. Moreover, text-based tasks are more challenging and diverse than image-based ones, so small corrections in the denoising process have a stronger impact, highlighting MIRROR's power through both qualitative and quantitative improvements.

Hope our responses address your concerns. A detailed revision will be in the appendix. Thanks again for your comprehensive feedback.

We sincerely appreciate your valuable suggestions and the recognition of our work in methodology, theoretical proof, and experimental design. Below are our responses to your concerns, and we hope they help clarify any doubts you may have.

Q1. More Metrics

A1. On the one hand, for fairness, we adopt the same quantitative metrics as baselines (including PSNR, SSIM, LPIPS, and CLIP Score), which are also widely used in current practice of multi-view generation. The quantitative results align well with the qualitative observations, further supporting the reliability of these metrics. In addition, Reviewer y23b acknowledges the quantitative metrics we used in the fourth line of their review in the "Claims and Evidence".

On the other hand, we additionally employ Chamfer Distance and Volume IoU to evaluate the 3D consistency of the reconstructed geometry from the generated multi-view images as follows:

For text-based methods, following VideoMV, we sample half of the views at regular intervals to reconstruct a pseudo-ground truth mesh for evaluation. Other evaluation settings follow those in Appendix C.4. The 3D reconstruction metrics in the table further validate the effectiveness of our method in improving the quality and consistency of both multi-view images and 3D reconstruction.

Q2. Depth Estimation

A2. All versions of Depth-Anything-V2 (DV2) achieve over 95% accuracy, with no significant performance differences observed in our task. Considering both accuracy and model size, we adopt DV2-Small as the depth estimation module.

Moreover, based on the fixed camera distance used during training of each baseline, we convert the relative depth to metric depth using a consistent scale factor. Specifically, the baseline model, trained and inferred under circular camera poses, inherently provides a fixed depth scale, which ensures cross-view consistency. This allows the relative depth predicted by DV2 to be reliably transformed into absolute depth, ensuring consistent metric depth across multiple views.

We hope our response could address your concerns. Based on your suggestions, we will include additional metrics in the appendix. Thank you once again for your recognition of our work and providing such insightful recommendations!

Thank you for your thoughtful feedback. We truly appreciate your acknowledgment of our motivation, methods, theoretical proof, and experimental results. Your encouraging comments are highly valued, and we are grateful for your insights!

Q1. Comparison with Cross-view Attention

A1. Cross-view attention layers inherently struggle to enforce 3D consistency, as they lack explicit geometric constraints and rely solely on implicit learning through network weights. Given the limited availability of 3D data, such approaches are prone to local optima and often fail to generalize. Existing methods (e.g., MVDream, Consistent-1-to-3, Era3D) still suffer from issues like multi-face artifacts and content drifting at inference stage. Moreover, integrating our method into cross-view attention is impractical, as depth supervision is difficult to extract during training, and the attention layers demand substantial training costs in both computation and time.

In contrast, FRM offers a more direct and effective solution. Leveraging the progressive denoising process of diffusion models, we introduce explicit multi-view consistency constraints by injecting expert priors from baseline models. It operates without requiring additional 3D supervision and is fully decoupled from the diffusion framework, making it lightweight, plug-and-play, and easily adaptable, which is a more novel approach at a high level. This explicit, geometry-aware mechanism enables FRM to correct inconsistencies more efficiently and reliably than cross-view attention, leading to more stable and coherent multi-view generation results.

Q2. Handling of Non-Lambertian Surfaces

A2. This is a promising direction, and it could be addressed by decomposing the generation of non-Lambertian surfaces into two tasks.

First, our pipeline enhances multi-view consistency to reconstruct high-quality 3D models, improving the geometric details of objects, which forms a solid foundation for subsequent lighting and rendering.

Building on this, lighting models and physics-based rendering techniques can be applied to illuminate and render the 3D model, accurately capturing the reflective properties of shiny surfaces from various viewpoints.

In other words, while MIRROR enhances the performance of the first task, it also supports the rendering of shiny surfaces. Together, the two processes improve the generation of non-Lambertian surfaces.

Q3. Scale Consistency of Depth

A3. Although the depth information comes from the monocular estimation model, the depth ratio across different views remains consistent because the baseline model fixes the camera distance during training. Thus, the monocular model’s estimation capability is sufficient. Given that the depth scale perceived by the baseline model is fixed, we can use it as a scale factor to convert normalized relative depth into absolute depth, ensuring depth alignment across views.

Furthermore, we perform a grid search for scale factors across multiple viewpoints and apply dual-anchor-based fusion of block features in FRM to mitigate potential scale shifts in TTM. We believe this approach is in line with your suggestion, and we will add these discussions in the appendix for further clarification.

We hope our response has effectively addressed your concerns. Your suggestions have offered valuable insights that will greatly inform our future work, and we deeply appreciate them. Thank you once again for your constructive comments.