Thank you for your valuable comments and suggestions. We provide our response to each question below. Please click the PDF to view our revision paper.

Weakness 1 (a): The presentation.

Thanks for the helpful comments on our paper. We have polished our presentation in our revision paper (highlighted in green). More specifically, we make a clearer explanation of the multi-view diffusion models and our MV-D3PO algorithm in Section 3. In Section 4 and 5, we provide the intuition of contrastive guidance (for 3D generation) and contrastive feedback (for multi-view image generation).

Weakness 1 (b)&2 (a). The discussion on the 3D contrastive guidance part.

The $X^{3D}$ means a 3D object with dimension $D$. For the 2D image, we use $d$ as its dimension (Hence, the $M$ different view images have dimension $M\times d$) Since the 3D object can be viewed as a combination of multi-view images ($A$ in Assumption 1) and different view images share the same latent information ($z$ in Assumption) (such as 3D representation), we have Assumption 1. For the linear assumption, it is widely used in theoretical works [1] [2].

In this part, we make a clearer discussion on the role of contrastive guidance and the relationship with our contrastive feedback. For the guidance section, we focus on the 3D generation task and discuss how to use suitable guidance to "render" a certain view from a 3D object (which is generated by a 3D diffusion model). In this process, we introduce the intuition of contrastive guidance.

Since the different view images rendering from a 3D object is 3D consistent, we think the intuition of contrastive guidance can be shared with the multi-view generation task. However, the multi-view generation task does not have a generated 3D object, which means we can not calculate $\nabla\_{X\_0^{3D}}$ and $\nabla\_{X\_t^{3D}}$ (two core parts of the contrastive guidance). An executable solution sharing a similar idea is to design a contrastive 3D feedback (Sec.5) and fine-tune the pre-trained multi-view models (Sec. 6). As shown in the experiment part, our MV-D3PO-C achieve a great performance from the qualitative and quantitative perspective. We have now added the above discussion at the beginning of Section 5 in our revision paper (highlighted in green).

Weakness 2 (b). The 3D consistency.

The generated quality of different view images is guaranteed by the classical NVS metrics such as LIPIPS, SSIM and PSNR. For the 3D consistency, as shown in our multi-object cases (Figure 5), the samples generated by the pre-trained models contain multi-object, which means the 3D object (reconstructed by a large reconstruction model given different view images) still has multiple parts. On the contrary, our generated samples only have one object, which is more 3D consistent compared with the pre-trained models. For the eyes case, the multi-view images generated by pre-trained models lose the feature of the pupil. On the contrary, our samples preserve the feature.

Furthermore, our CL score (which gives a higher score for the 3D consistency pairs) also shows that our generated samples have a higher score.

Weakness 3. The additional experiments.

Thanks for the comments on the additional experiment. We also test the performance of zero123++, which achieves LIPIPS 0.318, SSIM 0.7812, PSNR 9.0268. Though the result of zero123++ is worse than the results in Table 1, we note that these results can not be directly compared with our results due to the different rendering parameters (as we discussed at the end of the baseline part). We note that our contrastive 3D feedback and DPO framework is uniformly applicable to multi-view diffusion models and it is an interesting future work to show universe improvement in different multi-view models.

---

Thank you for your valuable comments and suggestions. We provide our response to each question below.

Weakness 1. The 3D consistency.

Thanks again for the suggestion on the human feedback evaluation on the 3D consistency property, we will add this part in our next version paper. In this part, we show how to support the discussion "our MV-D3PO-C algorithm achieves better 3D consistency".

The generated quality of different view images is guaranteed by the classical NVS metrics such as LIPIPS, SSIM and PSNR. For the 3D consistency, as shown in our multi-object cases (Figure 5), the samples generated by the pre-trained models contain multi-object, which means the 3D object (reconstructed by a large reconstruction model given different view images) still has multiple parts. On the contrary, our generated samples only have one object, which is more 3D consistent compared with the pre-trained models. For the eyes case, the multi-view images generated by pre-trained models lose the feature of the pupil. On the contrary, our samples preserve the feature. Furthermore, our CL score (which gives a higher score for the 3D consistency pairs) also shows that our generated samples have higher scores.

Q1: The source of negative samples.

We note that as long as contrastive feedback is used, the 3D consistency property would be improved (with different negative models). As shown in Table 1, MV-D3PO-C (pre) and MV-D3PO-C all achieve better 3D consistency (CL score), where the first algorithm uses our pre-trained multi-view models as the negative model and the second algorithm uses Zero123 as the negative model. However, we also note that the choice of the negative models is also important. As shown in Table, though MV-D3PO-C (pre) achieve a better CL score compared with MV-D3PO-M, the other metrics are worse. On the contrary, the MV-D3PO-C (with zero123 as the negative model) achieves better results for all metrics.

Thank you for your valuable comments and suggestions. We provide our response to each question below. Please click the PDF to view our revision paper.

Weakness 1. The 3D consistency.

The generated quality of different view images is guaranteed by the classical NVS metrics such as LIPIPS, SSIM and PSNR. For the 3D consistency, as shown in our multi-object cases (Figure 5), the samples generated by the pre-trained models contain multi-object, which means the 3D object (reconstructed by a large reconstruction model given different view images) still has multiple parts. On the contrary, our generated samples only have one object, which is more 3D consistency compared with the pre-trained models. For the eyes case, the multi-view images generated by pre-trained models lose the feature of the pupil. On the contrary, our samples preserve the feature.

Furthermore, our CL score (which gives a higher score for the 3D consistency pairs) also shows that our generated samples have a higher score.

It is an interesting future work to achieve an end-to-end 3D reconstruction by using our generated multi-view images and well-trained large sparse-view large reconstruction model with contrastive 3D consistency feedback (with our rendering parameters). We have added this discussion in the future work part of our revision paper (highlighted in green).

Weakness 2: the eyes and multi-object problem.

As discussed in Weakness 1, these two problems are typical and easy-to-observe problems for the 3D consistency property. We have added more examples with more complex geometries in Appendix B.1 of our revision paper.

Q1: The discussion on the COLMAP.

Since we use the reference attention technique, we need to generate different view images at the same time. Since the resolution of our images is $512*512$, it is hard to generate more than $10$ different view images simultaneously in a single A100. Hence, we choose five typically orthographic views to show our improvement compared with the pure SFT method.

Q2: the image encoder with a stronger 3D prior.

Thanks for your helpful comments! The image encoder of our contrastive 3D feedback can be any encoder with suitable 3D prior and we use DINOV2 as an example in this work. It is an interesting future work to use an image encoder with stronger 3D prior to achieving more fine-grained feedback.

Thank you for your valuable comments and suggestions. We provide our response to each question below. Please click the PDF to view our revision paper.

Weakness 1&2. The discussion on the 3D contrastive guidance.

Thanks for the comments on the contrastive 3D guidance and feedback. In this part, we make a clearer discussion of the role of these terms. For the guidance section, we focus on the 3D generation task and discuss how to use suitable guidance to "render" a certain view from a 3D object (which is generated by a 3D diffusion model). In this process, we introduce the intuition of contrastive guidance.

Since the different view images rendering from a 3D object is 3D consistent, we think the intuition of contrastive guidance can be shared with the multi-view generation task. However, the multi-view generation task does not have a generated 3D object, which means we can not calculate $\nabla\_{X\_0^{3D}}$ and $\nabla\_{X\_t^{3D}}$ (two core parts of the contrastive guidance). An executable solution sharing a similar idea is to design a contrastive 3D feedback (Sec.5) and fine-tune the pre-trained multi-view models (Sec. 6). As shown in the experiment part, our MV-D3PO-C achieve a great performance from the qualitative and quantitative perspective. We have now added the above discussion at the beginning of Section 5 in our revision paper (highlighted in green).

Weakness 3: The presentation.

Thanks again for the helpful comments on the introduction of DPO for multi-view diffusion models. We have polished our presentation of this part (please see Sec. 3.2 of our revision paper, highlighted in green).

Weakness 4: the advantage of the proposed MV-D3PO-C algorithm.

For the performance, we show that our algorithm achieves greater quantitative results (LIPIPS, SSIM, PSNR and CL score). We also show that detailed cases (the eyes and multi-object problem) show great performance from the qualitative perspective.

For the resource efficiency perspective, as shown in Sec. 6.2, since the DPO-based method is more data-efficient, we only use $240$ datapoints to fine-tune the pre-trained models, which can run in one V100. On the contrary, CARVE3D (a PPO-based method) uses $48$ A100 to fine-tune the pre-trained method

Weakness 5. The additional experiments.

Thanks for the comments on the additional experiment. We also test the performance of zero123++, which achieves LIPIPS 0.318, SSIM 0.7812, PSNR 9.0268. Though the result of zero123++ is worse than the results in Table 1, we note that these results can not be directly compared with our results due to the different rendering parameters (as we discussed at the end of the baseline part). We note that our contrastive 3D feedback and DPO framework is uniformly applicable to multi-view diffusion models and it is an interesting future work to show universe improvement in different multi-view models.

Weakness 5 &Q1: The comparison with CARVE3D.

The CARVE3D use a sparse-view large reconstruction model (for example, they use closed-source Instant3D) to determine their reward. Then, they use DDPO (a PPO-based algorithm) to fine-tune the pre-trained model.

For the first point, their algorithm relies on a high-quality large reconstruction model (LRM), which requires large high-quality data and is unfriendly to users. Though it is possible to use open-source LRM (such as OpenLRM and LGM), the performance of DPPO with these models is unclear and the LRM models are large. On the contrary, our contrastive 3D feedback is lightweight and only uses $1000$ data to fine-tune the DINOV2 model.

For the second point, our method inherits the advantages of DPO and is data-efficient (as we discuss in the response to Weakness 4).

For the experiment results, since CARVE3D does not release their checkpoint, we do not provide their empirical performance, such as PSNR, SSIM, etc.

Q2: The training time of the pre-trained models.

The 400 A100 days is the time to train our pre-trained models. For our MV-D3PO-C fine-tuning algorithm, as shown in the response of Weakness 4, our method is data and time-efficient.

Q3: The different objects

The forklift is the ground-truth other view image corresponds to the reference images. The "vehicles" is the other view image generated by zero123++ given the reference images. The "vehicles" image did not contain the forklift feature may be because Zero123 does not capture this feature.