MV-Adapter: Multi-view Consistent Image Generation Made Easy

We appreciate your review but respectfully disagree with the characterization of our contributions as merely incremental modifications to T2I-Adapter. While T2I-Adapter is a component within our framework, our core contributions extend significantly beyond it.

Our core contributions are summaried as follows:

1. First Adapter-Based Solution for Multi-View Image Generation. We introduce the first adapter framework specifically designed for multi-view image generation—a challenging task that greatly benefits from this approach. The advantages you mentioned (efficiency, adaptability, versatility, performance) are not simply inherited from T2I-Adapter. Instead, they stem from the adapter concept and our innovative design within this framework.
2. Technical Innovations in Adapter Design. We propose a novel parallel attention architecture that efficiently decouples the learning of geometric knowledge from image priors (our ablation studies on L475~485 and Appendix A.3.4 thoroughly discuss and validate its effectiveness), and a unified condition embedding and encoder that enhances the model's flexibility.
3. Extensions to Arbitrary View Generation and Inspirations for Future Work. MV-Adapter can easily extend to arbitrary view generation, facilitating a wider range of downstream tasks. Details are provided in Appendix A3.6. Furthermore, MV-Adapter provides insights for modeling new knowledge (such as geometric, physical or temporal knowledge) based on image priors with our framework for decoupled learning.

We kindly refer you to our global response for a deep understanding of our work. We firmly believe that these contributions demonstrate significant innovation and provide valuable insights to the field. Our work offers a novel and effective approach to multi-view image generation that goes beyond incremental changes to existing methods. We hope this clarifies our position and the substantial contributions of our work. Thank you for your consideration.

We appreciate your focus on technical novelty. Let us precisely articulate the technical contributions in our work:

Technical Contributions

Our main contribution lies in the technically innovative adapter framework we propose for multi-view image generation, rather than in specific components like T2I-Adapter or Row-wise Attention.

1. Attention Architecture. We propose novel attention architecture to preserve the integrity of the base T2I model while enabling efficient learning of geometric knowledge through a parallel structure.
   • Technical details:
     • Duplication of Self-Attention Layers: Our design adheres to the principle of preserving the original network structure and feature space of the base T2I model. Existing methods like MVDream and Zero123++ modify the base model's self-attention layers to include multi-view or reference features, which disrupts the learned priors and requires full model fine-tuning. In contrast, we duplicate the self-attention layers (including architecture and weights) to create new multi-view attention and image cross-attention layers, initializing the output projections to zero. This allows the new layers to learn geometric knowledge without interfering with the original model's priors, ensuring excellent adaptability.
     • Parallel Attention Structure: The straightforward way to insert new layers is to append them serially after the original layers. However, it requires the new layers to learn from scratch and diminishes the effectiveness of pre-trained priors. Even if we initialize the new layers with the pre-trained self-attention parameters, the features input to new layers are in a different domain with those input to original self-attention layers, causing the initialization to be ineffective. Therefore, we design a parallel architecture. This ensures that the new layers can fully inherit the powerful priors of the pre-trained self-attention layers, enabling efficient learning of geometric knowledge.
   • Empirical Validation: Our ablation studies and experimental results validate the effectiveness of our design:
     • Figures 1, 5, 8, and 22 in our paper display visual results that demonstrate our adapter's strong adaptability and effectiveness. And Figure 9 provides visual comparisons with existing methods on the adaptability, showcasing the advantages of our adapter design.
     • Table 5 and Figure 17 presents quantitative and qualitative results of ablation study on attention architecture, demonstrating the superiority of our parallel design.
2. Unified Condition Embedding and Encoder. We introduce a unified condition embedding and encoder that seamlessly integrates camera parameters and geometric information into spatial map representations, enhancing the model's flexibility and applicability.
   • Technical details: To enable the adapter's versatility—supporting both camera conditions and geometry conditions for critical 3D applications like 3D generation and 3D texture generation—we innovatively unify camera parameters and geometric information into spatial map representations. We introduce a spatial-aligned condition encoder that encodes both types of maps using the same architecture. Previous methods have only encoded either camera information or geometric information but have not unified both within a single framework. By adopting this unified representation and encoder, our adapter gains enhanced flexibility and applicability, seamlessly handling various types of guidance within a unified framework.
   • Empirical Demonstration: Figures 6 and 7 in the paper illustrate camera-guided multi-view image generation, while Figures 8 and 10 demonstrate geometry-guided multi-view image generation and texture generation. These results confirm that our design enables the model to support multiple functionalities, which previous works have not achieved.

Valuable Insights

• Decoupling Image Prior and Geometric Knowledge: By decoupling the learning of geometric knowledge from the image prior, our framework efficiently integrates new knowledge without compromising the base model's rich visual capabilities.
• Inspiration for Future Research: This principle not only enhances learning from limited data but also inspires other tasks that build upon existing image priors to learn new types of knowledge. Our approach can be applied to learning zoom in/out effects, consistent lighting, and other viewpoint-dependent properties. It provides valuable insights for modeling physical or temporal knowledge based on image priors.

Each of these represents a concrete technical advancement, not just a combination of existing techniques. We respectfully request that you reconsider your assessment of our paper.

Thank you once again for your thoughtful feedback. If you have any further questions or concerns, we would be happy to address them.

Dear Reviewer:

Once again, we sincerely appreciate the time and effort you have dedicated to reviewing our paper. We kindly suggest that you visit our anonymous project page https://mv-adapter.github.io/ for a deeper understanding of our work.

As the discussion period concludes in two days, we would be grateful if you could review our rebuttal at your convenience, should your schedule allow. If there are any further points requiring clarification or improvement, please be assured that we are fully committed to addressing them promptly. Thank you once again for your invaluable feedback on our research.

Meeting ICLR Standards

We would also like to take this opportunity to address the comment in your initial review: “Therefore, I think the paper is publishable but maybe not at a top conference like ICLR.” We believe our work aligns with ICLR's core emphasis on bringing "new, relevant, impactful knowledge" to the community through:

• Novel Technical Contribution:
  1. First Systematic Adapter Framework for Multi-View Generation: We introduce the first comprehensive adapter-based framework specifically designed for multi-view image generation, effectively addressing the complexities of handling multiple views and ensuring consistency.
  2. Innovative Design: Our attention architecture and unified condition embedding enable efficient integration of geometric knowledge without altering pre-trained models.
  3. Thorough Ablation Studies: We provide extensive ablations validating our design choices, demonstrating their impact on performance.
• Well-motivated Problem & State-of-the-Art Performance:
  1. Addresses critical limitations: Our work tackles significant limitations in existing methods, such as adaptability, scalability and resolution constraints, enabling high-resolution (768×768) multi-view generation on large models like SDXL.
  2. SOTA Results: We achieve leading performance in multi-view generation, with improved consistency and quality.
• Impact and Utility for the Community:
  1. Adaptability and Versatility: Our MV-Adapter seamlessly integrates with various powerful or customized T2I model variants, enabling consistent multi-view generation and supporting multiple conditional inputs, which facilitates downstream applications like 3D generation and 3D texture synthesis.
  2. Inspiring Future Research: Our approach of learning new knowledge based on existing priors can inspire the modeling of other geometric (e.g., lighting effects), physical, and temporal knowledge, potentially benefiting a wide range of applications.

We hope that these points clarify how our work meets the standards of ICLR and contributes valuable advancements to the field. We respectfully request reconsideration of your assessment of our paper. If you have any further concerns or questions, we would be grateful for the opportunity to address them.

Warm regards, The Authors

2. Comparison with Reference Attention in Zero123++.

While both our image cross-attention mechanism and the reference attention in Zero123++ employ a pre-trained U-Net to encode the reference image, there is a fundamental difference in how they integrate these features:

• Zero123++: Extends the denoising U-Net's self-attention keys and values to include reference features, requiring full fine-tuning of the base Stable Diffusion model. This approach modifies the original network's feature space, potentially disrupting the base model's prior knowledge.
• Our Method: Duplicates the self-attention layers to create new image cross-attention layers (including network structure and weights), organizing them in a parallel architecture. By initializing the output projections to zero, we decouple the learning of reference image information from the base model. This design avoids full-model fine-tuning and preserves the base model's capabilities, thereby enabling our model compatible with the adapter principle.

Our approach integrates reference information without interfering with the pre-trained model, maintaining the integrity of the original feature space while efficiently learning from the reference images.

3. Advantages of Using MV-Adapter to reconstruct 3D.

The advantages of using MV-Adapter for 3D reconstruction lie in its adaptability and scalability:

• Adaptability: Users can attach MV-Adapter to various base model variants or customized models (e.g., with DreamShaper, stylized or personalized models, ControlNet) to achieve higher-quality, customized, and controllable multi-view generation, as demonstrated in our paper.
• Scalability: The efficient training of MV-Adapter makes it feasible to scale to larger and more powerful T2I base models like SDXL. This scalability enables modeling stronger consistency and generating higher-quality multi-view images, providing significant benefits to downstream tasks like 3D reconstruction.

These advantages enhance the usability and effectiveness of MV-Adapter in practical applications requiring high-quality 3D reconstructions from multi-view images.

4. Text Captions Used for Training.

The text captions used for training are sourced directly from the Cap3D dataset. We have clarified this point by adding a statement in the "Dataset" section of Appendix A2 (Implementation Details).

5. Consideration of Full-Model Fine-Tuning.

We believe that efficient training (i.e., reducing the number of trainable parameters) is also important in multi-view generation tasks for several reasons:

• Potential for Stronger Base Models: Efficient training allows adaptation to more powerful base T2I models. As shown in our results, switching the base model to SDXL brings significant gains in multi-view consistency and image quality.
• Preservation of Base Model's Feature Space: By avoiding modifications to the base model's original feature space, MV-Adapter exhibits strong adaptability, compatible with various model variants and plugins. This compatibility enhances controllability and usability in multi-view generation.

To address your suggestion, we conducted an experiment on text-to-multiview generation, comparing our MV-Adapter (SD2.1) with a version using full-model fine-tuning while keeping the current design unchanged. As shown in the following table, full-model fine-tuning leads to overfitting on the 3D dataset, resulting in poorer performance. This demonstrates that our approach, which avoids full-model fine-tuning, maintains the base model's generalization capabilities and achieves superior results without the need to fine-tune all parameters.

Table R1 | Setting | FID$\downarrow$ | IS$\uparrow$ | CLIP Score$\uparrow$ | | - | :-: | :-: | :-: | | MV-Adapter (SD2.1, Full-tuning) | 33.76 | 13.23 | 30.51 | | MV-Adapter (SD2.1, Ours) | 31.24 | 15.01 | 32.04 |

We hope these clarifications and additional results address your concerns and provide a deeper understanding of our work. Thank you again for your constructive feedback.

Thank you for your valuable feedback and for highlighting areas where we can improve our work. We address your concerns and questions point by point below.

1. Quantitative Results of 3D Reconstruction for Validating Multi-view Consistency.

To further evaluate the consistency of our multi-view generation and its applicability to downstream tasks, we have added a quantitative comparison of 3D reconstruction on the GSO dataset using MV-Adapter and Era3D (which shares a similar pipeline). As shown in Table 7 (see Appendix A5.2), the 3D reconstruction quality using MV-Adapter based on SD2.1 is comparable to that achieved with Era3D. Importantly, when using MV-Adapter based on SDXL, the reconstruction quality improves significantly, with notable enhancements in both Chamfer Distance and Volumetric IoU metrics. These results demonstrate the scalability of MV-Adapter and its ability to leverage the strengths of state-of-the-art T2I models, providing additional benefits to downstream tasks like 3D reconstruction.

Table 7 | Method | Chamfer Distance$\downarrow$ | Volumetric IoU$\uparrow$ | | - | :-: | :-: | | Era3D | 0.0329 | 0.5118 | | Ours (SD2.1) | 0.0317 | 0.5173 | | Ours (SDXL) | 0.0206 | 0.5682 |

Furthermore, to provide a more in-depth analysis of our quantitative results on image-to-multi-view generation, we conducted a user study. Details of this study and its findings are presented in Appendix A5.1.

Thank you for your positive feedback regarding the clarity of our paper, the comprehensiveness of our experiments, and the versatility of our proposed method.

1. Details about T2I-UNet for Image Cross-Attention.

Yes, the T2I U-Net used for extracting reference image features is identical to the diffusion U-Net. We input the reference image without noise and set the timestep to 0 to extract the features. We have added these details to the methodology section of our revised paper.

2. Similarity to T2I-Adapter and Possibility of Combination.

While our condition guider component resembles that of the T2I-Adapter, our overall adapter framework is innovative. We kindly refer you to our global response for a deeper understanding of our contributions. Additionally, our MV-Adapter can indeed be combined with a pre-trained T2I-Adapter. We have included results demonstrating this combination in Figure 22 of the Appendix.

3. Calculation of CFG with Multiple Conditions.

Our method for calculating Classifier-Free Guidance (CFG) with multiple conditions is similar to that used in TOSS [1]. Let pred_uncond, pred_image, and pred_image_text denote the model's predictions under unconditional input, image-only condition, and both image and text conditions, respectively. The CFG calculation is as follows:

pred_uncond + cfg_image * (pred_image - pred_uncond) + cfg_text * (pred_image_text - pred_image)

We have provided a formal description of this calculation in the "Inference" paragraph of Appendix A.2.

[1] TOSS: High-quality Text-Guided Novel View Synthesis from a Single Image [ICLR 2024]

Thank you for your thoughtful review and valuable feedback. We address your concerns point by point below.

1. Enabling Arbitrary View Synthesis.

While our main text focuses on generating a fixed set of six views for 3D generation and 3D texture generation, our adapter framework can be extended to synthesize multiple views from arbitrary angles. In Appendix A3.6, we have detailed the scheme and experiments demonstrating this capability.

Our approach is similar to CAT3D: we first generate eight anchor views and then use these as conditions to guide the generation of additional views at arbitrary angles. The MV-Adapter framework remains largely unchanged—we adjust the number of input images to one or four (concatenate multiple images into a long image if the number is set to four), and replace the row-wise attention with full self-attention in the multi-view attention module. More details and experimental results are provided in Appendix A3.6 and on our anonymous project page https://mv-adapter.github.io/ .

2. Stronger Quantitative Evidence on Image-to-Multi-view.

To provide a more in-depth analysis of our quantitative results, we have supplemented Table 2 with metrics from our MV-Adapter model based on SDXL, showing significant performance improvements. Additionally, we conducted an ablation study comparing MV-Adapter (based on SD2.1) with baseline methods. We evaluated multi-view consistency and image quality preferences over 30 samples, collecting responses from 50 participants. The results and additional qualitative comparisons are documented in Appendix A5.1.

Results of User Study | Method | Multi-view Consistency Preference (%) $\uparrow$ | Image Quality Preference (%) $\uparrow$ | | - | :-: | :-: | | ImageDream | 6.27% | 0.93% | | Zero123++ | 18.00% | 16.87% | | CRM | 10.27% | 6.00% | | SV3D | 10.13% | 12.20% | | Ouroboros3D | 7.93% | 10.00% | | Era3D | 22.33% | 17.20% | | Ours (SD2.1) | 25.07% | 36.80% |

3. Use of Existing Components.

We acknowledge that components like multi-view self-attention and reference attention have been used in prior work. However, our core contribution lies in our innovative adapter-based framework for multi-view image generation, not just in individual components. Specifically:

• First Adapter-Based Solution: We introduce the first adapter framework for this task, achieving efficiency, versatility, and adaptability.
• Innovative Adapter Framework Design: Our design includes a novel parallel attention architecture and a unified condition embedding and encoder, which enables the versatility of MV-Adapter.
• Extensions to Arbitrary View Generation and Inspirations for Future Work: MV-Adapter can extend to arbitrary view generation, and provides insights for modeling geometric, physical or temporal knowledge based on image priors with our framework for decoupled learning.

We kindly refer you to our global response for further elaboration.

4. Details on Experimental Setup for GSO Dataset Comparison.

For the image-to-multiview comparison on the GSO dataset, we selected 100 assets covering multiple object categories. For each asset, we rendered an input image from a frontal view within azimuth angles of -45° to 45° and elevation angles of -10° to 30°. The evaluation views did not include extreme top or bottom angles. Additional details are provided in "Comparison with baselines" of Appendix A.2.

We hope these clarifications address your concerns. Thank you again for your valuable feedback.

It's a really nice rebuttal with tons of results and new visualizations. Very impressive job!

Overall, I am happy with every point in the rebuttal, and many of my concerns are well-addressed. Though I remain concerned about the incremental nature and novelty, the performance aspect is now mostly in very good shape with sufficient validation.

As stated in my review, my initial assessment was more like a 4 (weak reject); now I am inclined to change it to a 5 (borderline reject). I cannot assign a higher score due to the high standards of ICLR.

Thank you for acknowledging our impressive results.

1. Clarify and Emphasize Novel Contributions.

Our key contributions extend beyond merely applying self-attention and are outlined below: (1) We introduce the first adapter framework specifically designed for multi-view image generation. This framework offers efficiency, adaptability, and versatility by integrating geometric knowledge without altering pre-trained T2I models, demonstrating high extensibility and performance. (2) Our innovative adapter framework features an effective parallel attention architecture and a unified camera and geometric condition encoder. These innovations enable effective decoupling and integration of diverse conditions, enhancing the overall performance and flexibility of the model. (3) MV-Adapter can extend to arbitrary view generation, and provides insights for modeling geometric, physical or temporal knowledge based on image priors with our framework for decoupled learning.

We are revising the paper to emphasize these contributions and refer you to our global response for detailed explanations.

2. Evaluation Comparing Parallel and Serial Architectures.

We provided qualitative comparisons in the original version, and have added quantitative comparisons and additional qualitative results in Table 5 and Figure 17 of the revised paper, demonstrating the advantages of our parallel design.

Table 5 | Attention Architecture (Based on SDXL) | PSNR $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ | | - | :-: | :-: | :-: | | Serial | 20.687 | 0.8681 | 0.1149 | | Parallel (Ours) | 22.131 | 0.8816 | 0.1002 |

3. Enhance the Novelty Aspect Compared to Existing Methods.

Our method offers distinct contributions: 1) an efficient adapter framework design that decouples geometric knowledge modeling from the base T2I models in a non-invasive manner, which preserves the image priors and enables efficient training on larger models; 2) improved usability and customizability, which means higher generation quality, applicability to larger base models like SDXL, strong versatility and adaptability to various base model variants.

4. Highlight Potential Applications.

MV-Adapter's adaptability enables new applications in image creation and personalization, such as integrating with personalized T2I models for customization. For instance, creators can integrate MV-Adapter with their personalized T2I models—customized for specific identities or artistic styles—to generate multi-view images that capture consistent perspectives of their unique concepts. Furthermore, Its successful practice of decoupling image prior and geometric knowledge also provides insights for tasks requiring modeling of geometry (like lighting, zoom in/out) or other novel aspects (like physics or temporal knowledge) based on T2I models (see Appendix A3.5).

We hope that our responses address your concerns and clarify the contributions and novelty of our work. Thank you again for your valuable feedback.

Dear Authors,

Thank you for your response. It seems that most of the points I mentioned have been addressed, and I sincerely appreciate the authors' efforts. I will maintain my original rating ('6: marginally above the acceptance threshold').

Best regards, Reviewer Fnks

Dear Reviewer:

Once again, we sincerely appreciate the time and effort you have dedicated to reviewing our paper!

As the discussion period concludes in two days, we would be grateful if you could review our rebuttal at your convenience, should your schedule allow. If there are any further points requiring clarification or improvement, please be assured that we are fully committed to addressing them promptly. Thank you once again for your invaluable feedbacks to our research!

Warm regards, The Authors

Dear Yoonwoo Jeong,

Thank you very much for your thoughtful feedback and for bringing the overlaps with NVS-Adapter to our attention. We sincerely apologize for the oversight and appreciate your contribution to this field. We fully agree that acknowledging and comparing our work with NVS-Adapter is important for providing a comprehensive discussion of the related approaches.

After carefully reading and comparing the two versions of your paper—the initial arXiv version (2312.07315v1) and the latest ECCV 2024 version (posted on 10 Aug 2024)—we have identified several key points that we believe highlight the differences between NVS-Adapter and MV-Adapter.

Different Versions of NVS-Adapter

1. Version 1 (arXiv, 2023): The primary motivation and contribution of NVS-Adapter in this version focus on "synthesizing novel multi-views of visual objects while fully exploiting the generalization capacity of T2I models." This version does not consider the compatibility with LoRA, which was mentioned in your comment. As such, we do not see "significant overlaps" between this version of NVS-Adapter and our work, as the emphasis in NVS-Adapter is more on the generalization aspect of T2I models, which is a common issue that many existing works have focused on.
2. Version 2 (August 2024): The latest version of NVS-Adapter, which is published in August 2024 and should be considered as a concurrent work according to ICLR's policy, aligns more closely with the comment you provided, focusing on "compatibility with existing modules such as ControlNet and LoRA". However, the compatibility with T2I extensions is only part of the adaptability of MV-Adapter.

Differences and Our Distinct Contributions

1. Task Scope:

• Multi-view generation refers to the process of generating multiple images of the same object or scene from different viewpoints. It is a more generalized task that involves synthesizing consistent views based on a variety of input conditions (text, images, camera parameters, or geometry). In this broader context, MV-Adapter is designed to tackle multi-view consistent image generation with flexible conditions, which is not limited to a more specific task of synthesizing novel views from a single image (as in NVS-Adapter). This makes MV-Adapter a first adapter-based solution designed to meet a wide range of multi-view generation needs.
• We believe this distinction in task scope sets our work apart and justifies calling MV-Adapter the first systematic adapter solution for multi-view image generation.

2. Motivation and Focus:

• The primary focus of NVS-Adapter (version 1) is on synthesizing novel views while fully exploiting the generalization capacity of T2I models, which involves freezing the base model. The version 2 focuses on the compatibility with T2I extensions, which is only part of the adaptability of MV-Adapter. Our work emphasizes efficiency, adaptability, and versatility through an adapter-based framework. We prioritize not just generating novel views but also ensuring that our approach can seamlessly handle various conditions (text, image, and geometry) and scale efficiently.
• While NVS-Adapter (version 2) considers a certain degree of adaptability, MV-Adapter's complete adaptability, scalability, and versatility in supporting diverse input conditions remain key differentiators.

3. Methodology:

• NVS-Adapter freezes the base model and models view consistency using an additional attention-based network, which can only be trained from scratch and leads to limited effectiveness. In contrast, MV-Adapter duplicates and augments the self-attention layers of the base T2I model, employing a parallel architecture for multi-view attention and image cross-attention. This allows our model to leverage the pre-existing priors to learn view consistency and reference knowledge without disruptive retraining, thus boosting efficiency and adaptability. Additionally, we introduces a unified encoding mechanism for camera and geometric conditions, enabling broader applicability and making the approach more flexible and versatile.
• The above comparison further confirms that the effectiveness of our proposed technical innovations in our architecture, which allows MV-Adapter to seamlessly integrate with various T2I models with their extensions and generate high-quality and consistent multi-view images.

Acknowledgment

We will certainly address the relationship between NVS-Adapter and MV-Adapter in the revised version of our paper, providing a more thorough comparison. We will highlight the distinctions in task scope, motivation, methodology, overall design, and generated results. Once again, we apologize for not addressing this overlap earlier, and we thank you for your constructive feedback. We are grateful for the opportunity to improve the clarity of our work and to acknowledge the contributions of NVS-Adapter.

Warm regards,

The Authors

Dear Authors.

Hello, I am Doyup Lee who is one of the corresponding authors of NVS-Adapter (ECCV'24). First, I sincerely appreciate the nice and interesting idea for versatile multi-view generation based on Adapter, and the authors' responses on our public comment. However, I want to discuss some of points more to clarify the contribution of this paper based on my understanding.

NVS-Adapter focuses on generating novel views from a single input view (NVS). In contrast, MV-Adapter addresses a much broader task, tackling multi-view consistent image generation with flexible conditions such as text, image, and geometric inputs.

Our NVS-Adapter also includes the experiments to generate multi-views and 3D assets also from user-provided text prompt.

This makes MV-Adapter a first adapter-based solution designed to meet a wide range of multi-view generation needs.

The authors should not claim that it is the "first" adapter-based solution as the originality, following ICLR's Code of Ethics, "Researchers should therefore credit the creators of ideas, inventions, work, and artefacts, and respect copyrights, patents, trade secrets, license agreements, and other methods of protecting authors' works." If the authors want to claim the contributions, comparing with our first version, this paper's main contributions should be elaborating and extending the adapter-based multi-view generation method, which is previously proposed.

Meanwhile, I strongly believe that our second version, published in ECCV'24, is a "natural" extension from the first version, not a proposal of new value. Note that the core value of our NVS-Adapter is not the number of experiments, but lies in"composability and generalization ability". In addition, the original paper of T2I Adapter clearly mentions "Further, the proposed T2I-Adapters have attractive properties of practical value, such as composability and generalization ability." in its abstract. Although our first version lacks enough experiments to show how to fully enjoy the "composability and generalization," our second version is a natural extension to add more experiments, keeping the original value "how adapter-based multi-view generation can support various scenarios without fine-tuning of existing T2I models."

3. Methodology

I agree that the paper has a difference in the model architecture, and I like the core idea of parallel attentions. Meanwhile, note that NVS-Adapter's View-Consistency Cross-Attention include self-attention of each target view, because learnable tokens aggregate all information of target views. I also agree that NVS-Adapter do not have camera and geometric conditions but only have camera conditions.

should be considered as a concurrent work according to ICLR's policy,

Note that the ICLR policy also recommend "Authors are encouraged to cite and discuss all relevant papers". In addition, I'm surprised that the authors failed to find our paper, although the "naming" and "contributions" have severe overlaps and our first version's paper and code has been available from the last year. For you reference, our "published" paper was available 52 days before the ICLR submission.

Dear Dr. Lee,

Thank you for your feedback. We will further clarify the contributions of our work as follows.

1. Claim of MV-Adapter as the First Systematic Adapter-Based Solution for Multi-View Image Generation

We acknowledge your work on the NVS-Adapter and its contributions to novel view synthesis from a single input view. Our claim that MV-Adapter is the first systematic adapter-based solution for multi-view image generation is based on the broader scope of the definition of multi-view generation task. While NVS-Adapter focuses primarily on generating novel views from a single input view, MV-Adapter addresses a more generalized multi-view generation task that incorporates flexible conditions such as text, image, and geometric inputs, positioning it as a pioneering solution in this more comprehensive context.

Although, as you mentioned, your work also includes experiments to generate multi-views from text prompts, the approach involves first generating images from text and then generating novel views from these images, where NVS-Adapter only works for the latter stage (i.e. novel view synthesis from single view). Therefore, NVS-Adapter serves as an adapter for a specific task that is novel view synthesis from a single view.

Moreover, we emphasize that the text-conditioned adapter significantly enhances flexibility and adaptability. It allows MV-Adapter to integrate with different base model variants, such as personalized text-to-image models, achieving more applicable multi-view personalization or customization. In contrast, NVS-Adapter shows inflexible due to the limitation to image conditions, which inherently impose stronger appearance constraints. This limitation reduces the applicatibility of the model, as the generated results rely heavily on the predefined visual characteristics of the input images. For example, when generating multi-view images from a single image, it struggles to produce corresponding styles or customized concepts even using personalized LoRA. In contrast, our MV-Adapter is capable of generating multi-view consistent and customized images, thereby providing greater benefits to the community.

2. On the Evolution of NVS-Adapter from Version 1 to Version 2

Regarding your assertion that version 2 of NVS-Adapter is a natural extension of version 1 and these two versions introduce the same advantages on adaptability, we respectfully disagree. Version 1 of NVS-Adapter lacks comprehensive adaptability experiments, which is difficult to reflect the characteristics of the "adapter". Therefore, version 2 is a significant improvement over version 1. Furthermore, even the adaptability experiments in version 2, such as compatibility with T2I extensions like ControlNet and LoRA, constitute only a portion of the adaptability aspects addressed in our MV-Adapter. Importantly, our work introduces a text-to-multiview adapter mechanism, a task setting that has not been addressed in your NVS-Adapter. This represents a significant advancement beyond what is covered in both versions of NVS-Adapter, thereby highlighting the novel contributions of our MV-Adapter.

3. Concurrent Work and Citation

We acknowledge the importance of recognizing concurrent work and adhering to ICLR’s policy on citing and discussing relevant papers. We regret any oversight in our initial submission and will ensure that our revised version includes a comprehensive discussion and citation of your work on NVS-Adapter. This will provide a clearer context for our contributions and highlight the distinct advancements our MV-Adapter offers.

Once again, thank you for your valuable feedback. We are committed to refining our work to accurately reflect its contributions and to honor the advancements made by your research.

Best regards,

The Authors