3DOT: Texture Transfer for 3DGS Objects from a Single Reference Image

We thank the reviewer for the positive feedback and the insightful questions. We address each of your questions in the following sections.

1. User Study Participant Size

We agree that increasing the number of participants would strengthen the significance of the user study results, and we will expand the participant pool in the revised version. Now, we append results of three new participants in the end.

2. Request for Video Demonstration

We agree that a video demonstration would enhance the presentation of our results. We will include video demos on our project GitHub and release them alongside the code.

Q1: Effect of CLIP-Based Token Initialization vs. Random Initialization

We conduct an ablation study to compare two initialization strategies for the prompt used in the CLIP-guided gradient mechanism:

• Random Initialization: The prompt is randomly initialized and then optimized during fine-tuning.
• CLIP-Based Initialization: The prompt is initialized by directly projecting the texture difference into CLIP space.

As shown in the table below, initializing the prompt with CLIP-based features leads to significantly better performance in terms of LPIPS (Alex), LPIPS (VGG), and CLIP Score:

This result indicates that CLIP-based initialization helps the prompt converge faster and guides the diffusion more effectively toward the desired texture characteristics, resulting in better texture fidelity and alignment with the reference.

Q2: Inference Speed Comparison

We compare the inference time of our method against two baselines: the Gaussian Splatting base model (GaussCtrl) and a representative NeRF-based approach. Our method maintains the efficiency benefits of Gaussian Splatting while introducing only modest additional overhead due to our view-consistency and prompt-tuning-based guidance.

In particular, the total additional time required by our proposed guidance components is approximately 8 minutes. Since our full pipeline involves two iterations of image editing, the added cost per iteration is around 4 minutes, which we consider efficient given the improvements in texture fidelity and consistency.

Appended User Study

We have added 3 new participants as shown in following tables:

We appreciate the detailed review and questions. Our responses are provided in the following sections.

1. Clarification on Connection Between Depth-based Generation and Initial Reference Image

We thank the reviewer for pointing out the ambiguity. We would like to clarify that during progressive generation stage, the depth-based generation process is not primarily driven by prompts, but instead is explicitly conditioned on the visual content of the single 2D reference image and depth. Specifically, we employ a depth-based ControlNet diffusion model that takes both the depth map and the latent embedding of the reference image as inputs. To preserve the texture information from the reference image, we use a k-step partial denoising strategy: the reference image is encoded and reversed into noise for k steps (we set k = 400), and then denoised starting from that noisy state (see lines 46, 128, and 141). This ensures that the exact texture and shape information from the reference image is retained, rather than synthesized anew from a text prompt.

Regarding the reviewer’s concern about whether the process reduces to “text-to-image-to-texture”, we clarify that this is only one optional reference acquisition mode and not central to our method. We further clarify Section 3.1:

• In the manual image-to-image mode, users paste the 2D reference texture image directly onto the object’s silhouette to create a reference image. (Image-to-image-to-texture).
• Further, we provide an automatic image-to-image mode, where users provide a 2D reference image (which may differ in shape), which is then processed via partial denoising conditioned on depth, without manual intervention. (Image-to-image-to-tetxture).

These reference images, regardless of how they are obtained, are then used in the diffusion process to propagate the texture across views, not regenerated from scratch using a prompt. Text-prompted initial reference generation is only utilized when there is no desired texture.

2. Use of Flipped Reference Image and Asymmetry Concern

Indeed, our method includes a flipped variant $F(I_\tau)$ to improve consistency across distant views (Section 3.1). This design implicitly assumes some degree of left-right symmetry, which does not always hold. We now clarify this limitation and explain how our method mitigates the potential issues:

• Mitigation via cross-attention: When the object is not symmetric, the flipped image contributes little due to low similarity ($Query \times Key$) in the cross-attention mechanism. The attention weights naturally diminish for dissimilar content, preventing flipped references from introducing significant errors.
• Ablation results: As shown in our experiment table below on the “bear” scene, the flipped reference improves performance for the symmetric (back) side, while slightly degrading it for the asymmetric (front/head) side. Importantly, the overall performance still improves, as shown by the net gains in LPIPS and stable CLIP scores. Note, lower LPIPS = better and higher CLIP score = better.
• Optional inclusion: The flipped reference is not strictly required. In scenarios involving strongly asymmetric objects, users or future extensions of our system could disable this component without affecting the core pipeline.

We will include a discussion of this limitation and tradeoff in the revised paper to improve clarity.

3. No Explicit Defination of $e$ in $Attn_{e,e}$

Thank you for pointing this out. Index $e$ refers to the image that is currently being edited.

4. Clarification on “Negative Prompt” Terminology in Eq.(2)

We agree that the terminology in lines 144–145 may be misleading. Our intention was to refer to the unconditioned branch of Classifier-Free Guidance, i.e., the absence of a prompt or a null prompt, not an explicit negative textual instruction. We will revise this phrasing in the final version to use the more precise term “unconditioned prompt” or “null conditioning”, in line with standard CFG terminology.

We thank the reviewer for the constructive feedback and thoughtful questions. Each point is addressed in detail in the following sections.

1. Clarification on Similarity with Multi-View Diffusion Attention Mechanisms

While our modified generative model may appear similar to existing multi-view diffusion methods at a high level, we emphasize that our task settings, constraints, and diffusion components are fundamentally different.

Specifically, our method takes as input a single 2D reference image and a fixed 3D Gaussian Splatting (3DGS) representation, and performs high-fidelity texture transfer onto this existing geometry. In contrast, multi-view diffusion approaches are typically designed to generate novel views or reconstruct 3D scenes from a few same texture input images, which are not anchored to an explicit 3D representation. These methods synthesize geometry implicitly through learned priors, which makes them unsuitable for editing tasks that require consistency with a given 3D structure. In addition, multi-view diffusion methods assume cross-view texture consistency, which does not hold in the texture transfer setting where the reference image and the 3D representation exhibit different textures. Directly applying them as input leads to collapsed reconstructions.

Regarding the designed diffusion components, our proposed view-consistency guidance and prompt-tuning-based guidance are training-free plugins while finetuning is required for modifications of existing multi-view diffusions.

We will revise the final version of the paper to make this distinction clearer, particularly in relation to the design and function of our attention mechanism within the 3D editing context.

2. Justification of Progressive Generation via Ablation Study

To evaluate the effectiveness of the proposed progressive generation process, we conduct an ablation study where we disable progressive view propagation and perform editing using only the initial reference image. In this ablated setting, each novel view is generated independently, without leveraging intermediate edited views. The results, averaged across several scenes, are summarized below:

The detailed ablation study results on progressive generation process are listed in following three tables:

These results demonstrate that the progressive generation process significantly improves texture fidelity (lower LPIPS) and semantic alignment with the reference (higher CLIP score). The improvement highlights the importance of propagating texture across neighboring views to enforce view consistency and mitigate artifacts from single-view editing.

3. Clarification on Pipeline and Texture Transfer Process

We acknowledge that the current presentation of our pipeline in the method section and figure could be clearer. We will revise both the pipeline figure and accompanying text in the revised paper to better illustrate the flow of the texture transfer process.

Specifically, our pipeline proceeds as follows: (1) We first obtain the reference image(s), either manually (e.g., by cropping/pasting) or automatically (e.g., via depth-conditioned generation). (2) The selected reference image is used to initialize the progressive generation process, where edited images for multiple views are produced using our proposed view-consistency and prompt-tuning-based gradient guidance. (3) These high-quality edited views are then used to fine-tune the 3D Gaussian Splatting (3DGS) representation. (4) We re-render the updated 3D object into new images and repeat steps (2) and (3) for further refinement.

Typically, two iterations are sufficient to achieve high-fidelity, view-consistent texture transfer across the full object. We will incorporate this clarified explanation and an updated pipeline figure in the revised paper to improve the overall readability of the method section.

4. No Definition of $e$

Thanks for pointing this out. Index $e$ refers to the image that is currently being edited.

5. Clarification of Figure 2 (Pipeline)

We agree that certain components of Figure 2 could be made more informative. Specifically:

(1) We will revise the leftmost and bottom panels to better convey the reference image acquisition process and the rationale behind using flipped variants and sparse cross-attention.

(2) We also acknowledge that the rightmost subfigure (showing the iterative editing loop) lacks sufficient explanation in the main text. We will revise the manuscript to clarify that:

• The reference image is obtained either manually or via depth-conditioned generation.
• It is used to initiate the progressive generation stage, guided by both view-consistency and prompt-tuning-based gradient guidance.
• The resulting edited images are then used to fine-tune the 3D Gaussian Splatting representation.
• This process is iterative: we re-render the updated 3D objects and apply the same editing process again. Typically, two iterations are sufficient to achieve high-quality texture transfer.

We will revise both the figure and the method description accordingly to ensure that these steps are clearly communicated.

Q1: Clarification on “3D Objects” and Generalizability Across Modalities

We agree that the term “3D objects” in the title may be too broad, and we are open to revising it to “3DGS-OTT: Texture Transfer for 3D Gaussian Splatting from a Single Reference Image” to more accurately reflect our current implementation.

While our experiments focus on 3D Gaussian Splatting (3DGS) due to its compatibility with GaussCtrl and efficient rendering, the core contributions of our method: progressive generation, view-consistency gradient guidance, and prompt-tuning-based gradient guidance, are representation-agnostic. They operate independently of the underlying 3D format and can be adapted to other modalities such as NeRF, point clouds, or meshes, by treating diffusion-based generation and 3D model finetuning as two decoupled stages.

We also plan to release a modular version of our pipeline that includes only the diffusion-based editing stage, allowing users to integrate it with other 3D representations depending on their downstream applications.

Q2 (Multi-view Diffusion Methods) & Q3 (Ablation for Progressive Generation):

We believe these questions are addressed in our responses to previous discussions on the distinction between our progressive generation and existing multi-view diffusion methods, and where we provide an ablation study validating the effectiveness of the progressive generation process.

Thank you for your positive feedback. We’re glad to hear that your major concerns have been addressed, including (i) the difference between Progressive Generation and existing multi-view diffusion methods, and (ii) the ablation study for Progressive Generation.

We also agree with your suggestions regarding the title, Figure 2, and other points, and plan to incorporate them into the updated version. Thanks again for taking the time to review our rebuttal. We appreciate your support during the review process.

Should the revisions meet your expectations, we would greatly appreciate your consideration in raising your final score.

We thank the reviewer for the thoughtful comments and the positive feedback. Detailed responses for each question are provided in the following sections.

1. Originality Concern

While our method builds upon existing diffusion and prompt-based techniques, we believe our contributions offer meaningful and novel extensions for the 3D texture editing setting. In particular, we propose three key innovations:

a) Progressive Generation Process: To the best of our knowledge, our method is the first to introduce a progressive generation strategy in the context of 3D texture editing. Unlike existing multi-view diffusion methods that generate all views in parallel, our approach incrementally propagates texture edits across views by using previously edited views as reference inputs. This step-wise refinement promotes spatial coherence, enhances view consistency, and supports integration with 3D Gaussian Splatting.

b) View-Consistency Gradient Guidance: We propose a novel classifier-free guidance format formulation that explicitly incorporates multi-view coherence signals. This is achieved by comparing multi-view-conditional and unconditional noise predictions, allowing us to extract and reinforce view-consistent information during the diffusion process. To our knowledge, this idea has not been applied in diffusion-based 3D editing.

c) Prompt-Tuning-Based Gradient Guidance: While prompt tuning is a well-established technique, we propose a novel application: encoding the texture difference between the original object and the reference image as a learned prompt token. We further extract the texture difference information by the proposed equation and apply it to guide the diffusion process in a consistent stylistic direction. We believe this is the first use of prompt tuning for controlling texture appearance in a 3D-aware diffusion editing pipeline.

In summary, we introduce three new mechanisms that are carefully tailored to address the unique challenges of 3D texture transfer, specifically view consistency and faithful texture preservation, which standard components fail to address. We believe these innovations constitute a non-trivial contribution to the field.

2. Dark Border Artifact Around Edited Regions

The dark border artifact is primarily caused by the use of language-based object segmentation methods (e.g., LangSAM) to generate masks for isolating objects in the reference views. These segmentation methods often include a narrow band of background pixels near object boundaries due to imperfect boundary localization.

As a result, during the diffusion-based editing stage, this narrow band is misinterpreted as valid texture content, leading to the appearance of dark borders in the final renderings. To mitigate this, we apply dilation and Gaussian blurring to soften mask boundaries and reduce the visibility of such artifacts. While these steps alleviate the issue to some extent, traces may still remain in some challenging cases.

We acknowledge this segmentation field limitation and plan to address it more thoroughly in future work by incorporating more advanced and boundary-aware segmentation techniques.

Q1: Geometry Differences Between Reference Images and 3D Objects

Geometry mismatches between the 2D reference image and the 3D object can occur when the reference is generated via a depth-conditioned diffusion model with a small condition scale factor. We address this in two common scenarios:

(a) Slightly Different Geometry: In this case, the reference image shows small geometric deviations (e.g., slight pose or scale differences), such as a standing bear with legs in a slightly different position. Our method is robust to such differences due to the following mechanisms:

• During progressive generation, texture features are extracted via cross-attention while being constrained by both the depth map and the partially reversed latent features which preserve appearance clues of the original image. This encourages edits that align with both the 2D reference and 3D structure.
• During 3D fine-tuning, view-consistent textures are enforced across adjacent edited views. Overlapping regions from neighboring views iteratively correct geometry discrepancies and reinforce accurate texture transfer.

(b) Significantly Different Geometry: If the reference image shows a completely different shape (e.g., a running bear instead of a standing one), transfer quality may degrade, as discussed in Section 4.4 of our paper. However, such poor references typically arise from failure in the depth-conditioned option generation stage. In practice, users can easily detect these cases during the reference selection stage (which is interactive), and regenerate better references using our partial denoising strategy, which conditions on depth and corrects shape biases during generation.

In summary, our method is robust to minor geometric mismatches and provides mechanisms to mitigate large discrepancies through reference regeneration and iterative correction during 3D fine-tuning.

Dear Reviewer qSJ7, thank you for your initial review. They were insightful and helped us strengthen the paper. We just want to check if our response has addressed your points. If any additional clarification or supporting results are needed, we’d be happy to provide them. We appreciate your positive feedback, and we hope the rebuttal we’ve provided has resolved your concerns.

Thank you for your efforts of reviewing! Please remember to read the rebuttal as soon as possible, and discuss with the authors for questions not resolved. According to the guideline this year, reviewers need to participate in discussions with authors before submitting “Mandatory Acknowledgement”.