GuideFlow3D: Optimization-Guided Rectified Flow For Appearance Transfer

We thank Reviewer zJCJ for their positive feedback, highlighting the high quality of our results, the well-written and easy to follow presentation, and the interesting and promising nature of our training-free, universal guidance-based approach to the important task of editing 3D assets… vital for large-scale applications. Below we address all concerns:

Q1: My main concern is that the baselines are too weak. Though there are not many 3D appearance transfer works, there are tons of works that excel on 2D style transfer.

Following reviewer suggestions, we added three new baselines: (1) EASI-Tex (SIGGRAPH 2024) [1] is a conditional diffusion-based method that uses edge-aware conditioning and ControlNet to texture an existing 3D mesh from a single RGB image, (2) Cross Image Attention (SIGGRAPH 2024) [2] builds upon the self-attention layers of generative models and introduces a cross-image attention mechanism that implicitly establishes semantic correspondences across images; (3) Mamba-ST (WACV 2025) [3] adapts the behavior of State Space Models to simulate the behavior of cross-attention layers, which are able to combine two separate embeddings into a single output. Please note that (2) and (3) are 2D style transfer methods which we adapt to our use-case following your suggestion. We conduct an experiment with image condition using $\mathcal{L}_{\text{appearance}}$ on the simple-complex set (same setting as Main Paper: Tab. 1). Below are the results:

GuideFlow3D achieves strong improvements in both appearance transfer and structural clarity preservation. This demonstrates the ability of our approach to effectively translate visual and semantic cues into coherent, high-fidelity textures across different 3D objects. We also conducted a user study to measure the correlation between the LLM’s judgments and those of human evaluators and found that GPT-4 performs comparably to humans. Please refer to our response to Reviewer dDnT Q1 for a detailed explanation. We will add these results alongside comprehensive benchmarking evaluation and qualitative comparison with the baselines on all of our experimental settings to the final manuscript.

We appreciate your suggestions for adapting 2D style transfer methods to 3D and, if desired, we will also provide the text-based results during the discussion phase, as a way to better situate the context of our work in the evolving editing landscape.

Q2: I do not fully get what the baseline "TRELLIS" in L258 means.

Since TRELLIS supports flexible local editing based on their manuscript, the idea of this baseline is to understand how the pretrained model performs without any guidance. We pass the input image/text condition and the active voxels intersecting with the surface of the input 3D object mesh (structure) to the structured latents (SLATs) generation (second stage) of TRELLIS model to generate the output 3D asset.

[1] Perla, Sai Raj Kishore et al. "EASI-Tex: Edge-Aware Mesh Texturing from Single Image", SIGGRAPH 2024.

[2] Alaluf, Yuval et al. "Cross-Image Attention for Zero-Shot Appearance Transfer", SIGGRAPH 2024.

[3] Botti, Filippo et al. "Mamba-ST: State Space Model for Efficient Style Transfer", WACV 2025.

We thank Reviewer 2LHX for highlighting the detailed explanations for easy reimplementation, good visual quality, and noting that the in-the-wild appearance transfer in the Appendix (Fig 4) looks great. Below, we address all concerns:

Q1: The authors could adopt the editing method from another paper (even though the object representation could be different) and make comparison with the proposed method.

Following reviewer suggestions, we added three new baselines: (1) EASI-Tex (SIGGRAPH 2024) [1] is a conditional diffusion-based method that uses edge-aware conditioning and ControlNet to texture an existing 3D mesh from a single RGB image, (2) Cross Image Attention (SIGGRAPH 2024) [2] builds upon the self-attention layers of generative models and introduces a cross-image attention mechanism that implicitly establishes semantic correspondences across images; (3) Mamba-ST (WACV 2025) [3] adapts the behavior of State Space Models to simulate the behavior of cross-attention layers, which are able to combine two separate embeddings into a single output. Please note that (2) and (3) are 2D style transfer methods which we adapt to our use-case following the suggestion of Reviewer zJCJ. We conduct an experiment with image condition using $\mathcal{L}_{\text{appearance}}$ on the simple-complex set (same setting as Main Paper: Tab. 1). Below are the results:

GuideFlow3D achieves strong improvements in both appearance transfer and structural clarity preservation. This demonstrates the ability of our approach to effectively translate visual and semantic cues into coherent, high-fidelity textures across different 3D objects. While our method leverages the pretrained Trellis model, the core novelty lies in our training-free inference-time optimization framework, which combines universal guidance with structured latent flow sampling. This is enabled by our two proposed guidance objectives—part-aware appearance loss and self-similarity loss—specifically tailored to handle geometric dissimilarity in 3D appearance transfer. The strong performance over both 2D and 3D baselines empirically validates this contribution. We also conducted a user study to measure the correlation between the LLM’s judgments and those of human evaluators and found that GPT-4 performs comparably to humans. Please refer to our response to Reviewer dDnT Q1 for a detailed explanation. We will add these results alongside comprehensive benchmarking evaluation and qualitative comparison with the baselines on all of our experimental settings to the final manuscript.

Q2: How is the scene editing in Scene Appendix C implemented? Do you use GT pose or another off-the-shelf pose estimator?

We use an off-the-shelf CAD model retrieval and alignment algorithm to get per-object pose estimations and obtain 3D mesh geometries of objects in ScanNet [4, 5].

Q3: Will object categories have big impacts on co-segmentation based objective?

While the co-segmentation based objective is designed using PartField, in practice, we get the PartField feature vectors per 3D point/voxel and run K-means clustering. Instead of PartField features, we can also use SLAT features for clustering (Main Paper: Tab. 3, Ablation Study). While a slight performance drop is expected, our method does not rely on 1-1 correspondences, more on approximate part matching across objects. As shown with in-the-wild appearance transfer results (Appendix: Fig 4), we are able to support diverse object categories, for example, patterns from the body of the giraffe are transferred to the seat, arms, and backrest of the chair, while the appearance of giraffe legs is transferred to chair legs.

Q4: I would suggest adding some image pairs in Fig 4 of Appendix to the main paper.

Thank you for the suggestion, we will revise the final manuscript with some in-the-wild appearance transfer results in the main paper.

[1] Perla, Sai Raj Kishore et al. "EASI-Tex: Edge-Aware Mesh Texturing from Single Image", SIGGRAPH 2024.

[2] Alaluf, Yuval et al. "Cross-Image Attention for Zero-Shot Appearance Transfer", SIGGRAPH 2024.

[3] Botti, Filippo et al. "Mamba-ST: State Space Model for Efficient Style Transfer", WACV 2025.

[4] Stekovic, Sinisa et al. "Pytorchgeonodes: Enabling differentiable shape programs for 3d shape reconstruction." CVPR 2025.

[5] Ainetter, Stefan et al. "Automatically annotating indoor images with cad models via rgb-d scans". WACV 2023.

We thank Reviewer dDnT for their detailed review and for recognizing the high significance and clear motivation of our work, the originality of our training-free framework, and the clever and powerful nature of our interleaved optimization-flow scheme. We also appreciate the acknowledgment of our well-motivated and tailored guidance functions and the flexibility our method offers by avoiding the need for costly retraining or complex architectural modifications. Below, we address the concerns:

Q1: Reliance on an Un-validated and Potentially Biased Evaluation Metric.

Traditional metrics like CLIP similarity, LPIPS, or FID fail to capture semantic consistency, geometry-aware stylization, or perceptual realism in cross-category 3D appearance transfer [1]. Recent works across 3D generation and stylization tasks have shown that GPT-4V exhibits strong alignment with human preferences [50, 67, 43, 76], and is increasingly adopted as a practical tool for perceptual evaluation in absence of ground truth. However, we agree that without rigorous validation, relying solely on an LLM can be a major weakness. Based on the suggestion, we conducted a user study on Amazon Mechanical Turk with 21 participants using 51 randomly selected object and image pairs from the simple-complex evaluation set, on the image condition setting. The participants were shown the appearance image and two views (front and back at a camera angle of 45 degrees) each of the structure mesh + outputs from different methods, without any knowledge about the methods. Then, they were provided with the same prompts as GPT (Appendix: Fig 6) and asked to rate the best performing considering overall quality. Based on 1070 responses, GuideFlow3D achieved the highest win rate (34.8%) compared to Cross Image Attention (26.7%), EASI-Tex (21.6%), and Mamba-ST (16.9%), consistent with the GPT-4V rankings (please refer to Q2 for a discussion on the baselines and GPT evaluation). These findings validate the reliability of our LLM-based evaluation, demonstrating strong alignment with human perception in assessing texture fidelity, structural preservation, and overall appearance quality. We will include a summary of this user study in the main paper and provide full details in the appendix.

Q2: Limited Comparison to State-of-the-Art Baselines.

Following reviewer suggestions, we added three new baselines: (1) EASI-Tex (SIGGRAPH 2024) [1] is a conditional diffusion-based method that uses edge-aware conditioning and ControlNet to texture an existing 3D mesh from a single RGB image, (2) Cross Image Attention (SIGGRAPH 2024) [2] builds upon the self-attention layers of generative models and introduces a cross-image attention mechanism that implicitly establishes semantic correspondences across images; (3) Mamba-ST (WACV 2025) [3] adapts the behavior of State Space Models to simulate the behavior of cross-attention layers, which are able to combine two separate embeddings into a single output. Please note that (2) and (3) are 2D style transfer methods which we adapt to our use-case following the suggestion of Reviewer zJCJ. We conduct an experiment with image condition using $\mathcal{L}_{\text{appearance}}$ on the simple-complex set (same setting as Main Paper: Tab. 1). Below are the results:

GuideFlow3D achieves strong improvements in both appearance transfer and structural clarity preservation. This demonstrates the ability of our approach to effectively translate visual and semantic cues into coherent, high-fidelity textures across different 3D objects. We will add these results alongside comprehensive benchmarking evaluation and qualitative comparison with the baselines on all of our experimental settings to the final manuscript.

Q3: Narrow Experimental Scope and Unsubstantiated Generalization.

We agree that generalizability is important and appreciate the reviewer noting the quality of our open-domain examples. Our focus on furniture categories in the main paper is intentional, driven by practical downstream applications (e.g., AR/VR, asset libraries) where editable 3D meshes are commonly used. This also enables controlled, semantically coherent benchmarks, making user studies and GPT-based evaluations more interpretable and consistent. We also provide in-the-wild transfer results (Appendix: Fig 4) across diverse categories demonstrating that GuideFlow3D generalizes to semantically and geometrically dissimilar objects. We will make this motivation clearer in the revised manuscript.

Q4: To better situate this work, could you please elaborate on the relationship between GuideFlow3D and other recent state-of-the-art 3D style transfer methods like StyleRF or StyleGaussian?

Recent methods like StyleRF [4] and StyleGaussian [5] represent important progress in 3D style transfer. However, these approaches operate on implicit or point-based representations (NeRFs, Gaussians), whereas GuideFlow3D works on explicit textured meshes, enabling direct editing, part-aware control, and downstream usability. Notably, StyleGaussian stylizes only color while keeping geometry fixed, limiting its ability to reflect fine structural patterns. It also scales poorly with the number of Gaussians due to memory and compute constraints. In contrast, GuideFlow3D offers geometry-aware stylization with training-free inference, and fixed compute cost via structured latent sampling. We will revise the related works section with such conceptual tradeoffs, highlighting how mesh-based controllable appearance transfer can be useful.

Q5: How does GuideFlow3D’s performance degrade if the segmentation quality is poor?

While the co-segmentation based objective is designed using PartField, in practice, we get the PartField feature vectors per 3D point/voxel and run K-means clustering. Instead of PartField features, we can also use SLAT features for clustering (Main Paper: Tab. 3, Ablation Study). While a slight performance drop is expected, our method does not rely on 1-1 correspondences, more on approximate part matching across objects. As shown with in-the-wild appearance transfer results (Appendix: Fig 4), we are able to support diverse object categories, for example, patterns from the body of the giraffe are transferred to the seat, arms, and backrest of the chair, while the appearance of giraffe legs is transferred to chair legs. However, in extreme diversity cases with no semantic meaning between two meshes, the co-segmentation can fail due to inconsistent features, either SLAT or PartField. We will add qualitative analysis of co-segmentation failure cases to our manuscript for the final version.

Q6: How does the self-similarity loss handle such cases, and does it risk homogenizing the texture across all geometrically similar regions, even if the text prompt implies variation?

$\mathcal{L}_{\text{structure}}$ encourages intra-cluster consistency and inter-cluster contrast in appearance by operating on geometry-derived feature clusters. However, it does not enforce global texture uniformity across all geometrically similar parts. Instead, the contrastive formulation (Eq. 6) preserves local appearance consistency while allowing global variation across different clusters. Clustering is performed in the latent geometry-feature space, which captures more than raw shape similarity (e.g., spatial location, curvature context). In practice, this often separates semantically distinct parts even if they share structural traits.

[1] Perla, Sai Raj Kishore et al. "EASI-Tex: Edge-Aware Mesh Texturing from Single Image", SIGGRAPH 2024.

[2] Alaluf, Yuval et al. "Cross-Image Attention for Zero-Shot Appearance Transfer", SIGGRAPH 2024.

[3] Botti, Filippo et al. "Mamba-ST: State Space Model for Efficient Style Transfer", WACV 2025.

[4] Liu, Kunhao, et al. "Stylerf: Zero-shot 3d style transfer of neural radiance fields." CVPR 2023.

[5] Liu, Kunhao, et al. "Stylegaussian: Instant 3d style transfer with gaussian splatting." SIGGRAPH Asia 2024.

Thank you for acknowledging our efforts and raising your score. We appreciate all your comments.

Regarding evaluation bias, we would like to note that we conducted a full-scale unbiased study on Amazon Mechanical Turk (AMT) with 21 participants rating 51 examples, using the same rubric as GPT-4. The rankings were consistent with our initial LLM-based evaluation, confirming that our setup is unbiased and human-aligned, similar to recent work in text/image-to-3D generative domain [1, 2, 3].

We chose furniture due to its high structural variability, for example, chairs and tables, have different numbers of legs, shapes (e.g, an armchair vs. an office chair), and supports (e.g, round coffee table vs. a rectangular work desk). This makes part correspondence and appearance transfer particularly challenging. In contrast, categories like animals or vehicles often have more regularized part structure, with most variation occurring in appearance rather than geometry. Results in Appendix: Fig. 4 showcase strong generalization beyond furniture-to-furniture scenarios.

As Reviewer zJcJ suggested, we will include more qualitative results to provide readers with a comprehensive understanding of the outcomes across different approaches. We welcome any suggestions you may have on improving the two aspects of the paper you mentioned.

[1] Peng, Yuang, et al. "Dreambench++: A human-aligned benchmark for personalized image generation." ICLR 2025.

[2] Wu, Tong, et al. "Gpt-4v(ision) is a human-aligned evaluator for text-to-3d generation." CVPR 2024.

[3] Maiti, Shalini, et al. "Gen3deval: Using vllms for automatic evaluation of generated 3d objects." CVPR 2025.

We thank Reviewer nLwK for finding our paper clearly structured and our method's training-free nature, highly flexible for diverse 3D appearance transfer. Below, we address all the questions:

Q1: The core methodology of GuideFlow3D can be perceived as an incremental combination of existing techniques.

While guidance is a well-established topic in 2D denoising models, our work extends this concept for transfering appearance (texture and geometric details) between two objects in 3D. Our guidance formulation repurposes a given generative model for a new task (different from the one it was originally trained for) in order to leverage the inductive bias of a pretrained rectified flow model. We propose a novel inference-time mechanism that interleaves rectified flow sampling with latent-space optimization, incorporating custom guidance losses into the denoising trajectory of structured latents. This formulation enables conditioning the generative process without retraining or modifying the base model, extending universal guidance in 3D. This approach elegantly avoids the need for costly retraining or complex architectural modifications, e.g., ControlNet-like adapters (Reviewer dDnT), positioning it as a flexible, training-free solution (Reviewer dDnT, zJCJ). The proposed loss functions are tailored to the specific use case of 3D appearance transfer, by establishing a semantically meaningful part-aware correspondence and preserving intrinsic structures in absence of explicit geometric supervision (Reviewer dDnT).

Q2: A major limitation is the absence of direct comparisons with recent state-of-the-art 3D appearance and style transfer methods.

While we compare with Trellis to show how guidance can improve its local editing capabilities to support 3D appearance transfer, we understand that it is challenging to ascertain competitive advantage without benchmarking against recent style transfer works. Direct comparison with StyleRF [1] and StyleGaussian [2] is not feasible due to fundamental differences in representation and output format—these methods operate on implicit 3D scenes (NeRFs, Gaussians) and produce render-only outputs, while GuideFlow3D works with explicit textured meshes. Additionally, they rely on per-instance optimization and often require multi-view supervision, whereas our method is training-free and fast at inference (please also refer to our response towards Reviewer dDnT Q4 for an explanation on how we can inherently address these two methods' limitations). While TEXTure [3] tackles the same task as us, we compare with EASI-Tex [4], a more recent faster and scalable approach shown to perform better than TEXTure.

Following reviewer suggestions, we added three new baselines: (1) EASI-Tex (SIGGRAPH 2024) [4] is a conditional diffusion-based method that uses edge-aware conditioning and ControlNet to texture an existing 3D mesh from a single RGB image, (2) Cross Image Attention (SIGGRAPH 2024) [5] builds upon the self-attention layers of generative models and introduces a cross-image attention mechanism that implicitly establishes semantic correspondences across images; (3) Mamba-ST (WACV 2025) [6] adapts the behavior of State Space Models to simulate the behavior of cross-attention layers, which are able to combine two separate embeddings into a single output. Please note that (2) and (3) are 2D style transfer methods which we adapt to our use-case following the suggestion of Reviewer zJCJ. We conduct an experiment with image condition using $\mathcal{L}_{\text{appearance}}$ on the simple-complex set (same setting as Main Paper: Tab. 1). Below are the results:

GuideFlow3D achieves strong improvements in both appearance transfer and structural clarity preservation. This demonstrates the ability of our approach to effectively translate visual and semantic cues into coherent, high-fidelity textures across different 3D objects. We will add these results alongside comprehensive benchmarking evaluation and qualitative comparison with the baselines on all of our experimental settings to the final manuscript.

Q3: The prompts and scenarios used in the experimental evaluation lack complexity and diversity.

The main paper focuses on controlled intra- and inter-category furniture benchmarks to enable systematic evaluation. However, we also show the generalizability of GuideFlow3D in open-domain scenarios (Appendix D: In-the-wild Appearance Transfer). These examples involve transferring appearance across semantically and geometrically dissimilar object categories such as giraffe → chair and cabinet → airplane covering varied object domains including animals, tools, and vehicles (Appendix: Fig. 4). Despite the increased complexity and lack of semantic alignment, our method successfully preserves material styles, maintains structural coherence via part-aware matching in the latent feature space, and produces realistic outputs. This robustness is also noted by reviewers, the in-the-wild appearance transfer in the Appendix looks great (Reviewer 2LHX), and the method tackles an important task… vital for large-scale applications (Reviewer zJCJ), highlighting GuideFlow3D’s scalability towards complex meshes and scenarios.

Q4: The results shown typically use relatively clean input 3D meshes and clear appearance conditions.

Our evaluation benchmark is designed for demonstrating content creation possibilities, hence we use clean input meshes to have a fair measure of performance. We also demonstrate the application of GuideFlow3D to real-world scenes in Appendix C: Scene Editing, where it stylizes reconstructed ScanNet objects with pose noise, occlusion, and class ambiguity. Without retraining or scene-level modeling, our method applies object-wise appearance transfer to maintain stability across pose and mesh variation highlighting its suitability for practical AR/VR scenarios and digital asset creation.

[1] Liu, Kunhao, et al. "Stylegaussian: Instant 3d style transfer with gaussian splatting." SIGGRAPH Asia 2024.

[2] Liu, Kunhao, et al. "Stylerf: Zero-shot 3d style transfer of neural radiance fields." CVPR 2023.

[3] Richardson, Elad, et al. "TEXTure: Text-Guided Texturing of 3D Shapes". SIGGRAPH 2023.

[4] Perla, Sai Raj Kishore et al. "EASI-Tex: Edge-Aware Mesh Texturing from Single Image", SIGGRAPH 2024.

[5] Alaluf, Yuval et al. "Cross-Image Attention for Zero-Shot Appearance Transfer", SIGGRAPH 2024.

[6] Botti, Filippo et al. "Mamba-ST: State Space Model for Efficient Style Transfer", WACV 2025.