Wukong's 72 Transformations: High-fidelity Textured 3D Morphing via Flow Models

We thank the reviewer for the constructive comments. We address the concerns below.

Question 1: How is the cost matrix for the Wasserstein barycenter computed between source and target token sets? Is it Euclidean distance in CLIP/DINOv2 space? Have you tried cosine or learned distances?

A1: The cost matrix is computed using the squared Euclidean distance in the CLIP (for text) or DINOv2 (for image) embedding space. We also experimented with cosine distance(i.e., $C_{i j}=1-\cos \left(x_i, y_j\right)$ ). Using the same evaluation pipeline described in Section 4.1.1, the quantitative results for FID, PPL, and V-CLIP are 4.37, 2.93, and 0.90, respectively, which are worse than using Euclidean distance. Learning the distance via a network might be a good direction. However, we don't have enough (or such) datasets to support this type of training. We will continue to explore this in the future and would appreciate any suggestions from the reviewer.

Question 2: Can you elaborate on the optimization solver used for computing the barycenter? Are you using LP, Sinkhorn, or other entropic OT solvers?

A2: We use the free-support Wasserstein barycenter solver from the POT library (ot.lp.free_support_barycenter), which is based on linear programming (LP). We also experimented with Sinkhorn-based barycenters, but found that Sinkhorn-based barycenters tend to blur the interpolants, especially in early morphing stages. LP-based methods provide more spatially coherent transitions with better geometry preservation.

Question 3: What happens when the source and target shapes differ significantly in topology or part structure (e.g., human ↔ airplane)? Are there failure cases?

A3.1: In practice, our method is able to handle significant topological and structural differences between source and target shapes in many cases. For instance, as shown in Figure 3, morphing from “Elephant” to “Pyramid”, or “Digger” to “Pirate Ship”. Another example is included in Appendix Figure A5, showing a successful morphing from “Wukong” to “Train”, which again involves a major shift in object class and part layout. More videos in the form of rotating videos can be found in the supplementary demo video.

A3.2: Our method may struggle in cases involving explicit splitting or merging of semantic parts. A representative failure case is “Sphere” → “Dining Table with Chairs”: The source is a single-mass geometry, while the target consists of multiple, semantically distinct components. During morphing, the model fails to resolve how to meaningfully split the sphere into independent regions. As a result, the intermediate shapes appear blobby or tangled, with chairs fusing into the tabletop or floating detached from the structure.

Question 4: How sensitive is your method to the choice of pre-trained encoder (e.g., CLIP vs. DINOv2) or the underlying 3D generator (Trellis)?

A4: Our method is designed to be modular and works with different encoders and generators. We tested both CLIP (for text prompts) and DINOv2 (for image prompts), and observed consistent performance. For 3D generators, we implemented our pipeline on both Trellis, TripoSG (Appendix Table A1), and GaussianAnything (Rebuttal Table "Quantitative comparison with GaussianAnything as backbone"); results show that our interpolation strategy consistently improves morphing quality across both backbones, confirming low sensitivity to the specific pretrained components.

Weakness 1:Limited novelty in core technique: Using Wasserstein barycenters for shape or texture interpolation is not new.

A5: We thank the reviewer for pointing out the rich history of optimal transport (OT) methods in shape and style interpolation. Indeed, works such as Solomon et al. (2015) and Kolkin et al. (2019) have shown the power of OT and Wasserstein barycenters in both explicit geometric spaces (e.g., meshes, voxel grids) and 2D image features.

Our contribution is not simply the use of barycenters, but in adapting OT within a conditional 3D latent generative process:

• We operate not on mesh vertices, but on condition token distributions from multimodal encoders (CLIP / DINOv2) inside a flow-based 3D generative pipeline—a fundamentally different domain with different constraints.
• Prior OT-based morphing methods assume static geometric domains or shallow features. In contrast, our barycenter interpolation drives high-resolution textured 3D synthesis, integrated with recursive dynamics.
• Furthermore, our method is training-free, leveraging generative priors while avoiding the need for curated training pairs

Weakness 2: Heuristic design choices: The recursive initialization strategy and similarity-guided interpolation heuristics, though empirically effective, lack theoretical justification or ablation on robustness. The texture fusion rule based on cosine similarity threshold feels ad hoc. How sensitive is the result to this threshold?

A6: Recursive initialization is motivated by practical instability in latent barycenter optimization, particularly in high-dimensional latent token spaces. Linear initialization often falls into non-smooth or semantically invalid regions, as shown in our ablation (Fig. 6 and 8). The recursive method promotes trajectory continuity—a concept inspired by gradient flows on Wasserstein space.

We conduct quantitative evaluations with different threshold τ values and present the results below:

We observe that the performance is robust across a reasonable range of thresholds (0.2-0.8). We set a default threshold 0.3 in our evaluation.

Weakness 3: Dependency on pre-trained components: The success of WUKONG is highly dependent on the quality and generalizability of the pre-trained encoder and 3D generator (e.g., Trellis). It is unclear how well the method generalizes to out-of-distribution prompts or categories not seen during Trellis’ training.

A7: We appreciate the reviewer’s concern. We agree that the quality and bias of pretrained components (e.g., CLIP, DINOv2, Trellis) inevitably affect generalization, especially for out-of-distribution or long-tail categories. That said, this dependency is inherent to all current prompt-driven 3D generation frameworks, including prior works like 3DRM, which also rely on a pretrained generator (e.g., GaussianAnything). Our method introduces no additional trainable components, which makes it lightweight, modular, and fully compatible with future generative backbones. As newer and more generalizable models emerge, our morphing framework can benefit directly without retraining.

To test real-world generalization, we applied our method to the Headspace dataset containing high-quality real 3D face scans. Even though Trellis was not trained on such data, our method achieved strong results:

• FID: 3.97, PPL: 2.88, V-CLIP: 0.91 —outperforming both MorphFlow and 3DRM under the same evaluation protocol (see "Quantitative results on Headspace dataset").

These results demonstrate that despite its reliance on pretrained components, WUKONG exhibits robust generalization to real 3D data and is well-positioned to support broader domains as pretrained models continue to improve.

Limitations 1: The proposed method performs well under curated prompts and well-aligned source/target categories, but might struggle with topologically dissimilar or fine-grained prompt changes.

A8: Please refer to A3 for a more detailed discussion on our method's ability to handle topologically dissimilar shapes/cross category objects. In practice, our method has successfully managed significant structural differences in several cases, such as “Elephant” to “Pyramid” and “Digger” to “Pirate Ship” (Figure 3). These examples demonstrate our method’s robustness to topological changes.

Limitations 2: The morphing pathway is driven purely by token interpolation in feature space, which may lack semantic control (e.g., preserving pose vs. identity).

A9: While the core of our method relies on token interpolation in feature space, we designed this approach with the goal of maintaining both structural coherence and texture fidelity across morphs. The selective interpolation strategy enables us to preserve key features (e.g., identity) while controlling texture blending more accurately. However, we agree that pose preservation is a critical concern in 3D morphing. We plan to explore introducing pose priors or a pose encoder to anchor the morphing process.

Limitations 3: Heavy reliance on pre-trained backbones makes it hard to decouple where the quality improvements come from—WUKONG's algorithm vs. the generator or encoder.

A10: Our core contributions lie in the novel morphing strategy (e.g., Wasserstein barycenter-based interpolation, recursive initialization, and selective texture interpolation). These algorithms provide significant performance improvements even when the backbone is swapped. To clarify this, we implemented "Ours*" on the GaussianAnything backbone (the same generator used by 3DRM), and showed that our method outperforms 3DRM even when both models use the same backbone (Rebuttal Table "Quantitative comparison with GaussianAnything as backbone"), demonstrating that the improvements come from our morphing strategy rather than just the backbone itself. We will clarify these contributions further in the revised version of the paper by:

• Including additional ablations to isolate the impact of our algorithm versus the pretrained encoder and generator.
• Providing comparative performance when using different backbones to demonstrate the generality of our approach.

We appreciate the reviewer’s feedback. We are pleased that the high-quality 3D results and efficient generation time of our method were recognized, highlighting its potential for 3D asset generation in graphics pipelines. We address the concerns below.

Question 1: Prior method 3DRM [4] uses the CLIP-conditioned 3D generator GaussianAnything [5] as a backbone. Would the authors be able to run their proposed method using [5], so that we can compare apples-to-apples results with [4]?

A1: Thanks for this important question. To ensure a fair, apples-to-apples comparison with 3DRM [4], we implemented our full morphing method on top of the GaussianAnything framework [5]—the same 3D generator used in 3DRM. We denote this variant as “Ours*” in the table below.

Across all evaluation metrics, including FID, PPL, V-CLIP, and GPT-based perceptual scores (STP-GPT, SEP-GPT), our method consistently outperforms 3DRM, even when both share the exact same backbone. This clearly demonstrates that the improvement is not solely due to the use of a stronger generator like Trellis, but rather stems from our core morphing algorithm. Note that the GPT-based results may differ from those presented in Sec. 4.1.1, as they are computed using only the four methods reported here.

Quantitative comparison with GaussianAnything as backbone

Question 2: Would the authors be able to add quantitative ablation results to validate the selective interpolation strategy, into table A1?

A2: The reference in line 295 was intended to refer to an ablation analysis that was unfortunately omitted from the appendix in the initial version. We apologize for the confusion. To address this, we conducted quantitative ablation experiments under the same evaluation protocol described in Sec. 4.1.1, and compared three settings:

• W/o interp: Directly copying the source texture tokens across all steps without any interpolation.
• Linear: Applying standard linear interpolation between source and target texture tokens.
• Ours: Using our proposed selective interpolation strategy based on similarity thresholding.

Ablation study on texture interpolation strategy

Our results show that the selective interpolation strategy consistently outperforms both baselines across all metrics, confirming its effectiveness in preserving structure, appearance fidelity, and semantic consistency throughout the morphing process. We will include these quantitative results in the revised paper to make this analysis and comparison clearly visible.

Question 3: It was mentioned in Appendix H - Limitation, line 137, that the method encounters difficulties on cases involving extreme topological changes, eg splitting or merging. Could the authors show a couple of examples, to illustrate this failure case?

A3: Since figures cannot be included in the rebuttal, we briefly describe an example of failed case:

The source object ("wind chime") consists of multiple dangling rods suspended separately, while the target ("apple") is a smooth, unified solid. During morphing, the generator struggles to merge the disconnected chime parts into a cohesive apple shape. Intermediate outputs exhibit floating, misaligned fragments, and lack global structural coherence.

Question 4: Could the authors elaborate on what a plausible use case might be for morphing between unrelated object classes like a digger and a ship, as shown in figure 3?

A4: While morphing between unrelated object classes like a digger and a ship may seem unconventional, such transitions have practical value in creative industries, such as game cinematics, concept art, and visual storytelling, where surreal or symbolic transformations are often used to convey narrative or thematic shifts. Our framework enables such imaginative transitions in a controllable and high-fidelity manner.

Weakness 1: Quality.

A5: We kindly refer the reviewer to the experimental results in A1, where we implemented our full method using GaussianAnything—the same 3D generator used by 3DRM. This variant, denoted as “Ours*”, consistently outperforms 3DRM across all metrics, including FID, PPL, V-CLIP, STP-GPT, and SEP-GPT. This provides clear evidence demonstrating the effectiveness of our proposed techniques.

We respectfully emphasize that the core contribution of our work is not in proposing a new 3D generator, but in how we utilize and control the generative process through novel morphing strategies. Notably, all compared methods rely on pretrained generative priors. What distinguishes our method is the integration of:

• Latent space Wasserstein barycenter-based interpolation, replacing simplistic latent-space linear blending;
• Recursive initialization strategy, which ensures geometric stability and avoids degenerate intermediate shapes;
• Semantic-guided selective texture interpolation, which provides fine-grained control over appearance blending via thresholded feature similarity.

These contributions form a principled and generalizable framework for high-quality 3D morphing, independent of the specific backbone used.

Weakness 2: Clarity on Lipschitz continuous, DDIM and figure 7 layout.

A6: Lipschitz continuity. We acknowledge that the reference to a “mathematical derivation” of Lipschitz continuity in the appendix was misleading. Our intention was to reference the theoretical property of rectified flow models, which are designed to be deterministic and Lipschitz-continuous by construction, as discussed in standard differential equations literature. We will revise the text to clarify that no formal derivation is provided, and instead cite the relevant structural property of rectified flows.

A6: Regarding the comment on diffusion models and DDIM . We agree that deterministic sampling methods like DDIM are well-known and valid. Our statement in the appendix was not meant to suggest that all diffusion models are inherently stochastic, but to explain why we chose rectified flow models: they offer deterministic forward integration with a guaranteed likelihood formulation, which aligns well with our need for stable, continuous interpolation in latent space.

A6: Regarding the Figure 7 layout . We agree that the caption and layout of Figure 7 could be clearer. Each row individually represents a separate morphing process with different interpolation thresholds, and the leftmost column remains fixed as the source shape. The results along the vertical axis (column) are independent. The far-left column doesn't exhibit texture changes because they are all the same 3D sources generated from the input source image. We will revise the caption and figure layout to make this clearer and indicate that the figure should be viewed horizontally, and clarify the relationship between the rows and the varying thresholds.

Weakness 3: Significance.

A7: While our approach builds on established components (e.g., flow-based generation and optimal transport), it is far from a simple combination of existing tools. Instead, it is a carefully designed framework that specifically addresses the unique challenges of textured 3D morphing.

• The sequential initialization strategy serves as a trajectory memory mechanism in high-dimensional latent spaces, which helps stabilize barycentric optimization and avoids abrupt geometric artifacts—a failure commonly observed in naïve interpolation (see Fig. 6 and Fig. 8). Without this step, as shown in Appendix Table A1, visual quality significantly declines, highlighting its importance in ensuring smooth transitions.
• The similarity-guided selective interpolation is not a trivial heuristic, but a principled mechanism that enables localized control over texture blending, which is critical for semantic consistency and identity preservation across the morphing sequence. Removing this step, as shown in the “Ablation study on texture interpolation strategy”, leads to blurry and incoherent textures, diminishing overall visual fidelity.

Weakness 4: Originality.

A8: We appreciate the reviewer’s comment and agree that optimal transport (OT) has been explored in prior works on 3D shape analysis and morphing. However, we would like to clarify that the core novelty of our work does not lie in using OT per se, but in how we formulate and apply free-support Wasserstein barycenter optimization within a flow-based generative framework for textured 3D morphing. To the best of our knowledge, we are the first to leverage OT barycenters in the latent token space of a conditional 3D generator, enabling:

• Semantically meaningful, geometry-aware interpolation in a high-dimensional flow space
• Fine-grained texture control through selective semantic blending—something not addressed in prior OT-based morphing methods

In contrast, previous works typically：

• Apply OT to fixed geometric representations, which do not incorporate semantic texture control or handle arbitrary image and text inputs in a unified framework.
• Do not operate on latent token spaces with full texture control, limiting their ability to handle complex texture details in high-dimensional latent spaces.

We sincerely thank the reviewer for their thoughtful and positive feedback. We are pleased that the novelty and effectiveness of our approach, particularly in enabling smooth shape and texture interpolation, was recognized. We address the concerns below.

Concern 1: The method seems to work on some fictional images or artistic prompts. If one wants to use it for real 3D data, what would happen, and what modifications can be applied?

A1.1: Current model's ability for real 3D data.

To validate the current model's ability for real 3D data, we conducted extensive tests on real human face scans from the Headspace dataset [1], as well as cross-domain conditions like real photo → stylized identity (e.g., human face → Ironman). Our method handled these scenarios successfully.

1. Experiments between two real human faces (Headspace dataset [1]): To evaluate the method’s generalization to real 3D data, we conducted experiments using human face renderings from the Headspace dataset, which contains high-quality 3D face scans along with corresponding rendered RGB images.

   Despite the domain gap between real scanned faces and the Trellis training distribution, our method produced smooth and coherent 3D morphing sequences between different real identities. Quantitatively, using the same evaluation metrics as in Sec. 4.1.1, we obtained: FID: 3.97, PPL: 2.88, V-CLIP: 0.91 (see "Quantitative results on Headspace dataset"). These results are on par with our main test set results (main paper table 1), suggesting that our method generalizes well to real 3D data, even under a domain shift and without any additional learning. However, despite the good quantitative results, we still observe minor qualitative degradation for the fine-grained geometric details (e.g., wrinkles or subtle facial curvature). This is most likely due to the domain bias inherited from the pre-trained trellis model.

2. Experiments between a real human face and an artistic target: We also tested morphing between a real human face image (photo) and a stylized image of Ironman’s helmet. The resulting 3D sequence showed a smooth and visually consistent transition, with clear semantic interpolation from the human facial structure to metallic plating, visor formation, and color/style blending. The quality is visually comparable to our result in Figure 4 (last row: "bust of Caesar" → "Darth Vader helmet"). Due to the limited number of such stylized test cases, we do not provide a quantitative average, but the results we obtained were qualitatively compelling and robust, reinforcing the method’s flexibility in handling cross-domain, real-to-stylized scenarios.

Note that we conducted these experiments without any architectural changes or further fine-tuning. We will include these additional qualitative results and metrics in the updated version of the paper to better illustrate the method’s effectiveness under real-to-stylized morphing and real 3D scan domains.

3. Our current implementation is based on Trellis, which is trained mainly on synthetic textured 3D objects. Therefore, it's reasonable to perform best when the input domain aligns with the training distribution. However, we want to highlight that our method (WUKONG) is a modular, training-free morphing framework, not tied to any specific backbone. During application, it can be easily adapted to more generalizable or domain-specific 3D generators without retraining.

A1.2: Modifications can be applied. To further improve performance on real 3D data, the following modifications can be tried:

• Model adaptation: Fine-tune the encoder/decoder on real 3D dataset such as human face scans.
• Incorporating 3D scan priors (e.g., via FLAME or DECA) to regularize the geometry flow in our framework.
• If real 3D scans are available, the source can be used directly in Trellis’s latent geometry representation to avoid re-encoding from image.

Concern 2: What could be the reason for T-pose human artifacts when using linear interpolation (Fig. 6, first row)? Any insights on these issues?

A2: In flow-based 3D generators like Trellis, condition tokens (e.g., from CLIP/DINOv2) guide the generation by modulating cross-attention at each layer. Linear interpolation between latent features explores intermediate regions of the latent space. However, when the source and target features are semantically distant (e.g., from “T-Rex” to “cat”) and lack appropriate supervision, the model's conditioning may become ambiguous or collapse into mode-averaged representations. This often results in "hallucinated" generic outputs, such as T-poses or default humanoid templates commonly seen in generative models.

This issue highlights the importance of our barycentric interpolation + sequential initialization strategy, which ensures that: 1. intermediate tokens remain on a semantically valid manifold, 2. abrupt jumps across unrelated modes are avoided. We will further clarify this insight in the revised paper and illustrate it with additional examples.

[1] Pears, N. E. (Creator), Duncan, C. (Creator), Smith, W. A. P. (Contributor), Dai, H. (Contributor) (6 Jun 2018). The Headspace dataset. University of York. 10.15124/6efa9588-b715-44ec-b7bb-f10dff7ca93e

We thank the reviewer for the constructive comments. We are pleased that our method's ability to achieve smooth shape transitions and detailed texture preservation, as well as its generalization across object categories with both image and text inputs, was recognized. We address the concerns below.

Concern 1: While the training-free nature of WUKONG is a major strength in terms of usability and scalability, it also introduces limitations. Specifically, the method depends entirely on the capacity and priors of existing pretrained models (e.g., Trellis, CLIP, DINOv2). This reliance restricts adaptability. & Experiments focus on curated or synthetic data; real-world performance is not fully explored.

A1: We appreciate the reviewer’s thoughtful analysis. We would like to emphasize that our framework is intentionally modular and training-free by design, which brings several unique strengths and flexibility:

1. Future-proof and upgradable via backbone improvements: Our method can immediately benefit from advances in foundational models. For example, replacing Trellis with future more powerful generators can directly improve morphing quality without retraining. This flexibility also allows adaptation to models pretrained on different domains. For instance, as shown in Appendix Table A1, we applied our morphing pipeline on top of TripoSG without other modifications, and the results are still the best among all other methods. This demonstrates the plug-and-play nature of our framework and its ability to inherit capabilities from better 3D priors as they emerge.

2. Challenges for training-based 3D morphing methods: We note that building a learning-based 3D morphing system in the current landscape faces significant practical challenges. Specifically, there is no readily available dataset containing sufficient 3D morphing pairs—that is, sequences of high-quality 3D models capturing smooth transitions between two given concepts, with consistent geometry and texture supervision across frames. Constructing such a dataset is particularly difficult in open-vocabulary or cross-domain scenarios (e.g., “cat → dragon” or “human → cartoon Ironman”), where intermediate forms are ambiguous or subjective.

   In contrast, WUKONG’s training-free design provides a flexible, data-efficient alternative. It enables 3D morphing between arbitrary inputs (text or image) using only pretrained generative and perceptual models, without relying on morphing trajectories as supervision. This makes it immediately applicable to a wide range of prompt combinations, including those beyond curated or canonical categories.

3. Generalization to real 3D data: To evaluate the method’s generalization to real 3D data, we conducted experiments using the Headspace dataset[1], which contains high-quality 3D face scans along with corresponding rendered RGB images. In our pipeline, we used these rendered images as inputs and passed them through the DINOv2 encoder to extract texture and semantic features for morphing. The outputs were generated by the standard WUKONG pipeline without any architectural changes or fine-tuning. Despite relying on pretrained components, our method shows strong generalization to real-world 3D scans. Our method outperformed both MorphFlow and 3DRM(Our own implementation) on the same evaluation protocol. Quantitative results are shown below:

Quantitative results on Headspace dataset

We will clarify and expand these examples in the revision to emphasize real-world applicability.

[1] Pears, N. E. (Creator), Duncan, C. (Creator), Smith, W. A. P. (Contributor), Dai, H. (Contributor) (6 Jun 2018). The Headspace dataset. University of York. 10.15124/6efa9588-b715-44ec-b7bb-f10dff7ca93e

Concern 2: the components (e.g., barycenter optimization, flow-based generation) are largely drawn from prior works. As a result, the paper’s novelty lies more in integration and application than in methodological invention.

A2: While the building blocks are rooted in existing methods, their composition and adaptation in this context form a methodologically novel and non-trivial contribution that we believe will inspire further work in learning-free 3D content manipulation.

1. We are, to our knowledge, the first to formulate 3D morphing as a free-support Wasserstein barycenter problem, enabling semantically meaningful, geometry-aware transitions in a continuous latent space.

2. We introduce a sequential initialization scheme that is critical for ensuring stable interpolation in high-dimensional flow latent spaces, addressing issues not tackled by prior work in either 2D morphing or general OT literature.

3. We further propose a similarity-guided selective texture interpolation mechanism, offering controllable and detail-preserving transitions in 3D appearance, going beyond naïve interpolation schemes used in prior art. 1.1-1.3 one point.

Concern 3:Inference time (30s per sample on A100) may limit practical scalability. It is suggested to compare with other methods on the inference speed and computation burden.

A3: We acknowledge that we did not include inference time comparisons in the main paper and will add this comparison in the revised edition. We report the time consumption in the following table. Our method is the most efficient one among all methods.