We sincerely thank the reviewer for the positive feedback and insightful summary. We appreciate the recognition of our work's key strengths: the non-trivial extension of 3DGS Feature Fields to humans, the state-of-the-art performance achieved, and our commitment to releasing the code and dataset to foster future research in the community. We sincerely hope our responses below adequately address your concerns.

Q1: Polish Writing.

Thank you for your careful and detailed review. We will correct all typos and inconsistencies in the revised manuscript, including the specific points you raised:

• L68: The missing parenthesis ")" will be corrected.
• L252: The capitalization of "We" will be fixed.
• L243: The inconsistent sentence will be rewritten for clarity. Furthermore, we will conduct a thorough proofread of the entire paper to improve its overall polish.

Q2: Clarification of the Novelty

We appreciate the opportunity to clarify our motivation and contributions. Our primary insight and contribution is beyond it, as an unified pipeline synergizing 3D reconstruction and semantic segmentation. This proposed philosophy can unlock many novel and practical downstream applications, such as semantic-guided 3D reasoning, context-aware human behavior analysis, and interactive semantic editing in immersive AR/VR environments, which have not yet been thoroughly explored. To achieve this goal, we first devote substantial effort to curating data-label pairs, recognizing the extreme scarcity of labeled 3D datasets. Furthermore, we are pleased to observe the following advantageous properties emerging from our unique design:

1. $\color{#8B2E22}{**Segmentation can benefit reconstruction**}$: We leverage Pixel-Align Aggregation to enhance 3D reconstruction by incorporating semantic information on the 2K2K Dataset. Specifically, we utilize two types of semantic information: (1) pre-trained DINOv2 features and (2) manually annotated semantic maps. Both have been shown to improve reconstruction quality, as semantic segmentation helps the model prioritize specific parts and clothing regions.

2. $\color{#8B2E22}{**Reconstruction can benefit segmentation**}$: Limited segmentation data often struggles to achieve convergence for 3D consistency segmentation maps. To address this, we naturally incorporate reconstruction priors, which significantly enhance 3D consistency and improve mIOU from 0.724 to 0.840 (+16%). Notably, on human mesh datasets, our approach achieves better 3D consistency and coherence compared to 2D foundation models (e.g., Sapiens).

Additionally, our ablation studies validate the impact of key design components—such as loss functions and geometry guidance. We will release the entire pipeline (code and constructed dataset) to support community adoption.

Q3: Technical Details (Depth/Offset Prediction, Sapiens/Human3R in Table 1).

• Depth/Offset Prediction: Inspired by Splatter Image [1], the depth maps and 2D offsets are predicted by a small, two-branch MLP head attached to the output of our first (feature aggregation) Transformer. This head regresses per-pixel depth and offset values, which are then used to unproject pixels into an initial 3D XYZ.
• Sapiens + Human3Diffusion (2-view baseline): To create a strong baseline for two-view 3D segmentation, we adapted the official open-source implementation of Human3Diffusion. Since their framework already uses a multi-view diffusion model and a 3D Gaussian Splatting representation similar to ours, we "hacked" their 3DGS renderer to support feature rendering. This allowed us to establish a baseline with a comparable architecture. As shown in Table 1, Human3R achieves higher reconstruction fidelity with a lower inference time, demonstrating the effectiveness of our integrated approach.
• Human3R (2-view support): We simply bypass the diffusion-based view generation and feed the second ground-truth view directly into our feature aggregation module. This highlights our framework's flexibility. We will add these implementation details to the experimental setup section in our revision.

Q4: Using newer correspondence models (e.g., DUSt3R).

This is an excellent and insightful suggestion. We would like to clarify that our primary baseline, LSM, is already built upon DUSt3R. As mentioned in lines 226-227 of our paper, we fine-tuned it on our human dataset to ensure a fair and direct comparison.

Regarding the newer models you mentioned (VGGT [2], DUNE [3]), they were not yet established or widely adopted at the time we were conducting our primary experiments, and integrating them would require substantial effort to adapt and fine-tune for our specific downstream task. We thank the reviewer for this idea and will add it to our future work/discussion section.

Q5: Appendices in supplementary material.

Thank you for careful review. We will move all appendices to the supplementary material for the camera-ready version, as per NeurIPS formatting guidelines.

---

Reference:

[1] Splatter Image: Ultra-Fast Single-View 3D Reconstruction

[2] VGGT: Visual Geometry Grounded Transformer, CVPR 2025

[3] DUNE: Distilling a Universal Encoder from Heterogeneous 2D and 3D Teachers, CVPR 2025

We thank Reviewer wsxq for the critical and detailed feedback, which has prompted us to conduct several new experiments to validate our core claims.

Q1: Why task monocular images as inputs ?

Thank you for this important question! We agree that monocular 3D reconstruction is a highly ill-posed and challenging problem. However, we believe its difficulty is precisely what makes it a fundamental and valuable area of research, for two main reasons:

1. Greater Flexibility and Efficiency: Inspired by HumanSplat and PShuman [1,2], a single-image to 3D pipeline offers significant flexibility. It allows users to reconstruct a full 3D representation from just one picture, which is essential for many downstream applications in fields like gaming, movies, fashion, and AR/VR, where multi-view data is often unavailable.

2. A Unified and Extensible Framework: The architecture is modular, allowing us to adapt it for sparse or multi-view inputs by replacing specific components (e.g., using a multi-view spml estimation model). In fact, our model can be extended to handle two input views by simply replacing the generated latent with those from the GT input latents (encode input images via a VAE model), as shown in our experiments (Tables 1 and 3). This shows that our single-view approach provides a adaptable foundation for sparse input scenarios.

Q2: Reliance on Diffusion and Lack of Novelty.

Your insight regarding the technical contribution is enlightening. We appreciate the opportunity to clarify our motivation and contributions.

We group these points as they relate to the core premise and contribution of our work. We respectfully argue that our work presents significant novelty beyond a simple aggregation of techniques. The core innovation lies in the synergistic integration and non-trivial adaptation of existing components to solve the novel and challenging task of joint, single-image human reconstruction and segmentation.

Our motivation is to tackle the highly ill-posed but practical single-image problem, which is now feasible due to powerful 2D generative priors. Our contribution is not the prior itself, but the framework that harnesses and lifts these 2D features into a coherent, semantic 3D representation.

We appreciate the opportunity to clarify our motivation and contributions. Our primary insight and contribution is beyond it, as an unified pipeline synergizing 3D reconstruction and semantic segmentation. This proposed philosophy can unlock many novel and practical downstream applications, such as semantic-guided 3D reasoning, context-aware human behavior analysis, and interactive semantic editing in immersive AR/VR environments, which have not yet been thoroughly explored. To achieve this goal, we first devote substantial effort to curating data-label pairs, recognizing the extreme scarcity of labeled 3D datasets. Furthermore, we are pleased to observe the following advantageous properties emerging from our unique design:

1. $\color{#8B2E22}{**Segmentation can benefit reconstruction**}$: We leverage Pixel-Align Aggregation to enhance 3D reconstruction by incorporating semantic information on the 2K2K Dataset. Specifically, we utilize two types of semantic information: (1) pre-trained DINOv2 features and (2) manually annotated semantic maps. Both have been shown to improve reconstruction quality, as semantic segmentation helps the model prioritize specific parts and clothing regions.

2. $\color{#8B2E22}{**Reconstruction can benefit segmentation**}$: Limited segmentation data often struggles to achieve convergence for 3D consistency segmentation maps. To address this, we naturally incorporate reconstruction priors, which significantly enhance 3D consistency and improve mIOU from 0.724 to 0.840 (+16%). Notably, on human mesh datasets, our approach achieves better 3D consistency and coherence compared to 2D foundation models (e.g., Sapiens).

Additionally, our ablation studies validate the impact of key design components—such as loss functions and geometry guidance. We will release the entire pipeline (code and constructed dataset) to support community adoption.

Q3: Why two Transformers instead of a single transformer?

This is an excellent question. The first Transformer (Feature Aggregation) focuses on the general task of aggregating multi-view geometric and appearance features. The second Transformer (Pixel-align Aggregation) is specialized, using an attention mechanism to translate these fused features into the structured parameters of our semantic 3D Gaussians. To validate this, we have conducted ablation studies with a single, monolithic Transformer, and the results confirm our design is superior:

Q4: Why an end-to-end model instead of per-subject optimization?

Thank you for your constructive feedback. We agree that optimization-based methods can achieve more high-quality results. However, we respectfully offer a different perspective on the primary goal of our work.

1. Unlike optimization-based methods that often require minutes to hours per subject, our model is designed for efficiency and scalability , enabling 3D reconstruction in a matter of seconds. We believe this is a critical feature for many real-world applications where speed is essential.

2. Furthermore, the two approaches are not mutually exclusive. As explored in recent work like InstantSplat [3], efficient, end-to-end models can serve as an excellent initialization for optimization-based methods, significantly accelerating their convergence. This potential synergy further highlights that developing fast, feed-forward models is a valuable research direction.

Q5: How is the SMPL model used?

Thank you for this important question. For an input image, we first estimate SMPL(-X) parameters with PIXIE [4]. The resulting mesh provides a structural guidance for the human's pose and shape.

Q6: How is the SMPL model used? We appreciate your important point. Our dataset will be built exclusively from permissively-licensed images (e.g., HGS-1M [5]). For any future work, we will adhere to strict ethical guidelines, including data source transparency and privacy filtering.

---

Reference:

[1] HumanSplat: Generalizable Single-Image Human Gaussian Splatting with Structure Priors, NeurIPS 2024

[2] PSHuman: Photorealistic Single-image 3D Human Reconstruction using Cross-Scale Multiview Diffusion and Explicit Remeshing, CVPR 2025

[3] InstantSplat: Sparse-view Gaussian Splatting in Seconds, Arxiv 2024

[4] Feng, Yao, et al. "Collaborative regression of expressive bodies using moderation.", 3DV 2021

[5] SIGMAN: Scaling 3D Human Gaussian Generation with Millions of Assets, ICCV 2025

Thank the authors for their detailed rebuttal, which addressed most of my concerns. I understand the key contribution lies in synergizing 3D reconstruction and semantic segmentation, and the additional results can support this claim. I suggest updating the introduction to clarify the motivation of this work, specifically why using monocular input, why using an end-to-end model, and why jointly learning reconstruction and semantics are important, as well as the method section to make the key technical contribution clearer.

In addition, besides the good performance and the detailed ablation, it's still not clear how the performance of this work would be dependent on the image-to-3D diffusion priors in the first stage. Could the authors provide an additional ablation experiment on how different priors will affect the final result? For example, using Zero-1-to-3 [1] instead of SV3D. Is the performance independent of the prior diffusion model, or will it benefit from a stronger prior?

[1] Liu et al. Zero-1-to-3: Zero-shot one image to 3d object. ICCV 2023.

Dear Reviewer wsxq,

We sincerely thank you for your positive and constructive feedback. It is inspiring to hear that our previous rebuttal addressed most of your concerns. As you suggested, we will update the Introduction section to make our motivation and contributions clearer. Regarding your concern about "how model performance depends on image-to-3D diffusion priors":

This is a very insightful question. To address it, we have made a concerted effort to conduct an ablation study analyzing the impact of the 2D diffusion prior on our final results.

Clarification: Due to Zero-1-to-3's [1] single-view limitation, we instead compare SV3D against MVDream [2], both multi-view diffusion models capable of generating consistent 3D content from single images. Our findings indicate that MVDream is prone to inconsistencies in its generated multi-view outputs, whereas SV3D demonstrates superior coherence. This technical alignment ensures a meaningful comparison of how different priors affect the final result.

We replaced SV3D with the MVdream model and evaluated the performance on the 2K2K dataset (with the same training datasets and settings). The results are presented in the table below:

Even with suboptimal priors (e.g., MVDream), our method consistently produces high-quality, robust results while reducing multi-view inconsistencies, as evidenced by the increased number of matching points (#Points ↑). It indicates that the core components of our architecture extract and utilize geometric and semantic information without solely depending on a single, specific generative prior. We will incorporate this ablation study and its analysis into revision.

Once again, we thank you for your time and for providing this valuable feedback. It has helped us significantly improve the clarity and completeness of our work.

Sincerely,
The Authors

[1] Liu et al. Zero-1-to-3: Zero-shot one image to 3d object. ICCV 2023.

[2] Shi et al. MVDream: Multi-view Diffusion for 3D Generation. ICLR 2024

We thank the reviewer for their positive feedback and for recognizing the qualitative results and the effectiveness of our Pixel-align Aggregation module. We appreciate the opportunity to contribute to the community and we have provided detailed clarifications and responses below to address your concerns.

W1: Lack of discussion on failure cases.

We thank the reviewer for this suggestion! While HUMAN3R demonstrates robust generalization across various challenging scenarios, the model's performance may degrade under two primary conditions:

• Extreme Poses: For extreme poses that are challenging for pose estimation models [1], our model may generate minor misalignments in fine details (e.g., in the arms). However, the overall limb structure remains intact, a robustness afforded by the geometric prior.

• Low-Quality Inputs: When processing low-resolution or heavily compressed images, the input may lack the necessary high-frequency information for high-fidelity reconstruction. It leads to a degradation of fine-grained details, such as facial features or clothing textures in the outputs.

We will include this in the "Limitations and Failure Cases" section. Thank you for your valuable feedback!

W2: Comparison with general 3D generation models.

We thank the reviewer for their constructive feedback!

First, we would like to clarify that our method was compared against recent 3D generation models, including LGM, GRM, and InstantMesh (which is the base model for Hunyuan3D 1.0). To ensure a fair comparison, we fine-tuned both the LGM and GRM models on our dataset.

Furthermore, to fully address your concern, we have benchmarked our method against Stability AI's recent model, SPAR3D, on the test set of the 2K2K dataset. The results below further demonstrate the effectiveness of our approach.

We will incorporate these new results into the revised version of the paper. Thank you for your valuable suggestion.

W3: Clarify the contribution.

We appreciate the opportunity to clarify our motivation and contributions.

Our primary insight and contribution is beyond it, as an unified pipeline synergizing 3D reconstruction and semantic segmentation. This proposed philosophy can unlock many novel and practical downstream applications, such as semantic-guided 3D reasoning, context-aware human behavior analysis, and interactive semantic editing in immersive AR/VR environments, which have not yet been thoroughly explored. To achieve this goal, we first devote substantial effort to curating data-label pairs, recognizing the extreme scarcity of labeled 3D datasets. Furthermore, we are pleased to observe the following advantageous properties emerging from our unique design:

1. $\color{#8B2E22}{**Segmentation can benefit reconstruction**}$: We leverage Pixel-Align Aggregation to enhance 3D reconstruction by incorporating semantic information on the 2K2K Dataset. Specifically, we utilize two types of semantic information: (1) pre-trained DINOv2 features and (2) manually annotated semantic maps. Both have been shown to improve reconstruction quality, as semantic segmentation helps the model prioritize specific parts and clothing regions.

2. $\color{#8B2E22}{**Reconstruction can benefit segmentation**}$: Limited segmentation data often struggles to achieve convergence for 3D consistency segmentation maps. To address this, we naturally incorporate reconstruction priors, which significantly enhance 3D consistency and improve mIOU from 0.724 to 0.840 (+16%). Notably, on human mesh datasets, our approach achieves better 3D consistency and coherence compared to 2D foundation models (e.g., Sapiens).

Additionally, our ablation studies validate the impact of key design components—such as loss functions and geometry guidance. We will release the entire pipeline (code and constructed dataset) to support community adoption.

W4: Why not evaluate the 3D segmentation tasks on third-party datasets?

We agree that evaluation of diverse datasets is important. However, to the best of our knowledge, our work was the first work to explore a large-scale, 3D human dataset with detailed, multi-class part-segmentation labels (28 classes). This is precisely why we invested significant effort in curating our dataset. We believe our released dataset will serve as a valuable new benchmark for the community and help drive future research in this domain.

---

Reference:

[1] Feng, Yao, et al. "Collaborative regression of expressive bodies using moderation.", 3DV 2021

[2] SPAR3D: Stable Point-Aware Reconstruction of 3D Objects from Single Images, arXiv 2025

[3] SiFU:Side-view Conditioned Implicit Function for Real-world Usable Clothed Human Reconstruction, CVPR 2024

[4] HumanSplat: Generalizable Single-Image Human Gaussian Splatting with Structure Priors, NeurIPS 2024

[5] PSHuman: Photorealistic Single-image 3D Human Reconstruction using Cross-Scale Multiview Diffusion and Explicit Remeshing, CVPR 2025

Thank you for your detailed observation regarding the segmentation artifacts in Figure 3. This is a good question. Thank you for bringing it up for discussion. We clarify these phenomena as follows:

• Sapiens' Black Regions: As a purely 2D image-based model pre-trained on natural images, Sapiens exhibits a domain gap when applied to rendered human mesh datasets. For instance, it misclassifies yellow/white clothing regions as "background" or "other categories", or shows degraded segmentation performance for back views due to the long-tail data distribution.

• LSM's Black Regions: These occur primarily in (1) textureless regions (e.g., solid-color clothing) where feature matching fails and (2) input views with large camera distances that cause failure of LSM's matching mechanism and subsequent performance degradation.

However, this can be compensated by our human body prior, resulting in better performance through SMPL(-X) parameters. For instance, in Fig 3 row 3, our method can still recover reasonable segmentation outcomes by leveraging the human body prior.

We will include additional segmentation results ( 360° camera trajectory) of three methods in the revised manuscript and supplementary materials to clarify the observed phenomena. Thank you again for your time and valuable input!

Sincerely,
The Authors

We thank the reviewer for the insightful and thorough feedback and for acknowledging our core contributions, which introduce a unified and single-pass Transformer framework for joint 3D reconstruction, segmentation, and synthesis. Below, we provide clarifications addressing your concerns.

W1: The technical details of PIXIE [1]?

Thank you for this sharp observation! The reviewer is correct that HUMAN3R utilizes PIXIE [1] to estimate parameters for the SMPL-X model. The high-fidelity results for the hands and face benefit from the expressive priors provided by SMPL-X. We have corrected "SMPL" to "SMPL-X" in the revision.

W2: Details of the In-the-wild dataset?

We apologize for this omission. Our in-the-wild test set contains 8 images from public CC0 sources, selected to test generalizability under challenging scenarios (e.g., diverse poses and identities). The foreground human masks are automatically generated using the Segment Anything Model (SAM)[2] . We will add a subsection to the supplementary materials with these details.

W3: Why freezing image encoders?

Thank you for this important question. We freeze the image encoders based on DINOv2 [3]'s best practices to leverage its pre-trained features and to maintain the training efficiency by only training a lightweight decoder. We validated the design choice with an ablation study on the 2K2K dataset, which shows that fine-tuning the image encoder only provides marginal gains (+0.032 PSNR) in our setting.

We will incorporate it into the ablation study in the revision.

W4: Missing qualitative results on evaluation benchmarks.

We thank the reviewer for this constructive feedback. However, according to NeurIPS 2025 policy, we are prohibited from providing images, videos, or external links during the rebuttal period.

To ensure clarity and transparency, we summarize our qualitative results: Qualitatively, on the THuman2.1 and 2K2K benchmarks, our method excels at capturing details (e.g, face and clothing), yielding higher-fidelity results compared to the overly smooth or blurry surfaces from baseline methods. We are committed to adding qualitative comparisons in the revision.

---

Reference:

[1] Feng, Yao, et al. "Collaborative regression of expressive bodies using moderation.", 3DV 2021

[2] Kirillov, Alexander, et al. "Segment anything.", ICCV 2023

[3] DINOv2: Learning Robust Visual Features without Supervision, TMLR 2024