Thanks for your time and feedback. We also thank the reviewer for finding our method more applicable and useful for downstream tasks. We answer questions next.

QE1. The paper does not provide qualitative results on in-the-wild images; all evaluations are conducted on datasets collected in controlled laboratory settings. Given that the method does not require pose information, it would be particularly valuable to demonstrate its performance on in-the-wild data to better assess its robustness and practical applicability.

Only plain text is allowed for the rebuttal, so we cannot show more rendering results. We instead refer to the presented results in our submission. In Part 4 of the supplementary webpage index.html, we demonstrate the cross-domain generalization on PeopleSnapshot and UBCFashion datasets. Among those datasets, UBCFashion consists of try-on videos like those on fashion retail websites, close to the in-the-wild scenarios. We kindly hope the reviewer reassesses the remarks on in-the-wild results.

QE2. Only a limited number of novel views of a single generated subject are presented. It would be meaningful to evaluate and demonstrate multi-view consistency in this setting to further validate the quality and coherence of the reconstruction across viewpoints.

In Part 1 and Part 2 of the supplementary webpage index.html, we render 360 degree images and compare to baselines. The results demonstrate the multiview consistency as well as our improvements over baselines.

QE3. Animating Gaussian primitives using Linear Blend Skinning (LBS) may lead to noticeable artifacts, particularly for subjects wearing loose or complex garments such as dresses. This limits the realism and generalization of the animation results for diverse clothing types.

LBS is a standard practice in pose-driven 3D animation: It is adopted by all the state-of-the-art baselines including LIFe-GoM, IDOL and LHM. LBS can handle most of the animations, see Part 3 in the supplementary webpage index.html. However, artifacts exist when retargeting loose garments to extreme poses, e.g., legs are widely open. We will revise the limitations to mention this point. We would like to emphasize that the main contribution of this paper is to eliminate the need for pose priors during reconstruction at test time. This is crucial since it is difficult to obtain accurate poses at test time, and pose-dependent methods are sensitive to the quality of the input poses. Meanwhile, the proposed method can also reconstruct loose clothes, as is shown in examples 4, 6, 7, 8 of Part 2 of the supplementary webpage.

QE4. How does the model handle long garments, such as dresses, when predicting residuals in canonical pose? Are there specific mechanisms to account for the complex deformation of such clothing?

In the template branch, we predict residuals which accounts for the difference between the SMPL-X shape and the shape of an avatar with clothes, including complex garments. In the image branches, the model directly predicts the absolute positions that depict the shape of the dresses. We do not use any specific mechanism to model complex clothing. We find that this simple strategy works when trained on datasets that include complex clothing.

QE5. Could the authors clarify the role of the canonical UV map in the pipeline? Specifically, how is it used, and what advantages does it provide?

The canonical UV map is the SMPL-X mesh in its average shape projected in the 2D UV space. It serves as the “initialization” of the Gaussians’ means in the template branch. Since the canonical UV map does not model the clothes and hair of the person, the template branch predicts the residuals of the means to the UV map to model those. Without the canonical UV map, the template branch is required to predict the absolute Gaussians’ means and LBS weights, which is more challenging than predicting the residuals.

QE6. Does the method generalize well to animating subjects wearing dresses in diverse poses (especially for complex leg motion)? If so, could the authors provide examples or further discussion?

See QE3.

QE7. The approach relies on a Gaussian primitive template to inpaint missing content in unseen regions, which could lead to blurry renderings when only a single image or multi-view images with limited coverage of the subject are available. Thus the method performs best when sufficient multi-view coverage is provided, while its effectiveness is reduced in sparse or narrowly-viewed scenarios.

Our approach can inpaint unobserved regions. We demonstrate this in Part 2 of the supplementary webpage index.html, where models only take a single front-facing image as input and the entire back side requires inpainting. In the first example, our approach hallucinates the wrinkles on the back while LHM fails. As mentioned in the limitations, we agree that the inpainting can be blurry. We think it is mainly due to the regression-based losses during the training. This is a shared issue in IDOL, LHM, LIFe-GoM, and our method. A possible solution is to use diffusion models to inpaint high-frequency details, which we leave for future work.

Thank the author for clear explanation. Please try to include some qualitative result comparison to other baselines for in the wild images in the main paper as this will complete a more comprehensive evaluation, to help the reader understanding the performance of the proposed method.

Thanks for your time and feedback. We also thank the reviewer for finding our method well-motivated, effective and interesting, and the results strong.

QD1. The foundation of this paper is somewhat unclear. In the introduction, the authors primarily compare their method against previous approaches that rely on ground-truth or accurately estimated poses, claiming superior performance due to the removal of this dependency. However, recent methods such as LHM and IDOL also eliminate the need for pose inputs. Although the proposed method outperforms these baselines in the experiments, the authors do not provide sufficient analysis or explanation for this performance gain in the introduction.

Thanks for pointing this out! We did not mention LHM and IDOL in the introduction since both works focus on large-scale training of human models, not pose-free reconstruction. Instead, we opted to highlight them in the related work section. We will revise our paper and emphasize the differences in the introduction.

QD2. In the quantitative results, the method is not compared against IDOL and LHM on the THuman dataset.

We follow the split of prior works (e.g., GHG and LIFe-GoM) and compare to multiview baselines on THuman. While IDOL provides the checkpoint in its official repository and the metrics on THuman + HuGe100K’s test set in the paper, we are not able to fairly compare to it since 1) our single-view model reported in Tab. 3 is trained on HuGe100K solely, but IDOL’s checkpoint is trained on HuGe100K + THuman; and 2) we are not able to finish the training on HuGe100K + THuman due to limited resources. Moreover, LHM does not provide the training script so we will only evaluate it on the same setting without training, using the same way as in Tab.3. We will add the comparisons in our final paper.

QD3. Unlike LHM, your method does not include a dedicated module (input) for facial modeling. Have you observed any degradation in quality or identity shift, specifically in the face region?

We do not observe identity shift or degradation in face quality, even without a dedicated facial module, as can be seen in Part 2 of the supplementary webpage index.html. Note that our image branches output pixel-aligned Gaussians, which model more details than the SMPLX template branch, especially for fine-grained regions like faces.

QD4. While most limitations are discussed in the supplementary materials, I believe there are additional concerns worth noting. One issue is identity preservation — the reconstructed avatars may exhibit identity shifts, which could hinder real-world applications. Moreover, accurately reconstructing and animating loose clothing remains an open challenge in this field and is not yet adequately addressed by the proposed method.

Please see QD3 for the identity shift. Our approach can reconstruct loose clothing reasonably, e.g., examples 4, 6, 7, 8 in Part 2 of the supplementary webpage index.html. Our approach adopts LBS for animation. Since loose clothes do not strictly follow LBS, sometimes it may fail to animate challenging poses. This is a common issue, also in baselines such as LIFe-GoM, IDOL and LHM, which we leave for future work. We’ll revise the limitation section to discuss this.

The authors' rebuttals cannot convince me.

1. The authors say in the rebuttal: Thanks for pointing this out! We did not mention LHM and IDOL in the introduction since both works focus on large-scale training of human models, not pose-free reconstruction.

However, this argument is unconvincing to me. While it is true that LHM and IDOL do not explicitly target "pose-free reconstruction," they have already demonstrated the capability to achieve it effectively. Therefore, a thorough analysis is essential—either in the introduction or the experimental section—to clearly explain why the proposed method outperforms LHM and IDOL. General or vague statements such as "we will revise the paper and emphasize the differences" are insufficient. The rebuttal must explicitly articulate the technical distinctions and provide evidence or reasoning (maybe without the need of solid experiments) as to why the proposed approach is superior comparing with LHM and IDOL.

2. The response to QD2 is unconvincing. In Table 3 of the paper, IDOL is successfully trained on HuGe100K for comparison—so why is it not possible to train IDOL on THuman as well? Moreover, the proposed method is trained on both THuman (Table 2) and HuGe100K (Table 3), yet the rebuttal claims that training on both datasets altogether introduces resource limitations. This contradiction needs to be addressed clearly. Furthermore, in Section 4 of the website provided by the authors, it is explicitly stated that "The model is trained on THuman2.1 + HuGe100K," which directly contradicts the claims made in the rebuttal.

As for LHM, it should still be feasible to follow the same evaluation protocol as in Table 3 and report its performance on the THuman dataset. These comparisons are critical for a fair and thorough evaluation and should not be omitted without a concrete justification.

3. It is difficult to assess identity shift from the results in the supplementary material, as the faces are rendered too small. Close-up views are necessary for clearer visualization and meaningful comparison, particularly on real-world (THuman) and in-the-wild examples. At the very least, the THuman results (section 3 in webpage) show a noticeable drop in facial quality, and the cross-domain results (e.g., People-Snapshot in section 4 of webpage) show clear identity shifts—even the head shape changes significantly.

QC1. Dataset limitations: The datasets used for training and testing include THuman2.0, THuman2.1, and HuGe100K, with additional testing on XHuman. However, I would like to see more real-world, in-the-wild scenarios evaluated, as well as any limitations on the input person's actions, such as whether they can only be in a standing position, etc. These aspects are crucial for practical applications.

Only plain text is allowed for the rebuttal, so we cannot provide more rendering results. We instead refer to the presented results in our submission. In Part 4 of the supplementary webpage index.html, we demonstrate cross-domain generalization on PeopleSnapshot and UBCFashion datasets. Among the datasets, UBCFashion consists of try-on videos like those on fashion retail websites, close to in-the-wild scenarios. Our approach is not limited to standing poses. As results on THuman2.0 data show, our method performs well on input images showing diverse poses.

QC2. Insufficient detail modeling: In the results presented by the authors, the details of hands and faces are lacking. The authors also mentioned the reasons in the limitations section, such as the small part of image occupied by hands which causes challenges in predicting LBS weights and corresponding 3D locations in the canonical T-pose for each pixel in the image branches, and also possible solutions, train extra models for hands and faces beside the body part. Considering the importance of faces and hands in practical applications, I would like to ask if there are any appropriate metrics to evaluate this part for future research?

Note that the details are lacking when hands and faces are heavily occluded. Otherwise, the image branches which predict pixel-aligned Gaussians depict details. To evaluate the hands and faces, a possible solution is to compute the PSNR/SSIM/LPIPS* in cropped regions based on hand and face masks.

QC3. Rendering condition limitations: Although the input does not require poses, the rendering still needs GT poses. Is this GT pose the same size skeleton as the person being rendered? If so, this may not be the case in practical applications. Is it robust under noisy conditions?

During rendering, we require users to provide poses for animation. These can be any desired poses. We use the GT poses in the tables of the paper only for evaluation purposes, so that we can compare the renderings with the ground-truth images and baselines. This evaluation setting is identical to prior baselines.

QC4. Can you show the requirements for input images in real-world scenarios, such as whether the pose needs to be standing, and how the results are in such cases?

See QC1.

QC5. Are there any better suggestions for evaluating the accuracy of faces and hands?

A possible solution is to compute the PSNR/SSIM/LPIPS* in cropped regions based on hand and face masks.

QC6. During training, is the target pose for rendering GT, or is it robust to noise? Is there any data to support this?

We use the GT poses provided by the dataset during training. To analyze the robustness to poses during training, we add synthetic Gaussian noise. We find that our approach is robust to some noise, e.g., Gaussian noise of std=0.1: On THuman2.0, it achieves a PSNR/LPIPS*/FID of 22.59/110.12/49.39, compared to 22.49/105.45/42.19 with GT training poses reported in Tab. 1. If we further perturb the GT poses with more significant Gaussian noise of std=0.3, the PSNR/LPIP*/FID drops to 20.58/138.67/73.20. We do not think robustness to the target poses in training is a big concern. Collecting high-quality training data is a one-time effort. Once trained, the model does not rely on poses for reconstruction at inference time. This is different from pose-dependent methods, which still require high-quality input poses during inference for reconstruction.

Thanks for your time and feedback. We also thank the reviewer for finding our method superior and the test-time pose optimization interesting. We answer questions next.

QB1. Although the paper does not require pose, it does require providing a mask. This demands on the quality of the mask.

Mask prediction is usually more robust than pose estimation. This is intuitive because masks are directly related to input images while poses can be ambiguous, e.g., because of clothing or occlusion. Therefore, we aim to avoid poses as input. Moreover, our approach is robust to the mask quality. Note that we use the predicted masks for training and inference on HuGe100K. To further evaluate the robustness, we additionally corrupt the masks by dilating with kernels of size 3 and 5. Our approach achieves a PSNR/LPIPS*/FID of 22.92/93.04/15.69 when the dilation kernel size is 3, and 22.83/95.03/15.80 when the kernel size is 5. Compared to 23.15/90.63/15.56 reported in Tab.3, the FID scores stay almost the same, demonstrating the robustness of our approach w.r.t. mask quality.

QB2. Applying a separate branch to each image may lead to an increase in computational cost.

Providing more input images increases computation but also improves quality, as is shown in Tab. 4. Note that our approach is significantly faster than LHM and comparable to IDOL, as shown in Tab. 3. Even with 3 input images of resolution 1024x1024, our approach still achieves 1.32s (Line 259), faster than LHM-500M’s 2.69s obtained with a single image of 896×640 resolution.

QB3. There is a lack of complex action image input. The paper uses an internal normalization strategy instead of relying on GT pose. This is good, but it raises questions about the normalization process because it cannot guarantee overall performance compared to precise input pose or the normalization results under complex action image input.

We clarify that our method is evaluated on complex action image inputs using the THuman2.0 dataset, which includes challenging poses and occlusions. Pose estimators often fail on THuman2.0, degrading pose-dependent baselines—as shown in Part 1 of our supplementary webpage. This highlights the difficulty of relying on pose estimators for precise input pose under complex motion. Despite not using ground-truth poses, our method achieves comparable or better performance. In Tab. 2, we compare our approach to the baselines with precise GT poses and show comparable results with better LPIPS* and FID, indicating better visual quality. Therefore, the overall performance is compelling even for complex action image input. We would like to emphasize that in the real world, input poses are either less precise, or are time-consuming (e.g., 20min) to acquire. So it is impractical to assume precise input poses.

QB4. Reconstruction stability regarding standard poses. Results for complex input images (even for use cases where pose estimators trained specifically for them have problems).

In Fig. 1 and in Tab. 7 of the appendix, we evaluate stability of approaches w.r.t. pose quality. We examine the stability by adding Gaussian noise with different standard deviations or by using predicted poses. Our approach is comparable to the baselines with ground-truth poses as inputs. Please see QB3 for results on complex input images.

QB5. Comparison with methods using high-precision pose input in the case of simple input images.

For simple input images where pose estimation is less likely to fail, our approach achieves superior visual quality compared to pose-dependent approaches. We compare our approach to LIFe-GoM on a subset of HuGe100K’s test data which contains 400 subjects in the simplest A-pose. Both methods are trained on THuman2.1+HuGe100K. Our approach achieves a PSNR/LPIPS*/FID of 22.69/98.80/20.65, significantly outperforming LIFe-GoM’s 18.28/121.63/26.12. We find that LIFe-GoM predicts spatially inconsistent colors due to view-dependent effects in the input images, an issue our method avoids. Additionally, we find LIFe-GoM performs worse than ours in inpainting unobserved regions.

QB6. Provide scaling experiments regarding computational cost.

1) Scaling the training data as in Tab. 6 in the appendix won’t increase the computational cost during inference. 2) Increasing the number of input images from 1 to 3 increases reconstruction time from 495ms to 1.32s on one NVIDIA A100 when using a resolution of 1024x1024. But it also improves quality, e.g., LPIPS* from 124.36 to 106.98, as shown in Tab. 4.

QB7. Current results are good with sparse image input, but overall there is a lot of room for improvement. With better input conditions, such as increasing the number of image inputs, what are the performance boundaries? Can it surpass existing methods?

Our paper focuses on the setting of sparse input images as it is more convenient for real-world applications. In this setting, we improve upon state-of-the-art methods. Our approach also demonstrates better performance with 3 input images compared to using a single image (Tab. 4). We expect further quality improvements with more input images since there is less to hallucinate. Our approach can operate on as many as 18 input images while using ~97GB GPU memory. Unfortunately, due to the limited resources, we are unable to finish this training.

QB8. Why use T-pose but not da-pose?

We use T-pose since it is the SMPL-X defined canonical space and LIFe-GoM also adopts it. Due to the limited rebuttal time, we are not able to finish the ablation. We suspect a different format won’t make a big difference, as the poses are related via a transformation that can be learnt by our deep nets.

Thanks for your time and feedback. We also thank the reviewer for finding our method well-justified, elegant and effective, and the results solid. We answer questions below.

QA1. I would like to see more results on long dresses or loose clothing. These cases are often challenging for avatar reconstruction, and it would help demonstrate the robustness of your method.

We find our method reconstructs long dresses or loose clothing well particularly if samples with loose clothes are part of the training data. We showcase an example of long dresses in Fig. 4. We also provided freeview rendering of long dresses in Part 2 of the supplementary file index.html.

QA2. For the comparison with LHM and IDOL in Fig. 4, could you clarify what the input image is for each method? Were the same images used across all methods? Additional detail here would improve the fairness and interpretability of the comparison.

The input image is the front-facing image of the subject. The same image is used for all methods and the comparison is completely fair. We will emphasize this by revising Fig. 4.

QA3. A recent paper has tackled a similar problem. I would appreciate it if you could include a comparison with their results. [1] (PF-LHM) It would help position your contributions more clearly within the current landscape of pose-free avatar reconstruction.

Thanks for the reference. We are happy to cite it in the final version. Please note that the paper appeared on arXiv on June 16, i.e., after the NeurIPs submission deadline. The codes have not been released yet. We compare our approach to its previous version (LHM) in Tab. 3.