[W4] Paper Writing.

Response:

a. Thanks for your suggestion. I think your categorization is more clear. We have revised it in the updated PDF.

b. Zou et al. follows PIFu to prepare the pixel-algined feature, I have revised the illustration.

c. The captions of Table 1 and Table 3 have already clearly shown that the Thuman dataset is used.

[Q1] Why selecting diffusion model?

Response: As we claimed in the paper, the indirect paradigm can only support training a regression model, which usually stucks in local-optima, hence, we selected the diffusion model because of its strong ability to learn the data distribution, and thus we need the direct paradigm. In this paper, we aim to learn the data distribution of the 3D Gaussian attributes. Since we constrain the diffusion model with strong SMPL and back image conditions, it cannot produce diverse reconstruction results.

[Q2] Why are faces blurred in Figure 8? Why are SHERF results dark in Figure 7?

Response: Our model was only trained on 480 human scans. Owing to the relatively limited training data, it could not predict unseen areas with high resolution. Hence, the face can be blurry if it does not appear in the input image. The results of SHERF are also dark in the original paper (see SHERF’s Figure. 3, ZJU MoCap).

[Rebuttal to Reviewer 73DC]

[W1] Discussion about direct vs indirect supervision.

Response:

a.We kindly disagree with your conclusion that the direct paradigm is a hasty conclusion. As you mentioned, there are no methods using 3DGS for single-view human reconstruction; hence, we build an indirect paradigm baseline to validate the effectiveness of our proposed direct paradigm. Note that, even the indirect paradigm achieves the best performance when compared with other methods. Both quantitative and qualitative results strongly support that our direct paradigm is better than the indirect paradigm.

Moreover, the indirect paradigm typically only supports regression training, however, such a direct paradigm supports training a diffusion model which is more efficient than the regression-based model to learn the 3D Guassian attribute distribution (see Figure 8).

b. We have quantitatively compared the results of indirect and direct paradigms in Table 1 and Table 3. Our proposed direct paradigm achieves better performance than the indirect paradigm, moreover, the visual results in Figure 4(a) and Figure 8(a) shows that the direct paradigm can achieve better visual quality. These results support that our direct paradigm is better than the indirect paradigm.

c.The randomness of the vanilla-3DGS is very large, the two-stage workflow aims to reduce the randomness of the 3DGS as much as possible, we never claim that we can completely solve it. When the randomness of the indirect paradigm is reduced, we can learn the model easier in a smaller solve space with the direct paradigm. Note that, there won’t be enormous points because the total number of 3DGS is fixed as N (according to Eq. 7).

[W2] Distribution unification.

Response: Thank you for your feedback. I believe there is a significant misunderstanding regarding our paper, particularly with the interpretation of Figure 4. To clarify, Figure 4 does not illustrate the effectiveness of the distribution unification stage (L400-401 in the original paper). Instead, the left column of Figure 4(d) shows the result of the vanilla-3DGS, while the right column shows the result after the per-scene overfitting stage (the first stage).

Our proxy-3DGS ground truth construction consists of two stages: the per-scene overfitting stage and the distribution unification stage.

Per-scene overfitting stage: In this stage, the vanilla 3DGS results are highly random and the internal distribution of each individual 3DGS is non-uniform (left column of Figure 4(d)). To address this, we leverage the smoothness of the neural network (as noted by reviewer jqqm) to regularize the optimization process. This allows us to achieve a more uniform and structured 3DGS, which is shown in the right column of Fig. 4(d).

Distribution unification stage (Figure 5): Although the per-scene overfitting stage improves the internal structure of individual 3DGSs, the external distribution among different 3DGSs remains non-uniform, as each 3DGS is optimized independently. To further align the distribution across different 3DGSs, we introduce the distribution unification stage. This second stage also utilizes the smoothness of the neural network to ensure that the 3DGSs from different scenes share a more consistent and unified distribution. More details and explanations about the distribution unification stage can be found in Figure 5 and L400-416 in the original paper.

[W3] Evaluation of 3D geometry.

Response: As you suggested, we have conducted the geometry evaluation on the Thuman and CustomHuman datasets. (SHERF does not reconstruct the 3D geometry). As reported in the Table below, our Human-DAD can have much better 3D geometry reconstruction performance than other methods. Even so, we want to emphasizethat this work focuses on the novel view synthesis, the reconstructed 3D geometry is not the objective of this paper.

[W1] Discussion about direct vs indirect supervision.

As a reviewer interested in the direct vs indirect paradigm discussion, I'd like to point out to this answer.

The author's answer is self-evident, but I argue that there are three logical fallacies.

1. Inappropriate example

Your answer is a good example for prediction/regression tasks where the network output must be accurate.

However, the main task of your paper is 'novel-view synthesis'.

For this task, how natural the rendering (splatted Gaussians) is is more important than the accuracy of the 3D Gaussian estimate.

In your answer, a more appropriate example is comparing the aggregated samples "output.mean()".

I think, this is what reviewer jqqm's sentence "Empirically, we are finally evaluating the NVS results in image space." means.

2. There is no groundtruths of 3D Gaussians.

Your answer is based on the assumption that a precise GT exists (value: 256).

In the paper's scenario, 3D Gaussians are not GT and arise from network estimation. (more close to psudeo-GTs)

So, in your example, it would be correct to supervise an incomplete value (around 256) obtained by guessing, rather than directly supervising the value 256.

3. Direct supervision does not completely eliminate ill-posedness.

This is because there are infinitely many correct answers for the target subject.

For example, in the case of super-resolution, the answer image corresponding to the input image is infinite.

This problem is not solved by direct supervision of high-resolution images.

Likewise in your task, an infinite number of 3D Gaussians can be mapped to a single person.

For me, the direct vs indirect paradigm is very interesting and important in 3D literature. However, as with the questions I raised, it seems there are still some unresolved questions.

I would appreciate a constructive discussion on my questions.

[W2] Distribution unification.

I still feel like it's an empirical interpretation.

I agree that the network provides blurry output.

However, the main goal is to make the distribution consistent, as in Figure 5, which is a different goal.

If your argument is correct, numerous domain gap problem in DL should be solvable with your approach.

Ex) By training the dataset of domain A and domain B together, the distribution between the data is set similarly and achieves higher performance.

Are there any previous studies that can justify your method in terms of 'data distribution unification'?

If you can provide them, I think it will be easier for me to understand.

I would like to clarify my comment.

As I understand it, your toy example is as follows:

model output, 3D Gaussians --- outputs

rendered images --- outputs.sum()

GT images --- 256

pseudo-GT 3D Gaussians --- torch.ones(256,1)

• Indirect paradigm

GT images --- (supervision) ----> model

• Direct paradigm

GT images --- (construction) ---> pseudo-GT 3D Gaussians --- (supervision) ---> model

I would like to point out that the (construction) process is ignored in your toy example.

The construction process is not complete.

However, your toy example proceeds assuming that torch.ones(256,1) is completely constructed.

3D Gaussian is not GT but pseudo-GT, and your paper also addresses that.

GT images <---> pseudo-GT 3D Gaussian is a many-to-one relationship, because of two issues.

1. Inherent ill-posedness: Imagine a 3D Gaussian that is occluded in all view directions. That Gaussian can be removed or added to reconstruct the GT images in the same way.

2. Network error: The network used to create pseudo-GT 3D Gaussians can have errors. If you change the random seed, the learning process will change, and the 3D Gaussians that are created will also change.

I have consistently argued that direct supervision also has its drawbacks.

Your main argument is that "direct supervision is better for the monocular human rendering task".

I believe that the direct vs indirect discussion should not be complete without considering this drawback.

Could you kindly clarify how to validate that the indirect paradigm is better than the direct paradigm for our monocular human rendering task, given our specific pipeline? Since the proposed method focuses solely on this task, it is essential to frame the comparison within this context.

Based on your suggestion, we conducted a toy experiment using a neural network to generate a tensor of shape (256, 1), where the elements are no longer fixed to 1.

model output, 3D Gaussians --- outputs

rendered images --- outputs.mean()

GT images --- 256

pseudo-GT 3D Gaussians --- network generated tensor (256,1)

The correspondence is fully aligned. Even under this adjusted setup, the direct paradigm consistently outperformed the indirect paradigm.

There are two possible scenarios where an object cannot be observed from all view directions:

1. The object does not exist.
2. The object is entirely obscured by something, e.g., a bag.

If an object cannot be observed from all GT image view directions, it should be considered nonexistent, and the 3DGS does not need to overfit it.

We also ensured full control over randomness by fixing all seeds (Python, NumPy, PyTorch, CUDA, etc.). As shown in Figure 19 , the rendered image quality is extremely high, making the network error negligible. The proxy-ground-truth 3D Gaussians are fixed after the construction. Hence, the "many-to-one" problem never occurs within the direct paradigm.

The contradiction in your statement lies in the fact that you first assert that there is no GT for 3DGS and that even our constructed proxy GT cannot be deemed as a GT. However, you subsequently imply that there could potentially be multiple 3DGS GTs for the same scene.

We agree that the direct paradigm has limitations. However, we are not addressing super-resolution problems or any other 3DGS tasks beyond monocular human rendering.

Could you provide specific drawbacks of the direct paradigm in the context of our monocular human rendering pipeline? This would help refine the discussion and clarify the scope of the work.

[Rebuttal to Reviewer 8d88]

[W1] The supervision gap between the direct and indirect paradigm is not clear.

Response: During the training process, the models of direct paradigm and indirect paradigm have similar memory consumption. During the two-stage proxy-ground-truth preparation process, we used almost 1 day to overfit 480 scans of Thuman. The cost time is acceptable. We have conducted several experiments to see whether more iteration or larger batch size can improve the performance. As shown in the Table below, the actual improvement is limited.

Actually, we consider the direct paradigm to be better than the indirect paradigm because it reduces the solution space and makes the model easier to train. We have demonstrated that the photometric loss will lead to many different solutions for one particular scene, as shown in Fig. 4(b). Moreover, one pixel of an image may be decided by several different Gaussians, and each Gaussian is decided by the position, SHs, rotation, scale, and opacity attributes. Such a large solution space is very difficult to learn. Hence, we use a two-stage construction process to achieve a fixed proxy-ground-truth 3D Gaussian attributes, thus reducing the solution space, which makes the model easier to train.

Moreover, the indirect paradigm typically only supports regression training, however, such a direct paradigm supports training a diffusion model which is more efficient than the regression-based model to learn the 3D Guassian attribute distribution (see Figure 8).

[W2] Missing comparison with Human3Diffusion.

Response: Thank you for the suggestion. We have added the comparison with Human3Diffusion both quantitatively and qualitatively, as shown in the Table below and Figure 23 in the updated paper.

[W3] The figures are too small.

Response: The PDF has high quality, please zoom in for details.

[Q1] Performance on in-the-wild and out-of-distribution images.

Response: First, the evaluation of in-the-wild scenarios has already been performed in Figure 7 of our original paper version. Second, our model is trained on Thuman, and then tested on other three datasets of CityuHuman, 2K2K, and CustomHuman, which form out-of-distribution data. Please refer to Table 2 and Figure 6 of our paper.

[Q2] Will Human-DAD support multi-view?

Response: Our targeted monocular setup is harder than the multi-view setup. There is no theoretical obstacle in adapting our framework to the multi-view setup. We will briefly mention this as potential future work in our paper.

Concern about direct vs indirect

We would like to clarify the definitions of the direct paradigm and indirect paradigm again:

• Direct paradigm: Given a single-view image, the goal is to generate 3D Gaussians directly, using the proxy-ground-truth 3D Gaussians as supervision.
• Indirect paradigm: Given a single-view image, the goal is to generate 3D Gaussians, transform the 3D Gaussians into images, and use ground truth images for supervision.

The issue with the indirect paradigm is not the supervision in the image space or the loss design but the many-to-one problem. This means there are multiple solutions for rendering the same image. For instance, two completely different sets of 3D Gaussians can result in the same rendered image, as demonstrated in Figure 4(b). This many-to-one phenomenon has also been observed in prior work [1].

To address this concern, we designed a toy experiment. The objective of a simple neural network in this experiment is to produce an output tensor with the shape (256, 1), such that the sum of its elements equals 256 (output.sum() = 256).

The objective of a simple neural network in this experiment is to produce an output tensor with the shape (256, 1), such that the sum of its elements equals 256 (output.sum() = 256).

input = torch.randn(256)
output = toymodel(input)

There are numerous solutions to achieve a sum of 256. For example:

• Solution 1: All elements are equal, e.g., [1, 1, 1, ..., 1] (256 elements of value 1).
• Solution 2: Non-uniform distribution, e.g., [2, 2, 2, ..., 0.5, 0.5] (most elements are 2, with a few elements summing to the remainder).
• Solution 3: Random distribution, e.g., [10, 5, 0, ..., 0.1, 1.9] (values vary widely but still sum to 256).

This highlights the many-to-one problem, as multiple configurations of the tensor can achieve the same target sum, making supervision ambiguous.

We can train the toy model using both the direct paradigm and the indirect paradigm as follows:

Direct Paradigm: In the direct paradigm, we use a tensor [1, 1, 1, ..., 1] with shape (256, 1) where all elements are 1 as the ground truth. The optimization process ensures each element matches the corresponding element in the ground truth tensor.

ground_truth = torch.ones(256,1)
output = toymodel(input) 
loss = torch.abs(output-ground_truth).mean()
optimizer.zero_grad()
loss.backward()
optimizer.step()

indirect paradigm: In the indirect paradigm, we sum the output tensor and compute the difference between the sum and the target value, 256. The optimization process ensures the total sum matches the target, without considering individual element values.

output = toymodel(input)
loss = torch.abs(output.sum()-256)
optimizer.zero_grad()
loss.backward()
optimizer.step()

I have run it 5 times, the results are:

The error of the direct paradigm is

while the error of the indirect paradigm is

The error of the direct method is two orders of magnitude lower than that of the indirect method.

To further clarify, I have provided the Jupyter notebook of this toy experiment. You can run the experiment on your local machine.

The results of the toy experiment clearly show that, although using torch.abs(output.sum() - 256) is more straightforward, the many-to-one problem significantly affects performance, even in such a simple toy scenario. To address this challenge, we first construct proxy-ground-truth 3D Gaussian attributes to enable direct training.

Thank you for dedicating your time and effort to reviewing our work. We have carefully considered and addressed all the concerns you raised in your review, as outlined in our response and reflected in the updated manuscript. As the Reviewer-Author discussion phase is nearing its conclusion, we eagerly await any further feedback from you. Should you have any additional questions, we would be delighted to provide detailed responses.

[Q1] Evaluation with ground truth 3D shapes.

Response: It seems only feasible on GTA and SiTH by providing ground truth occupancy or SDF. (LGM generates 3D shapes from multiview images, and SHERF needs SMPL). The quantitative and qualitative results are in the Table below and Figure 21 of the updated paper.

[Q2] Optimizing a 3DGS model on frontal and back images.

Response: Yes, we simply map the front and back view to the point cloud. We provide results of optimizing 3DGS on only the front and the back images in Figure 22 of the updated paper. The results are full of randomness, while our Human-DAD can significantly reduce the randomness and improve the visual quality.

Ethics Concerns: We follow the instructions of the datasets to present the results. The kid is the 00005 person in 2K2K; their original website also presents the kids’ faces.

[Rebuttal to Reviewer jqqm]

[W1] Whether the neural network at the first stage is necessary?

Response: The utilization of a neural network at the first stage is indeed necessary, since the distributional randomness of vanilla-3DGS outputs cannot be fully regulated by the second stage. Here we made an additional experiment as shown in Figure 20 of the updated paper. When vanilla-3DGS is used in the first stage, the overall itting results only reach 34.76 PSNR, while the adoption of a neural network can achieve 40.83 with better appearance details.

[W2&W3] Whether the direct paradigm is better than the indirect paradigm?

Response: First, we need to point out that both PSNR and SSIM metrics are the higher the better while the LPIPS metric is the lower the better. Hence, our direct (last row) supervision is PSNR 1.01 and LPIPS 0.005 better than the direct supervision (first row) in Table. 3 (same SSIM performance).

We have shown that the photometric loss will lead to many different solutions (see Fig. 4(b)). Moreover, one pixel of an image may be decided by several different Gaussians, and each Gaussian is decided by position, SHs, rotation, scale, and opacity attributes. Such large solution space can be hard to learn. Hence, we use a two-stage construction process to achieve a fixed proxy-ground-truth 3D Gaussian attributes, reducing the solution space and making the model easier to train.

Moreover, the indirect paradigm typically only supports regression training, however, such a direct paradigm supports training a diffusion model which is more efficient than the regression-based model to learn the 3D Guassian attribute distribution (see Figure 8).

[W4&Q3] The diffusion model design is very confusing, especially for the extra steps for alpha, scale, and quaternion attributes.

Response: During the training process, we find that jointly diffusing all 3D Gaussian attributes usually leads to collapse training (mainly because the dimension of the spherical harmonics is much more than other attributes). Hence, we use a diffusion model to learn the distribution of the spherical harmonics attribute, which fundamentally decides the appearance of the human body. We consider the spherical harmonics generated by this diffusion model to be coarse spherical harmonic attributes. Empirically, PDR and HaP have demonstrated that the generated point clouds by diffusion models usually still contain some noise, hence, we set up the extrastep to learn the remaining noise of the coarse spherical harmonic attributes and remove it. Because we are still learning a noise term which will be minus by the coarse spherical harmonic attributes, the extra-step is still the noise-diffusing. Also, we predict the scale, rotation and opacity attributes at this step, these attributes using the $L1$ regression loss.

[W5] Stabe diffusion and HaP are not contributions.

Response: The stable diffusion model or the position generator alone cannot form our technical contribution. In addition to the novel direct attribute-level supervision paradigm, the overall construction of our conditional Gaussian diffusion framework should be regarded as a systematic effort. Especially in the field of human-centric tasks (e.g., either reconstruction or NVS) featured by strong prior, it is rather common that the overall processing pipeline in a new work is composed of several aspects of tools (e.g., learning architectures, parametric human body models, depth/normal estimation, etc). For example, GTA utilizes implicit function, triplane modulem, and transformer module. SiTH utilizes implicit function, stable diffusion, and ControlNet. The core problem lies in whether one can combine different components (possibly with necessary adaptations and modifications) in a better way for applying to specific settings and scenarios.

Concern about the diffusion model.

noise_t = diffusion_model(noised_sh_ground_truth, condition_features, time_step)

The noised_ground_truth is the proxy-ground-truth 3D Gaussian spherical harmonics with added noise, the condition_features are the human-centric features (SMPL-semantic and backimage).

The diffusion model predicts the noise_t at given time step, and we gradually remove the noises from a gaussian noise to achieve a final 3D Gaussian attribute, in our paper, we achieve the spherical harmonics at the diffusion inference process. (We find that jointly diffusing all 3D Gaussian attributes usually leads to collapse training, mainly because the dimension of the spherical harmonics is much more than other attributes). Hence, we use a diffusion model to learn the distribution of the spherical harmonics attribute)

Emperically[2][3], the point cloud-based diffusion models usually cannot remove all the gaussian noises and the direct achieved spherical harmonics still remain some noise, hence, we need a further step to remove these remained noise.

Actually, the extra-step is not novel and it has already been demonstrated its effectiveness in both PDR[2] and HaP[3]. In PDR[2] and HaP[3], they name the extra-step as refinement.

spherical_harmonics_coarse = inference_diffusion_model(condition_features)
noise_extra_step, scale, rotation, opacity = model_extra_step(condition_features, spherical_harmonics_coarse)
spherical_harmonics_final = spherical_harmonics_coarse - noise_extra_step

You can consider model_extra_step as a regression model, we call this as the extra-step because we are still learning the noise noise_extra_step remained in spherical_harmonics_coarse.

[2]. Lyu, Z., Kong, Z., Xu, X., Pan, L., & Lin, D. (2021). A conditional point diffusion-refinement paradigm for 3d point cloud completion. ICLR 2022 .

[3]. Tang, Y., Zhang, Q., Hou, J., & Liu, Y. (2023). Human as Points: Explicit Point-based 3D Human Reconstruction from Single-view RGB Images. arXiv preprint arXiv:2311.02892 .

Concern about the direct paradigm.

We would like to clarify the definitions of the direct paradigm and indirect paradigm again:

• Direct paradigm: Given a single-view image, the goal is to generate 3D Gaussians directly, using the proxy-ground-truth 3D Gaussians as supervision.
• Indirect paradigm: Given a single-view image, the goal is to generate 3D Gaussians, transform the 3D Gaussians into images, and use ground truth images for supervision.

The issue with the indirect paradigm is not the supervision in the image space or the loss design but the many-to-one problem. This means there are multiple solutions for rendering the same image. For instance, two completely different sets of 3D Gaussians can result in the same rendered image, as demonstrated in Figure 4(b). This many-to-one phenomenon has also been observed in prior work [1].

To address this concern, we designed a toy experiment. The objective of a simple neural network in this experiment is to produce an output tensor with the shape (256, 1), such that the sum of its elements equals 256 (output.sum() = 256).

input = torch.randn(256)
output = toymodel(input)

There are numerous solutions to achieve a sum of 256. For example:

• Solution 1: All elements are equal, e.g., [1, 1, 1, ..., 1] (256 elements of value 1).
• Solution 2: Non-uniform distribution, e.g., [2, 2, 2, ..., 0.5, 0.5] (most elements are 2, with a few elements summing to the remainder).
• Solution 3: Random distribution, e.g., [10, 5, 0, ..., 0.1, 1.9] (values vary widely but still sum to 256).

This highlights the many-to-one problem, as multiple configurations of the tensor can achieve the same target sum, making supervision ambiguous.

We can train the toy model using both the direct paradigm and the indirect paradigm as follows:

Direct Paradigm: In the direct paradigm, we use a tensor [1, 1, 1, ..., 1] with shape (256, 1) where all elements are 1 as the ground truth. The optimization process ensures each element matches the corresponding element in the ground truth tensor.

ground_truth = torch.ones(256,1)
output = toymodel(input) 
loss = torch.abs(output-ground_truth).mean()
optimizer.zero_grad()
loss.backward()
optimizer.step()

indirect paradigm: In the indirect paradigm, we sum the output tensor and compute the difference between the sum and the target value, 256. The optimization process ensures the total sum matches the target, without considering individual element values.

output = toymodel(input)
loss = torch.abs(output.sum()-256)
optimizer.zero_grad()
loss.backward()
optimizer.step()

I have run it 5 times, the results are:

The error of the direct paradigm is $10^{-4}$, while the error of the indirect paradigm is $10^{-2}$

The error of the direct method is two orders of magnitude lower than that of the indirect method.

To further clarify, I have provided the Jupyter notebook of this toy experiment in the ZIP. You can run the experiment on your local machine.

The results of the toy experiment clearly show that, although using torch.abs(output.sum() - 256) is more straightforward, the many-to-one problem significantly affects performance, even in such a simple toy scenario. To address this challenge, we first construct proxy-ground-truth 3D Gaussian attributes to enable direct training.

[1]. Qin, D., Lin, H., Zhang, Q., Qiao, K., Zhang, L., Zhao, Z., ... & Komura, T. (2024). Instant Facial Gaussians Translator for Relightable and Interactable Facial Rendering. arXiv preprint arXiv:2409.07441 .

[Rebuttal to Reviewer kXVE]

[W1&W2&W4] Wrong index and missing reference.

Response: We are sorry for these mistakes. They have been fixed in the updated paper version.

[W3] Ablation study on image-conditioned diffusion model.

Response: Thanks very much for your constructive suggestion. We have supplemented an additional ablation study, as shown in Figure 18 of the updated paper. It can be observed that such scheme completely fails, with no meaningful outputs obtained. Obviously, the overall diffusion process and network components must be carefully designed.

[W5] Applying the two-stage construction on other datasets.

Response: There is no problem in applying the proposed proxy ground-truth data construction process to other datasets. Note that the scale issue can be trivially addressed by normalizing all scans into the bounding scope. We additionally applied the construction process on 2K2K and CustomHuman datasets. Please refer to Figure 19 in the updated paper.

[W6] What auxiliary constraints are imposed should be elaborated at line 248 in the main paper.

Response: The auxiliary constraints are as follows:

$\Sigma_{n=1}^N\|\mathtt{radius}(\mathbf{s}_n)- \mathtt{kdist}(\mathbf{p}_n) \|^2$

$\Sigma_{n=1}^N \| \mathbf{o}_n -1 \|^2$ where $p$ is the 3D Gaussian position, $s$ is the scale, $o$ is the opacity, $radius(·)$ and $kdist(·)$ are the operations to get the radiuses of the Gaussians and the mean distances between the Gaussians and their neighbors.

[Q1] How are semantic labels embedded in the latent space through MLPs?

Response: The one-dimensional SMPL semantic label is fed into a positional encoding layer, whose output vector is further passed into the subsequent MLPs for feature learning. Note that all such MLPs are trained together with the diffusion model.

[Q2] Artifacts of thin dark lines in Figure 6.

Response: These thin dark lines are the boundary lines of the ground truth images. We visualized the alpha channel map of ground truth images and overlapped it with our results to analyze the performance of HumanDAD. These boundary lines have been removed in the updated paper.

[Rebuttal to Reviewer GRmh]

[W1&W2] 1) Unclear claim about the “direct” paradigm. 2) Unclear writing.

Response: In fact, the concepts of "direct" and "indirect" paradigms are irrelevant to the specific implementations (such as the number of stages as the reviewer concerns) of the intermediate learning process. We have explicitly explained in our paper (e.g., in the middle of the Abstract, in the third paragraph of the Introduction, etc) that the criteria lies in whether the supervision is imposed over the 3D Gaussian attributes or over the rendered 2D image pixels.

It is reasonable to make such definitions because, when building generalizable 3DGS learning frameworks, the actually-desired network outputs are the 3D Gaussian attributes. Once a certain Gaussian attribute set is given, the splat-based rasterizer is just a principled (rule-based, without any parameters to be learned) rendering process, and thus the rendered image at arbitrary view is uniquely determined by the given Gaussian attribute set. This means that, although the rendered image is the direct result for people to watch, it is the indirect result in terms of the learning target. Corresponding to your comments, we added some necessary highlights in our paper to make the concepts clearer.

[W3] 1) It is unhow ablation studies verify the effectiveness of each component. 2) It is hard to understand where the significant improvements come from.

Response: There might be some misunderstandings about the organization structure of our experiments. From the reviewer's descriptions like ``they present single cases of failure ...'', it seems that the reviewer sees Table 1 as our ablation study, but it's not. Our ablation results are reported in Table 3, where we remove each component to see the performance change. Note that the results in Table 1 are produced from simpler baseline architectures with ground-truth point clouds as Gaussian positions, just in order to compare indirect supervision and different ways of preparing proxy ground-truth Gaussian attributes for direct supervision.

[W4&Q2] Providing animation results.

Response: Thanks very much for the valuable suggestion. There is no problem for our approach to achieve animation. We have supplemented necessary animation experiments for novel pose synthesis in the newly-revised paper version (see Figure 17).

[W5] The overall contribution/novelty is not well supported.

Response: As explicitly highlighted in the end of Introduction, in addition to performance gains, our technical contributions are two-fold:

1. The direct attribute-level supervision paradigm achieved by the creation of proxy ground-truth Gaussian attribute sets.
2. The integrated generalizable human 3DGS learning framework consisting of two sub-problems: a. Producing the required conditioning signals from the input monocular human image. b. Designing the conditional Gaussian attribute diffusion process.

Here, the reviewer GRmh mainly challenges our claimed second contribution. However, we beg to differ on the judgment that our contribution is invalid only because “transformers, stable diffusion, pixel-aligned feature learning, etc, have been involved in prior works”. In fact, our second contribution should be regarded as a systematic effort. Especially in the field of human-centric tasks (e.g., either reconstruction or NVS) featured by strong prior, it is rather common that the overall processing pipeline in a new work is composed of several aspects of tools (e.g., learning architectures, parametric human body models, depth/normal estimation, etc). For example, GTA utilizes implicit function, triplane modulem, and transformer module. SiTH utilizes implicit function, stable diffusion, and ControlNet. The core problem lies in whether one can combine different components (possibly with necessary adaptations and modifications) in a better way for applying to specific settings and scenarios.

[Q1] Reasons for our large numerical improvements.

Response: The large performance gains exactly come from our two aspects of technical contributions, which are explicitly claimed in our paper, i.e., 1) a novel proposal of direct attribute supervision, and 2) a systematic effort on constructing a Gaussian attribute diffusion framework involving producing the conditional signals and designing the diffusion procedures.

While our analyses have been presented in Section 4.4, here we further detailedly discuss how to comprehend the 5 different rows of quantitative results as reported in Table 3 to help facilitate the reviewer' understanding.

-- Observing the first row: We changed the direct attribute-level supervision scheme and turned to impose the indirect pixel-level supervision. The resulting performance still outperforms previous state-of-the-arts whose performances are reported in Table 2. This supports our claimed contribution of the whole attribute diffusion framework by effectively adapting and combining different components and carefully designing the conditional diffusion procedures. We kindly remind that this contribution (i.e., the second item as summarized in the end of Introduction) shouldn't be totally overlooked

-- Comparing the first and last rows: Based on our constructed powerful attribute diffusion framework, the adoption of direct attribute supervision further brings about 1dB performance improvement. Here, an important reminder is that the absolute numerical value of performance gain must be evaluated in a relative manner. For example, lifting 29dB to 30dB is typically much harder than lifting 25dB to 26dB.

-- Comparing the third, fourth, and last rows: Starting from the full model, gradually removing the "Back Image" and "SMPL Semantic" components leads to continuous performance degradation. This demonstrates the effectiveness of our designed conditioning signal extraction process.

-- Comparing the second and last rows: This demonstrates the necessity of diffusing the desired Gaussian attributes instead of regressing them. Note that, although the regression-based model variant still achieves highly competitive performance benefiting from direct supervision and effective conditioning signal extraction, its visual effect is usually problematic, as illustrated in Figure 8. This observation is consistent with the numerous diffusion-based generative modeling approaches in various fields.

Thank you for dedicating your time and effort to reviewing our work. We have carefully considered and addressed all the concerns you raised in your review, as outlined in our response and reflected in the updated manuscript. As the Reviewer-Author discussion phase is nearing its conclusion, we eagerly await any further feedback from you. Should you have any additional questions, we would be delighted to provide detailed responses.