Perm: A Parametric Representation for Multi-Style 3D Hair Modeling

Dear Reviewer BeFM,

Thank you for your positive review and constructive feedback! Below, we address your concerns point by point:

Q: “A potential improvement direction, from my perspective, is to demonstrate the proposed representation on 3D generation tasks / other amortized pipelines, such as X-conditioned 3D avatar (with controllable hair) generation.”

A: Thank you for this insightful suggestion. While this direction is beyond the primary scope of our paper, we agree that our proposed representation could be integrated into 3D diffusion training based on our 2D uv texture design. With more 3D data available, it would certainly be feasible to train a 3D diffusion model for full-head synthesis with controllable hair modeling. We consider it as a promising research direction and look forward to seeing progress in this area by future research endeavors.

---

Q: “Regarding the overview pipeline, is the residual branch necessary, since the StyleGAN model with U-Net SR module can already generate pretty fine-grained results (intuitively).”

A: The residual branch is necessary because the output from our SR module lacks all high-frequency details, which needs to be complemented by the synthesized residual textures. In our revised paper, we include an example generated without the residual texture synthesis in Figure 22, which only recovers the global structure of the ground truth hairstyle. After adding the synthesized residual textures, the curl patterns are generated to better match the ground truth. Besides, our design with this residual branch also enables applications such as hairstyle interpolation with different granularity (Figure 28). We can separately change the curliness of the hairstyle, thus generating novel hairstyles ranging from straight to curly while maintaining a nearly identical haircut.

Dear Reviewer BeFM:

As the rebuttal phase approaches its end with only two days remaining for PDF revisions, we would like to kindly remind you of our responses to your comments and questions.

We have carefully addressed all your feedback and have made thoughtful updates to the manuscript to address your concerns. We hope that our efforts meet your expectations and provide clarity on the points you raised.

If you have any remaining questions or concerns, we would be happy to discuss them further and make additional revisions as needed.

Thank you again for your time, thoughtful feedback, and invaluable contributions to improving our work.

Dear Reviewer vjoV,

Thank you for your positive review and feedback! Below, we address your concerns point by point:

Q: “The hair details are still misaligned. Is this restricted by model capacity given the current parameterization?. Are there any suggestions to improve the parameterization?”

A: Thank you for raising this important question! In the revised paper, we have included a discussion of these limitations in Appendix E. Briefly, we hypothesize that the observed artifacts primarily arise from two key issues:

1. Our training data is limited, as high-quality 3D hair assets are notoriously challenging to acquire. This limited data distribution may restrict the generative capabilities of our model. Addressing this issue requires the development of a systematic framework to efficiently capture a diverse range of real-world data -- a critical challenge that remains unresolved in the hair modeling area.
2. Our current single-view hair reconstruction pipeline adopts a relatively straightforward approach, relying on an optimization process that primarily focuses on per-pixel strand information encoded in the 2D rendered images. These supervisions may fail to capture fine strand-level details, resulting in suboptimal output when the hair is tied in certain hairstyles such as buns or heavily occluded by accessories such as hats. To enhance reconstruction quality, better geometric clues and more advanced optimization techniques could be explored.

Developing methods to model the random strands in input images is definitely another promising future research direction, as it increases the expressiveness of our parametric model. Achieving this, however, would require a clean separation of these random strands from the main hairstyle, as well as a suitable parameterization for them. Moreover, we would need some specific 3D training data, as most existing 3D hair models do not include such random flyaway strands. In conclusion, while this is an interesting direction to explore, it requires significant effort to make these solutions feasible.

---

Q: “Is it possible to combine the proposed method with NeRF/3DGS for realistic full-head/full-body reconstruction?”

A: Yes, this is another exciting direction for future work! As we discussed in the related work section, existing approaches such as GaussianHaircut, GaussianHair, GroomCap, and HVH have successfully combined strand-based hair with 3DGS/NeRF for 3D hair capture, where 3D Gaussians or neural primitives are attached to individual strands to model their appearance. In this context, our Perm model could serve as a pre-trained geometry prior, enabling the generation of plausible strands in invisible parts of a hairstyle (e.g., interior or occluded regions). This would provide a more robust geometric proxy for NeRF and 3DGS, thereby enhancing full-head and full-body reconstruction with a complete and controllable hair area.

Dear Reviewer vjoV:

As the rebuttal phase approaches its end with only two days remaining for PDF revisions, we would like to kindly remind you of our responses to your comments and questions.

We have carefully addressed all your feedback and have made thoughtful updates to the manuscript to address your concerns. We hope that our efforts meet your expectations and provide clarity on the points you raised.

If you have any remaining questions or concerns, we would be happy to discuss them further and make additional revisions as needed.

Thank you again for your time, thoughtful feedback, and invaluable contributions to improving our work.

Dear Reviewer vjoV:

We sincerely appreciate the time and effort you have dedicated to reviewing our manuscript and for providing thoughtful and constructive feedback. With the deadline for revising the manuscript approaching on November 27th, we kindly ask if you have any remaining questions or require further clarification. We would be more than happy to provide additional information or address any remaining concerns.

Thank you again for your thorough review and valuable insights.

Dear Reviewer vjoV:

Thanks for your support of our work! We sincerely appreciate your thoughtful feedback and are truly grateful for your positive recommendation!

Dear Reviewer SDZK,

Thank you for your detailed and thoughtful review! We sincerely appreciate your feedback and have carefully revised our paper to address your concerns. Below, we provide point-by-point responses to your comments:

Q: “The paper only compared Groom-Gen and presented a limited number of samples.”

A: Thanks for pointing it out! We choose GroomGen as it’s the most closely related work to ours (an unconditional generative model of 3D hair). To better illustrate the differences between GroomGen and our approach, we have included more random sample results in Figure 27 of our revised paper (14 samples for each method).

---

Q: “In the comparison with GroomGen, it is evident that all random sampling results generated by GroomGen exhibit markedly unrealistic strand structures. How can this phenomenon be explained?”

A: We have provided some possible reasons in Appendix C.4. To summarize:

1. While GroomGen’s strand VAE formulation with frequency features seems theoretically sound, its training strategy or loss design makes the trained model hard to fully represent the strand geometry. As demonstrated in Figure 23, both the official GroomGen strand VAE and our reimplementation failed to accurately reconstruct given strands.
2. GroomGen’s guide strands are quantitatively downsampled from the full hair, thus preserving all high-frequency details. This design choice increases the variation of guide strands, making the Hairstyle VAE more difficult to train. As shown in Figure 24, the trained Hairstyle VAE may produce some unnatural guide strands during random sampling.
3. GroomGen’s neural sampler uses a CNN-based GAN architecture, which we found very unstable during training. Consequently, the upsampled hairstyles often include unnatural curls and flyaway strands, as shown in Figure 27. In contrast, our method formulates this process as a deterministic mapping using U-Net, enabling stable supervised training and more natural upsampling results.

---

Q: “In Figure 5, the rendering results of the PCA method and the authors' proposed Freq. PCA (Ours) methods are similar, to the extent that they are almost indistinguishable by visual inspection. Could the authors further elaborate on the differences and merits of these two methods in terms of specific rendering details?”

A: Thanks for raising this important question! We choose Freq. PCA as our design because it is formulated in the frequency domain, which better aligns with the cyclical growth behavior of human hair. Our experiments also show that this approach preserves strand curliness more effectively than spatial-domain PCA, as reflected in the curvature error reported in Table 1. To further clarify this difference, we added a visual comparison in Figure 15 in our revised paper, where strands are color-coded by their curvature error. The quantitative values are also provided, which are 2.671 and 2.341, respectively. These experiments further demonstrate that Freq. PCA better preserves strand curliness both qualitatively and quantitatively.

---

Q: “What is the extent of the impact of Guide Texture Upsampling on the final rendering results? Incorporating an ablation study on the upsampling module is crucial for comprehensively evaluating its significance and contribution.”

A: This experiment is already included in Appendix C.2 and Figure 21, where we compare our super-resolution module with alternative interpolation methods. As shown in Figure 21, naive approaches such as nearest neighbor and bilinear interpolation consistently produce various artifacts, and our method achieves the most natural results that match the ground truth hair best.

---

Q: “I suggest that the authors showcase failure cases encountered in random generated and single-view reconstruction tasks, and provide a thorough analysis and discussion of such cases. This not only helps to uncover potential limitations of the current method but also offers valuable insights and references for future research endeavors.”

A: Thanks for this valuable suggestion! In the revised paper, we have added examples of typical failure cases in Appendix E (Figure 34 and Figure 35), along with a detailed discussion of their causes and potential solutions. To summarize, these artifacts are primarily caused by limitations in our training data and the simplicity of our current optimization scheme. Expanding our training data with more realistic hairstyles and designing more advanced optimization techniques would likely improve both the generation and reconstruction quality of our model.

Dear Reviewer SDZK:

As the rebuttal phase approaches its end with only two days remaining for PDF revisions, we would like to kindly remind you of our responses to your comments and questions.

We have carefully addressed all your feedback and have made thoughtful updates to the manuscript to address your concerns. We hope that our efforts meet your expectations and provide clarity on the points you raised.

If you have any remaining questions or concerns, we would be happy to discuss them further and make additional revisions as needed.

Thank you again for your time, thoughtful feedback, and invaluable contributions to improving our work.

Dear Reviewer 4JLz:

Thank you for your detailed and thoughtful review! We truly appreciate your feedback and have carefully revised our paper to address your concerns. Below, we provide point-by-point responses to your comments:

Q: “While the motivation of this paper is clear and the introduction is enjoyable to read, I am extremely frustrated with the forward-referencing (or no-referencing) writing style in Section 3”

A: We apologize for the confusions caused by our writing style. We acknowledge that our previous writing is a bit misleading and unclear. In our revised paper, we have rewritten Section 3 to improve its clarity and flow. Specifically, we now introduce intuitive explanations for key terms (e.g., "guide textures," "residual textures," etc.) at the beginning of this section and include hyperlinks to the specific subsections where each concept is explained in detail. Please let us know if this version is clear enough to you. We keep "Strand Representation" as Section 3.1 because it introduces the concept of strand PCA coefficients, which is later used by all the textures.

---

Q: “The paper presents the method as a parametric model, but unlike deterministic models such as SMPL or FLAME, it lacks a clear deterministic nature as it involves stochastic elements (e.g., generative processes or randomness in texture synthesis).”

A: Sorry but we believe this is a misunderstanding. Unlike Diffusion Models, GANs and VAEs are deterministic models. This means that, given a fixed latent code, the model will always produce the same output once trained. The process of finding the latent code corresponding to a target image, often called GAN Inversion, is a well-studied problem, with a comprehensive survey in [1]. In our case, we have already performed such experiments in Section 4.2 and Figure 8, where we encode hair geometry into a fixed set of Perm parameters through optimization. The results in Figure 8 demonstrate a close match between the optimized outputs and the ground truth, validating the deterministic and expressive nature of our Perm model.

---

Q: “Is S = {p1 , p2 , . . . , pL } the output of Equation 2?”

A: Yes it is the output of Equation 2. Sorry for the confucions caused by our writing and we have updated the text to clarify this point – please see Lines 216–217 in our revised paper.

---

Q: “Analysis of PCA coefficients”

A: We apologize for the oversight in the previous manuscript. We have now included this analysis in Appendix B.4 and Figure 19. In this analysis, we show a series of hairstyles reconstructed using different numbers of strand PCA coefficients. The results indicate that using 10 coefficients per strand strikes the best balance between smoothness and global structure preservation. This is why we chose to use 10 coefficients for the global structure and 54 coefficients for high-frequency details.

---

[1] Xia et al. "Gan inversion: A survey." IEEE transactions on pattern analysis and machine intelligence 45.3 (2022): 3121-3138.

Dear Reviewer 4JLz:

As the rebuttal phase approaches its end with only two days remaining for PDF revisions, we would like to kindly remind you of our responses to your comments and questions.

We have carefully addressed all your feedback and have made thoughtful updates to the manuscript to address your concerns. We hope that our efforts meet your expectations and provide clarity on the points you raised.

If you have any remaining questions or concerns, we would be happy to discuss them further and make additional revisions as needed. Otherwise, if you find our updates satisfactory, we kindly invite you to consider reevaluating your score.

Thank you again for your time, thoughtful feedback, and invaluable contributions to improving our work.

Hi Yuxiao,

Thank you for initiating this important discussion. We sincerely apologize if any technical aspects of GroomGen were overlooked, and we truly appreciate your engagement and the opportunity to clarify our contributions.

1. Regarding Our Design Choices and Novelty

Our core contribution lies in the PCA-based strand representation and the interpretable disentanglement of full hair generation into guide textures and residue textures, enabled by PCA parameterization. While we do incorporate frequency features to achieve an improvement in curvature preservation from an independent research finding, we have never claimed the use of frequency features as our novel contribution, out of respect for GroomGen. For clarity, our mention of GroomGen’s use of frequency features can be found in Section 4.1.

Similarly, we do not present the upsampling scheme as a novel contribution, nor do we consider it as a novel contribution of GroomGen. Hair upsampling through neural interpolation is a widely adopted practice, first introduced in “HairNet: Single-View Hair Reconstruction Using Convolutional Neural Networks” (Zhou et al., ECCV 2018), and subsequently used in works such as “Real-Time Hair Simulation with Neural Interpolation” (Lyu et al., TVCG 2020) and “CT2Hair: High-Fidelity 3D Hair Modeling using Computed Tomography” (Shen et al., SIGGRAPH 2023). We reference GroomGen alongside other works in the first paragraph of Section 3.2.2 when discussing interpolation strategies.

2. Regarding Experimental Results and the GroomGen Implementation

Since you did not release the full model and dataset – only the frequency-based strand VAE and hairstyle VAE – we reimplemented the full pipeline and trained it on our publicly available dataset (https://github.com/c-he/GroomGen). If you identify any inaccuracies in our implementation, we sincerely welcome your feedback and are fully open to revising the code and re-running relevant experiments.

Concerning Figure 23, which compares your released VAE and our replication, we do not believe it is unfair as the testing hairstyle is from CT2Hair, a third-party dataset we did not use for training either. Moreover, GroomHair contains more curly samples than USC-HairSalon, thus the result should favor the reconstruction of curly hairstyles as the one we tested. Regardless of dataset bias, the reconstruction results from both the official GroomGen model and our reimplementation underperform compared to our PCA-based representation (see Figure 5), pointing to a consistent issue with the frequency-based strand VAE design. Although we do acknowledge that using frequency features can help preserve strand curvature, and our experiment between PCA and Freq. PCA also supports this, experimentally we found the frequency-based strand VAE in GroomGen cannot faithfully reconstruct hairstyles in our testing cases.

If you still find unfairness, could you let us know which public dataset we should use to train our replications and which dataset we should use for testing the official version and our replicated version?

Regarding Figure 24, our goal was to compare generated results from your released VAE and our replication using the same Gaussian noise. This is why we did not use your provided seed or test script. If you believe this comparison does not adequately reflect GroomGen’s generative capacity, we are happy to provide additional random sampling examples. We have also included our testing script for your review.

3. Regarding the Number of Points for Kinky Hair

We appreciate your correction regarding our earlier claim. After reviewing the kinky hairstyle you referenced, we agree that 100 points are sufficient to represent such styles within a certain length range. However, for longer kinky hairstyles, we find 100 points often insufficient for preserving high-frequency details and visual fidelity. In our private dataset, styles of these complex types typically require over 200 points per strand, and a model parameterized with 100 points cannot faithfully represent the hairstyle, even with frequency features. We apologize for any confusion and will revise the corresponding statement to clarify this point.

In general, we greatly value your contributions to this field and are committed to constructive, transparent dialogue. If we have misunderstood any point or if further clarification is needed, please do not hesitate to let us know – we are happy to continue the discussion.

# generate new hair models

for seed in range(100):
    hairstyle_latents = np.random.RandomState(seed).randn(1, 1, 1, 512).astype(np.float32)
    latent_maps, masks = hae_model.decode(hairstyle_latents)
    strands = sae_model.decode_latent_map_to_strands(latent_maps[0], masks[0], mask_threshold=0.9)
    strands = np.concatenate([strands, np.ones_like(strands[..., :1])], axis=-1)
    strands = (strands @ groomgen_to_pinscreen.transpose((0, 2, 1)))[..., :3]
    save_hair(f"outputs/seed{seed:04d}.abc", strands)

Hi Chengan,

Thank you for your response.

I believe there may have been a slight misunderstanding. Please allow me to clarify: my intention is not to challenge the novelty or contributions of Perm (why do I?) - in fact, I am genuinely glad to learn that PCA can do well in your work.

Rather, as wrote in my initial comment, I want to "share a few clarifying comments regarding the comparisons with GroomGen presented in this paper", because "it is disappointing to see GroomGen misrepresented while we have already released publicly available resources".

To keep things organized, I'll follow-up in the same order as I asked, because the order reflects the objectivity of the questions:

1. Since you've acknowledged that Figure 25 was a misuse, I kindly ask you to remove that example and revise the corresponding claim in Section 4.2 and Section C.4. Using an image from our paper and asserting it as an impossible case is quite misleading. I mentioned Figure 24 for a similar reason: to show that 100 vertices can indeed represent curly hair models to a reasonable degree.

2. Regarding Figure 23, I'm afraid the main point might have been overlooked. My concern is that "(comparing with Perm's re-implementation) our official result is perceptually more faithful to the ground truth, particularly in preserving curl structure". In contrast, the result shown in the middle of Figure 23 appears noticeably broken, which suggests that GroomGen may not have been properly re-implemented. Perm states that the re-implementation was verified against the official checkpoints, and used Figure 25 "to assess whether our implementation of GroomGen can reproduce kinky hairstyles similar to those shown in the GroomGen teaser". Given the visual results in Figure 23 and the acknowledged misuse of Figure 25, I find it difficult to fully agree with that claim. It might be helpful to clearly differentiate between Perm's re-implementation and our original release in the paper to avoid potential confusion.

3. Lastly, when presenting non-trivial design choices, I always try to cite closely related work and briefly discuss the differences to clarify the motivations and novelty (if any) behind my approach. I understand that you may have a different writing philosophy, and of course, that is entirely your prerogative.

Just to reiterate, my intention was never to challenge the novelty of your work, but rather to clarify GroomGen. Thank you again for the discussion, and I appreciate your time.

Best regards, Yuxiao