HairFastGAN: Realistic and Robust Hair Transfer with a Fast Encoder-Based Approach

We thank Reviewer RjWU for their favorable review of our paper. We appreciate the reviewer's recognition of the value in our research and their contribution to the peer review process. The reviewer's support is significant in advancing our field of study.

We thank Reviewer fS7X for their valuable input and the time they have invested in reviewing our work.

Can your method edit hair length, hairstyle type (e.g., wavy, curly), and facial pose post-hairstyle transfer, while preserving the editability of other facial features without overfitting?

Our method is indeed capable of modifying hair shape, including length, through the use of sliders. This functionality is achieved by training the Shape Adaptor to project hair shape attributes into a small number of independent normal distributions. In our work, we take these attributes from another image, but they can also be edited by hand using the sliders. For more detailed information on this approach, we refer to the CtrlHair method [1].

Regarding attributes such as hair waviness, curliness, and facial pose, these can be edited using complementary techniques like StyleFeatureEditor [2]. This method can be applied to our output image as a post-processing step, allowing for further refinement of these specific attributes.

It's important to note that our method maintains the editability of other facial features without overfitting, as the hair editing process is separate from other facial attribute manipulations.

[1] Xuyang Guo, Meina Kan, Tianle Chen, Shiguang Shan, GAN with Multivariate Disentangling for Controllable Hair Editing. ECCV 2022

[2] Denis Bobkov, Vadim Titov, Aibek Alanov, Dmitry Vetrov, The Devil is in the Details: StyleFeatureEditor for Detail-Rich StyleGAN Inversion and High Quality Image Editing. CVPR 2024.

Have you used ArcFace or similar models to quantitatively assess identity preservation?

Thank you for your insightful comment. While we did not include this analysis in the original manuscript, we have conducted additional quantitative assessments of identity preservation. These results are available in the rebuttal file on Tab 2, and we intend to incorporate them into the final version of the paper.

For this evaluation, we utilized the SFace model rather than ArcFace, as the latter was employed in training some of our encoders. Our findings demonstrate that our method outperforms most other approaches in the main experiments with respect to identity preservation.

How does your method ensure continued editability of facial features after hairstyle transfer, and what measures prevent degradation of initial edits?

While our current work has not specifically addressed this aspect of the problem, we acknowledge its importance and intend to explore it in future research.

Can your method effectively handle varying attributes like hair length, waviness, and curliness? How does its flexibility compare with existing methods?

Our method demonstrates considerable effectiveness in modifying hair length. However, similar to existing approaches, it encounters challenges in accurately transferring texture attributes such as waviness and curliness. We have included additional comparisons illustrating changes in these attributes in Figure 2 of the rebuttal document for a more comprehensive evaluation.

We thank Reviewer JffK for their thoughtful comments and questions. Their insightful feedback has provided us with valuable perspectives to improve our paper. We appreciate the time and effort the reviewer has dedicated to this review.

1. Since "Fast" is a key highlight of this work, why is there a lack of in-depth computational complexity comparisons, such as FLOPs and parameter count?

Thanks for your comments, we acknowledge the importance of these metrics and have addressed this by calculating the FLOPs and parameter counts for our method and the compared approaches. These values have been included in Table 1 of our general rebuttal document, and we will incorporate them into the final version of the paper.

Our initial decision to omit these comparisons stemmed from the observation that they do not always accurately reflect real-world runtime performance. For instance, the CtrlHair method, despite using significantly fewer FLOPs, actually runs approximately 10 times slower due to its use of Poisson blending in their own inefficient CPU-based implementation. However, we recognize the value of providing this information for a comprehensive evaluation.

It's worth noting that despite these discrepancies between theoretical complexity and practical runtime, the overall ranking of methods in terms of speed remained consistent with our original findings.

2. How does the HairFast model compare to other state-of-the-art hairstyle transfer methods based on the diffusion model in terms of accuracy, speed, and user-friendliness?

The task of hairstyle transfer is highly challenging, and until recently, there were no diffusion-based methods addressing this problem. To the best of our knowledge, only one paper on this topic has been published, which appeared on July 19, 2024, significantly after our submission deadline. This paper claims to be the first diffusion-based framework for hairstyle transfer, likely making it the only work to date in this specific domain. To maintain anonymity, we cannot directly cite this work in our response.

As the code for this method is not yet available, our comparison is based on limited information. The method demonstrates effectiveness in transferring complex hairstyles and gradient hair colors. Due to the properties of diffusion models, it also performs well in inpainting tasks while maintaining good reconstruction. The authors conducted a user study comparing their work to ours, and according to their results, our method is only marginally inferior in terms of Accuracy, Preservation, and Naturalness.

However, this approach does not directly address the challenge of transferring hairstyles with significant pose differences. In terms of computational efficiency, their method will be much slower, as it requires running Stable Diffusion v1.5 twice for 30 steps. Additionally, their approach may be less flexible and user-friendly, as it transfers hair shape and color from a single reference, unlike our method which allows for multiple reference inputs.

3. In line 173, what is the objective basis for setting the value of \alpha to 0.95?

The value of \alpha=0.95 was determined through a systematic ablation study and manual fine-tuning. In our ablation configurations C and D, we initially set \alpha=0, which resulted in the hair color not being transmitted and even leaking from the target, causing various artifacts. This issue persisted for values up to \alpha=1. We then conducted a visual comparison across different \alpha values, ultimately selecting 0.95 as it provided the best balance between preserving the desired texture in hair reconstruction and effectively transferring the color. This value empirically demonstrated superior performance in maintaining the integrity of the reconstructed hair while still allowing for successful color transfer.

4. How does the HairFast model handle hairstyles that are not well-represented in its training dataset?

Our HairFast model, which operates on feature space (FS) image reconstructions, has the capability to store and transfer even highly complex attributes that were not present in the FFHQ dataset used for training StyleGAN and our encoders. The primary challenge in handling unusual hairstyles arises from the BiSeNet segmentation model. When faced with an unfamiliar domain or particularly complex facial images, BiSeNet may incorrectly select regions, leading to inaccurate hair transfer results.

To demonstrate our method's effectiveness in handling such cases, we conducted additional experiments involving cross-domain hair transfer. The results of these experiments are presented in Fig. 1 of the main rebuttal file. These findings illustrate that our method can successfully perform hair transfer even in domains that were not represented in the training data, showcasing its robustness and adaptability to novel hairstyles.

Weakness discussion

It is important to note that the identified weaknesses in points 1-3 are not unique to our approach but are common challenges faced by other methods addressing this problem as well. These limitations reflect the current state of the field and highlight areas for future research and improvement across all related techniques.

Thank you for the clarification. The diffusion-based model we are talking about uses a two-stage approach:

1. Generation of a bald proxy image using Latent ControlNet
2. Hair transfer utilizing a Hair Extractor based on U-Net, with features injected into the SD model via cross-attention layers

Each stage requires a specific training dataset and network training. While the authors provide comprehensive information, reproducing their work accurately would be time-consuming and resource-intensive.

We aim to include comparisons with this model in our paper's final version, either through our own implementation or by analyzing their published sample images.

Please let us know if you have any further questions or concerns. We're happy to provide additional information.