Thank you for the valuable comments and suggestions. Before we address the reviewer's questions, we'd like to summarize the motivation and main contribution of our paper below.

Motivation, Contribution, and Relationship to Video-Driven Methods:

In our work, we observed that existing audio-driven 3D talking head methods cannot achieve highly realistic and vivid talking head results. The quality is far worse than the 2D talking face counterpart. After examination, we found that most existing methods apply 3D parametric models, in particular, FLAME, for expression control, following a generic "audio->FLAME expression->deformed 3D model" paradigm. However, existing 3D parametric models like FLAME are constructed by registering 3D scans – typically from just a few hundred subjects and without sophisticated expression design – and then applying PCA. They have intrinsic limitations in handling nuanced motion and subtle expressions of real human faces when talking and expressing emotions. So we propose to use a latent space learned on massive 2D face videos to drive 3D avatars, an attempt not made by any previous methods. We also propose a framework to map the learned latents to 3D Gaussian variations (Sec. 3.2) and a set of carefully designed losses to handle the training on synthesized data and addressing the challenges they incur (Line 170).

We have demonstrated the clear advantage of our method compared to using FLAME parameters. We also achieved high-quality and naturalistic audio-driven 3D talking head animation results from a single image, significantly pushing the state-of-the-art quality in this domain and minimizing the performance gap to audio-driven 2D talking head methods.

We also would like to re-emphasize that the task we handle in this work is audio-driven talking face generation. We acknowledge that there is rapid progress in 3D avatar reconstruction and video-based reenactment, such as those works pointed out by the reviewer. However, please note that audio-driven and video-driven 3D head avatar generation are often treated as distinct problems. Adapting video-driven methods to the audio-driven scenario is not trivial and achieving high realism and vividness in the talking scenario introduces challenges not fully addressed by these methods. Taking our base model GaussianAvatars [17] as an example, their video results on expression transfer (with FLAME expression codes) look convincing (see demos on their website). However, when it comes to audio-synchronized motion behaviors and the expression of facial nuances subtle yet critical for expressing emotions, FLAME's expression space is insufficient, as shown in our paper. In fact, these 3D avatar reconstruction and video-based animation methods represent technical advancements orthogonal to ours and they can be applied as our base model, similar to GaussianAvatars.

---

We now respond to the reviewer's comments below

W1: Due to the sheer number of papers on this topic, we opted to discuss the previous work based on the properties they share, rather than discuss each of them individually. For example, the 3D Gaussian representations used by GaussianAvatars and GAGAvatar are mentioned in Line 81-83 of the Related Work. For 3D head reconstruction, the multi-view correspondence used by GaussianAvatars is mentioned in Line 89-90, and the one-shot approach by GAGAvatar using a parametric head model as a strong prior is described in Line 90-92. For head animation, it is explained in Line 104-108 that GaussianAvatars and GAGAvatar utilize parametric models for animation encoding and control, while the animations of VASA-3D are instead driven by the VASA-1 motion latent.

We feel that presenting the related work in this holistic manner allows us to position our method more clearly within the global context of the recent literature. However, to further address the reviewer’s concern, we could add the following at Line 106: “For example, models based on 3D Gaussian splats such as GaussianAvatar [17] and GAGAvatar [20] control facial motion by modulating the expression parameters of the FLAME and 3DMM models, respectively, on which the splats are bound.” Please let us know if this would be an acceptable solution.

W2: Our paper indeed focuses on audio-driven 3D head avatar generation as the reviewer noted. As we mentioned earlier, audio-driven and video-driven 3D head avatar generation are often treated as distinct problems. FlashAvatar and GBS are both video-driven 3D head avatar models, and they both use FLAME's expression parameters to drive the avatars, unlike ours that proposes using a latent space learned from massive 2D videos. These works are orthogonal to ours and can be potentially applied as our base model as well, similar to GaussianAvatars.

W3: We wish to emphasize that the novelty of this work lies mostly in its new approach for driving animation of a 3D head avatar, rather than in the technical details for implementing this idea. Nevertheless, there also exist important technical innovations, as discussed below.

Regarding our model framework, our instantiation is based on GaussianAvatars and we introduced a new module with two region-wise MLPs (neck and face) conditioned on VASA latent and head pose to regress additional Gaussian variations.

Regarding our loss functions, they are designed to handle the unique challenges we had when training on the synthesized videos: temporal inconsistency, view angle limitation, and model overfitting (see Line 170-177). To our knowledge, no prior method has dealt with the same problem as ours. Although some loss terms might have been seen in different papers, our devised combination is a well-crafted and effective way to overcome the special challenges presented by our training data. Our novel-view render-consistency loss $L_{consist}$ with a $stop$_$gradient$ operator is a new one specially designed to regularize our final Gaussian output and mitigate the side effect brought by the SDS loss.

W4: Thanks for pointing out the typos. We will correct them in the revised paper.

Q1: Thank you for the suggestion regarding open source. We plan to release the code and some synthetic training data of VASA-3D for research purposes, which will facilitate checking the details of our VASA-3D implementations and exactly reproducing some typical VASA-3D avatars and the driving sequences shown in our paper.

Q2: As we discussed above, our key motivation is that FLAME parameters are insufficient for capturing nuanced motion and subtle expressions of real human faces. This limitation is especially pronounced for the talking scenario where humans are sensitive to emotional expression and demand a high degree of motion realism. The latent space of VASA-1 is learned on a large volume of face videos to capture rich facial dynamics in a disentangled manner. The 2D talking face generation results of the VASA-1 also shows the capability of VASA-1 latent to represent nuanced motion and diverse talking behaviors. In light of this, we propose to apply it for 3D Gaussian head motion control and demonstrated its advantage in this work (Table 1, Fig. 3, etc.).

Directly regressing dense 3D motions from VASA-1 latent vectors would pose significant challenges in learning. Therefore, we decouple the deformation to a base deformation guided by FLAME and a dense, free-from deformation (VAS deformation) guided by VASA-1 latent. This way, we properly leveraged the priors in FLAME to ease the learning but go beyond it to use VASA-1 latent as the ultimate driving signal.

Q3: We tried this alternative approach on one case where we only predict position and rotation offsets in VAS deformation while keeping color, scale, and opacity fixed. A quality drop is observed as shown in the table below. And we did not observe obvious improvements related to generalization.

Q4: Our training data can be generated using either audios or face videos. The 10-hour training data used to conduct ablation study and validate algorithm designs was generated using face videos (Line 234). Specifically, we used video clips randomly sampled from VoxCeleb2 where each clip is typically of 5-10 seconds. The pose variation in the generated data is large. We also used audios to generate training data in Sec. 4.2 and 4.3 (Line 270, 292), where we use 20-minute continuous audios to generate training videos (some of the compared methods require continuous video for training and cannot utilize the collection of video clips in our 10-hour data). For this case, we did not apply additional pose augmentation for simplicity and the results look satisfactory. We conjecture that VASA-1-generated pose contains adequate variations and our loss designs are effective to facilitate the reconstruction with limited angles. Applying pose variations is nevertheless an excellent idea, and we thank the reviewer for this suggestion!

Q5: Yes, the FLAME model we used includes teeth geometry. Each 3D Gaussian is bound to a triangle of the FLAME mesh, following GaussianAvatars (Line 123-125).

---

Please let us know if you have any further questions.

Dear reviewer,

Thank you for the additional feedback. We are glad to see that most of your concerns have been addressed! Regarding open-sourcing, as we mentioned in the rebuttal, we will offer our code and synthetic training data for research purposes. This will allow users to fully examine the details of our VASA-3D implementations and exactly reproduce our typical VASA-3D avatar samples and the driving sequences.

In terms of contribution, we believe our core idea, which involves using a latent space learned from 2D videos to drive 3D talking faces, offers significant advantages over the prevailing approach of using 3DMMs. Our instantiation also contains important innovations to address the unique challenges we encountered, making our work the first of its kind to use a single real image and synthesized, imperfect videos to construct high-quality 3D talking faces. We believe that both our core idea and its implementation offer valuable insights that make a significant contribution to the community.

Thank you again and please feel free to let us know if this response does not satisfactorily address your remaining concern or if there's any clarification we shall further provide that could assist with the final assessment. Thanks again.

Authors

Dear reviewer,

Thank you again for your thoughtful feedback. As the discussion phase nears its end, we wanted to kindly bring your attention to our rebuttal and would be glad to provide any further clarification if needed.

Authors

Thank you for the constructive feedback, and we address the questions as follows.

W1: Thank you for the comment. We’d like to clarify that the video sequences used for model training do not necessarily need to be long. In fact, the model can be trained on only a collection of individual images (no videos) or a collection of short video clips (5-10 seconds long as done with random VoxCeleb2 samples in Sec. 4.1). Although we set the default training video length to 10 hours, it is shown in Fig. 3 that using only 2 hours leads to just a marginal performance drop, and we also used only 20-minute audio-generated continuous training video to compare with existing methods in Sec 4.3 (some of the compared methods require continuous video data due to their data preprocessing design and training loss implementation). Under these tested settings, the computational requirements are reduced significantly without much sacrifice in quality.

We are unsure if this satisfactorily addresses your concern on reasons that may limit the "applicability, generalization ability, and efficiency". Please let us know if it doesn't or you have any further questions.

W2 & Q1: We thank the reviewer for pointing out the related works SEGA [1] (a concurrent and arXiv work, April 2025) and ExAvatar [2], which we will cite in our revised paper. However, with due respect, we find significant differences between these works and ours, in terms of their motivations and solutions to enhance the expressiveness of 3D parametric models.

The key motivation of our work is that the parameter space of existing 3D parametric models is intrinsically limited and cannot capture nuanced motion and subtle expressions of real human faces when talking (See Line 4-5, Line 25-33, Line 40-42). So we advocate the use of the powerful latent space learned on massive 2D talking face videos to drive 3D Gaussian avatars. To the best of our knowledge, this is a novel technical contribution not explored by previous works, and we have demonstrated its clear advantage (Table 1, Fig. 5, etc.).

Please note that both [1] and [2] still rely on FLAME's expression parameters to drive their 3D models. Even though their dynamic offsets are learned from the subject’s data, their raw driving signals still stem from the FLAME expression space. These offsets lead to results that more closely adhere to the subject’s appearance, but they do little to enhance the expression space to encompass subtle expressions and nuanced motions. In contrast, our 3D model is driven by learned latent codes (from VASA-1), not FLAME expression codes.

Regarding open-sourcing, we plan to release the code and some synthetic training data of VASA-3D for research purposes, which will facilitate checking the details of our VASA-3D implementations and exactly reproducing some typical VASA-3D avatars and the driving sequences shown in our paper.

W3: While more training data was used in our method (minimum 20-minute videos containing 30K frames tested our paper) than HRAvatar [3] or RGBAvatar [4] (typically 2K-5K frames), there are key differences in task setup and data assumptions. These two methods use carefully captured monocular videos with exaggerated expressions and diverse poses to improve data coverage. Moreover, their training and testing data often come from highly similar distributions, enabling efficient data use.

In contrast, we simply take random real-world talking videos or audios to generate our training data, which represent an easier and more scalable data capture setup than these methods. We also use random real-world audios for testing. Under such situation, it is natural to use more training data to cover a wide motion distribution including nuanced facial expressions.

Additionally, our method must handle the inconsistencies in VASA-1-generated synthetic videos, further increasing the complexity. This makes our setting different and more challenging than these 3DGS avatar works.

W4: We thank the reviewer for suggesting this potential baseline approach. We are unable to add this baseline in a short time window. However, we wish to make a few arguments:

• Once again, our key motivation is to build expressive 3D head avatars animated with powerful latent spaces beyond 3DMM parameters. We have demonstrated the limitations of 3DMM parameters in expressing nuanced motions (Table 1, Fig. 5, etc.). As far as we can see, both [3,4] still use FLAME parameters to drive 3D facial expression, similar to the base model we used, i.e., GaussianAvatars. In fact, we view [3,4] as technical advancements orthogonal to ours and they can be applied as our base model in lieu of GaussianAvatars.

• Using a large, diffusion-based talking head generation model such as [5,6] to generate data will incur significantly increased computation cost compared to using VASA-1 (e.g., <1 fps generation using [6]). Moreover, using such data still does not resolve the aforementioned challenge.

• It's not clear to us how to adapt such baselines to the audio-driven scenarios. The task we handle in this work is audio-driven 3D talking face generation, with the target application being virtual, autonomous conversational agents. We provide more discussions related to audio-driven and video-driven methods at the end of this response.

W5: The main goal of the SDS loss is to mitigate visual artifacts in side views (Line 189). Since the enhanced regions only occupy small image areas in side views, their improvement is not well reflected by the numerical values computed on all images with both frontal and side views. However, the visual improvements can be clearly observed (Fig. 4). We will add explanations in the revision, and we thank the reviewer for this comment.

Q1: Please see above.

Q2: There are multiple advantages of VASA-3D compared to VASA-1 in terms of 3D-related capabilities:

• There's no true camera viewpoint control in VASA-1. The head pose control therein primarily steers the head angle with shoulder and torso regions largely unchanged. The camera distance control was approximated with 2D head region size as conditioning signal. In contrast, VASA-3D supports true 3D camera control and free-view rendering.

• VASA-3D can be naturally integrated into 3D environment (VR/AR/Games) as it's inherently a 3D model with 3DGS rendering.

• Last but not least, even when rendered as 2D videos, VASA-3D's results exhibit strong texture consistency. In contrast, VASA-1 shows noticeable texture sticking artifacts under head rotations and large facial motions, especially around regions with high-frequency textures such as hair and beard.

We will add more related discussions in our revised paper, and we thank the reviewer for raising this question.

---

Motivation, Contribution, and Relationship to Video-Driven Methods:

Here we'd like to rephrase the motivation and main contribution of our paper. First, we observed that existing audio-driven 3D talking head methods cannot achieve highly realistic and vivid talking head results. The quality is far worse than the 2D talking face counterpart. After examination, we found that most existing methods apply 3D parametric models, in particular, FLAME, for expression control, following a generic "audio->FLAME expression->deformed 3D model" paradigm. However, existing 3D parametric models like FLAME are constructed by registering 3D scans – typically from just a few hundred subjects and without sophisticated expression design – and then applying PCA. They have intrinsic limitations in handling nuanced motion and subtle expressions of real human faces when talking and expressing emotions. So we propose to use a latent space learned on massive 2D face videos to drive 3D avatars, an attempt not made by any previous methods. We also propose a framework to map the learned latents to 3D Gaussian variations (Sec. 3.2) and a set of carefully designed losses to handle the training on synthesized data and addressing the challenges they incur (Line 170).

We have demonstrated the clear advantage of our method compared to using FLAME parameters. We also achieved high-quality and naturalistic audio-driven 3D talking head animation results from a single image, significantly pushing the state-of-the-art quality in this domain and minimizing the performance gap to audio-driven 2D talking head methods.

We also would like to re-emphasize that the task we handle in this work is audio-driven talking face generation. We acknowledge that there is rapid progress in 3D avatar reconstruction and video-based reenactment, such as those works pointed out by the reviewer. However, please note that audio-driven and video-driven 3D head avatar generation are often treated as distinct problems. As we mentioned earlier, those works can potentially be applied as our base models. In fact, we believe that adapting these works to the audio-driven scenario is not trivial and achieving high realism and vividness in the talking scenario introduces challenges not fully addressed by these works. Taking our base model GaussianAvatars as an example, their video results on expression transfer (with FLAME expression codes) look convincing (see demos on their website). However, when it comes to audio-synchronized motion behaviors and the expression of facial nuances subtle yet critical for expressing emotions, FLAME's expression space is insufficient, as shown in our paper.

We are open to any discussion and please let us know if you have any further comments.

Dear reviewer,

Thank you again for your thoughtful feedback. As the discussion phase nears its end, we wanted to kindly bring your attention to our rebuttal and would be glad to provide any further clarification if needed.

Authors

Thank you for the positive feedback on our work. We appreciate your recognition of the contribution and effectiveness of our method. We address your questions as follows.

W1: Given an audio clip and a trained VASA-3D model, our method can generate rendered frames at 75 fps (i.e., ~13ms for one frame) with a 65ms latency. The speed and latency were measured on one RTX 4090 GPU; see Line 282 of the main paper. So yes, it supports real-time audio-driven synthesis. A recorded real-time demo is also provided in the supplementary video ("1.Realtime_Demo.mp4").

Q1: The face and neck regions are pre-defined on the FLAME template mesh. Since 3D Gaussians are bound to the triangles of the deformed FLAME mesh, the regions they belong to can be easily derived.

---

Kindly let us know if you have any further questions.

Thanks for your recognition of our work. We appreciate your positive feedback and address your questions as follows.

W1 & L1: Thank you for the comment. Indeed, using synthetic data created by VASA-1 (or other talking head video generation method) may introduce distribution gaps to real data. We'll add more related discussions in the limitation section. Nevertheless, we found that our training data enables VASA-3D to generate vivid and realistic talking heads that exhibit clear superiority over other techniques. Also note that VASA-3D could also leverage future advances in 2D talking head generation for improved training.

W2 & L2: Since training is done offline, efficiency was not our main focus. However, we show in the following that strong results can still be achieved with much less computation.

Training data size and generation speed:

Although 10 hours of training video is the default setting, it is shown in Fig. 3 that similar results could be obtained with just 2 or 5 hours of data. Note that training data generation is fast: <1 hour for 10 hours of video (Line 244), or ~10 min for 2 hours of data.

Furthermore, our comparisons with other methods in Sec. 4.3 used only 20 minutes of audio-generated video data for training (because some of the methods require continuous video and cannot utilize the collection of video clips in our 10-hour data). The results show that VASA-3D trained with 20-min data still generates highly realistic 3D talking heads, surpassing the other methods by a wide margin across metrics evaluating lip-sync, video quality, etc. (see "5. Methd_Comparisons.mp4" and Table 3). The 20-min data can be generated in <2min.

Training time:

We wish to point out that 18 hours of training time is reasonable for 3D model optimization on videos. For reference, we report the training time of different methods in Table 3 of the main paper. For the compared methods, we stop the training when convergence was observed on our training data. Note that we kept their original implementation of running on a single GPU unchanged.

We did not carefully optimize the training framework or tune the hyperparameters for efficiency considerations. But the training time can be reduced with more GPUs or implementation optimizations (e.g., less frequent SDS loss computation which is the most costly part). For the comparison below, we found that our model trained with 20K iterations (1.8 hours) already significantly outperforms other methods and nearly reaches the performance of the model trained with 200K iterations.

In summary, the training cost is reasonable, and significant computational reductions (down to 20 minutes of training video and 1.8 hours of training) still lead to high quality results. We intend to study efficiency improvements in future work.

W3 & L3: There were experiments on real-world images, and the performance was equally good to StyleGAN2-generated portraits. Please see Line 36, Table I, and Fig. V in the supplementary pdf, and the video results in "2.Main_Results.mp4" (02:03-02:59).

W4: We plan to release the code and some processed synthetic training data of VASA-3D for research purposes, which will facilitate checking the details of our VASA-3D implementations and exactly reproducing some typical VASA-3D avatars and the driving sequences shown in our paper.

Q1: Identity preservation under novel expressions is primarily handled by VASA-1. The synthetic training videos generated by VASA-1 effectively preserve identity with novel head poses and expressions, as demonstrated in their paper.

Q2: FLAME is a well-established 3D morphable model that provides a parametric representation of human faces, and it's widely used in previous 3D Gaussian avatar methods such as [a,b,c]. We followed these works to use FLAME and did not try alternative parametric models. Other parametric models are applicable as well and we'll explore the trade-offs in our future work.

Q3: Yes, VASA-3D can generalize to non-frontal input images (with yaw angles up to ~45 degrees) and low-resolution portraits.

Q4: The choice of VASA-1 as the motion latent source is based on its proven ability to generate expressive talking videos with diverse head poses and expressions, which are essential for supervising our 3D Gaussian model training. Although we have not explicitly tested other motion latent sources, our method does not depend on any specific architecture of VASA-1 or any specific properties of VASA-1's latent space.

[a] S Qian et al. GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussians

[b] PW Grassal et al. Neural Head Avatars from Monocular RGB Videos

[c] Y Zheng et al. I M Avatar: Implicit Morphable Head Avatars from Videos

---

Thank you once again and please let us know if you have any further questions.