Long-Term TalkingFace Generation via Motion-Prior Conditional Diffusion Model

Dear Reviewer kTry:

Thank you very much for the detailed comments. We address your concerns as follows.

Q1: Lack of Clarity in Technical Descriptions

Response: Thank you for the helpful comments. We address each point below:

1. Memory Cost of Past Frames vs. Archived Sequences: Standard temporal self-attention stores all key-value pairs, resulting in quadratic memory growth with the number of frames. In contrast, our Archived-Clip Motion-Prior Module integrates with Memory-Efficient Temporal Attention (MTA), which maintains a fixed-size memory via incremental updates, avoiding GPU memory overflow in long-term generation. Please see our experimentalanalysis in Q2 of R2 (Reviwer S3hq).
2. Landmark Token Drop: Yes, we replace dropped landmark tokens with zeros.
3. Typo in Line 299: Sorry, the symbol “\alpha” was a typo, it should be the variable name “a” and is unrelated to Section 3.4.
4. Interpretation of Figure 5(a): We agree with your observation. Our intent was not to suggest identity drift. As stated in line 375, the absence of archived-clip priors leads to visible artifacts and inconsistencies in head, mouth, and expression, not identity change. We will clarify this wording in the revised version.

Q2: Issues in Subjective Evaluation Design

Response: Thank you for your thoughtful comments regarding the user study design. We address each concern below:

1. Rank Selection Protocol: Following prior works [1,2], we adopted a ranking-based evaluation strategy to effectively capture relative user preferences. While this approach may introduce some cognitive load, participants were allowed unlimited time and could freely replay videos to ensure fair and consistent judgments.

   [1] Ma et al., "Follow Your Emoji: Fine-Controllable and Expressive Freestyle Portrait Animation", SIGGRAPH Asia 2024.

   [2] Gao et al., "FaceShot: Bring Any Character into Life", ICLR 2025.

2. Time Control: There was no strict time limit. Participants were allowed to complete the task at their own pace.

3. Randomization and File Naming: We confirm that the method-to-letter mapping (e.g., A.mp4, B.mp4) was independently randomized for each session. The correspondence was stored using a dictionary to avoid fixed associations and mitigate bias.

4. Addition of MOS Study: Thank you for your thoughtful advice. Following your suggestions, we have additionally conducted a Mean Opinion Score (MOS) study with 20 participants. Each participant rated video quality on a 1–5 scale (5 = Excellent, 1 = Bad). The results are summarized below:

The experimental results demonstrate that our method achieved the highest MOS score, demonstrating strong user preference in terms of identity consistency and visual quality.

Q3: Writing and Terminology Issues

Response: Thank you for the reminder. We have fixed all mentioned issues, including terminology corrections, grammar errors, and formatting. We also reviewed the entire paper to correct other typos, table labels, and figure captions for consistency and clarity.

Q4: Concerns on Expressiveness and Visual Artifacts

Response: Thank you for the valuable feedback. We acknowledge that minor artifacts (e.g., around the chin or eyes) may appear in difficult cases involving large head movements, this remains an open challenge in the community. Our work takes a meaningful step forward by proposing a memory-based generation framework and releasing a 200-hour multilingual dataset to help mitigate these issues. Both quantitative and qualitative results validate the motivation of our approach, showing that our method consistently outperforms prior baselines in expressiveness and coherence. More visual comparisons, including EMO (closed-source, official demo inputs), are provided in link 1 and link 2, demonstrating superior identity consistency and visual quality.

Q5: Literature Framing and Terminology

Response: Thank you for the suggestion. We agree that referring to recent deep learning methods as “early” may be misleading. We have revised the introduction to more clearly distinguish early non-neural approaches from modern neural-based methods.

Lastly, thank you so much for helping us improve the paper and appreciate your open discussions! Please let us know if you have any further questions. We are actively available until the end of this rebuttal period. Looking forward to hearing back from you!

Dear Reviewer S3hq:

Thank you for your detailed and constructive review. We appreciate your recognition of our contributions, including the TalkingFace-Wild dataset and the proposed motion-prior framework. We address your concerns as follows.

Q1: Ambiguity in Audio-Motion Correlation

Response: Thank you for pointing this out. The original sentence "Reliance on static audio features and weak correlations between audio and 'motion' complicate the decoupling of identity and motion cues...". 'motion' refers specifically to head motion, not lip motion. Following [1-2], audio and head motion are only weakly correlated. We have clarified this distinction in the revised text.

[1] Wang et al., "V-express: Conditional dropout for progressive training of portrait video generation"

[2] Chen et al., "Audio-driven Talking-Head Generation with Rhythmic Head Motion" .

Q2: Missing Complexity and Efficiency Analysis for MTA

Response: Thank you for the detailed feedback. Standard temporal self-attention requires storing all key-value pairs, leading to quadratic memory growth and potential OOM issues in long sequences (e.g., V-Express in Table 7 is marked N/A due to this).

In contrast, our Memory-Efficient Temporal Attention (MTA) maintains a fixed-size memory via incremental updates, enabling stable and efficient long-term modeling.

Furthermore, we provide a comparison (T=4, batch size=1):

MTA reduces FLOPs by 54% and GPU memory by 43%, while also improving FID. We will include this result in the final version.

Q3: Lack of Explanation for Multimodal Causality

Response: Thank you for highlighting this point. We apologize for the lack of clarity. By multimodal causality (MC), we refer to modeling head motion, lip motion, and expression tokens jointly using a unified transformer conditioned on audio and reference image features. These conditioning tokens are prepended to the sequence and influence all motion tokens uniformly. Unlike modality-specific denoising, our model adopts a shared causal transformer to perform unified denoising across all motion cues. To quantify its effectiveness, we conducted an ablation study:

These results demonstrate that multimodal causality modeling leads to consistent improvements in identity preservation, synchronization, and overall generation quality.

Q4: Missing Comparison with Reference-Based and Seq2Seq Methods

Response: Thank you for the suggestion. As noted in Q2, reference-based methods (e.g.,EchoMimic) segment long videos to mitigate OOM, but often lead to identity inconsistency between clips. Seq2Seq models (e.g., SadTalker and EMO) lack memory mechanisms and are prone to error accumulation. Our unified memory-based framework avoids these issues by maintaining and updating a fixed-size memory for consistent long-term generation. We have added qualitative comparisons in the link 1, and link 2. which demonstrate superior identity consistency and visual quality.

Q5: Concerns About the User Study Design

Response: We would like to clarify that in our user study, the mapping between methods and filenames (e.g., A.mp4, B.mp4) was randomly shuffled for each evaluation task, and the correspondence was stored using a dictionary to ensure fair randomization and avoid naming bias.

The initial study involved 20 participants, which aligns with common practice in prior works [3]. To enhance statistical reliability, we extended the study to 40 participants. We made every effort to expand the subject pool within the available timeframe. The results are available at: link3.

[3] Zhang et al., "SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation".

In the end, thanks a lot for your detailed comments and thank you for helping us improve our work! We appreciate your thoughts on our work and we would be more than happy to discuss more during or after the rebuttal to explore more in this direction. Please let us know if you have any further questions. We are actively available until the end of this rebuttal period.

Dear Reviewer Vnrq:

Thank you for your thoughtful and encouraging review. We appreciate your recognition of our work as “an interesting idea” with “rich and comprehensive experiments.” We have carefully considered all your suggestions. Please find our point-by-point responses below.

Q1: Ablation Study on Historical Frame Length

Response: Thank you for the insightful question. We conducted an ablation study on the number of archived frames, as shown in Table 8 of the supplementary material, covering 2, 4, 8, and 16 frames. Additionally, we extended the study to 24 frames. Due to memory constraints, further increases were infeasible.

The experimental results show that even with only 2 frames, our method outperforms MegActor-Σ across all metrics, indicating strong robustness under limited historical context. Furthermore, increasing the number of archived frames consistently enhances identity preservation and motion quality. We will highlight this ablation more clearly in the revised manuscript.

Q2: Lack of Qualitative Comparison with Related Works [1][2], Recent Open-Sourced Hallo2, and Other Methods

Response: Thank you for the valuable suggestion. We have added qualitative comparisons between our proposed MCDM and both open-source methods (SadTalker, EchoMimic, and Hallo2) and closed-source methods (Loopy [1] and EMO [2]), using the official audio and image inputs provided on the project pages of Loopy and EMO. Please refer to the following demo links: Loopy: Demo 1, Demo 2 and EMO: Demo 1, Demo 2.

Our method demonstrates stronger identity consistency and better motion synchronization compared to SadTalker, EchoMimic, and Hallo2, while achieving visual quality comparable to Loopy. Additionally, it generates more natural expressions and overall enhanced visual realism compared to the original EMO outputs.

Q3: On the Distinction Between GAN-based and Diffusion-based Methods

Response: Thank you for the helpful suggestion. We have revised the introduction to improve the logical flow and clarify the distinction between GAN-based and diffusion-based methods. GANs often suffer from training instability and artifacts such as flickering, especially in long-form generation. In contrast, diffusion models provide more stable training and improved visual realism via multi-step denoising and stronger identity preservation, making them better suited for this task. We have added this clarification and cited relevant works in the revised version.

Q4: On Pose Constraints and Expression Diversity

Response: Thank you for the constructive feedback. (1) Our original intent was to highlight that when the reference image and driving pose come from different identities, strong pose constraints (e.g., facial landmarks) may cause identity distortion or unnatural warping. (2) While facial expressions are generally independent of head pose, over-constraining pose can inadvertently limit expression diversity. For example, methods like EchoMimic use dense 3D landmarks from tools such as MediaPipe, which may implicitly encode facial details and constrain expression variation. We have clarified this point in the revised introduction.

Q5: Misuse of the Term “Static Audio Feature”

Response: Thank you for pointing this out. The term “static audio feature” was a misphrasing. Our intended meaning was to describe the limitation of relying solely on audio features without incorporating complementary cues such as lip or head motion. We have clarified this wording in the revised manuscript.

Q6: Redundancy in Contribution Statements

Response: We agree with the reviewer’s observation. Contributions 2, 3, and 4 all describe technical components of our framework. We have merged them into a single contribution for improved clarity.

Again, thank you so much for helping us improve the paper! Please let us know if you have any further questions. We are actively available until the end of this rebuttal period. Looking forward to hearing back from you!