Aligning Human Motion Generation with Human Perceptions

Thank you to the author for the rebuttal, but I still feel that the overall approach is not compelling enough, and there are logical issues in the explanation.

1. Your feedback annotation is based on text-to-motion, which certainly involves text alignment factors. Even if, as you mentioned, using human preferences to emphasize physicality for improving generative model performance seems plausible, it is not entirely convincing. Physicality should instead focus on physical simulations and joint motion, as joint motion data can directly calculate physical performance.

2. Thank you for providing the extra visualizations, but upon closer inspection, the results are not satisfactory. For instance, some high-scoring samples still show obvious jittering and sliding steps, while some lower-scoring samples exhibit fewer issues.

3.''Utilizes a critic model to fine-tune the motion generation model in a more flexible, unconstrained manner, which does not hurt multimodality '' is not convincing. The critic model is essentially just a discriminator. This statement does not provide any meaningful insight.

We thank reviewer pzfu for the thoughtful feedback. We are pleased to know that our rebuttal addressed most of your initial concerns, such as the diversity of generation results and the rationale for why higher accuracy is better in Table 1. However, we are confused by the fact that the score was changed from 5 to 3 within two minutes of the comment saying, “I will maintain my score.” Was this a misunderstanding or a misclick? We would greatly appreciate clarification.

Now, we will address your new concerns point by point.

Q1: Text alignment factors

A1: Evaluation of motion generation typically involves multiple dimensions, such as naturalness/fidelity, diversity, multi-modality, and alignment with text/audio/scene. In this work, we specifically focus on improving one critical aspect of motion generation: motion quality (e.g., naturalness, smoothness, plausibility), which we align with human perception. As explained in Appendix A.3, we deliberately exclude the text descriptions of the motion during the annotation process. This is because our primary objective is to evaluate the quality of the motion itself, independently of its alignment with textual descriptions.

Q2: Physicality and joint motion data

A2: While physicality is an important aspect of motion quality, it is not equivalent to the human perception of motion quality, which includes a broader range of factors.

• Conceptually: Our study goes beyond objective physical and biomechanical plausibility to include subjective factors like naturalness and smoothness, which are strongly influenced by human perception and difficult to quantify using purely physical metrics. This is why we conducted a large-scale human perceptual study to learn these standards in a data-driven manner.

  For example, a motion that satisfies physical constraints (e.g., no joint violations or unnatural forces) may still feel unnatural to humans—such as a rapid, jerky movement that, while physically plausible, appears awkward and unrealistic.

• Experimentally: As demonstrated in Section 5.2, our proposed MotionCritic significantly outperforms prior handcrafted physical metrics (e.g., Person-Ground Contact, Foot-Floor Penetration, Physical Foot Contact) in reflecting human preferences. Additionally, as detailed in Appendix B.5, MotionCritic implicitly incorporates information from these physical metrics through its data-driven framework.

Nonetheless, We agree that combining physical simulation priors with our model's human perceptual priors is a meaningful direction for future exploration. By integrating these two complementary aspects, we can potentially develop models that are both physically grounded and perceptually realistic, leading to more natural and convincing motion evaluation and synthesis.

Q3: Extra visualization results

A3: As per your earlier suggestion, we visualized the motion both before and after fine-tuning, which is intended to demonstrate that our fine-tuning process does not negatively impact generation diversity. It is worth noting that these results are completely free of cherry-picking.

Regarding your observation that some lower-scoring samples may appear better than some higher-scoring ones—this is entirely reasonable. As reported in Table 1, our metric’s accuracy is not 100%, so occasional failure cases are expected. However, statistically, our proposed metric demonstrates significantly better alignment with human preferences compared to all existing metrics, as shown by the comprehensive evaluations.

We also examined these failure cases in Appendix B.2 and found that they are relatively rare. In such cases, the differences in Critic scores tend to be smaller, and the motions are generally more ambiguous and harder to judge. This reflects the inherent subjectivity in human perception, where different individuals may interpret the same motion differently, especially for borderline cases. Thank you again for your constructive feedback, and we hope this explanation clarifies the intent and outcomes of our visualizations.

Q4: “Utilizes a critic model to fine-tune the motion generation model in a more flexible, unconstrained manner, which does not hurt multimodality” is not convincing.

A4: The statement was made in comparison to the work you referenced, HuTuMotion [1], and we would like to expand on this discussion below:

• Flexibility: In HuTuMotion [1], the base motion generation diffusion model remains unchanged, as their approach does not fine-tune the motion generation model itself. In contrast, our method fine-tunes the motion generation diffusion model directly, making our approach more flexible.
• Unconstrained: HuTuMotion [1] constructs an optimal prior distribution for texts that are near a given representative text (RT). In their implementation, they use ChatGPT to generate a predefined set of RTs (typically around 5–50). This predefined set inherently constrains the diversity of the generated results (multimodality). In contrast, our method does not impose such presets, making it unconstrained in terms of generation.

We acknowledge that HuTuMotion [1] has its own advantages, such as providing a minimal feedback-based approach, as we have mentioned earlier. However, based on the theoretical analysis above, our approach is more flexible and unconstrained, which explains why it does not reduce multimodality. This claim is further supported by our experimental results (see Table 2).

Additionally, your earlier statement, "They only select the best result, which significantly reduces diversity", is not entirely accurate. Selecting the "best" result based on human perception does not inherently conflict with motion diversity. High-quality motion inherently has multiple forms of expression, does it not? Our method does not assume that generating high-quality motion leads to a reduction in diversity, and this is further validated by our experimental results.