We thank the reviewer for the valuable and positive feedback with constructive suggestions! We hope our responses adequately address the following questions raised about our work. Please let us know if there is anything we can clarify further.

---

W1: Discussion on the previous work [1]

Thanks for the helpful suggestions. While [1] also uses LLM to decouple body motion into part motions, we emphasize the following differences:

• The motivations and objectives are different. The work [1] focuses on the motion generation task, aiming to enhance model generation performance through part motion conditioned generation, which can serve for fine-grained controllable generation. In contrast, we target motion-text retrieval and cross-modal alignment learning, aiming to learn a shared semantic space through explicit part decoupling to mitigate potential over-fitting problems.
• In terms of the LLM knowledge generation, [1] adopts a complex prompting strategy, i.e., PoseScript+LLM, which we have thoroughly considered and discussed (Line 118-123). To our knowledge, PoseScript, as a non-deep learning method, is highly sensitive to hyperparameters and exhibits fragile performance across different motion domain data, significantly limiting its practical application potential. On the other hand, the PoseScript+LLM generation strategy requires the simultaneous availability of motion and text data, which is only possible during the training stage and contradicts our proposed GAR paradigm. In comparison, we employ a more universal and generalized strategy of few-shot + special token generation only dependent on texts, enabling our model to perform GAR augmentation during the testing stage.
• The utilization of part text data differs. [1] employs the traditional generative condition injection learning approach, i.e., auto-regressive generation, whereas we propose part mixture decorrelation learning and directional alignment to fully leverage part information for alignment.

We will add this discussion in the final version.

[1] CoMo: Controllable Motion Generation through Language Guided Pose Code Editing, ECCV 2024.

---

Q1: More comparison with Lex

Thanks for your advice. Lex [19] is the recent work, while the authors do not make the code open-sourced. Due to its complex training stages and designs, it is difficult to reproduce it well. Therefore, we have to directly copy the reported results of Lex for comparison (only All protocol is reported in the Lex paper). We have provided the additional comparison of All protocol on the KIT-ML dataset in the following table, from which we can see he effectiveness of our method, especially for the motion-to-text retrieval. Meanwhile, from an empirical perspective, the All protocol is well representative of the results of other protocols.

Table 1: Comparion with Lex on motion-text retrieval.

---

Q2: More comparison on motion generation with LaMP and ChronNet

Thanks for your question. We have provided additional results on motion generation, including LaMP and ChronNet with one of the recent SOTA methods, MoMask [1], as baseline. As shown in the following tables, our method can benefit the current motion generation methods significantly. Moreover, compared to ChronNet, our method has overall better performance. While compred to LaMP (Note LaMP adopts the larger text encoder, which can lead to unfair comparison), our method can achieve better FID scores with smaller DistillBERT text encoder.

Table 2: Comparison with MoMask and T2MGPT as the backbones.

*1) For MoMask, we made efforts to reproduce MoMask during rebuttal with the official code. However, we cannot reproduce the performance reported in the original paper in terms of FID metric (0.073 vs. 0.045). We have found that a few researches also encountered similar problems posted in the github issue. Therefore, during rebuttal, we report the currently reproduced results for reference.
2) LaMP utilizes a larger text encoder, i.e., BERT, while ChronNet and our method SGAR use the smaller DistillBERT.

We also provide more comprehensive comparison of LaMP and ChronNet on motion understanding and zero-shot action recognition in the W4 of Reviewer pgZA to demonstrate the advantages of our method.

[1]MoMask: Generative Masked Modeling of 3D Human Motions, CVPR 2024.

---

Q3: Explain t-SNE visualization

Sorry for the confusion. We make efforts for cross-modal feature visualization to provide more intuitive comparison results, although this might not be easy. As explainations, for the left two figures, it is to demonstrate our method has a more aligned motion-text representation space because the line segments (connecting paired motion-text data) are shorter. Then, for the third column figure, it can be viewed as an ablation study to demonstrate the difficulty in performing retrieval only based on part information caused by the less diverse texts and false negative problem in contrastive learning. Therefore, we can conclude that only part-level alignment without global information is insufficient to achieve desirable retrieval performance, as verified in Table 11. Based on this, our idea is to use the part information as a beneficial supplement to enhance the global alignment during testing.

Sorry for the confusion again. We are considering some better presentations, e.g., replacing this figure with the feature space of full body+right-arm in the final version to better demonstrate the improvement effect of part-level alignment on the feature space. We will refine this in our final version.

We thank the reviewer for the valuable feedback and constructive suggestions! We hope our responses adequately address the following questions raised about our work. Please let us know if there is anything we can clarify further.

---

W1:About [udf] indicator and implementation

Thanks for your helpful question. We have calculated that ~15.9% of samples include this indicator and the corresponding texts are excluded in HumanML 3D, while ~18.2% in KIT-ML.

In implementation, we utilize the designed prompt to guide the LLM to output the [udf] indicator for the parts with high uncertainty. This is inspired by previous research findings [1-5] that LLMs can effectively decompose knowledge and model its uncertainties, thereby perceiving potentially more fine-grained concepts within it. Specifically, we utilize the few-shot prompting strategy to better guide the LLM to perform this uncertainty modeling. The related prompts are (full prompts can be referred to Figure 11 in Appendix):

... If the specific movement of a body part cannot be determined, denote its motion as "undefined".
Q: [a man is bowling with right hand.] [a man swung his right hand backward, then threw something forward.]
A: {
The right arm is bending from straight to bent under the shoulder when preparing to release the ball and then extending from bent to straight when releasing the ball.
The left arm remains relaxed under the shoulder by the side of the body.
The right leg is undefined.
The left leg is undefined.
The torso leans forward slightly during the release of the bowling ball, aligning with the direction of the throw.
}
Q: [xxx] [xxx] ...
A:

[1]Large language models are versatile decomposers: Decomposing evidence and questions for table-based reasoning, SIGIR 2023
[2]Least-to-Most Prompting Enables Complex Reasoning in Large Language Models, ICLR 2023.
[3]Mldt: Multi-level decomposition for complex long-horizon robotic task planning with open-source large language model, arXiv 2024.
[4]Reasoning Abilities of Large Language Models: In-Depth Analysis on the Abstraction and Reasoning Corpus, ACM TIST 2025.
[5]A Survey of Uncertainty Estimation in LLMs: Theory Meets Practice. arXiv 2024

---

W2: Explaination on Euclidean distance in formulas 1 and 2

Sorry for the confusion. In fact, we calculate the inner product of the feature vectors in the normalized space. In this case, maximizing this inner product is equivalent to minimizing the l2 norm in the Euclidean space. Specifically, given two vectors x and y that are l2-normalized ($||x|| = ||y|| = 1$) $\text{argmin} \|\|\mathbf{x} - \mathbf{y}\|\|_2^2 = \text{argmin} (\mathbf{x}^\top \mathbf{x} + \mathbf{y}^\top \mathbf{y} - 2\mathbf{x}^\top \mathbf{y}) = \text{argmin} (1 + 1 - 2\mathbf{x}^\top \mathbf{y}) = \text{argmax} \ \mathbf{x}^\top \mathbf{y}.$ Therefore, the formulas 1 and 2 can encourage the consistency learning of positive motion-text pairs. We will clarify this in paper.

---

W3: If the text encoder of SGAR include LLMs?

Our text encoder does not involve an LLM. Instead, we keep the text encoder implementation the same as previous works [1-3], i.e., a pre-trained Distill-BERT, for fairness. LLMs are only for knowledge generation/augmentation as an additional external expert, which is an optional choice and independent of the text encoder (Distill-BERT). Therefore, when the LLMs are available or accessible during testing, they can be directly used for augmentation. If not, part texts can be provided by users (text-to-motion retrieval scenario), or just turn off this part-based augmentation option.

[1]Exploring vision transformers for 3D human motion language models with motion patches, CVPR 2024.
[2]TMR: Text-to-motion retrieval using contrastive 3D human motion synthesis, ICCV 2023.
[3]Chronologically accurate retrieval for temporal grounding of motion-language models, ECCV 2024.

---

W4: More comparison results in application experiments

Thanks for your advice. We try our best during rebuttal to conduct additional experiments to include recent methods for comparison, ChronRet (ECCV 24) and LaMP (ICLR 25), following the experimental settings of Section 5. Note that ChronRet has reported the motion generation results with its text encoder and we directly report their original results. Other results are reproduced using the officially released model weights under our experimental setting.

Specifically, as shown in the following tables,

1. ChronRet is mainly designed for temporally aware retrieval task, which does not demonstrate advantages for single action recognition. LaMP introduces a new motion-text matching task based on naive contrastive learning paradigm. However, this simple design can still not alleviate the over-fitting problem effectively, leading to sub-optimal alignment learning and generation capacity. Therefore, our method can achieve better perormance on generalized motion understanding, e.g., transfer learning or cross-dataset retrieval.
2. Meanwhile, we find that Lamp and ChronRet present poor performance in zero-shot action recognition with their text encoders. This indicates that these methods struggle to learn generalized text representations with limited data. In contrast, our method involves a more fine-grained part motion text alignment and regularization designs, to achieve more meaningful and generalized text representations.
3. For text-guided motion generation on HumanML3D dataset, we find these methods can well boost the performance compared with CLIP baseline, and our method has more advantages on R-Precision metrics compared with ChronNet (Note LaMP adopts and finetunes a larger text encoder, BERT, which can lead to unfair comparison under a in-domain setting).
4. For the main comparsion on motion-text retrieval, our method can significantly surpass ChronNet and LaMP as shown in Table 1 in the main paper.

Table 1: Transfer learning results for action recognition on BABEL.

Table 2: Cross-dataset motion retrieval results.

Table 3: Text encoder for zero-shot action recognition.

*LaMP utilizes a larger text encoder, i.e., BERT, while ChronNet and our method SGAR use the smaller DistillBERT.

Table 4: Text encoder for motion generation on HumanML3D.

*LaMP utilizes a larger text encoder, i.e., BERT, while ChronNet and our method SGAR use the smaller DistillBERT.

---

W5: Exaplain k in Formula 2

Sorry for the confusion. In Formula 2, $p^\tau$ is a similarity distribution that consists of multiple similarity scores between $z$ and each embedding anchor, i.e., $p^\tau = \{p^\tau_1, p^\tau_2 ..., p^\tau_N\}$ where N is the batch size in our implementation. Therefore, $k$ is a subscript in Eq. (2) and $p^\tau_k (z,z^j_{t,*})$ denotes the similarity score between $z$ and $z^{j}_{t,k}$ after softmax with temperature $\tau$. We will polish this presentation in the final version.

We thank the reviewer for the valuable feedback and constructive suggestions! We hope our responses adequately address the following questions raised about our work. Please let us know if there is anything we can clarify further.

---

W1: Computional Efficiency

As suggested, we study the computational efficiency with FLOPs and Runtime. All experiments are conducted on one NVIDIA 4090 GPU and Intel(R) Xeon(R) Platinum 8358P CPU @ 2.60GHz.

As shown in the following table, our SGAR has similar complexity and runtime to other methods. When testing GAR is enabled (SGAR++), the main additional cost is the encoding of the extra part texts (1.05G for once forward of text encoder, here we take 5 parts for example). In terms of the runtime, it should be noted that the encoding of part texts is highly parallel and does not incur much time cost.

Table 1: Comparison of computational complexity and runtime.

---

W2: Comparing with other part-based motion retrieval methods

Thanks for your advice. We have studied the previous works for 3D motion-text retrieval, and take the following works that are relevant to body parts as discussions. We distinguish our method and novelty from the following aspects:

• First, in terms of the paradigm, we argue that our method explores the part motion modeling under a new paradigm of generation-augmented retrieval (GAR), which is different from [1] and [2]. This enables the testing augmentation option for performance boosting.
• For detailed methodology, [1] considers the word-level features with a gating mechanism, to align with fine-grained motion features. However, it fails to model separate part motions on the motion encoder side with the coupled encoding of different parts. Meanwhile, it is difficult to learn this fine-grained motion-text correspondence due to the lack of explicit decoupling of linguistic knowledge as guidance. Besides, [2] explicitly considers the part motion description generation as extra knowledge. However, a naive solution for part alignment is adopted, i.e., directly using a contrastive learning objective. This ignores the difference of part motions compared with full-body motions, which are more coupled with each other due to the concurrence of different part motion patterns. In contrast, we apply LLM-based knowledge decomposition and propose part mixture decorrelation learning with directional relation regularization design, presenting a comprehensive part motion modeling framework.
• Finally, we compare the performance of the above works in the following table, where we can see that our method has a desirable performance advantage.

Table 2: Comparison of motion-text retrieval on HumanML3D dataset.

*The codes of [1,2] are not available. Therefore, we follow their test protocol that ignores the misjudgments caused by mirror augmentation for evaluation correction, and report our performance under this setting for fairness.

[1] Hierarchical Semantics Alignment for 3D Human Motion Retrieval, SIGIR 2024.
[2] KinMo: Kinematic-aware Human Motion Understanding and Generation, arXiv 2024.

---

W3 & Q1: How to ensure LLM generated query is correct?

Thanks for the insightful question. This is an important problem that we have made efforts to study the corresponding design. We explain our consideration from the following aspects:

• First, the generation of part motion texts based on the given full-body motion texts can be viewed as a process of knowledge decomposition or refinement from coarse to fine. From the perspective of users, the provided queries should include all the content that is intended to be emphasized. Otherwise, any other unrestricted associated content should be regarded as reasonable. Therefore, at this point, the LLM only needs to perform relatively simple part-wise decoupling of the provided coupled full-body motion texts. Many recent studies [1-5] have demonstrated that LLMs possess such capabilities, even in tasks that are more complex than human motion text data, which provides support for our design.
• Technically, we carefully design the LLM prompts, i.e., a few-shot strategy with [udf] indicator for uncertainty, to improve the quality of generation.
• On the other hand, due to the current limitations of LLMs and the existence of hallucination problems, this automated method cannot achieve 100% correct output results. Therefore, we randomly selected 200 samples and conducted manual checks, with an error rate of approximately 9%. Fortunately, our method has demonstrated a certain degree of robustness to the quality of part texts, as shown in Table 10 in the Appendix, which enables this design to bring expected performance improvement to motion-language models.

[1] Large language models are versatile decomposers: Decomposing evidence and questions for table-based reasoning, SIGIR 2023
[2] Least-to-Most Prompting Enables Complex Reasoning in Large Language Models, ICLR 2023
[3] Mldt: Multi-level decomposition for complex long-horizon robotic task planning with open-source large language model, arXiv 2024
[4] Reasoning Abilities of Large Language Models: In-Depth Analysis on the Abstraction and Reasoning Corpus, ACM TIST 2025
[5] Tree of Thoughts: Deliberate Problem Solving with Large Language Models, NeurIPS 2023

---

Q2: Explain why testing is better without LLM

This can be explained from three aspects:

• The key motivation is to decompose and model the part motion consistency to benefit the general motion-text alignment learning. Taking "walks and waves hand" as an example, with single global alignment learning, the model may incorrectly correspond the motion pattern of arm swinging to the verb "walk" rather than "wave hand", especially when training with a limited data scale. This means the difficulty of motion-text retrieval arises not only from the cross-modal alignment but also from the reorganization and decomposition of semantic concepts within the same modality. In this context, learning the correspondence of local concepts as primitives can lead to more precise, generalized local alignment, and ultimately enhance global understanding.
• It is also beneficial to introduce more diverse part texts generated by LLMs, which alleviates the over-fitting problem with more data.
• Moreover, as shown in the ablation study Table 2, the proposed part mixture learning and directional relation regularization further boost the performance with no LLM during testing. These designs introduce additional knowledge, e.g., novel mixed motion patterns and relational knowledge, to boost the overall performance and generalization.

---

Q3: Efficiency on the motion-to-text retrieval with LLM

Thanks for this valuable question! Actually, in motion-to-text retrieval, numerous inferences of LLM can degrade the real-time efficiency. Fortunately, we can alleviate this problem through some simple strategies. For example, the following two strategies can be considered

• Pre-computed Text Augmentation: During the database construction stage, LLM can be used in advance to generate part motion descriptions as supplementary (or the database can use LLM to generate it offline during idle time). The augmented part texts are vectorized with the index established. When querying, we can directly retrieve the pre-augmented database to avoid the overhead of real-time generation.
• Hierarchical Query: Another solution is to use hierarchical query. For instance, we can first use only the full-body motion texts to retrieve the Top-k candidates in the database, and then perform more fine-grained LLM augmentation on these Top-k texts with the GAR strategy. This can effectively reduce the number of LLM inference. To support this scheme, we provide the results of the hierarchical query when using different k (Top-k candidates). We can see that a small k (10~20) can also boost the performance significantly.

Table 3: Results using hierarchical query with different k on motion-to-text retrieval on HumanML3D.

We thank the reviewer for the valuable feedback and constructive suggestions! We hope our responses adequately address the following questions raised about our work. Please let us know if there is anything we can clarify further.

---

W1: About reproducibility, i.e., code, performance error bars

Thanks for your helpful comments. First, the code would be available upon publication. Due to the author's policy of NeurIPS, we are sorry that we can not provide any link to a demo or code during rebuttal. Alternatively, we report the average performance with error bar to better demonstrate the performance fluctuations in the experiment under 5 runs. As mentioned, retrieval results could be sensitive to randomness, while we can see that our method can consistently achieve the SOTA performance, verifying the effectiveness of our method.

Table 1: Average performance with the error bar.

---

W2: Further explanation on part-mixture learning design

Thanks for your valuable question. We design the part mixture learning inspired from both the success of previous image-based methods but also the characteristics of skeleton data.

Specifically,

• On the one hand, this can be referred in the image-based mixing contrastive learning works [1-3]. Similarly, they copy-and-replace the image patches like Cut-Mix, using linear interpolation of the embeddings of the images before mixing as positives for contrastive learning. Similarly, mixing ratio is adopted as the interpolation coefficient. In this case, the mentioned problems can still occur, that is, replacing image patches of the same size in the upper left corner or lower right corner may result in the same interpolated latents for contrastive learning. This seems to be an inherent defect of random mixing operations, whether it is for supervised learning or self-supervised contrastive learning. However, despite this, the success of these works has proved the effectiveness of the design. This can mainly be interpreted as an effective regularization of the feature space in statistical sense by applying linear equivariant transformation constraints during training [3]. To help understand, interpolation in the InfoNCE-objective-applied feature space can be viewed similar to interpolation in the prediction label space of supervised learning, (like original Cut-Mix for classification), which both perform interpolation / smoothing in the deep feature space of models.
• In addition, previous work [4] has revealed the essence of local motion pattern modeling for skeleton data, which means a subtler meaningful semantic unit as body parts. This makes different body parts all potentially contribute to the extraction of a certain global motion concept. Therefore, the mixture at the input data level can be expected to achieve interpolated features in the high-level semantic latent space. Meanwhile, as you understand, although we do not restrict which parts to be mixed (the importance of different parts to certain global motion may vary), this design is still statistically reasonable and empirically effective as discussed above. Therefore, we adopt this simple part-wise replacement strategy, retaining the meaningful motion semantic unit while encouraging a smoother representation space as regularization. Furthermore, from the authors' personal experience, mixing augmentation improves performance mainly through the representation smoothing constraint as regularization rather than obtaining strictly precise linear interpolation coefficients.

[1]i-Mix: A Domain-Agnostic Strategy for Contrastive Representation Learning, ICLR 2021.
[2]MixSiam: A Mixture-based Approach to Self-supervised Representation Learning. arXiv 2021.
[3]Un-Mix: Rethinking Image Mixtures for Unsupervised Visual Representation Learning, AAAI 2022.
[4]Actionlet-Dependent Contrastive Learning for Unsupervised Skeleton-Based Action Recognition, CVPR 2023.

---

Q1: Explain why only Part CL is better

Only part CL (no LLM is used under testing) still boosts global-motion-based retrieval, which can be explained as follows:

• The key motivation is to decompose and model the part motion consistency to benefit the general motion-text alignment learning. Taking "walks and waves hand" as an example, with single global alignment learning, the model may incorrectly correspond the motion pattern of arm swinging to the verb "walk" rather than "wave hand", especially when training with a limited data scale. This means the difficulty of motion-text retrieval arises not only from the cross-modal alignment but also from the reorganization and decomposition of semantic concepts within the same modality. In this context, learning the correspondence of local concepts as primitives can lead to more precise, generalized local alignment, and ultimately enhance global understanding.
• It is also beneficial to introduce more diverse part texts generated by LLMs, which alleviates the over-fitting problem with more data.