CacheFlow: Fast Human Motion Prediction by Cached Normalizing Flow

We sincerely appreciate the reviewer’s thoughtful feedback and are encouraged by the recognition of our method as “a neat way” and “general.” Below, we address each point in detail:

How much memory is required for storing the embedded representation?

Thank you for highlighting the importance of memory efficiency in real-world applications. We have added an evaluation of both computational and memory costs for storing cached embeddings. We used Intel(R) Xeon(R) Gold 6154 for CPU calculation and single NVIDIA TITAN RTX (note that different from the main paper's setting NVIDIA A100) for GPU calculation. As shown in Table 1, the cached embeddings for the entire AMASS dataset require only 46.90 MB, and the total GPU memory usage is 875 MB. This demonstrates that our method is well-suited for edge robotics platforms with constrained resources. For instance, even the most affordable NVIDIA Jetson Orin Nano Super offers 8 GB RAM, making our approach highly practical.

Table1. Conputation and memory costs of CacheFlow.

How applicable is the solution to CPU-based systems?

We agree that CPU performance is critical for broader deployment. As shown in Table 1, our method achieves 6.9 ms inference time on CPU for AMASS, which remains efficient despite being slower than GPU-based inference. This efficiency stems from our formulation, which minimizes computational overhead. Thus, our method is also viable for CPU-only systems.

What other benefits does the likelihood estimation bring?

The likelihood estimation enhances the discriminative power of our model beyond diverse generation. It enables the system to differentiate between likely and unlikely motions, which is crucial for applications such as collision avoidance and safe human-robot interaction. This added capability contributes to both safety and reliability in deployment scenarios.

Running GMM directly on embeddings (No flow matching).

We appreciate the suggestion and conducted an ablation study by removing flow matching and applying GMM directly to the embeddings. As shown in Table 2, this approach significantly degrades performance across all metrics. The drop in accuracy is due to the inability of anisotropic Gaussians to model the complex latent distribution effectively. These results underscore the necessity of flow matching in our framework.

Table 2. Comparison to the GMM only model (without Flow Matching) on Human3.6M.

More qualitative results are favorable.

We acknowledge the value of qualitative results and agree they can further illustrate the strengths of our method. Due to rebuttal phase constraints, we are unable to include images or videos. However, we commit to adding more visualizations and demo clips in the camera-ready version and accompanying GitHub repository.

We sincerely thank the reviewer for their insightful feedback. We are encouraged by the recognition that our “idea is interesting” and that the paper is “well written and organized.” Below, we address each point in detail:

Generalization capability.

Thank you for raising this important point. To clarify, our method does not sample directly from the training motions. Instead, it generates novel future motions by decoding a combination of the input past motion and a sampled behavioral embedding from the Behavioral Latent Space (BLS) [1]. This design enables generalization beyond the training data.

The decoder in CacheFlow is deterministic and does not involve probabilistic sampling. However, because the behavioral embeddings are sampled, each new input past motion can yield multiple diverse future predictions. This mechanism allows CacheFlow to generate new and plausible motions, rather than merely retrieving or interpolating from the training set.

Performance gap on diversity metric APD

We appreciate the concern regarding APD. While APD measures diversity via pairwise distances between generated samples, higher APD does not always imply better diversity. In fact, excessively high APD can indicate unrealistic or noisy predictions that ignore the input context.

For example, GSPS and DivSamp report very high APDs (14.757 and 15.310), but we observed failure cases such as unrealistic floating motions, as also mentioned in other paper[2]. In contrast, recent diffusion-based models (BeLFusion, HumanMAC, CoMusion) prioritize plausibility and contextual consistency, resulting in moderate APDs (7.602, 6.301, 7.632).

Our method achieves a comparable APD (6.101) to these diffusion-based methods, indicating a good balance between diversity and realism. Furthermore, we evaluated Frechet Inception Distance (FID), which better captures semantic and distributional similarity to real data. CacheFlow achieves significantly better FID than GSPS and DivSamp, supporting our claim that our method generates realistic and diverse future motions.

Table 1. The APD (Average Pairwise Distance) and FID (Frechet Inception Distance) on the previous methods and CacheFlow on Human3.6M.

Concern on large overlap between training and test sets

We understand the concern about potential overlap between training and test sets. However, the AMASS benchmark is constructed from multiple MoCap datasets with distinct subjects and motion categories. The train-test split is performed dataset-wise, ensuring no overlap in subjects or motion sequences.

Moreover, as discussed above, CacheFlow generates novel future motions conditioned on novel input sequences, further confirming that our performance is not due to memorization or overlap.

Typos.

Thank you for your careful reading. We will correct the identified typos and ensure the manuscript is polished for the camera-ready version.

References

[1] German Barquero, Sergio Escalera, and Cristina Palmero. Belfusion: Latent diffusion for behavior-driven human motion prediction. In ICCV, pages 2317–2327, 2023.

[2] Ling-Hao Chen, Jiawei Zhang, Yewen Li, Yiren Pang, Xiaobo Xia, and Tongliang Liu. Humanmac: Masked motion completion for human motion prediction. In ICCV, pages 9544–9555, 2023

I appreciate the authors' clear explanation. All of my concerns have been addressed. I would like to raise my rating.

We sincerely thank the reviewer for their thoughtful remarks. We are encouraged by the recognition that our method is “practical” and achieves “competitive accuracy with state-of-the-art methods.” Below, we address each point in detail:

Impact on the diversity and accuracy.

We agree that sampling from a fixed set can potentially limit diversity and accuracy. However, CacheFlow does not rely on simple retrieval. Instead, it generates novel future motions by decoding the combination of the input past motion and a sampled behavioral embedding from the Behavioral Latent Space (BLS) [1].

Diversity: The diversity arises from the rich combinations of input motions and precomputed embeddings, not merely the number of cached entries. Accuracy: The generated motions are well-aligned with the input context, thanks to the structured decoding process. Therefore, CacheFlow offers greater diversity and accuracy than nearest-neighbor retrieval, as it synthesizes new motions rather than selecting existing ones.

Physically motivated constraints.

Thank you for this excellent suggestion. We agree that incorporating physically motivated constraints can enhance the realism and plausibility of predicted motions. In this work, we used BLS as-is, but we plan to integrate such constraints into the training of both the encoder and decoder in future iterations.

Regarding qualitative results, we are currently unable to include images or videos during the rebuttal phase. However, we will provide additional visualizations and demo clips in the camera-ready version and on our GitHub repository.

Cross-action generalization.

The AMASS benchmark comprises multiple MoCap datasets with diverse motion categories and subjects. The train-test split is performed dataset-wise, ensuring no overlap in subjects or motion sequences. Thus, the results on AMASS reflect cross-dataset and cross-action generalization.

CacheFlow achieves high accuracy on AMASS while maintaining exceptional inference efficiency, demonstrating its robustness across varied motion types.

Augmenting the cache.

We appreciate the suggestion to explore cache augmentation. We investigated two strategies:

Dense Sampling: We varied the stride used to sample training motions (from every 10 frames down to every frame). Cache Interpolation: We interpolated between cached embeddings to increase coverage.

Table 1. The effect of cache augmentation on Human3.6M.

While augmentation slightly improves accuracy, it also increases computational cost due to the larger cache size.

Additionally, for test-time adaptation, one can assume motion sequences are streamed over time. As one of the augmentation methods, Cached embeddings of these past motions can be added dynamically. Embedding computation is efficient and parallelizable, making this a promising direction for future work.

Typos and formatting suggestion.

Thank you for your careful reading. We will correct the typo “FlowPrecomp” and improve the formatting of Table 1 by adding appropriate highlighting.

References

[1] German Barquero, Sergio Escalera, and Cristina Palmero. Belfusion: Latent diffusion for behavior-driven human motion prediction. In ICCV, pages 2317–2327, 2023.

We sincerely thank the reviewer for their helpful and constructive comments. We are encouraged by the recognition that our paper “does an excellent job of motivating the research” and that the “concept is presented clearly.” Below, we address each point in detail:

Computational and memory costs.

Following your suggestion, we evaluated CacheFlow’s computational and memory efficiency using an Intel(R) Xeon(R) Gold 6154 CPU and a single NVIDIA TITAN RTX GPU (note: different from the A100 used in the main paper).

Table 1. Conputation and memory costs of CacheFlow.

(a) Cache Generation Time: Cache creation is efficient—just 1h10min on GPU and 3h40min on CPU for the full AMASS training set (sampled every 10 frames). This is feasible due to the compact 8-dimensional latent space, derived from Behavioral Latent Space [1] and linear factorization (e.g., SLD [2]).

(b) Each cached triplet $\\{z, |det J_{f_\theta}(z)|^{-1}, x\\}$ uses 68 bytes (17 float32). Thus, even the full AMASS cache occupies only 46.90 MB—extremely lightweight.

(c) Inference requires 811 MB (Human3.6M) and 875 MB (AMASS), which is well within the limits of edge devices like the NVIDIA Jetson Orin Nano Super (8 GB RAM).

In summary, CacheFlow is computationally efficient and memory-friendly, making it suitable for deployment in resource-constrained environments such as robotics and AR devices.

How robust to noisy input data?

Thank you for this important suggestion. We evaluated CacheFlow and BeLFusion under varying levels of Gaussian noise added to the input past motion. The highest noise level (100%) corresponds to the MPJPE error (67.5 mm) of the SoTA 3D pose estimator [3] on 3DPW.

Table 2. Prediction accuracy with different noise levels.

CacheFlow shows a sharper drop in accuracy under noise compared to BeLFusion, likely due to its reliance on clean cached embeddings. This highlights a domain shift issue.

Mitigation Strategies:

• Smoothing: Techniques like moving average or advanced methods (e.g., diffusion-based recovery [4], physics-based restoration [5]) can reduce noise.
• Training with Noisy Data: Adding noise during training and cache generation can help CacheFlow learn to denoise implicitly. We plan to include results with these strategies during the discussion phase.

How is the cache updated or adapted?

We argue that the full regeneration of the entire cache doesn't require a substantial cost, which is different from the suggenstion of the reviewer. As shown in the Table 1, cache generation is significantly cheaper than model training. This flexibility allows CacheFlow to adapt to new domains without costly fine-tuning.

One can perform test-time cache adaptation for the both senarios (a) and (b), or augmentation via disentanglement for the senario (a).

Adaptation Strategies:

• Test-Time Cache Adaptation: Motion data is typically streamed during inference. Since generating one cache entry takes ~0.5 seconds, the cache can be updated dynamically to adapt test-time motions.
• Augmentation via Disentanglement: Using annotations (e.g., subject, action), we can disentangle feature spaces and synthesize new cache entries for unseen combinations (new user - existing motion type, existing user - new motion type).

These strategies enhance CacheFlow’s adaptability to new users, tasks, and environments.

The benchmarks lack diversity.

We appreciate the suggestion to explore broader mobility contexts (elderly, with disabilities). While we plan to incorporate musculoskeletal motion synthesis for different mobility in future work, the AMASS benchmark already includes 15 diverse MoCap datasets with varied motion categories, subjects, and cultural backgrounds. We believe this provides a reasonable level of diversity in terms of age, body type, and context.

Performance on the underrepresented motions

Thank you for this insightful point. We argue that this is not a specific limitation of CacheFlow. Models without caching may overlook rare motions due to data imbalance. In contrast, CacheFlow can memorize and leverage all training motions, potentially improving performance on underrepresented cases. We welcome further discussion on this topic.

References

[1] German Barquero, Sergio Escalera, and Cristina Palmero. Belfusion: Latent diffusion for behavior-driven human motion prediction. In ICCV, pages 2317–2327, 2023.

[2] Xu, Guowei, et al. "Learning semantic latent directions for accurate and controllable human motion prediction." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.

[3] Fiche, Guénolé, et al. "MEGA: Masked Generative Autoencoder for Human Mesh Recovery." Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.

[4] Zhang, Siwei, et al. "Rohm: Robust human motion reconstruction via diffusion." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.

[5] Zhang, Youliang, et al. "A Plug-and-Play Physical Motion Restoration Approach for In-the-Wild High-Difficulty Motions." arXiv preprint arXiv:2412.17377 (2024).