RoPECraft: Training-Free Motion Transfer with Trajectory-Guided RoPE Optimization on Diffusion Transformers

We thank the reviewer for their feedback, finding our work novel and clear.

Regarding runtime comparisons: Over a total of 360 individual runs each taken place on WAN 2.1 based our implementation with floating point 16 precision. Each run has been executed in a single H100 gpu. the configuration of each run is as follows:

• number of optimization steps = 5
• height of video = 480
• width of video = 832
• number of frames = 49
• number of inference steps = 50

According to this configuration, you can find the runtime information in the table below:

As shown in the table, all methods except ConMo have comparable runtimes, with our method running at 109.2 ± 3.1 seconds. While not the fastest, it remains within the typical range observed across recent approaches, balancing efficiency with improved realism. We appreciate the reviewer’s suggestion regarding runtime analysis. We will add the comparison table and a brief discussion in the revised manuscript.

Regarding more challenging datasets: Our method is also successful in more challenging scenarios beyond the DAVIS dataset, such as RealCamVid [1]. However, due to this year’s new rules, we cannot provide links or a PDF to showcase our results. You will be able to see the additional experiments in the updated Supplementary, in case of acceptance, thank you for your understanding. To provide evidence that our method can handle camera motion and multiple object cases better compared to current baselines, we have randomly sampled 20 videos from RealCamVid [1] dataset ensuring they contain camera motion, multi object and relatively harder samples compared to DAVIS. We have generated 2 prompts per video using the same approach described in the main paper. The computed metrics for these models is as follows:

As it can be seen, our model surpasses all of its competitors on text and motion alignment in this dataset.

In addition, regarding the breakdance example in the DAVIS dataset, our method performs significantly better than the others. We appreciate the reviewer’s suggestion and will add harder examples in the revised manuscript.

[1] Zheng, Guangcong, et al. "Realcam-vid: High-resolution video dataset with dynamic scenes and metric-scale camera movements." arXiv preprint arXiv:2504.08212 (2025).

We thank the reviewer for their feedback, finding our work easy to follow, original, and interesting.

Regarding FTD: We agree with the observation that RoPECraft produces more realistic results, whereas GWTF adheres more strictly to the conditioning motion. This likely explains why GWTF slightly outperforms RoPECraft on the FTD metric, as noted by the Reviewer. Since FTD is specifically designed to evaluate motion alignment rather than visual realism, the difference in scores reflects this trade-off. However, our results also show that strict adherence to the conditioning motion does not always lead to better outcomes. GWTF has higher adherence to the reference motion, we observe that this is not an advantage for GWTF, leading to distortions and conflicts with the text prompt. For example, when generating a flock of animals from a single-object reference video, GWTF often overfits to the single instance, producing unrealistic and distorted outputs (see “CogVideoX results” on the website, in the supplementary ZIP).

We believe that both motion adherence (e.g., FTD) and realism/naturalness (e.g., user study and CD-FVD scores) are essential for a holistic evaluation of motion transfer methods. This is a very important point, and we will include this discussion in the final manuscript, thank you for the suggestion.

Regarding user study: As suggested, we conducted a user study with 21 participants to evaluate both realism and proposed metrics alignment with human preferences. The user study is conducted as follows:

For sample selection, we randomly sampled 20 prompts and reference videos from DAVIS and their custom prompts described in the paper, and generated outputs using all compared methods (including ours), and presented users with the resulting videos for each prompt-reference pair. Then, we calculated the FTD and MF score for each generated video-ground truth video pair. For reference, some portion of the data is given below:

In the first part of the survey, users were shown the 7 generated videos per prompt and asked to select the top 3 that best matched the motion in the reference video. Participants were instructed to focus solely on motion alignment, ignoring visual quality. This section aimed to compare how well FTD and MF align with human perception of motion. For each prompt and video sample the procedure is as follows:

• Provide all 7 video samples to users, for each question (10 in total).
• Users pick top-3 samples among 7.
• For each video, we also get the subset of ground-truth FTD and MF sorted scores.
• We compare their user and ground-truth match, and assign percentages, to get a sub-table for each user.
• We average and normalize each sub-table of each user to get the final table.

For Part 1, on the average of all prompts and all users, the results are as follows:

The table clearly reveals that our proposed FTD metric is more aligned with human perception, further suggesting its success.

In the second section of the survey, users were asked to order the best 3 among the outputs based on visual quality, text prompt alignment, and motion alignment. This aimed to evaluate overall method quality. For each prompt and video sample the procedure is as follows:

• Provide all 7 video samples to users, for each question (10 in total).
• Users pick top-3 samples among 7.
• Average all 3 choices in themselves (each row would sum up to 100%)

For Part 2, on the average of all prompts and all users, the results are as follows:

The table shows that our proposed method, RoPECraft, is consistently rated as more visually appealing, accounting for visual quality, text prompt alignment, and motion alignment all at the same time. RoPECraft was consistently preferred over other methods, indicating that our approach is better than the current literature in human perception.

The results reveal two key insights: (1) FTD correlates better than MF in evaluating motion fidelity, and (2) RoPECraft is consistently rated as more visually appealing while also achieving good motion alignment.

We would also like to again point out why FTD may be preferred over MF. FTD can be more reliable than MF for evaluating motion alignment because it considers the entire motion trajectory over time, capturing the global structure and temporal consistency of motion. Unlike MF, which focuses on frame-to-frame displacements and overlooks long-term trajectory patterns, FTD evaluates the overall shape of the motion path, making it sensitive to deviations across multiple frames. FTD also effectively handles occlusions by reassigning or discarding invalid trajectories, ensuring that only valid motion paths contribute to the score. In contrast, MF does not account for occlusions or tracking errors, leading to potentially misleading results. Furthermore, while MF normalizes frame-wise displacements and ignores their magnitude, FTD preserves such variations, making it more robust in capturing complex motion dynamics. Overall, we argue that FTD offers a more holistic and accurate reflection of motion similarity, especially in dynamic or occlusion-heavy scenes.

We thank the reviewer for their positive feedback and for recognizing our work as insightful, interesting, and well presented.

Regarding multi-object cases: RoPECraft is capable of handling multi-object motion transfer. However, due to this year’s new rules, we cannot provide links or a PDF to showcase our results. You will be able to see the additional experiments in the updated Supplementary, in case of acceptance, thank you for your understanding. To provide evidence that our method can handle camera motion and multiple object cases better compared to current baselines, we have randomly sampled 20 videos from RealCamVid [1] dataset ensuring they contain camera motion, multi object and relatively harder samples compared to DAVIS. We have generated 2 prompts per video using the same approach described in the main paper. The computed metrics for these models is as follows:

As it can be seen, our model surpasses all of its competitors on text and motion alignment in this dataset.

Regarding preserving background: The goal of motion transfer is to transfer the motion from a reference video to a newly generated one, guided by a text prompt. While the prompt may not always specify the background, DiT models (owing to their attention mechanisms and training data) naturally generate coherent and realistic backgrounds.

That said, by applying flow-matching optimization for longer and using background masks, it is possible to preserve the original background while transferring only the motion. In fact, we verified that it works with our parameters in Supplementary A.3; but with the mask and by changing s=30 and t=20 for this sub-task. We will include the additional results to the Supplementary, thank you for suggesting this.

Regarding typos: In Fig. 2, the caption for Column 4 should read Column 5. We will go through the manuscript and fix the errors, thank you.

[1] Zheng, Guangcong, et al. "Realcam-vid: High-resolution video dataset with dynamic scenes and metric-scale camera movements." arXiv preprint arXiv:2504.08212 (2025).

We thank the reviewer for their positive feedback and for highlighting the significance of our problem setting, the advantages of our training-free approach, the qualitative improvements over prior work, and the value of our ablation studies.

Regarding the motion directions of other methods: Our experiments with MOFT, DiTFlow, and ConMo are based directly on the official DiTFlow [CVPR25] codebase. We did not re-implement these methods but used the provided code implementation in Wan 2.1 as-is. We have carefully double-checked the results and can confirm that the observed motion direction differences stem from limitations of the baseline methods, not from any implementation issues.

While the motion direction is generally correct across methods, we observe that object orientation is often incorrect in other competing methods. This can be more clearly seen with the background motion, which suggests that motion is being transferred to incorrect subjects. Our flow-matching and phase constraints address these issues, leading to more accurate motion transfer. We will add this discussion to the final manuscript, thank you for pointing it out.

Regarding theoretical analysis and applicability to RoPE-based models: We agree that our study is primarily empirical and specifically applies to models using RoPE as their positional encoding within DiT architectures. However, we believe this does not diminish the contribution of our work. RoPE is increasingly adopted in modern generative models, and our experimental insights provide a timely and valuable foundation for future research. There is growing interest in exploring RoPE-based mechanisms in recent literature. For example, recent work has extended its use to camera matrices [1] or there have been recent works that aim to update RoPE for longer video generation [2] or their integration to the video models [3]. Similarly, our approach (modifying RoPE tensors directly with flow values) may inspire further advancements, such as consistent long video generation or video editing.

Regarding user study: As suggested, we conducted a user study with 21 participants to evaluate both realism and proposed metrics alignment with human preferences. The user study is conducted as follows:

For sample selection, we randomly sampled 20 prompts and reference videos from DAVIS and their custom prompts described in the paper, and generated outputs using all compared methods (including ours), and presented users with the resulting videos for each prompt-reference pair. Then, we calculated the FTD and MF score for each generated video-ground truth video pair. For reference, some portion of the data is given below:

In the first part of the survey, users were shown the 7 generated videos per prompt and asked to select the top 3 that best matched the motion in the reference video. Participants were instructed to focus solely on motion alignment, ignoring visual quality. This section aimed to compare how well FTD and MF align with human perception of motion. For each prompt and video sample the procedure is as follows:

• Provide all 7 video samples to users, for each question (10 in total).
• Users pick top-3 samples among 7.
• For each video, we also get the subset of ground-truth FTD and MF sorted scores.
• We compare their user and ground-truth match, and assign percentages, to get a sub-table for each user.
• We average and normalize each sub-table of each user to get the final table.

For Part 1, on the average of all prompts and all users, the results are as follows:

The table clearly reveals that our proposed FTD metric is more aligned with human perception, further suggesting its success.

In the second section of the survey, users were asked to order the best 3 among the outputs based on visual quality, text prompt alignment, and motion alignment. This aimed to evaluate overall method quality. For each prompt and video sample the procedure is as follows:

• Provide all 7 video samples to users, for each question (10 in total).
• Users pick top-3 samples among 7.
• Average all 3 choices in themselves (each row would sum up to 100%)

For Part 2, on the average of all prompts and all users, the results are as follows:

The table shows that our proposed method, RoPECraft, is consistently rated as more visually appealing, accounting for visual quality, text prompt alignment, and motion alignment all at the same time. RoPECraft was consistently preferred over other methods, indicating that our approach is better than the current literature in human perception.

The results reveal two key insights: (1) FTD correlates better than MF in evaluating motion fidelity, and (2) RoPECraft is consistently rated as more visually appealing while also achieving good motion alignment.

We would also like to again point out why FTD may be preferred over MF. FTD can be more reliable than MF for evaluating motion alignment because it considers the entire motion trajectory over time, capturing the global structure and temporal consistency of motion. Unlike MF, which focuses on frame-to-frame displacements and overlooks long-term trajectory patterns, FTD evaluates the overall shape of the motion path, making it sensitive to deviations across multiple frames. FTD also effectively handles occlusions by reassigning or discarding invalid trajectories, ensuring that only valid motion paths contribute to the score. In contrast, MF does not account for occlusions or tracking errors, leading to potentially misleading results. Furthermore, while MF normalizes frame-wise displacements and ignores their magnitude, FTD preserves such variations, making it more robust in capturing complex motion dynamics. Overall, we argue that FTD offers a more holistic and accurate reflection of motion similarity, especially in dynamic or occlusion-heavy scenes.

Regarding camera motion: Our method effectively transfers camera motion. However, due to this year’s new rules, we cannot provide links or a PDF to showcase our results. You will be able to see the additional experiments in the updated Supplementary, in case of acceptance, thank you for your understanding. To provide evidence that our method can handle camera motion and multiple object cases better compared to current baselines, we have randomly sampled 20 videos from RealCamVid [4] dataset ensuring they contain camera motion, multi object and relatively harder samples compared to DAVIS. We have generated 2 prompts per video using the same approach described in the main paper. The computed metrics for these models is as follows:

As it can be seen, our model surpasses all of its competitors on text and motion alignment in this dataset.

[1] Li et al., Cameras as Relative Positional Encoding, 2025.

[2] RIFLEx: A Free Lunch for Length Extrapolation in Video Diffusion Transformers, ICML 25

[3] VideoRoPE: What Makes for Good Video Rotary Position Embedding?, ICML 25, Oral

[4] Zheng, Guangcong, et al. "Realcam-vid: High-resolution video dataset with dynamic scenes and metric-scale camera movements." arXiv preprint arXiv:2504.08212 (2025).