We thank reviewer 6B9z again for reviewing our paper and providing helpful feedback! Here we address your concerns one by one.

The framework may struggle with ambiguities in language instructions or unexpected changes in the environment.

We point out that ambiguous language instruction and unexpected environmental changes fall outside the scope of this study. This work is based on the static MDP assumption, where we assume a language instruction can clearly specify the goal, and the MDP is not changed during the model-based planning and policy execution.

Depending on the computational demands of the multi-modal modules, real-time performance in dynamic environments could be a concern. Evaluating the speed and efficiency of planning under real-time constraints would be crucial.

Thanks for your question. Yes, real-time (which means about 30 HZ) inference speed is a question of our model. This is mainly constrained by: 1) the denoising process of the dynamics module; 2) the model-based planning process with three networks. However, with the development of faster denoising techniques such as [1], we believe we can achieve near real-time planning performance in the future.

[1] Lu, Cheng, and Yang Song. "Simplifying, Stabilizing and Scaling Continuous-Time Consistency Models." arXiv preprint arXiv:2410.11081 (2024).

How does FLIP compare to other state-of-the-art planning frameworks in terms of efficiency and effectiveness?

We compare FLIP to two generative planning methods (UniPi [1] and IRASim [2]) in Table 1 and Table 2. We hope these results can address your concerns.

What are the limitations of using image flow as an action representation? In which scenarios/tasks, this would not be ideal?

Thanks for your good question! There are two main limitations of using image flows as action representation: 1) 3D motions that have very small movements in the 2D space. For example, an arrow flying in the direction perpendicular to the screen. 2) objects with a very small size in the image. For example, if some object is put very far away from the camera, there could be no query points sampled on that object because it only occupies several pixels. Thus its movement will not be predicted.

How does FLIP handle ambiguities in language instructions?

Currently, we use the same language instruction during testing.

The language embedding is extracted from Meta Llama 3.1, thus we leave the ambiguities in language instructions to LLMs. We show zero-shot results in our website of the pretrained FLIP model on LIBERO-90 to LIBERO-LONG tasks, where the language instruction and the whole scene are new to the model. Since all of the three modules of FLIP is scalable, we hope in the future, with internet-level data, it can handle the ambiguities in language instructions.

Are there any results on the real robot that show the applicability on world models even for quasi-static tasks?

Thanks for your question. We add real robot experiments in Appendix B.4.

How will the model's performance be affected in the presence of noise or visual obstructions?

Thanks for your good question! We add visual obstructions experiments in the LIBERO-LONG experiments. We manually add a picture of an apple into the scene during the planning, and from the results, we can see that our flow generation model can resist visual distraction, and the video generation model can generate dynamics for a few planning steps (x16 for execution steps), while it will generation distorted images after that. This shows that flows are a good choice to represent high-level actions that can be robust to visual distractions. Since our model is only trained on specific sense data, we believe with internet-level data, the video generation model will also perform better.

How does the performance of FLIP depend upon or scale with the size of the available data, numerical analysis for the same would be helpful to understand the efficiency of the proposed approach.

Thanks for your question. We add a data scalability experiment in Appendix B.5. From Table 9, we can see that with more data for each task, the planning success rates become better.

We thank reviewer 6B9z again for reviewing our paper and providing helpful feedback! We hope the additional experiments and clarifications have addressed the reviewer’s concerns. If the reviewer has any further concerns or questions, please let us know before the rebuttal period ends, and we will be happy to address them.

The authors response clarify my concerns and I update my score to 6.

We thank reviewer R3ys again for reviewing our paper and providing helpful feedback! Here we address your concerns one by one.

There are no real-robot experiments to validate the effectiveness of the proposed method in policy learning in the real world.

We have added real-world experiments in Appendix B.4. Please check this section and our website for more real-world results.

Including a comparison with recent policy learning methods, like OpenVLA [1] or Octo [2], would further strengthen the paper.

Thanks for your suggestion! We add an experiment of OpenVLA on LIBERO-LONG to compare its results with FLIP. We show the results of zero-shot and fine-tuned with 50 demonstrations for each task of OpenVLA in Appendix B.2. We can see that OpenVLA cannot handle the long-horizon tasks of LIBERO-LONG either with zero-shot or fine-tuned models, showing there is still a long way to go for general-purpose vision-language-action models.

It is worth noting that, in the original OpenVLA paper, they also fine-tuned the pretrained model on LIBERO-LONG tasks and archived a 53.7 ± 1.3% success rate. We think the success of their results comes from two aspects, which cannot be true in our setting: 1) we are using a resolution of 128×128, which may not be large enough to represent the details in the scene. In comparison, OpenVLA uses a resolution of 256×256. 2) We are using the official demonstrations provided by the LIBERO paper, which may not be as good as the re-collected demonstrations in their demonstrations.

The paper compares with IRASim on a text-to-video task. However, IRASim generates video based on a trajectory instead of a text. For the text-to-video task, it would be better to compare with a text-to-video method (e.g. [3]).

Thanks for your question. The LVDM [4] baseline we used is a text-to-video method. We hope this can answer your question.

Is it possible to provide more details on how FLIP-NC performs beam search without a value guidance?

Thanks for your question. FLIP-NC performs a beam search in an autoregressive manner, which means it initializes the same number of beams as FLIP and generates the long-horizon flows and videos iteratively within each beam, without multiple action generation for each beam.

Is it possible to provide a more detailed description on the typical failure modes of FLIP in policy learning (Sec. 5.3) ?

Thanks for your question. We provide some failure videos of the trained policy in our attached videos and on our website. The typical failure mode is that the robot has a small action error when it is going to grasp something, which leads to an unsuccessful grasp and the robot will do this action repeatedly. Interestingly, our flow generation model can still generate reasonable future flows in these out-of-distribution areas to guide the robot to the correct regions. However, it sometimes cannot accomplish this in given episode timestep limits.

What information is provided for the two comparing baseline methods (LVDM and IRASim)?

Thanks for your question. For LVDM, there is no additional information provided. For IRASim, it is provided with the end-effector trajectory extracted with SAM2.

Are the flow generation model and the video generation model trained individually or jointly?

They are trained individually.

We thank reviewer R3ys again for reviewing our paper and providing helpful feedback! We hope the additional experiments and clarifications have addressed the reviewer’s concerns. If the reviewer has any further concerns or questions, please let us know before the rebuttal period ends, and we will be happy to address them.

We have carefully revised the manuscript to address the concerns that led to your lowered score, and we sincerely hope the changes will allow for a more favorable consideration. Thanks!

We thank reviewer 9xux again for reviewing our paper and providing helpful feedback! Here we address your concerns one by one.

The paper could have compared to some stronger baselines ... UniSim [1] ... DIAMOND [2]. Both of these paper use just a diffusion model without the need to first extract action flows, which seems to contradict to the key proposal of this paper, which is to make the model flow-centric. Some discussion on this would be appreciated.

Thanks for your question. The baseline used in our paper is LVDM [3], UniPI[4], and IRASim [5], which cover UniSim and DIAMOND in network architectures.

For UniSim, it uses a standard DiT [6] architecture, which is the same as IRASim, and we have shown the advantage of our newly designed dynamics module in Table 3. The originality of UniSim is that they use an internet-scale video dataset to train, which is beyond the scope of our method.

For DIAMOND, it uses a U-Net structure, which is also used in UniPI and LVDM, and we have already compared them in Table 1, Table 2, and Table 3. The novelty of DIAMOND is that they use EDM [7] to train the diffusion model for a faster denoising process, and train a reinforcement learning agent in the dreamed world, which are not the main points of this paper.

Thus, we think it is not necessary to add these two baselines. We have cited them in our paper.

For the flow generation model, why using a C-VAE instead of a diffusion model?

Thanks for your question. We use CVAE because it has a shorter inference time than diffusion models. Here we add an ablation experiment in Appendix B.3 to show if diffusion models can achieve better results for flow generation. The architecture is a DiT [6]. From the results in Table 7, we can see that there is not too much difference between CVAE and diffusion models in LIBERO- LONG and Bridge-V2 data, which shows that for such short-horizon flow generation tasks, CVAE is enough to represent them.

It is worth noting that there are some specially designed flow generation architectures with diffusion models, and here we think this is out of the scope of this paper and we leave this question for future works. We believe in larger datasets with diverse flows, diffusion models can be better than CVAE. For example, in [8], they first use a pretrained Stable-Diffusion model to extract the embedding of the flow, then use AnimateDiff [9] as the backbone for flow diffusion.

please specify the quality of the video demonstrations.

Thanks for your question. The demonstrations used to train the FLIP model are all expert videos. This is because the value module is aligned with the language instruction, which is the goal of the whole video. If there are some random policies during the video, the value module will not be able to identify them and assign some values according to their corresponding task progress. We believe that with an internet-level training dataset, this issue could be alleviated.

[1] Yang, Mengjiao, et al. "Learning interactive real-world simulators." arXiv preprint arXiv:2310.06114 (2023).

[2] Alonso, Eloi, et al. "Diffusion for World Modeling: Visual Details Matter in Atari." arXiv preprint arXiv:2405.12399 (2024).

[3] He, Yingqing, et al. "Latent video diffusion models for high-fidelity long video generation." arXiv preprint arXiv:2211.13221 (2022).

[4] Du, Yilun, et al. "Learning universal policies via text-guided video generation." Advances in Neural Information Processing Systems 36 (2024).

[5] Zhu, Fangqi, et al. "IRASim: Learning Interactive Real-Robot Action Simulators." arXiv preprint arXiv:2406.14540 (2024).

[6] Peebles, William, and Saining Xie. "Scalable diffusion models with transformers." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.

[7] Karras, Tero, et al. "Elucidating the design space of diffusion-based generative models." Advances in neural information processing systems 35 (2022): 26565-26577.

[8] Xu, Mengda, et al. "Flow as the cross-domain manipulation interface." arXiv preprint arXiv:2407.15208 (2024).

[9] Guo, Yuwei, et al. "Animatediff: Animate your personalized text-to-image diffusion models without specific tuning." arXiv preprint arXiv:2307.04725 (2023).

We thank reviewer 9xux again for reviewing our paper and providing helpful feedback! We hope the additional experiments and clarifications have addressed the reviewer’s concerns. If the reviewer has any further concerns or questions, please let us know before the rebuttal period ends, and we will be happy to address them. We deeply appreciate your thorough review and constructive comments, which have been invaluable in refining the paper's presentation.

We have carefully revised the manuscript to address the concerns that led to your lowered score, and we sincerely hope the changes will allow for a more favorable consideration. Thanks!

I would have liked to see the authors train the model on the full LIBERO benchmark and then finetuning on a subset of tasks for policy learning (or even internet-scale pertaining)

We have trained the agent view world model on the LIBERO-90 (although with a smaller resolution of 64×64) and shown the zero-shot generative planning results on LIBERO-LONG in Figure 9, as well as on the website. We are sorry that we do not have enough resources to retrain it with a resolution of 128×128 or train for the eye-in-hand view on LIBERO-90 during this rebuttal, but we will do this after this paper gets accepted.

To show the fine-tuning world model for policy learning, we finetune this pretrained agent view model with 50 actionless demonstrations for each task of LIBERO-LONG with a resolution of 64×64, and use 10 demonstrations with action labels for policy learning. The architecture is the same in the first question. Results are in Figure 12 in Appendix B.2. We can see that Ours-90 performs similarly to Ours-F and Ours-FV, showing that pretraining in other tasks may not bring significant improvement for low-level policy learning. This comforts with the lifelong learning results in the original LIBERO paper, where they also show that pretraining cannot help (sometimes even hurt) the policy training results.

I would have liked to see a larger emphasis on the actual analysis for manipulation tasks.

We use the success rates of the low-level policy as the quantitative metric for our model. For qualitative analysis, the advantage of FLIP over other generative planning methods on manipulation tasks is two-fold, as stated in the Introduction Section: 1) be able to represent various kinds of movements across diverse objects, robots, and tasks in the whole scene (e.g., dexterous manipulation and deformable objects are very hard to describe with languages); 2) be easy to obtain or label a large amount of training data for scaling up.

Please also check our real-world experiments and results in Appendix B.4.

How much does the action space affect downstream performance?

In our original experiments, we used behavior cloning with delta translation, rotation, and aperture of the end-effector as the action space. In our new experiments, we use action chunking with the same delta action as our action space, and the results show that with the receding horizon policy, the results are better.

We thank reviewer H4xH again for reviewing our paper and providing helpful feedback! Here we address your concerns one by one.

Why doesn’t this method offer significant downstream benefits compared to ATM, which has no planning, on the selected benchmark?

Thanks for your question. Firstly we want to emphasize the major contribution of this paper is the high-level generative model-based planning framework, and thanks for your recognition of this point.

Secondly, we think the original low-level policy for FLIP suffers from two problems: 1) it is a simple behavior cloning algorithm, which may not be able to make good use of the temporal information of the future plans from our high-level model, because it only regresses one step action; 2) it only use one layer of MLP to extract the visual embedding from image patches, which may be future improved with pretraiend vision encoders. To better show the advantage of our FLIP model for low-level policy learning, we here train a diffusion policy [1] for the low-level policy with the action chunking mechanism [2], which can predict the future action sequence rather than regress single- step actions. Meanwhile, we also use pretrained ResNet to extract the full image embedding for conditioning. With these modifications, we compare our policies to the diffusion policy version of ATM (ATM-DP) and OpenVLA (asked by Reviewer R3ys).

From Figure 12 in Appendix B.2, we can see that with the new model architecture, our policy can achieve better results than previous results, and the success rates are better than ATM-DP. With this new architecture, Ours-FV is no longer the worst one among the three architectures, which shows that with an effective vision backbone, the model can have a better multi-modal capacity for different condition information. We can also see that using dense image flows can lead to a smaller variance in LIBERO-LONG than using videos.

Although the improvement of success rates of our policy compared to ATM-DP is not large (about 6%), we think this is not because our method is not good enough, but because this result has almost reached the upper limit of LIBERO-LONG under our setting (10 demonstrations with actions for each task). The SOTA results of this LIBERO-LONG task suite is [3], which achieves 53.7 ± 1.3% and the mean success rate is only about 4% higher than ours. We think there are two main reasons for limiting the further improvement of the success rates on LIBERO-LONG: 1) we are using a resolution of 128×128, which may not be large enough to represent the details in the scene. In comparison, in [3], they use a resolution of 256×256. 2) We are using the official demonstrations provided by the LIBERO paper, which may not be as good as the re-collected demonstrations in [3].

Finally, we show the results of our method on real robot experiments compared to Diffusion Policy and ATM to better demonstrate the superiority of our policy over baselines. Here we only test Ours- F because video generation is time-consuming during online replanning. From Appendix B.4, we can see that our policy is way better than the baselines, showing that in very difficult long-horizon tasks, our model can perform better with the help of model-based planning.

[1] Chi, Cheng, et al. ”Diffusion policy: Visuomotor policy learning via action diffusion.” The International Journal of Robotics Research (2023): 02783649241273668.

[2] Zhao, Tony Z., et al. ”Learning fine-grained bimanual manipulation with low-cost hardware.” arXiv preprint arXiv:2304.13705 (2023).

[3] Kim, Moo Jin, et al. ”OpenVLA: An Open-Source Vision-Language-Action Model.” arXiv preprint arXiv:2406.09246 (2024).

of particular concern is the Ours-FV results, which are not well-explained.

Here, we explain the (original) Ours-FV results in detail. In the original low-level policy learning experiments, Ours-FV is worse than Ours-F and Ours-V. We think this is because the visual features inherently contain richer and more diverse information. However, our previous architecture did not incorporate a dedicated feature extraction process for the visual modality. Instead, both visual and flow information were passed through a single, simplistic MLP layer independently. This lack of specialized processing for visual features resulted in challenges when integrating them effectively with flow features. This is also pointed out by OpenVLA and Pi0.

To address this problem, in our new low-level experiments, we employ a pre-trained ResNet to extract the visual features and then employ an additional transformer dedicated specifically to the feature fusion from visual conditions, flow conditions, and language conditions, thereby decoupling the the learning process of the policy transformer. From Figure 12 in Appendix B.2, we can see that the new results show that Ours-FV is better than the previous result, which shows the effectiveness of the new multi-modal architecture.

We thank reviewer H4xH again for reviewing our paper and providing helpful feedback! We hope the additional experiments and clarifications have addressed the reviewer’s concerns. If the reviewer has any further concerns or questions, please let us know before the rebuttal period ends, and we will be happy to address them. We deeply appreciate your thorough review and constructive comments, which have been invaluable in refining the paper's presentation.

We have carefully revised the manuscript to address the concerns that led to your lowered score, and we sincerely hope the changes will allow for a more favorable consideration. Thanks!