RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation

Q3: Regarding the computation of success rate for the 'wash cup' task with 'seen cup1': the success rate (SR) for 'get water' is 50, for 'pour water' is 87.5, and for 'place back cup' is also 87.5, yet the overall SR is listed as 50. Since the 'get water' subtask has an SR of 50, and the following subtasks have SRs below 100, how is the total SR calculated as 50?

We thank the reviewer for bringing up this question, which helped us improve our presentation. The success rate (SR) of a subtask here refers to the percentage: $P(\text{completing this subtask})$, not the percentage: $P(\text{completing this subtask}\mid \text{completing all subtasks before this subtask})$. There are also scenarios that a subtask is completed without completing all previous subtasks. In the particular example you mentioned, the 87.5% SR of "pour out water" means that in 87.5% of cases, the agent completed the action of "pour out water", 50% of which there was enough water in the cup ("get water" is successful), and 37.5% of which there were only a few drops of water in the cup ("get water" is failed). The 50% total SR means that in 50% of cases, the agent completed all the subtasks.

Q2: Does the RDT-1B fine-tuning use the entire self-collected dataset, and is it not fine-tuned separately for each task in the evaluation?

By default, all baseline models are fine-tuned using the entire self-collected dataset and tested by a multi-task setting (one model for all tasks). However, due to the limited model capacity of Octo and OpenVLA, they experience issues with insufficient convergence when fine-tuned this way (more details are elaborated in Q1 and App. H). As an alternative, we fine-tuned these models separately for each task to enhance their performance and tested each task with the model fine-tuned on the data of this task (one model for one task).

To facilitate a fair comparison, we also fine-tune RDT on one of the tasks. From the results above, we can find that RDT performs even better than the original, which means that the multi-task setting is much more challenging than the single-task setting. For the former, the model needs to have sufficient capacity to simultaneously fit the distribution of multiple tasks. So, it is fair to compare the performance of the multi-task RDT with that of the single-task versions of baselines.

Pour Water: unseen room 1 (Unseen Scene)

We sincerely thank you for your insightful review of our submission. Your feedback is invaluable, and we appreciate the time and effort you dedicated to evaluating our work.

In response, we have conducted additional experiments to strengthen our evaluation and address your concerns. Below, we provide the results and clarifications to resolve any confusion.

Q1: Compared to the robot datasets used for pre-training the baseline, such as OpenVLA, this paper appears to use a more diverse set of datasets, including additional bimanual manipulation datasets like ALOHA and Mobile ALOHA, contributing nearly 10% of the total datasets.

Thank you for pointing it out. We have now extended the pre-training of the officially released models (Octo and OpenVLA) using the largest open-source bi-manual manipulation datasets, including ALOHA (static and mobile) and our fine-tuning dataset, totaling over 7,000 episodes. However, even after this additional pre-training, the models still failed to converge to a deployable loss level, resulting in random or meaningless patterns of movement in real-world scenarios (such phenomenon is elaborated in App. H). The detailed results are depicted in Table a.

Table a: The performance comparison of different methods further pre-trained on bimanual dataset (ALOHA Static and ALOHA Mobile and our finetune dataset). They are further finetuned with whole fine-tuning dataset.

Wash Cup: seen cup 1 | unseen cup 1 | unseen cup 2 (Unseen Object)

Pour Water-L-1/3 | Pour Water-R-2/3 (Instruction Following)

Pour Water: unseen room 1 | unseen room 2 | unseen room 3 (Unseen Scene)

Fold Shorts (1-Shot)

Handover (5-Shot)

In summary, despite extensive efforts in re-implementing and fine-tuning these models for optimal performance, the issue of insufficient convergence persists. For OpenVLA, there might be a broader scientific challenge in scaling vision-language-action (VLA) models effectively for bi-manual manipulation, which could be a possible reason why they do not consider bi-manual manipulation [*1]. For Octo, other researchers have also reported the phenomenon of non-convergence and random behaviors of the robot when adapting Octo to bi-manual manipulation [*2]. We can conclude that bimanual manipulation is challenging for previous foundation models, which is addressed by designing a unified action space and a powerful architecture in our RDT. While beyond the scope of this work, we believe adapting VLA to bimanual manipulation will be a promising direction.

[*1] https://github.com/openvla/openvla/issues/48

[*2] https://github.com/octo-models/octo/issues/29

Dear Reviewer dG58,

We would like to express our sincere gratitude for your insightful feedback. We hope our response and additional experiments can meet your expectations and await any further comments you may have.

Best, Authors

Thank you for your thoughtful question. Please find our detailed response below:

1. Modifications for Bimanual Datasets

• Input Modifications:

  The original models were designed for robots using a third-person view camera, which provides a comprehensive view of the manipulation scene. In contrast, ALOHA employs a single first-person camera and two wrist cameras. Using only one camera can result in unexpected information loss and partial observability. Therefore, we modified the input to include three images: one from the front-facing camera and two from the wrist cameras, instead of relying solely on a third-person view

• Output Modifications:

  The original models predict the 7DOF end-effector pose for control, while ALOHA utilizes joint positions for control. Consequently, we adapted the models to predict target joint positions instead of end-effector poses.

2. Training and Fine-Tuning Details for OpenVLA

• Pretraining: We continued pretraining OpenVLA on an integrated bimanual dataset using 2x8 H100 GPUs with a global batch size of 64 for 300k steps. This process took approximately 3 days.
• Fine-tuning: For fine-tuning, we used the same resources, training the model for 50k steps, which took approximately 13 hours. The model achieved an average classification accuracy of 30%.

3. Training and Fine-Tuning Details for Octo

• Pretraining: Octo was pretrained on the bimanual dataset using a single 8x H100 setup with a batch size of 32 for 300k steps.
• Fine-tuning: For fine-tuning, we used the same setup (1x8 H100 with a batch size of 32) for 700k steps. The model achieved a validation set loss of approximately 5e-2.

We hope this clarifies our approach. Please feel free to reach out if you have any further questions or require additional details!

Q3: Showing how each component contributes to performance could clarify their impact on handling visual and language-conditioned tasks effectively.

Yes, we completely agree with you that more ablation studies will help to deepen our understanding of the model. Since pre-training requires a lot of GPUs and time and real-robot experiments also require effort (dozens of trials per task), with limited resources, we decided to perform ablation studies on our key architectural innovations (e.g., improved normalization, MLP decoder, alternating condition injection) to highlight the necessity of our contribution. In future work, we will conduct ablation experiments on more design choices once additional resources are permitted.

Q4: There's a typo at L76, it should be "data" instead of "date".

Thanks for your careful reminder. We have corrected it in the revised manuscript.

Q5: Could the authors elaborate on why diffusion models were specifically chosen over other generative methods like VAEs or GANs for this task?

This is a very good question to gain deeper insights into our probabilistic modeling choice. VAEs assume that the data are related to a Gaussian latent variable and sample the data by Gaussian distribution with learnable mappings. Such assumptions can reduce sampling quality [*1], which is critical for dexterous tasks demanding high action precision.

GANs involve adversarial training between a generator and a discriminator. This setup needs carefully tuned loss functions to avoid problems like mode collapse, where the generator produces limited diversity. Adversarial training is generally unstable, which requires specialized stability techniques like gradient penalty and spectral normalization [*2].

In contrast, diffusion models excel in both sampling quality and training stability. Usually, its drawback is the sampling speed, which is minor in our settings since actions have a much lower dimension than images and videos. Therefore, diffusion models become an ideal choice for robot learning.

[*1] Dai, B., & Wipf, D. (2019). Diagnosing and enhancing VAE models. arXiv preprint arXiv:1903.05789.

[*2] https://www.sapien.io/blog/gans-vs-diffusion-models-a-comparative-analysis

Q6: The Physically Interpretable Unified Action Space is innovative, but how does it handle robots with vastly different kinematics or action constraints?

This is an insightful question. To handle robots with vastly different kinematics or action constraints, we need to feed the structure parameters of the robot arm into the model, although these parameters are not part of the dataset and are challenging to encode. Nevertheless, our action space ensures that the model learns shared physics and common end-effector patterns even without explicit structural parameters. This is achieved by aligning various physical quantities and maximizing the preservation of shared physical information across different robots.

Q2: Expanding the evaluation to more varied and complex real-world tasks and more hardware could further validate RDT's robustness.

Thank you for your thoughtful feedback and insightful suggestions. In response to your recommendation, we have expanded our evaluation to include a broader range of real-world tasks that are both diverse and more challenging. Despite constraints in time and resources, we designed four additional experiments to assess three key dimensions: Unseen Scene, Unseen Object, and Instruction Following.

Task Definition

Below, we provide an overview of each task.

Task visualizations can be found at https://anonymous.4open.science/r/rdt-demo-543D/addtional_experiment_visualization.png

Implementation Details

The implementation details for each method are described as follows:

1. Training Details for ACT

ACT was fully trained using our finetune dataset. Since ACT does not support language instructions as input, it was excluded from experiments involving Instruction Following tasks.

2. Training and Fine-Tuning Details for OpenVLA

• Pretraining: We continued pretraining OpenVLA on an integrated bimanual dataset (a combination of Mobile ALOHA dataset and our finetune dataset).
• Fine-tuning: We fine-tuned OpenVLA on the entire fine-tuning dataset. The model achieved an average classification accuracy of 30%.

3. Training and Fine-Tuning Details for Octo

• Pretraining: Octo was pre-trained on the same bimanual dataset as OpenVLA.
• Fine-tuning: We fine-tuned Octo on the entire fine-tuning dataset. The model achieved a validation set loss of approximately 5e-2.

4. Implementation of RDT

For RDT, we utilized our previously fine-tuned checkpoints in all experiments, consistent with prior evaluations.

Experiment Results

Each task was evaluated over 8 trials. The quantitative results are shown below:

Results Analysis

We highlight the challenges of each task and evaluate method performance. Stack Tomato Can (STC) involves adapting to new environments and precise placement of the cylindrical, slippery tomato can. Stack Tomato Can Mahjong (STCM) requires generalization to unseen foam mahjong pieces. Put Orange Lunchbox (POL) and Put Apple Lunchbox (PAL) demand instruction-following to distinguish and pick the correct object.

In the seen scene, ACT can grasp and occasionally place the tomato can but fails in the unseen scene, showing erratic movements. Octo reliably reaches targets but fails to grasp due to ineffective gripper closure. OpenVLA can grasp in the seen scene but struggles with placement and frequently misidentifies objects, while also failing to generalize to unseen scenes. In contrast, RDT demonstrates robust generalization, successfully handling unseen objects and scenes while following instructions accurately.

Including Additional Hardware

We are actively working on it; however, due to shipping delays, there is a possibility that it may not arrive in time for further evaluations. We will incorporate these results into future updates as resources become available.

We sincerely thank you for your detailed and constructive feedback, which has greatly helped us improve our work. We appreciate your recognition of the novelty and practical value of our approach, as well as the thoroughness of our experiments.

In response to your comments, we have carefully addressed each point you mentioned. We hope our revisions and additional experiments meet your expectations. Below, we provide detailed responses to each of your points.

Q1: This could include examples where action standardization might lead to the loss of unique features across robots.

Thanks for your advice. The most direct example is the situation we encounter in our pre-training dataset. Our collection contains datasets of multiple robots, each of which is small in size and often contains limited types of tasks. If we standardize each dataset with its mean and variance, there may be a relatively large variance, which is not conducive to model learning. Moreover, independent standardization will cause the action space of each robot to be different, thereby destroying the alignment across robots, and making it difficult for the model to learn shared physical laws. In general, to align different action spaces, we need pairwise data: the trajectories of different robots performing the same task in the same scene. Such data is extremely difficult to obtain. Therefore, we believe that separate standardization is harmful.

Thanks for the authors' clarification. In general, I think it's a good paper.

Q4: Can you provide more insight into how the model design specifically addresses high-frequency change?

We agree that more explanation is helpful for readers to understand our design choice. According to the neural tangent kernel (NTK) theory [*4], we know that neural networks learn different frequencies at different speeds: they learn low-frequency components faster and fail to capture high-frequency behaviors. This causes the neural network to learn smooth actions and fail to represent fine-grained actions (such as aiming, twisting, and plugging). These fine-grained operations are crucial for dexterous tasks. To solve this problem, we introduce an MLP encoder with Fourier Features of different frequencies [*4] so that the model can learn high-frequency components more effectively.

[*4] Tancik, M., Srinivasan, P., Mildenhall, B., Fridovich-Keil, S., Raghavan, N., Singhal, U., ... & Ng, R. (2020). Fourier features let networks learn high-frequency functions in low-dimensional domains. Advances in neural information processing systems, 33, 7537-7547.

Q5: What are the performance gains from using all data through this unified action space compared to previous methods that only use robots with similar action spaces or retain subsets of inputs with a common structure?

We carefully point out that, among our baselines, OpenVLA is a representative of this type of method. Its action space only has EEF: for datasets that contain both EEF and joints, it removes the joints part; for datasets that only have joints, it removes the entire dataset. As can be seen from Fig. 1 and Table 3, our method has an average gain of more than 60% over it. This is because we can leverage more data with the help of the unified action space, achieving better generalizability.

Q6: How does the proposed padding strategy improve performance compared to zero padding?

We'd like to sincerely clarify that the zero padding technique is a part of our strategy. For a specific robot, we fill the physical quantities in its original action space into the corresponding positions in the unified action space (see the left side of Fig. 3), and pad the remaining unavailable positions with 0. In addition, we also feed in a 0-1 indicator vector to indicate whether each position in the unified action space is valid or not. This is to distinguish between the padded 0 and the physical 0 value that appears in the original action. For more details, please refer to the second paragraph of Appendix C and refer to Appendix F.

Q7: Showing more data points (models with different sizes, using different percentages of the dataset) will present more insights to the reader.

We appreciate the reviewer’s feedback and understand the desire for a more granular analysis across different model sizes and dataset percentages. However, due to the substantial computational cost and time constraints, conducting such experiments for additional data points was not feasible within the scope of this work. Pre-training large models, such as the 1B parameter model, requires significant GPU resources and time, while real-world robot experiments demand extensive testing (often requiring dozens of trials per task). Given these limitations, we believe that the binary experiments presented—comparing pre-trained vs. non-pre-trained models and large vs. small models—sufficiently highlight our key contributions. Furthermore, we acknowledge the importance of scaling laws and plan to explore this in future work, once additional funding/resources become available. We hope the reviewer can understand that while more data points would indeed provide valuable insights, the current analysis remains a strong demonstration of the major findings.

Q8: How do these normalization layers integrate into the transformer architecture, and how do they address instability? Have you tested different configurations of these layers?

Thank you for your insightful advice. For RMSNorm, we replace each LayerNorm with RMSNorm in the original Transformer architecture. For QKNorm, we normalize query and key vectors before calculating attention in each attention layer. We refer to Appendix B for more details. In our revised manuscript, we compare the loss curves with and without our improved normalization technique in Figure 4(a). We can see that our normalization technique has improved the training stability and avoided the loss explosion. Besides, as discussed in Q7, we believe our current ablation study in Figure 4(a) can highlight the core contribution of our improved normalization technique given limited resources. In future work, once additional resources are ready, we would like to test different configurations of these layers to get more insights.

Q3: What are the MLP block performance gains on other tasks?

To comprehensively evaluate the advantage of employing an MLP decoder, we conduct three additional ablation experiments in this response. To ensure the fairness and reduce the resources needed by pre-training, all the RDT variants are trained directly on the fine-tuning dataset without large-scale pre-training.

Here are results of the original experiment in Fig. 4(b).

Robot Dog (Dexterity):

Here are results of three additional experiments.

Pour Water-L-1/3 (Instruction Following):

Pour Water: unseen room 2 (Unseen Scene):

Fold Shorts (1-Shot Learning):

Through the above results, we can conclude that replacing the linear decoder with an MLP decoder is beneficial, which supports our analysis in Q1.

We gratefully appreciate the time and effort you have dedicated to reviewing our manuscript. Your insightful feedback has greatly improved our work and inspired a deeper exploration of our model architecture.

In this revision, we have thoroughly addressed each of your comments and questions. Below, we provide detailed responses to your points to clarify your concerns and further strengthen our work.

Q1: Given that the diffusion transformer already contains multiple nonlinear layers, why does adding an additional layer improve performance? Have you conducted ablation studies to support this choice?

First, we would like to humbly emphasize that the MLPs in Transformer layers are different from the MLP as a decoder (in the final layer). The former is a function on the latent space, while the latter is a mapping from the high-dimensional latent space back to the low-dimensional physical space. Thus, politely, we do not suggest considering the decoder as "an additional layer".

Next, let's explain why nonlinear decoders are beneficial. If the decoder is linear, then we can easily prove that the nonlinearity of the physical space is consistent with the latent space. If we use a nonlinear decoder (such as MLP), then it is possible to greatly weaken the nonlinearity when coming to the latent space, thereby allowing the model to better fit nonlinear actions in the latent space. The idea of using nonlinear mappings between low-dimensional nonlinear space and high-dimensional space is very common. This often effectively decreases nonlinearity, and even only linear layers are needed to fit in high-dimensional latent space (the Koopman operators are classic examples [*1-*3]).

Finally, we respectfully note that we did conduct ablation studies to support our choice. Please refer to Fig. 4(b) in our manuscript, where the experiments show that when the decoder is a linear layer, the model will have a significant performance drop on dexterity tasks.

[*1] Mauroy, A., Susuki, Y., & Mezic, I. (2020). Koopman operator in systems and control. Berlin, Germany: Springer International Publishing.

[*2] Korda, M., & Mezić, I. (2018). Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive control. Automatica, 93, 149-160.

[*3] Lusch, B., Kutz, J. N., & Brunton, S. L. (2018). Deep learning for universal linear embeddings of nonlinear dynamics. Nature communications, 9(1), 4950.

Q2: In the original UNet diffusion policy, the last layer is a Conv1dBlock. Have you compared the performance of your MLP block with this alternative?

We modestly think that Conv1dBlock is usually used with CNN-based architectures such as UNet, but not with Transformer. This is because the self-attention layer in the Transformer already includes convolution in terms of expressiveness. You can see that in the original Transformer diffusion policy, the last layer, or equivalently, the decoder is a linear layer instead of Conv1dBlock. Since our backbone is the Transformer, we mainly compare our MLP decoder with the mainstream choice -- the linear decoder.

Dear Reviewer fucs,

We are grateful for the thoughtful and constructive feedback you have provided. We have carefully considered each of your comments and have made the necessary revisions to our manuscript. We hope you will find the response satisfactory and are eager to receive your further feedback.

Best, Authors

Q3: Some of the sentences are too long.

Thanks for pointing it out. We have made some long sentences shorter, thus improving the readability. See our revised manuscript for modifications (highlighted in blue).

Q4: Mistakes in citing papers.

We thank the reviewer so much for pointing this out. We have corrected the wrong citations, as shown in lines 83 and 319 of the revised manuscript (highlighted in blue).

Q5: Why does the model use frozen vision encoders?

You raise a very good question, which we may have forgotten to explain in the manuscript. We think that it is better to freeze the weights of the vision encoder. The frozen vision encoder can retain the robust, generalizable features acquired through extensive Internet-scale pre-training. In contrast, fine-tuning the encoder on much smaller and domain-specific robotic datasets may cause overfitting, thereby learning less generalizable features. Similar observations have also been documented in video generation models (See e.g., [*1]).

[*1] Polyak, A., Zohar, A., Brown, A., Tjandra, A., Sinha, A., Lee, A., ... & Du, Y. (2024). Movie gen: A cast of media foundation models. arXiv preprint arXiv:2410.13720.

Q6: How does the strategy of taking control frequency as conditioning work in practice?

Unlike video data, the frame rate (or equivalently, control frequency) of robotic data is widely distributed. In our case, the control frequency is almost evenly distributed from 1Hz to 66Hz. It is unreasonable to simply align all datasets to 1Hz and it is unacceptable for some robots that need to react quickly. Therefore, we adopt the conditioning in RDT and feed the control frequency into the model as an important context, as illustrated in Fig. 3. In this way, RDT knows whether the action sequence to generate is dense in time (i.e., high control frequency) or sparse in time (i.e., low control frequency). Without such context, the entire action prediction problem will be ill-defined.

Q2: It seems that the baselines are not trained on the complete fine-tuning dataset.

We want to humbly point out that we have carefully fine-tuned OpenVLA and Octo on the complete multi-task fine-tuning dataset, adhering closely to the settings described in their respective original papers. Despite these efforts, we observed that both models failed to converge to a deployable loss level, resulting in random or meaningless patterns of movement in real-world scenarios. Consequently, we opted to fine-tune the models on specific tasks individually. Further details on our implementation can be found in App. H.

Reviewer DG58 proposed a valuable hypothesis that these models, being primarily pre-trained on dual-arm robot datasets, may face a significant domain gap that hinders successful adaptation to bi-manual manipulation. To investigate this, we extended the pre-training of the officially released models using open-source bi-manual datasets, including ALOHA (static and mobile) and our fine-tuning dataset, totaling over 7K episodes. However, even after this additional pre-training, the models still failed to converge during fine-tuning across the entire dataset, as depicted in Table a.

Table a: The performance comparison of different methods further pre-trained on bimanual dataset (ALOHA Static and ALOHA Mobile and our finetune dataset). They are further finetuned with whole fine-tuning dataset.

Wash Cup: seen cup 1 | unseen cup 1 | unseen cup 2 (Unseen Object)

Pour Water-L-1/3 | Pour Water-R-2/3 (Instruction Following)

Pour Water: unseen room 1 | unseen room 2 | unseen room 3 (Unseen Scene)

Fold Shorts (1-Shot)

Handover (5-Shot)

In summary, despite extensive efforts in re-implementing and fine-tuning these models for optimal performance, the issue of insufficient convergence persists. For OpenVLA, there might be a broader scientific challenge in scaling vision-language-action (VLA) models effectively for bi-manual manipulation, which could be a possible reason why they do not consider bi-manual manipulation [*1]. For Octo, other researchers have also reported the phenomenon of non-convergence and random behaviors of the robot when adapting Octo to bi-manual manipulation [*2]. We can conclude that bimanual manipulation is challenging for previous foundation models, which is addressed by designing a unified action space and a powerful architecture in our RDT. While beyond the scope of this work, we believe adapting VLA to bimanual manipulation will be a promising direction.

[*1] https://github.com/openvla/openvla/issues/48

[*2] https://github.com/octo-models/octo/issues/29

We sincerely thank you for your time and effort in reviewing our manuscript. Your insightful feedback has been invaluable in improving our work, and we appreciate your suggestion for additional experiments to gain more insights.

In this revised submission, we have carefully addressed all your comments and questions, including the requested experiments. Below, we provide detailed responses to each of your points to clarify and further strengthen our work.

Q1: Evaluations on more tasks and existing benchmark tasks will complete the results.

Thank you for your valuable feedback and insightful advice. We have expanded our evaluation to cover a broader range of real-world tasks, which are even more challenging. Given the constraints of time and resources, we designed four additional experiments from three dimensions: Unseen Scene, Unseen Object, and Instruction Following.

Task Definition

Below, we provide an overview of each task.

Task visualizations can be found at https://anonymous.4open.science/r/rdt-demo-543D/addtional_experiment_visualization.png

Implementation Details

The implementation details for each method are described as follows:

1. Training Details for ACT

ACT was fully trained using our finetune dataset. Since ACT does not support language instructions as input, it was excluded from experiments involving Instruction Following tasks.

2. Training and Fine-Tuning Details for OpenVLA

• Pretraining: We continued pretraining OpenVLA on an integrated bimanual dataset (a combination of Mobile ALOHA dataset and our finetune dataset).
• Fine-tuning: We fine-tuned OpenVLA on the entire fine-tuning dataset. The model achieved an average classification accuracy of 30%.

3. Training and Fine-Tuning Details for Octo

• Pretraining: Octo was pre-trained on the same bimanual dataset as OpenVLA.
• Fine-tuning: We fine-tuned Octo on the entire fine-tuning dataset. The model achieved a validation set loss of approximately 5e-2.

4. Implementation of RDT

For RDT, we utilized our previously fine-tuned checkpoints in all experiments, consistent with prior evaluations.

Experiment Results

Each task was evaluated over 8 trials. The quantitative results are shown below:

Results Analysis

We highlight the challenges of each task and evaluate method performance. Stack Tomato Can (STC) involves adapting to new environments and precise placement of the cylindrical, slippery tomato can. Stack Tomato Can Mahjong (STCM) requires generalization to unseen foam mahjong pieces. Put Orange Lunchbox (POL) and Put Apple Lunchbox (PAL) demand instruction-following to distinguish and pick the correct object.

In the seen scene, ACT can grasp and occasionally place the tomato can but fails in the unseen scene, showing erratic movements. Octo reliably reaches targets but fails to grasp due to ineffective gripper closure. OpenVLA can grasp in the seen scene but struggles with placement and frequently misidentifies objects, while also failing to generalize to unseen scenes.

In contrast, RDT demonstrates robust generalization, successfully handling unseen objects and scenes while following instructions accurately.

I would thank the authors for the clarifications and additional experiments despite a tight timeframe. I think it's overall a good paper and I'm happy to accept it. I have updated my score.