AnyBimanual: Transferring Single-arm Policy for General Bimanual Manipulation

1. As emphasized in the paper, the main contribution is converting a single-arm policy into a bimanual policy. However, in the experiments, PerAct2 and PerAct-Lf already appear to be bimanual policies (e.g., the policy outputs left and right actions), which raises questions about how the proposed method is combined with such baselines. From my perspective, the experiments might be more insightful if baselines like PerAct were used instead. I expected to see that using a true single-arm policy with the proposed method could achieve comparable results with bimanual manipulation policies.

2. According to the original PerAct2 paper [1], one point of confusion is that the baseline success rates in this study seem much lower than those reported in the original paper. Am I missing some evaluation or condition differences?

3. One of the challenges in dual-arm versus single-arm manipulation is coordinating the two arms. The current method decomposes the dual-arm policy into two single-arm policies, seemingly without coordination between the arms; it seems that coordination is learned implicitly through imitation of expert demonstrations？ However, the supplementary demos provided seem to show significant asynchrony between the two arms' motions.

[1] Grotz, M., Shridhar, M., Asfour, T., & Fox, D. (2024). Peract2: A perceiver actor framework for bimanual manipulation tasks. arXiv preprint arXiv:2407.00278.

There are several concerns:

1). In the problem formulation, the authors specify that they collect K offline expert trajectories. However, they later mention that expert demonstrations are "lacking" due to high data collection costs, which is confusing—since they still start with K trajectories. Clarifying this point would help improve coherence in the problem setup. Additionally, recent studies (e.g., [1], [2]) have developed low-cost tools for gathering data across various manipulation tasks. It would be helpful to understand the authors' perspective on whether easy data collection tools for single-arm tasks could also benefit bi-manual tasks, given that humans can intuitively control both arms in data collection.

[1]. Chi, Cheng, et al. "Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots." arXiv preprint arXiv:2402.10329 (2024).

[2]. Yu, Kelin, et al. "MimicTouch: Leveraging Multi-modal Human Tactile Demonstrations for Contact-rich Manipulation." 8th Annual Conference on Robot Learning.

2). The authors use the collected K expert bi-manual trajectories to train the skill manager, which dynamically outputs importance weights for the linear combination of learnable skill primitives. However, how is the mapping established between the generated language embedding for one arm and its corresponding low-level control commands? Is this mapping solely managed by pre-trained models? If so, including some explanation of these details—even if they’re based on prior work—would greatly enhance the readability and understanding of this work.

3). In the learning objective for skill primitives, does this objective improve the low-level performance of the single-arm policy? Since the skill manager ultimately outputs a composite language embedding, how does the quality of this embedding impact the effectiveness of the bi-manual policy? There appear to be gaps in explaining how the skill manager’s outputs—used as inputs to the pre-trained low-level single-arm policy—connect to the policy’s low-level performance. To my understanding, learning the skill primitives does not enhance the single-arm policy’s performance directly. Instead, this approach seems focused on dynamically assigning high-level manipulation roles to each arm for a cooperative task rather than enhancing bi-manual manipulation precision. If this is the case, then the actual low-level manipulation performance is primarily driven by the reconstruction loss upon the K demonstrations, which isn’t a contribution unique to this work.

4). Further, the composite language embedding is concatenated with the original bi-manual language embedding. Could this final embedding be somewhat out-of-distribution (OOD) for the pre-trained single-arm policy model? For instance, commands like “push two buttons” would be an unseen task during single-arm policy training, so how would the pre-trained model handle such cases?

5). Similarly, the single-arm policies use augmented visual embeddings, but is the global context from bi-manual manipulation potentially out-of-distribution (OOD) for the frozen single-arm policy during pre-training? Even if Transformer-based single-arm policies can handle a variable number of input tokens, the global context tokens would still be OOD in this scenario. While the use of KL divergence to learn better decomposed subspaces makes sense to me, I’m unclear about the direct concatenation of the global context. If I’ve misunderstood this approach, could the authors provide clarification?

6). The performance of all policies, including the one proposed in this work, is notably low. This raises questions about whether imitation learning (IL) methods are well-suited for such tasks; reinforcement learning (RL) might potentially yield better results, or perhaps more data augmentation could improve raw IL data. I’d like to understand why the policy performs so poorly across most tasks—particularly for simpler tasks like lifting a ball, where success rates are only 32% despite training on 100 demonstrations. Are there additional challenges that are not immediately intuitive? If so, describing these in the paper would help clarify. Otherwise, the low success rate could indicate issues with the quality of the collected demonstrations or possible implementation bugs.

7). In Table 2, the average performance using no proposed methods achieves a success rate of 14.33%, while utilizing all proposed methods only improves this to 21.67%. This increase does not seem significant. Similar to my previous question regarding the poor overall performance, I would appreciate some insights into why the performance remains low despite the implementation of these methods. Understanding the factors contributing to this limited improvement would be beneficial.

8). In Figure 5, I’m puzzled by the "open" operations in the simple task "bring me the item." Specifically, it’s unclear why the left arm grasping the block is treated similarly to the open drawer task (line 513). Additionally, the visualization of the voxel editor is challenging to evaluate. Would it be possible to employ a better visualization tool, or is this approach common in the literature? Clarification on these points would enhance understanding.

9). The real-robot results presented in this paper are insufficient, as the authors only mention them toward the end. Additionally, the reported success rate of 66% for the lift task feels quite limited to fully demonstrate the effectiveness of the method. There is also a lack of evaluation using qualitative or other quantitative metrics to provide a comprehensive assessment of performance. Since this paper proposes a framework aimed at addressing challenging bi-manual tasks, including more extensive real-robot results and diverse metrics would be necessary to strengthen the contributions.

• Lack of baseline: This paper includes only two baselines: one is PerAct, which is primarily designed for single-arm policies, and the other is PerAct-LF. Therefore, there is only one baseline that fully aligns with the setting of this paper. I suggest that the authors consider including more baselines for comparison. While I understand that there are fewer methods in the field that focus on dual-arm operations guided by language, the authors could still compare their work with pure dual-arm algorithms, such as DualAfford[1]. Since language-guided tasks are more challenging and this paper emphasizes cross-task generalization, the performance might not be as high as that of algorithms focused on cross-object generalization. However, these comparisons are essential and necessary to validate the effectiveness of the proposed method.
• Generalization design: This paper makes significant efforts and design choices to preserve generalization ability. However, the final training method adopted undermines this goal.
  • Specifically, the paper uses an end-to-end training approach with a cross-entropy loss calculated between the predicted action and the final action (although there are other loss designs, this is the primary one). This approach makes the method more akin to imitation learning, which is known for its poor generalization capabilities. As a result, it seems to undermine the generalization ability that the pretrained single-arm model originally possessed.
  • Moreover, the paper still uses 20 or 100 demonstrations per task to train a model, which makes the voxel editor and skill manager task-specific and difficult to transfer to other tasks. For task-specific imitation learning, there are already well-established methods in the field, such as DP3.
  • Lack of Generalization Validation: As mentioned above, the experimental content lacks substantial validation of generalization. I recommend that the authors combine all tasks and demonstrations to train a single model and then directly test the model's generalization on new tasks. Such a training approach would hopefully enable the voxel editor and skill manager to learn cross-task knowledge and apply it directly to new tasks.
• The design of pure implicit representation: The two main modules proposed in this paper are entirely implicit, which necessitates end-to-end supervision. For the skill manager, existing large models can explicitly address this task using language prompts. Therefore, we encourage the authors to include a comparison with this explicit approach using language prompts.

• To transfer the pretrained single-arm policies to multi-task bimanual manipulation, much of the contribution in this work focuses on input-related aspects, such as the linear combination of language embeddings and the alignment of visual observations. While I agree that these input designs can leverage the commonsense knowledge of the pretrained single-arm policies, this framework overlooks the cooperation between the two arms. In this design, the policies for the left and right arms are nearly independent of each other. Although this approach may be effective for certain bimanual tasks that assign fixed roles to each arm (e.g., one arm for stabilizing and the other for acting), it may not perform well in tasks requiring deeper collaboration between the arms (e.g., jointly lifting a large ball, where AnyBimanual achieves relatively lower scores). While I understand that such cooperation can be learned from data (i.e., the limited bimanual demonstrations), I believe it would be beneficial to explicitly ensure dual-arm cooperation within the framework design. In summary, I appreciate the efforts to transfer the single-arm policy to a dual-arm system, but there are still limitations regarding the cooperation of the two arms.
• In the last paragraph in Section 3.3, I agree with 'bimanual manipulation tasks often involves asynchronous collaboration that requires the left and right arm to attend to different areas of the whole workspace to act as different roles.' However, some tasks require both arms to work in the same small area, such as handing over a small item (as illustrated in Figure 5). In these cases, it is crucial to avoid collisions between the arms during the handover process; therefore, it would not be reasonable for the left arm to ignore the right arm.
• The authors are encouraged to conduct more real-world experiments on different tasks.