RoBERT: Low-Cost Bi-Directional Sequence Model for Flexible Robot Behavior Control

The presentation and writing of this paper makes it very difficult to assess the quality of this work. There are incoherent sentences (eg: By encouraging an agent to perform task in one direction, we are discouraging it from performing task in opposite direction. For example, by tweaking reward function to train a robot to teach dance smoothly, we’re restricting its ability to serve as a counter-example for showing how un-smooth dances look like., An intuitive idea here is as masks increasing, determinism of inverse dynamics would decrease and our setting would become a multi-modality Imitation Learning, as studied in Shafiullah et al. (2022).), incorrect use of terminology (eg: zero-shortly (change to in a zero short manner), ill-collected dataset (change to noisy dataset)) and grammatical errors (eg: unsupervise RL) making it challenging to parse the paper. The introduction of the paper is written in a manner that does not clearly explain the problem, and abruptly switches to listing the contributions. The methods section also lacks clarity as it is difficult to understand sections 3.3 and 3.4. Figure 1 also does not explain the method clearly as the inputs and the outputs of each phase are not shown clearly. Therefore, I would suggest that the authors rewrite certain sections of the paper to make it more understandable.

• The contribution of this work is unclear and the paper lacks novelty, as the proposed method simply combines unsupervised RL dataset with masked modeling without introducing any unique techniques.
• The motivation of adopting a bi-directional sequence modeling approach, akin to BERT-like methods, remains unclear in the context of the problem they are addressing.
• The paper mentions a low-cost aspect without providing details on how this is achieved or its relevance to the main contribution.
• The experiment section of the paper seems to be weak. It is not clear what the experiments aim to show, and there is an absence of any baseline comparisons. Presenting results solely for the proposed method without any comparative baselines seems weird given there are similar line of work such as [1, 2]. It would be better to include experimental results comparing the proposed method to other approaches to show the superiority of the suggested architecture or learning scheme.
• The argument that the proposed method is superior to the state-of-the-art in general robot control is weakened by the fact that their experimental setup focuses on only simple tasks rather than complex tasks (see RT-2[3]). Given the experimental results the authors show, the claim of requiring the least amount of human effort for general robot control/decision-making sounds too strong even though it uses unsupervised dataset.

[1] Philipp Wu et al., Masked Trajectory Models for Prediction, Representation, and Control. ICML, 2023.
[2] Fangchen Liu et al., Masked Autoencoding for Scalable and Generalizable Decision Making. NeurIPS, 2022.
[3] Anthony Brohan et al., RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control. arXiv, 2023.

The task seems easy to solve. Merely making the robot reach target states does not constitute general control, as this can be achieved with basic control techniques and without jittering observed in the supplementary videos. Moreover, for manipulation tasks using robot arms or dexterous hands, simply reaching target states is insufficient, as interactions with dynamic objects is more important and challenging. If the authors insist on exploring this direction, I recommend considering obstacle avoidance planning for robotic arms or social navigation; and exploring how to encode obstacles into the state to achieve target-reaching while avoiding obstacles.