Chain-of-Action: Trajectory Autoregressive Modeling for Robotic Manipulation

We appreciate your insightful questions and suggestions. Below, we provide detailed responses to your concerns:

w1[Why using 60 tasks rather than 18 tasks]

Thank you for the valuable suggestion. We would like to kindly clarify a possible misunderstanding regarding our evaluation protocol. Our full evaluation covers 60 diverse RLBench tasks rather than just the commonly used 18 standard tasks. While the 18 tasks are often used as a benchmark, they are neither the only nor a required standard for evaluation. In fact, several recent works—including Point Cloud Matters (NeurIPS 2024), Conditional Flow Matching (CoRL 2024), and Generative Image as Action Models (CoRL 2024)—also adopt customized subsets of RLBench tasks tailored to their specific settings.

The rationale behind this is that the standard 18 tasks are frequently employed to evaluate hierarchical policies that rely on high-precision 3D inputs and motion planners. Many of these tasks are quite challenging for RGB-only visuomotor policies, resulting in uniformly low success rates and limited discriminative power. For this reason, we excluded some of these tasks, although 7 out of the 18 standard tasks are still included in our larger 60-task set.

Moreover, to directly address your concern, we now provide results on 18 standard tasks. The results highlight two key points:

• A large performance gap exists between RGB-based imitation learning methods (CNN/ViT-BC, CoA, ACT) and 3D keypoint-based hierarchical models (RVT), which validates the difficulty of these tasks under 2D perception.
• CoA still achieves the best performance among RGB-based imitation learning methods, demonstrating its advantage within this category.
• The results for CNN-BC, ViT-BC, and RVT-2 are directly reported from the paper "RVT-2: Learning Precise Manipulation from Few Demonstrations", while the CoA and ACT results are reproduced by us.

w2 [Compare with OpenVLA]

While it would be beneficial to include comparisons to OpenVLA if time and resources permit, it is important to note that OpenVLA targets a different problem setting. It is a large-scale Vision-Language-Action model (~7 billion parameters) pretrained on the extensive Open-XE dataset, requiring substantial computational resources and data.

In contrast, our work focuses on visuomotor policy learning with relatively small-scale models (under 0.5 billion parameters), without reliance on massive pretraining datasets. As such, a direct comparison to OpenVLA would largely reflect differences in model scale and data availability rather than algorithmic advances.

w3[Understanding of experiments]

Thank you for pointing this out. Our goal in Section 5.3 is to evaluate how well CoA generalizes under spatial shifts in object positions. To this end, we evaluate the performance with respect to the object position. This way is also used in previous work like Generative Image as Action Models (CoRL2024), DemoGen: Synthetic Demonstration Generation for Data-Efficient Visuomotor Policy Learning (RSS2025).

It is important to clarify that RLBench primarily varies object positions between training and test scenes, while other attributes such as texture, shape, and color remain fixed. As a result, task success rates serve as a direct indicator of a policy’s spatial generalization capability.

In our setup, we generate 150 object configurations per task. Among these, we select the 100 samples nearest to the center and compute their convex hull. We then randomly sample:

• 50 configurations inside the hull as the training set,
• another 50 configurations within the hull as the interpolation test set, and
• the remaining 50 outside the hull as the extrapolation test set.

Section 5.3 is dedicated to analyzing how well CoA generalizes across these spatial regimes. Specifically:

• Interpolation refers to test object positions within the convex hull of the training distribution.
• Extrapolation refers to positions outside this region, requiring out-of-distribution generalization.

The core insight is that backward reasoning allows the policy to anchor each local action on the final task goal. This global-to-local structure mitigates compounding errors in long-horizon tasks and enhances environment-aware decision-making, especially under spatial shifts. Figures 5 and 6 provide both quantitative and qualitative evidence to support this claim.

q1[Compare with goal+motion planner]

This is a great question. The strategy of first predicting a goal pose and then using a motion planner to reach it is a well-established approach, commonly adopted in hierarchical modeling. We review such methods in our Related Work section, including: “A Multi-Task Transformer for Robotic Manipulation”, “RVT: Robotic View Transformer for 3D Object Manipulation”, “RVT-2: Learning Precise Manipulation from Few Demonstrations”, and “Coarse-to-Fine Q-Attention with Learned Path Ranking.”

CoA differs from these approaches in three key ways:

1. Closed-loop execution: Rather than relying on an open-loop planner, CoA generates action chunks that are continually updated based on real-time observations.
2. Context-aware control: Tasks with path constraints (e.g., sweeping dust or articulated object manipulation) often fail under naive go-to-pose strategies.
3. All hierarchical modeling depends on 3D observation

To further investigate this, we conducted a controlled study where we replaced CoA’s predicted trajectory with direct execution of its keyframe actions (i.e., the goal poses) via a motion planner:

• Open Box: Success rate drops from 72% to 0%, as the task requires executing a specific rotational trajectory to lift the hinged lid.
• Sweep Dust: Success rate drops from 92% to 8%, since merely reaching the end pose fails to guarantee coverage along the intended sweeping path.

We note that some hierarchical models perform well on these tasks using motion planners. However, they often benefit from high-fidelity 3D inputs, intensive computation to accurately infer the goal pose, finer-grained trajectory decomposition, and curved trajectory generation through specific motion planners, all of which differ significantly from our integrated and lightweight design.

q2[Reverse temporal ensemble]

Our reverse ensemble is conceptually similar to ACT’s temporal ensemble, but aligns predictions using the keyframe action as anchor, rather than forward time alignment.

During inference, we generate the trajectory autoregressively from the keyframe backward. Each method—including CoA and baselines—executes one action per inference call, enabling closed-loop control.

q3[computation overhead]

The inference cost is proportional to the predicted trajectory length, which depends on the distance between the gripper and the goal. During training, we determine a maximum sequence length T_max (see Implementation Details and Algorithm 2). Most tasks have a sequence length of 50-100 steps, and for CoA is a single-task imitation learning method, the specific length is up to task.

We sincerely appreciate your thoughtful questions and valuable suggestions. Below, we provide detailed responses to address your concerns:

w1[Limitation]

We thank the reviewer for the valuable suggestion. Indeed, the stopping critiria of CoA might face issues when generalize to other scenarios, e.g., dexterous manipulation. However, as a proof of concept, CoA focuses on quasi-static tasks and aims to boost the spatial generalization capabilities. We demonstrated its performance on 60 RLBench tasks. In our future work, we will extend this paradigm by learning the stopping signal.

w2 [Why latent consistency loss]

Overall, our goal is to establish a robust representation for autoregressive modeling in continuous action spaces. While ACT also uses a Transformer decoder, its action predictions are generated in parallel rather than autoregressively. In contrast to LLMs and VLMs, which operate over a fixed, discrete vocabulary, our scenario involves continuous action prediction, where the output cannot be constrained to a predefined token set. As a result, even small drifts in action embeddings can accumulate and amplify over autoregressive steps. To address this, we introduce a latent consistency loss that explicitly enforces the stability of action embeddings across time steps.

w3[Ablation of keyframe action + forward action chunk]

wait for the result

w4[statistical confidence]

This is a valuable question as it concerns whether the method truly works or merely reflects randomness. Variance is indeed reported in many prior works. We fully understand its importance and would like to clarify why we do not include it in our results. Our primary goal is to evaluate the generalization ability of the method across a diverse set of tasks. Therefore, we extend the evaluation to cover 60 different tasks and report the average performance. Conducting multiple training runs for all these tasks would be computationally expensive. To address variance, we perform ablation studies on 10 representative tasks, which can similarly help reduce the variance of the average success rate. In summary, this is a trade-off between computational cost and statistical rigor.

q1[The rationale of name]

Thank you for pointing this out. We agree that the classical notion of Chain-of-Thought (CoT) in language models typically refers to multi-step intermediate reasoning expressed in discrete text. However, the connotation of CoT has been evolving with technological advances, and recent works support our naming rationale.

For example, recent works such as CoT-VLA and Robotic Control via Embodied Chain-of-Thought Reasoning have extended the CoT concept beyond text, conditioning action prediction on image goals.

Similarly, another related work from NeurIPS 2022, Chain of Thought Imitation with Procedure Cloning, introduces procedure cloning without relying on intermediate discrete text reasoning.

Therefore, we follow this broader community understanding and apply the term “chain-of-action” to denote multi-step, intermediate action-level reasoning in continuous action spaces, which we believe is a reasonable and natural extension of the CoT paradigm.

q2[Correlation between segment length and success rate]

The average segment length of CoA across the 60 RLBench tasks is approximately 72, with most trajectories consisting of 2 execution segments. The Pearson correlation between average segment length and task success rate coefficient is −0.27 (p ≈ 0.037), indicating a statistically significant negative correlation: tasks with longer execution segments tend to have lower success rates, likely due to accumulated prediction errors.

This is matched with similar observation in Figure 5: we analyze task success rates under varying levels of spatial variance, which reflects the spread of object positions across demonstrations. Tasks with higher spatial variance generally require longer execution segments. The success rates of all methods decline as spatial variance increases, which is expected due to the increased difficulty and longer required trajectories.

q3[Failure case]

In summary, we did not observe consistent failure patterns across specific categories of tasks (e.g., high-precision or deformable-object tasks). However, distinct failure modes can often be identified at the individual task level, suggesting that performance limitations are more task-specific than type-specific.

Regarding tasks where CoA shows lower performance compared to ACT, DP, or Octo:

• Ocot with Pretrained model and different action space: CoA underperforms relative to Octo on this task. This is largely because Octo is a fine-tuned, pretrained model and operates in a delta joint action space, which is particularly advantageous for tasks requiring precise rotational control. Therefore, direct comparisons on such tasks remain limited in fairness.
• Incidental fluctuations due to training variability: For individual tasks such as Reach Target, Press Switch, and Sweep Dust, the relatively small performance gaps (<12%) can be attributed to non-systematic factors such as optimization variability. That is, re-training the model with the same hyperparameters may yield slightly different results—sometimes even better—and we report results from batch training without any manual selection over individual tasks. Such variability is commonly observed in imitation learning, especially with behavior cloning, and may lead to minor performance shifts across runs. In some cases, CoA may even outperform other methods on the same task under different training seeds. To mitigate these fluctuations, we report average performance over a broad task set (10 tasks for ablations and 60 for overall evaluation), ensuring that our conclusions reflect stable, aggregated trends. Similar instability has also been discussed in prior work, such as "What Matters in Learning from Offline Human Demonstrations for Robot Manipulation".

We appreciate your insightful questions and suggestions. Below, we provide detailed responses to your concerns:

w1[Motivation]

To clarify our motivation more precisely, we decompose it into two aspects:

1. Goal conditioning helps mitigate compounding errors.
2. Backward prediction can seamlessly incorporate goal-conditioned generation, whereas forward prediction cannot do so as naturally.

Evidence that goal conditioning reduces compounding errors:

Waypoint-Based Imitation Learning for Robotic Manipulation (CoRL 2023) demonstrates that decomposing trajectories into intermediate waypoints effectively mitigates cumulative errors. Similarly, hierarchical modeling approaches discussed in related work support this idea. For example, Hierarchical Diffusion Policy for Kinematics-Aware Multi-Task Robotic Manipulation (CVPR 2024) leverages keyframes and a diffusion-based planner, achieving higher success rates than ACT and Diffusion Policy.

Why goal conditioning with backward prediction is better:

Backward prediction can incorporate goal conditioning following the natural trajectory order, preserving the spatial dependencies between waypoints, which leads to better performance. Of course, forward prediction can also incorporate goal conditioning, as shown by the setting Hybrid in Table 2, where the goal is predicted first and then actions are generated in a forward manner. However, its performance is even worse than pure Forward. This is because backward prediction imposes a natural global-to-local constraint on the predicted actions. In contrast, the hybrid approach directly jumps from the goal prediction to the current action, lacking spatial causal continuity.

w2[Novelty]

Since our technical approach has not been extensively explored before, there may be concerns about its rationale. To address this, we highlight two possibly related works that partially support our idea:

1. There is a concurrent work titled Efficient Robotic Policy Learning via Latent Space Backward Planning (accepted to ICML 2025, posted on arXiv May 11, four days before NIPS DDL), which performs backward planning in the latent space by predicting future latent states from current observations. This is fundamentally different from our approach, as we conduct backward planning directly in the action space, enabling explicit autoregressive generation of action sequences from goal to start.
2. Another related work is Chain of Thought Imitation with Procedure Cloning, which introduces procedure cloning and conditions action prediction only on goal poses. However, it does not incorporate backward trajectory prediction.

In summary, to the best of our knowledge, ours is the first method to perform explicit reverse-order prediction in the action space.

On the naming rationale of “chain-of-action”: We believe the community widely accepts that “chain-of-…” terminology refers to the presence of an intermediate reasoning or procedural process between inputs and outputs, not necessarily limited to symbolic or language-based reasoning like in large language models (LLMs). For instance, recent work like CoT-VLA simply uses an image goal as a condition for action prediction, yet the naming convention is broadly accepted. This supports our use of “chain-of-action” to describe our method’s autoregressive backward action reasoning.

w3[Writing]

We apologize for the unclear description in the current manuscript and would like to update. To clarify, the start-of-sequence (SOS) token serves as a query input to the transformer decoder and then transferred to the keyframe action that initiates the autoregressive prediction sequence.

w4[Related work]

We would like to clarify that our work is specifically related to subgoal-conditioned policy learning in robotic manipulation, rather than the broader class of general goal-conditioned methods. Prior works that use goal poses as subgoals predominantly depend on explicit motion planners and 3D state observations, these methods have been comprehensively reviewed in “Hierarchical Modeling in Robotic Manipulation.”

For approaches utilizing alternative subgoal modalities—such as language, bounding boxes, or visual traces. We have already discussed relevant works in our review of CoT-style methods in robotic manipulation.

apart from these, we notice more works that employ subgoals for manipulation, including:

1. Waypoint-Based Imitation Learning for Robotic Manipulation
2. Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generation
3. Subgoal Diffuser: Coarse-to-Fine Subgoal Generation to Guide Model Predictive Control for Robot Manipulation

w5[Baseline]

We will improve our writing for clearer expression. In fact, we have already presented results for goal-conditioned forward prediction (the Hybrid setting in Table 2), as well as for ACT with keyframe action conditioning. As mentioned in Appendix D, adding keyframe action conditioning improves ACT’s performance from 48.8% to 51.6%. Additionally, we conducted an experiment showing that Diffusion Policy improves from 42.0% to 44.8% with keyframe conditioning. We will update these results accordingly in the manuscript.

q1[Failure case]

In summary, we did not observe consistent failure patterns across specific categories of tasks (e.g., high-precision or deformable-object tasks). However, distinct failure modes can often be identified at the individual task level, suggesting that performance limitations are more task-specific than type-specific.

Regarding tasks where CoA shows lower performance compared to ACT, DP, or Octo:

• Ocot with Pretrained model and different action space: CoA underperforms relative to Octo on this task. This is largely because Octo is a fine-tuned, pretrained model and operates in a delta joint action space, which is particularly advantageous for tasks requiring precise rotational control. Therefore, direct comparisons on such tasks remain limited in fairness.
• Incidental fluctuations due to training variability: For individual tasks such as Reach Target, Press Switch, and Sweep Dust, the relatively small performance gaps (<12%) can be attributed to non-systematic factors such as optimization variability. That is, re-training the model with the same hyperparameters may yield slightly different results—sometimes even better—and we report results from batch training without any manual selection over individual tasks. Such variability is commonly observed in imitation learning, especially with behavior cloning, and may lead to minor performance shifts across runs. In some cases, CoA may even outperform other methods on the same task under different training seeds. To mitigate these fluctuations, we report average performance over a broad task set (10 tasks for ablations and 60 for overall evaluation), ensuring that our conclusions reflect stable, aggregated trends. Similar instability has also been discussed in prior work, such as "What Matters in Learning from Offline Human Demonstrations for Robot Manipulation".

q2/q3[About CVAE and latent token]

We would like to clarify a potential misunderstanding and modify the writing to better reflect our approach. As shown in Figure 1, we do not use the CVAE component from ACT, so there is no latent variable from CAVE. The SOS token actually indicates a start token for the autoregressive transformer decoder.

CoA only share identical number of layers of transformer encoder and decoder with ACT

As detailed in Section 4 (Implementation Details – Network Architecture), our model do share architectural similarity with ACT, as both employ a 4-layer Transformer encoder and a 7-layer Transformer decoder. However, we intentionally removed the CVAE module to maintain a minimalist design. We will further clarify this distinction explicitly in our manuscript to avoid confusion.

Thank you for raising this insightful question, it highlights a key advantage CoA.

Summary

Regarding Q1, as briefly explained in line 289, while the Hybrid (goal + forward) paradigm does have a global-to-local constraint, it lacks spatial continuity. This significantly increases modeling difficulty, leading to weaker fitting ability compared to our reverse. It can be confirmed by our training loss analysis below, Reverse consistently achieves lower loss than Hybrid. Additionally, the Reverse paradigm facilitates dynamic stopping more effectively than Hybrid.

Regarding Q2: The terms continuity, spatial continuity, and spatial causal continuity are used interchangeably in our paper, they all refer to the same concept. A precise definition will be provided below for clarity.

Modeling Comparison

Why Reverse Performs Best:

CoA combines two essential structural properties:

1. Goal Conditioning: The entire trajectory is anchored at the goal ($a_T$), guiding all subsequent predictions and reducing compounding errors. Its benefits have already been well established in our earlier discussion.

2. Spatial Continuity (also refer to as spatial causal continuity in previous clarification): Predictions follow naturally adjacent actions, maintaining smooth and coherent transitions. We will elaborate on this aspect using formal equations in the next section.

Both Forward and Reverse maintain spatial continuity, but only Reverse also leverages goal conditioning. While Hybrid is goal-conditioned, it breaks the spatial continuity.

Spatial Continuity Analysis:

Reverse (refer to Formulation 1, lines 118–119):

$$p(a_{1:T} \mid O)=p(a_T \mid O) \cdot p(a_{T-1} \mid a_T, O) \cdot p(a_{T-2} \mid a_{T-1:T}, O) \dots p(a_1 \mid a_{2:T}, O)$$

Each prediction step is spatially adjacent, enabling local and coherent transitions anchored by the goal.

Hybrid:

$$p(a_{1:T} \mid O)=p(a_T \mid O) \cdot p(a_1 \mid a_T, O) \cdot p(a_2 \mid a_1, a_T, O) \dots p(a_{T-1} \mid a_{1:T-2}, a_T, O)$$

• While it is goal-conditioned, the prediction of $a_1$ after $a_T$ introduces a long-range spatial jump, which breaks the spatial continuity. Intuitively, predicting $p(a_1 \mid a_T, O)$ is clearly harder than predicting $p(a_{T-1} \mid a_T, O)$ as the latter preserves spatial continuity. This influence propagates through the entire sequence, degrading the model’s fitting ability. This can be justified by Training loss analysis below.

• Moreover, the lack of continuity makes dynamic stopping difficult to implement effectively ( In the Hybrid paradigm, the stopping criterion is modified to stop when the predicted pose approaches the goal pose, rather than the start pose). It is because in Hybrid a single incorrect stop can break the entire trajectory generation. In contrast, for Reverse, even if stopping occurs too early, the trajectory remains coherent and anchored to the goal. This can be justified by our visualization of trajectories. (sorry for it can not be shown here)

These two issues significantly increase the difficulty of generating a spatially continuous trajectory.

Training loss analysis:

This table shows the total training loss across 10 representative tasks. We observe that Reverse consistently achieves lower training loss compared to Hybrid on all tasks, with the average loss reduced from 0.00376 (Hybrid) to 0.00160 (Reverse).

We appreciate the opportunity to clarify this fundamental insight and will further emphasize this distinction in the revised manuscript.

We appreciate your insightful questions and suggestions. Below, we provide detailed responses to your concerns:

w1[Limitations in applicability]

We acknowledge that this limitation exists in our implementation in this paper. However, the core idea of CoA can be extended to other domains through domain-specific heuristics and design choices. For example, in quadruped locomotion, foot touchdown events can naturally serve as keyframes. Moreover, the dynamic stopping mechanism could also be learned depending on the specific scenario, rather than relying on a distance-based rule.

w2/q1[Regarding novelty]

While backward planning has been extensively studied in the classical motion planning and optimization communities, to the best of our knowledge, it has not been explored within learning-based visuomotor policies prior to our work.

However, we actually find a concurrent work titled “Efficient Robotic Policy Learning via Latent Space Backward Planning” (accepted to ICML 2025, released on arXiv on May 11, four days before the NeurIPS deadline) also adopts a backward planning strategy—but in the latent space, by predicting future latent states and sub latent goals from current observations and . This approach is fundamentally different from ours: we perform backward planning directly in the action space, enabling explicit autoregressive generation of a whole trajectory. Our advantage is that: 1. the modeling of CoA can serve as a effective alternative to action chunking for it is highly compatible with other existing methods 2. CoA offers better interpretability, as its generated trajectories can be visualized to reveal the underlying decision-making logic.

Another related work is “Chain of Thought Imitation with Procedure Cloning”(NIPS 2022), which proposes procedure cloning conditioned only on the final goal pose. However, it does not involve backward prediction of the full trajectory.

In summary, our method is the first manipulation imitation learning policy to perform backward prediction directly in the action space.

q2[Comparison with Diffusion Policy with longer action horizon]:

Thank you for raising this insightful point. To directly address your question, we conducted additional experiments to explicitly compare CoA with Diffusion Policy (DP) under longer action horizons. Specifically, we evaluated DP on 10 representative RLBench tasks (as previously used in Table 1's ablation studies) with extended action horizons of 100 and even 180 steps. Note that 180 exceeds the maximum full trajectory length observed across these 10 tasks (178.96 steps), as detailed below:

Full Trajectory Lengths (10 representative tasks):

The resulting success rates for DP with extended action horizons are summarized below:

We observed that extending DP’s action horizon from its original length of 20 steps to 100 or even 180 steps yielded minimal improvement (from 41.6% to only 42%), indicating that DP does not effectively leverage longer action horizons. This finding aligns with the original Diffusion Policy paper, which noted optimal performance at around 8-step horizons, beyond which performance deteriorates.

Thus, the superior performance of CoA compared to DP can be attributed to CoA's ability to effectively handle long-horizon trajectories and explicitly condition each local action sequence on the final goal via backward reasoning. In contrast, despite denoising steps implicitly considering future states, DP inherently struggles to capture long-horizon dependencies when the action horizon is significantly increased.

Key Differences in Methodology

---

Why Our Main Benchmark is 60 Tasks

The RLBench-18 subset originates from PerAct (CoRL 2022) and is tailored to 3D-based pipelines. Our 60-task RGB benchmark:

• Covers a broader and more diverse range of tasks (including 7 from the 18-task set).
• Better reflects the strengths and weaknesses of image-based policies. The 18 tasks are often tailored for 3D-based hierarchical pipelines. For RGB-only policies, several of these tasks are too hard and frequently result in zero success, reducing their evaluative value.
• Recent literature has adopted customized task subsets. For example, Point Cloud Matters (NeurIPS 2024), Conditional Flow Matching (CoRL 2024), and Generative Image as Action Models (CoRL 2024) all adopt their own task selections based on their model scope and evaluation needs.

---

In summary, we will revise the abstract to avoid overclaiming, and we have provided the requested side-by-side comparison. While CoA is not SOTA when compared to 3D-based hierarchical pipelines, it achieves the strongest performance within image-based visuomotor policies, which is the intended scope of our claim.

Dear Area Chair,

Thank you very much for raising this point.
We acknowledge that the current abstract might unintentionally imply a broader SOTA claim than intended. Our actual intention was to highlight CoA’s empirical strength within the scope of image-based visuomotor policies. We will revise the abstract to the following phrasing to avoid misunderstanding:

"Empirically, we observe that CoA outperforms representative imitation learning algorithms such as ACT and Diffusion Policy across 60 RLBench tasks and 8 real-world manipulation tasks."

---

Direct Comparison with RVT-2 and 3D Diffuser Actor

On the RLBench-18 subset used by RVT-2 and 3D Diffuser Actor, we report the results side-by-side below. As shown, 3D-based hierarchical methods indeed achieve higher overall success rates. However, within image-based visuomotor policies, CoA achieves the strongest performance, showing the advantage of reverse modeling.

Note:

• Results for CoA, ACT, and DP are from our implementations.
• All others (3D Diffuser Actor, RVT-2, Image-BC) are from RVT-2, 3D Diffuser Actor, and PerAct.
• For image-based policies, some tasks have near-zero success, this reflects task difficulty, not evaluation error.
  • Image-BC results are from PerAct (CoRL 2022).
  • Point Cloud Matters (NeurIPS 2024) also reported 0 success for ACT and DP on many tasks.