Hierarchical Equivariant Policy via Frame Transfer

We thank the reviewer for their response. We respond below:

---

“...The authors should provide more explanation to clarify the novelty of their approach.”

We agree with the reviewer that there exist various works on hierarchical policy learning. However, our method is the first to incorporate equivariant learning into hierarchical policy. This is both novel, and leads to sizable performance increases. Additionally, our proposed frame transfer interface ensures translational equivariance and imposes soft constraints on the low-level agent.

We refer Reviewer 4 to Reviewer 1’s assessment of our method's novelty

“Can the method be applied in scenarios with only image input, without requiring RGB-D inputs?”

Currently, our approach requires a 3D understanding of the environment, which we obtain using RGB-D inputs. Given the expense and inherent noise associated with RGB-D sensors, an interesting direction for future work would be to extend our framework to scenarios that rely solely on RGB images. Recent advances in depth-estimation foundation models [1,2] may enable the generation of accurate 3D representations from pure image data. Additionally, in tabletop settings where a top-down camera is available, pixel coordinates can be directly mapped to spatial (x,y) positions without requiring explicit depth measurements, thus potentially enabling SE(2) equivariance in the policy.

[1] Guo, Y, et al. "Depth Anything: Unleashing the Power of Large-Scale Image-Text Pretraining for Zero-Shot Depth Estimation." arXiv preprint arXiv:2311.16502, 2023.

[2] Wen, B, et al. "FoundationStereo: Zero-Shot Stereo Matching." arXiv preprint arXiv:2501.09898, 2025.

“...separate policy trained for each task? ...does the policy incorporate language?”

We are evaluating our policy under a single-task setting following former work [3] for fair comparison. However, it should be straightforward to operate in a multitask setting by adding a language feature to the observation, as is done in [4,5]. Evaluating the performance in multi-task settings can be an interesting direction for future work.

[3] Xian, Z et al "ChainedDiffuser: Unifying Trajectory Diffusion and Keypose Prediction for Robotic Manipulation." CoRL. PMLR, 2023.

[4] Ke, J, et al. "3D Diffuser Actor: Policy Diffusion with 3D Scene Representations." CoRL. PMLR, 2024.

[5] Shridhar, M, et al. "Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation." CoRL. PMLR, 2023.

---

“How is the Equivariant Diffusion Policy used?...sampling procedure differ from that of a standard Diffusion Policy?”

Our method improves upon standard diffusion policy in two key ways. Specifically, our policy is explicitly equivariant [6], meaning that it is automatically able to generalize to new problem instances related by symmetry. This is not inherent in Diffusion Policy [7]. Secondly, our method utilizes a hierarchical decomposition — namely, a high-level and low-level agent — not found in standard diffusion policy. As shown in our experiment result in Table 1 in our response to reviewer 3 cFSf (referenced here due to character limitations) and Table 1 here, these two changes enable our method to significantly outperform standard diffusion policy.

[6] Weiler, M. et al. "E(2)-Equivariant Neural Networks for Image Classification and Beyond." ICLR, 2019.

[7] Chi,C et al. "Diffusion Policy: Visuomotor Policy Learning via Action Diffusion." RSS, 2023.

---

“...baselines use 3D scene representations; a comparison with state-of-the-art image-based visuomotor policies would provide a more comprehensive evaluation.”

As shown in the prior work [9], image-based Diffusion Policy [7] performs poorly in RLBench tasks, where the 3d representation of the workspace is essential. We also performed an extra experiment comparing the image-based diffusion policy against our method and we report the result here:

Table 1. Performance on 5 Tasks from RLBench

We follow the hyperparameters from the original work, with the exception that we use RGB images from the same four cameras used in our setup (wrist, front, right shoulder, and left shoulder) and at the same resolution, to ensure a fair comparison.

[8] Wang ,D ,et al "Equivariant Diffusion Policy." CoRL. PMLR, 2024.

[9] Shridhar, M et al "Generative Image as Action Models." CoRL PMLR, 2024.

We thank the reviewer for their thoughtful feedback. We respond below:

---

"HEP addresses translational equivariance thoroughly... currently, the framework isn’t equipped to specify or leverage rotations... could limit performance on tasks where orientation is critical."

Thank you for bringing this up! We agree that predicting orientation subgoals at the high level could potentially further improve sample efficiency. However, we would like to clarify that our current low-level agent already predicts SE(3) actions, enabling it to handle tasks requiring complex end-effector orientations, such as door closing, fridge opening, and pot cleaning. That being said, we are actively working on an extension of this paper where the high-level explicitly predicts goal orientations. We will thoroughly investigate and compare the advantages and disadvantages of these two orientation-handling strategies in our follow-up work.

"Experiments... confined to tabletop manipulation... scalability to other domains remains unclear."

We appreciate the reviewer's insightful suggestion. Extending HEP to more complex robots, such as humanoids, is indeed a valuable direction for future research. We recognize that the generality of HEP in cross-embodied scenarios remains to be fully established. Nonetheless, we firmly believe that HEP effectively demonstrates two fundamental principles essential for efficient policy learning: special Euclidean group symmetry and hierarchical policy decomposition. These principles are expected to inform and inspire future developments across various robotic domains.

"HEP relies on demonstrations... could be a bottleneck... hyperparameter sensitivity was not thoroughly discussed."

We appreciate the reviewer's valuable insight. We agree that dependence on expert demonstrations can be a limiting factor, especially given the time-consuming nature of obtaining demonstrations. While this study primarily explores behavior cloning, we believe future research could effectively extend HEP into reinforcement learning frameworks, thus eliminating the necessity for human demonstrations. Regarding hyperparameter sensitivity, we would like to clarify that our approach did not involve any hyperparameter tuning. Specifically, we directly adopted the high-level hyperparameters from Act3D and the low-level hyperparameters from Equivariant Diffusion Policy without any finetuning. We believe this choice underscores the robustness of our method. We will also make this explicit in the final version of the manuscript.

"Related work is comprehensive... recent hierarchical RL or imitation learning works outside diffusion models should be acknowledged."

Thanks for your valuable advice. We added following works as our related works:

[1] Wang, C, et al. "MimicPlay: Long-Horizon Imitation Learning by Watching Human Play." CoRL. PMLR, 2023.

[2] Luo, J, et al. "Multi-Stage Cable Routing through Hierarchical Imitation Learning." IEEE Transactions on Robotics, 2024.

[3] Triantafyllidis, E, et al. "Hybrid Hierarchical Learning for Solving Complex Sequential Tasks using the Robotic Manipulation Network (ROMAN)." Nature Machine Intelligence, 2023.

[4] Bagaria, A, et al. "Effectively Learning Initiation Sets in Hierarchical Reinforcement Learning." NeurIPS, 2023.

[5] Shao, J, et al. "Pfrl: Pose-free reinforcement learning for 6d pose estimation." CVPR. 2020

---

Reviewer’s Questions:

"HEP showed strong one-shot generalization... Is it largely due to equivariant inductive bias?"

Thank you for bring this up. We believe the reason is that we use a 3D Unet as our high level (which is T(3) equivariant), but most importantly, our frame transfer interface passes the equivariance and generalization ability to our low level, leading to an improved generalization ability of the whole policy.

"Contribution of hierarchical decomposition alone (without equivariance) compared to flat equivariant policy?"

This is a great question. We conducted an ablation study with hierarchical decomposition. And we put it into our response to Reviewer 3 cFSf due to character limitation. Please see Table 1 in our response to reviewer 3 cFSf for more details. As shown in the table, removing the hierarchical decomposition lead to a significant drop in performance, demonstrating importance of a hierarchical structure especially on long horizon task.

"Improving robustness to depth sensor noise?"

We thank the reviewer for raising this important point. While the high sample efficiency of our policy allows us to train directly on real-world data—enabling the policy to adapt to sensor noise to some degree—we agree that introducing additional noise during training (e.g., applying a dropout layer to randomly remove points) could further enhance the robustness of the system to sensor noise. We will explore this idea in future experiments.

Thank you again for your insightful feedback!

We thank the reviewer for their response. We respond below:

“The theoretical claims seem correct under the assumptions stated, although they are impossible to verify due to the deep neural net.”

We thank the reviewer for raising this important point. While our theoretical claims appear valid under the stated assumptions, we acknowledge the difficulty of directly verifying these assumptions due to the complexity of deep neural networks. To address this concern empirically, we conducted an experiment measuring the equivariance error specifically on C4, a subgroup of SO(2). This allowed us to quantify the difference between the rotated output and the output from a rotated input. The experimental results are summarized in the table below:

Table 1. Equivariance Error Under Different Rotations

As you can see, the network is equivariant, modulo numerical error, to rotational transformation. Also, our network inherits the transnational equivariance through the Unet and frame transfer architecture.

---

“In open-loop training, the low-level target is constructed by interpolating between the consecutive keyframes. This interpolation may fail in a clustered environment due to the need for obstacle avoidance. This also introduces one of the main limitations of the experimental setup: the method is tested only on a simple, unclustered table-top setting.”

We thank the reviewer for the question. In Section 6 “Robust to Environment Variations”, we demonstrate that the trained policy would be robust to some disturbance objects in the workspace. We conducted an extra experiment evaluating the success rate of executing a block stacking task under human perturbation mimicking a dynamic environment and include the results here:

Table 2. Success Rate Under Human Perturbation

However, the reviewer makes a good point that if the demonstration data does not contain obstacle avoidance while an obstacle is introduced at test time, the trained policy might fail. In future work, we propose to add an additional hierarchy of a trajectory optimization layer, which would refine the policy for obstacle avoidance.

We thank the reviewer for their response. We respond below:

"...related baselines are missing like equivariant policies, e.g., Equivariant Diffusion Policy. I would like understand the difference of having a hierarchical policy and more simple versions"

"There should be work on equivariant policies, e.g., Equivariant Diffusion Policy, Wang et al."

"Could you show results of other equivariant policies?"

Thank you for bringing this to our attention. We did provide comparisons with equivariant policies, specifically the Equivariant Diffusion Policy [1], labeled as "EDP" in Table 1. However, we acknowledge that this reference may not have been sufficiently clear. In our revised manuscript, we will explicitly clarify that "EDP" refers to the Equivariant Diffusion Policy and emphasize this comparison more clearly to avoid confusion.

Additionally, we've conducted an ablation study on the hierarchical architecture to better understand its benefits:

Table 1. Revised Ablation Experiment

Our findings demonstrate that incorporating a hierarchical structure notably enhances performance, particularly on long-horizon tasks.

---

[1] Wang, Dian, Stephen Hart, David Surovik, Tarik Kelestemur, Haojie Huang, Haibo Zhao, Mark Yeatman, Jiuguang Wang, Robin Walters, and Robert Platt. "Equivariant Diffusion Policy." Conference on Robot Learning (CoRL). PMLR, 2024.