A Practical Guide for Incorporating Symmetry in Diffusion Policy

We thank the reviewer for their helpful comments. Please see our response below.

The action representations proposed in this paper appear to be specifically designed for eye-in-hand robotic setups, and it remains unclear how well these techniques would generalize to other types of tasks or embodiments.

Although relative actions with eye-in-hand perception can achieve a theoretical benefit as discussed in Proposition 2, our action representations are not limited in eye-in-hand robotic setups. As shown in Table 1 and Table 4, relative trajectory works very well with the in-hand + external observation setting. Regarding other types of embodiments, eye-in-hand setting might actually be better for cross-embodiment generalization, as the observation would remain invariant across different robot embodiments.

One common problem with incorporating symmetries is that the tasks usually do not fully respect the group structures in general, due to factors like imperfections of the observations. I think it would be beneficial to consider soft variants like [1]. [1] Finzi, Marc, Gregory Benton, and Andrew G. Wilson. "Residual pathway priors for soft equivariance constraints." Advances in Neural Information Processing Systems 34 (2021): 30037-30049.

This is a good thought, however, unlike the prior works in equivariant policy learning that use end-to-end equivariant networks and might encounter the symmetry breaking problems you mention, our work uses an equivariant encoder followed by a standard diffusion head (see Figure 2). As a result, our policy is not totally constrained by the symmetry of the network, and thus will not suffer from the symmetry mismatch problem. That said, we do agree with the reviewer that methods like the residual pathway would be an interesting future direction to explore fused equivariant and non-equivariant feature encoding. We will discuss this in the final version of our paper.

How important is FOV in terms of performance? It appears that there are no direct comparison between normal FOV and large FOV in the experiments, but only large FOV and normal FOV + external/voxel.

We performed a new experiment comparing the large FOV with the normal FOV. As shown in the table below, a normal FOV could work for simple tasks like Stack D1, however, when facing more challenging tasks with multiple objects in the scene like Stack Three D1, it suffers from incomplete information in the observation.

Related to (1), would it be possible to also incorporate equivariant CNNs in the normal FOV + external setting? Would it require significant changes to the architecture?

It is possible, and no changes to the architecture are required. Please notice that we did perform experiments using the proposed method with external cameras in Appendix E, where our method still outperforms the baselines with a significant margin. In fact, our implementation supports an arbitrary number of cameras, and our Table 4 and Table 2 were run using the same encoder class.

How sensitive is the method's performance to the discretization of SO(2) used in equivariant CNNs? (e.g., would improve/degrade performance)

Reducing the discretization will degrade performance. We validated this with our Equi Enc variation, where we changed the group associated with the network from C8 (containing 8 discrete rotations ranging from 0 to 2pi) to C4 (containing 4 discrete rotations) or C2. Please see the results in the table below.

We suspect that increasing the size of the sub-group from C8 to higher order might not future improve the performance. As experimented in the e2cnn paper, a cyclic group of order 8 is typically a sweet spot among the discrete subgroups of SO(2).

We thank the reviewer for their insightful suggestions. Please see our response below.

The novelty is limited, as the authors are simply combining and analysing different known methods

We would like to clarify three novelties of our paper:

1. A theoretical analysis of the symmetry of relative trajectory and eye-in-hand perception.
2. The use of an equivariant encoder but non-equi policy, rather than a fully equivariant policy as commonly done in prior works, for incorporating symmetry.
3. The use of Frame Averaging to bridge the benefit of equivariant encoding and pre-trained vision encoders.

To the best of our knowledge, we are the first to accomplish each of those three points in the robot learning literature. We thank the reviewer for raising this concern, we will clarify the novelty of our work in the revision.

The comparisons in Table 2 are not completely fair, as they are comparing their method (which uses an eye-in-hand camera) to EquiDiff (using absolute actions and multiple external cameras). Yes, it is a simpler setup, and the authors argue the setup is a key part of their solution, but I still find the comparison a bit awkward.

This is a good point, thanks for bringing it up. The reason for using the external camera for EquiDiff is two-folds: 1. This is the setting proposed by the prior work and 2. Using EquiDiff with only a large FOV in-hand camera will decrease its performance, due to the lack of equivariant signal from the external camera. However, we do agree with the reviewer that using a different camera setting can raise fairness concerns. To address this, we performed an additional experiment for EquiDiff using a large FOV in-hand camera. As shown in the table below, the performance of EquiDiff dropped significantly without accessing the external camera.

I think it would be interesting to see an experiment where the proposed Pretrain+FA was applied to the same multi-camera, absolute action setting as EquiDiff. Additionally, would it be possible to run EquiDiff in your eye-in-hand setup? I think having these two experiments would strengthen the paper

This is also a great point, thank you for the suggestion. Demonstrating the performance of Pretrain + FA with multi-camera, absolute action setting as EquiDiff would indeed demonstrate the generalizability of our proposal. We performed a new experiment in this suggested setting for two methods: (1) EquiDiff + Pretrain + FA: replacing the encoder in EquiDiff with our Pretrain + FA encoder. (2) Diff + Pretrain + FA: using our Pretrain + FA with the original diffusion head, similar as our method in Table 2. As shown in the table below, the Pretrain + FA encoder can be used with either EquiDiff or the standard diffusion head, both performing better than the baseline EquiDiff. In addition, we had an experiment of Pretrain + FA in the multi-camera (with relative action though) in Appendix E. We believe these results well demonstrate the generalizability of our Pretrain + FA encoder, improving baseline performance in various different settings.

We also performed the requested EquiDiff in eye-in-hand setup, please see our response to the question above.

We thank the reviewer for their careful review. Please see our response below.

Proposition 2 assumes the policy does not depend on the gripper pose , yet the authors themselves acknowledge this is not strictly true in practice. However, no ablation is provided to quantify how much performance degrades due to residual -dependence. As a result, the claim that “approximate invariance” suffices remains qualitative and lacks experimental support.

We performed a new ablation study to remove the gripper pose in the policy input. The resulting ablation better satisfies Proposition 2 at a cost of the proprioceptive information. As shown in the table below, removing the gripper pose decreases the performance, justifying our design choice of keeping the pose in the input.

The experimental sections evaluate combinations of techniques (e.g., relative trajectories with equivariant encoders), but do not fully isolate the contributions of each component via comprehensive ablations.

This is a good point. Although some of the ablations do exist in the paper, they are not organized as an ablation table. We add an additional ablation baseline that uses the Pretrain + FA encoder with absolute trajectory, and explicitly ablates the relative trajectory and equivariant encoder compared to the full model. As shown in the table below, both components contribute significantly to the overall performance.

How much additional computational cost is introduced by Frame Averaging, and how does this compare to training from scratch with a full equivariant architecture?

Frame averaging increases the GPU memory consumption by increasing the batch size by a factor of the frame size. In our case (8 frames), adding frame averaging to a pretrained encoder increases the GPU memory from 2970GB to 6476GB. Compared to Equi Enc’s 4942GB, frame averaging is slightly heavier (partially because the pretrained resnet-18 is slightly larger than our custom equivariant resnet-18), however, it remains relatively light and can be trained on most of the existing gaming GPU. For larger base models, one can reduce the size of the frame or the training batch size to reduce the GPU memory consumption.

The authors thank the reviewer for their thoughtful review. Please see our response below:

No real-world experiment to evaluate the proposed approach.

While we do not explicitly include real-world experiments in this paper, the assumptions that we are making in our simulation experiments are well-aligned with standard real-world experimental settings (i.e., the only information that we are acquiring from the simulation is the eye-in-hand image and the proprioceptive information of the gripper). Consequently, our findings should transfer easily to the real world. Moreover, one important goal of this paper is to systematically evaluate a number of practical methods for incorporating symmetry into Diffusion Policy. This requires abundant and rigorous comparisons that are more controllable in simulation than the real-world.

Limited experimental results. The result from using relative actions is not very interesting as it was already proposed in the oringal DP paper and widely used in diffusion policies these days.

The original DP did not use the same relative action definition as our paper. Instead, their relative action is a delta action defined in the world frame (referred as velocity control in the original DP paper), following the RoboMimic action space convention. This action formulation does not have any inherent symmetry, thus it underperforms the absolute action in the original DP paper. The relative action studied in this paper, however, is defined in the gripper frame, thus its symmetric property discussed in section 4.1 enables an outperformance compared with absolute control.

Action selection is well explored in the community, and the use of equivariant encoders and in-hand observations are not uncommon in robot policy learning.

Our method uses an equivariant encoder with a non-equivariant policy, as well as frame averaging for bridging pretrained encoders and equivariance, both of which have never been done before in robot learning. Prior works in equivariant robot learning all focus on defining an end-to-end equivariant policy, which is much more complex mathematically and implementationally compared with our approach. Regarding action selection, although this is indeed a question that has been studied a lot in the community, it remains uncertain what is the best action formulation in the context of Diffusion Policy. Moreover, we believe re-thinking about this question from a symmetry perspective will also offer the community some new insights.

Since the design incorporates symmetry, will it perform better in low-data regimes? All of the datasets used are fairly abundant in the experiments.

Yes, it will perform better in low-data regimes. In fact, all of our experiments use 100 demonstrations, which is already in the low-data regime (e.g., compared with the 200-300 demonstrations per task used in the original DP paper and the UMI gripper paper). If we reduce the number of demonstrations further, we believe the performance of all methods will decrease, but the impact on symmetric methods will be smaller.

As mentioned in the paper that frame averaging slows down the training process. Does it also slow down the inference process? Will it limit the performance of the policy in realworld settings if the latency of the model increases?

This is a good question. In our experiments, the inference for frame averaging is the same as the baseline. One advantage of frame averaging is that it typically does not slow down inference, especially in the robot policy inference where the inference batch size is normally 1. Frame averaging essentially increases the batch size by the size of the frame (8 in our case), which does not affect inference time due to batch processing.

It would be interesting to see how this work can be extended to external observation cameras.

This is a very important point, but we would like to clarify that we do have this experiment in Appendix E, where we found that the proposed method in external observation can also improve the performance by a significant margin.

In addition, we performed an additional experiment validating the Pretrain + FA encoder (with either equivariant diffusion head or standard diffusion head) in the external observation and absolute action space (i.e., same setting as EquiDiff). As shown in the table below,our method is also effective in the external observation and absolute action space setting.