Reducing Symmetry Mismatch Caused by Freely Placed Cameras in Robotic Learning

We thank the reviewer for their helpful comments.

The experiments are conducted only in six simple simulated environments, without any real-world validation.

We agree that real-world validation would strengthen this method. We are not able to provide results for real world tasks during the rebuttal period.

While the proposed methods reduce the performance gap with the oracle top-down view, they do not entirely close it. The occlusion of objects and grippers, especially in cluttered environments, remains an unsolved problem.

This is a good point. Keep in mind that the oracle view is an unrealistic ideal setting, since its very challenging to achieve the oracle view in a real-world setting. We are interested in closing the gap with the oracle further and are open to suggestions for experiments that would shed light on why this gap remains.

In the RGBD setting, how might more sophisticated inpainting or occlusion handling methods (e.g., learned inpainting) improve the performance gap with the oracle? Have the authors experimented with these techniques, and what were the results?

There are many effective inpainting methods available that could be used in place of interpolation. In our simulated results, interpolation worked well, due to the simplicity of the scene. We believe these inpainting methods would be more useful in real-world tasks with more complex backgrounds and objects.

While the experiments are simulated, can the authors elaborate on the challenges and potential modifications required to apply these preprocessing steps in real-world robot learning tasks with physical cameras and hardware?

We have begun real-world experiments in the RGBD setting. The main difference we observed was noisiness in the depth values on the gripper. After reprojection, the appearance of the gripper was very distorted (missing regions or missing entire finger), which we believe made it difficult for the model to learn. One option to help with this is to add a synthetically rendered gripper image to the input (similar to how we do it for the RGB setting).

We thank the reviewer for their helpful comments.

The problem this paper attempts to address may not be a genuine issue… a more natural approach might be to consider non-equivariant policy learning algorithms instead.

We directly compare our proposed approach against non-equivariant policy learning algorithms. The non-equivariant baselines perform much worse in terms of performance and sample efficiency (see “Sideview NonEqui” Figure 5 and Table 1). The non-equivariant methods were trained with data augmentation and still underperformed the equivariant versions. These results were also observed in [1].

The methods proposed in this paper lack originality… I view these techniques as pre-processing tricks rather than substantive innovations.

We agree that these techniques are simple and well-known, and we tried to make that clear in the writing. We see this simplicity as a benefit to our approach since it expands the problem settings where equivariant policy learning methods are useful without requiring different sensors or new network architectures.

The formulation for 3D reprojection in this paper is not entirely realistic. To perform reprojection, RGBD information is required.

We believe the reviewer is misunderstanding our work. We only perform 3D projection when RGBD information is available. When it is not available, we perform a perspective transform, which, as we note in the paper, deviates from a 3D reprojection for any visual features that are above the ground plane.

if 3D data is available, it would be more straightforward to use equivariant policy networks based on 3D groups$^{[1,2]}$ (such as SO(3), SE(3), or SIM(3))

You are correct. If we use an SO(3) equivariant network, then the original point cloud (in the sideview camera frame) could be used directly as input. In that setting, we would have to apply pooling at the end to reduce to an SO(2) equivariant representation for the output actions. One downside to this approach is the additional compute. Moving from SO(2) to SO(3)/SE(3) equivariance requires applying additional constraints that slow down training and increase memory. The other downside is that an SO(3)/SE(3) equivariant network cannot resolve the “gravity” direction (they are invariant to the input’s coordinate frame). In many robotic manipulation tasks, the “gravity” direction is important for determining the best action. There are ways to re-inject information about gravity, but it would take some experimentation to identify the best approach.

We believe running an experiment would be interesting. We are training a VectorNeuron [1] baseline and will try to add the results by the end of the discussion period.

In the experimental part, the author compares many baselines (equivariant, non-equivariant, 2D, 3D), but does not clearly write out the specific structure of each baseline and the group on which their equivariance properties are defined.

We include the symmetry group used and the specific structure of all networks in Appendix A.4. We refer to this part of the Appendix at the end of Baselines (Section 5.3).

All baseline methods in this paper are based on the same framework (SACfD). To demonstrate the effectiveness of this preprocessing approach in broader scenarios, I believe it would be beneficial to include comparisons with other state-of-the-art methods.

In this paper, we run experiments with SACfD (Figure 5 & 6) and behavior cloning (Table 1 & 2). Do you have a SOTA method in mind that we should run comparisons on?

[1] Deng, Congyue, et al. "Vector neurons: A general framework for so (3)-equivariant networks." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.

We thank the reviewer for their helpful comments.

How many equivariant methods could benefit from the proposed method? I would like the authors to discuss this question, and list as many papers as possible.

The point of this work is not to improve existing equivariant methods. In fact, we believe existing equivariant methods work very well within structured settings (e.g. those where the symmetric transformation of the observation is well-defined). Instead, we show that image-based equivariant methods can also excel in less structured settings (e.g. those with non top-down camera views) when they incorporate the simple pre-processing steps. Here is a list of robotic manipulation papers that fit into this less structured category, where our work could be beneficial

since the proposed method is mainly designed to tackle the challenge when deploying cameras in the real world, I think real-world experiments are indispensable

We agree that real-world results would strengthen the paper. We are not able to provide these results within the discussion period.

Would the proposed method also benefit general-purpose robot learning methods such as Diffusion Policy?

We believe the proposed method would help any SO(2) equivariant policy learning methods with non-top down images. The processed pre-processing steps align transformations of the input with transformations of the action, which leads to faster learning. As shown by a recent paper [1], SO(2) equivariant diffusion policy is useful for robotic manipulation, and we expect our preprocessing steps could enhance performance with sideview image inputs.

It seems that the compared point cloud baseline is using a single-view RGBD image. What if we have access to multi-view RGBD images? Consider the scenario in [1].

That is a good question. In theory, multiple RGBD images could be fused and reprojected to produce a top-down image. The resulting top-down image would have less occluded regions than the single-view RGBD case, which would boost performance. However, in the multi-view RGBD setting, the equivariant point cloud method is probably a better option since it can process a complete point cloud effective (reprojection collapses the information in the z-direction). We are interested in looking more into the multi-view RGB setting, such as the ALOHA system.

[1] Wang, Dian, et al. "Equivariant diffusion policy." arXiv preprint arXiv:2407.01812 (2024).

We thank the reviewer for their helpful comments.

Related works: I believe a small discussion on point cloud models (in the context of Image reprojection) should also be discussed.

We will add a discussion on point cloud models in the updated version of the paper.

I don't particularly find a difference between Point cloud equi and Reproj. equi in Fig 5. and Table 1. What are the benefits of Reproj. Equi over point cloud equi?

The performance of Reproj Equi and Point Cloud Equi are comparable. In general, point cloud networks are more compute and memory-intensive (more so when enforcing equivariance constraints), so we downsample the input to 1024 or 2048 points. So in settings where the task requires high-resolution observations (like peg-insertion or grasping mug handle), the point cloud model may struggle. In contrast, Reproj Equi uses an image encoder that can handle high-resolution inputs.

Sec 5.6 (Effects of camera angle) needs to also have the PointNet++ baseline (point cloud equi) for the RGB-D plots. Some works have suggested that point cloud RL policies are robust to viewpoint changes

We agree that this would be a good comparison. We will try to add the result by the end of the discussion period.

I would like to see an experiment with DrQ-v2 where image augmentaiton has shown significant sample-efficiency gains and am curious how that performs as compared to an explicit Equivariant policy.

Image augmentation is helpful for equivariant and non-equivariant policy learning, as shown by [1]. In our paper, all methods apply random image crops to observations during training as is the case in DrQ-v2. In the paper, we cite this technique as RAD [2], which was a simpler, concurrent work to DrQ (v1).

Does this formulation of having a gripper image generalize to a dextrous manipulation with a non-trivial gripper?

This is a great question. The gripper image is generated with the assumption that the gripper model is known. So generating the gripper image is possible as long as the gripper is fully actuated and composed of rigid parts, regardless of how dexterous the task is.

it would be better if the Fig 11 (from appendix) can be moved/integrated into the main paper.

We agree. We have added the gripper image to Figure 3, which is next to where we introduce the gripper image.

Are the models in Fig 7(a) and 7(b) test-only models are are they trained on individual camera angles?

The models in Figure 7 are tested on the same view angle as they were trained on. It would be interesting to see if the models could generalize to novel view angles (since the projection and perspective transform approximately canonicalize the perceived view to be topdown).

Are the class of Equivariant policies biased to the action space? Would the same set of architectures work for a other action spaces that are common in robotic manipulation such as end-effector pose, joint velocities, joint angle poisitions etc?

Equivariant policy networks are specific to an action space. Consider the equivariance equation (Eqn 1), we need to know the action of the group on the output space when we create the equivariant network. If we change the output space (action space), then we need to modify the equivariant constraints at the end of the network accordingly. This is easy for action spaces based on end effector pose or velocity, but difficult for joint space control. For instance, if we apply a 2D transformation to the scene, it is not clear what transformation should be applied to the joint angles to produce a similar action. This would be an interesting direction to pursue in the future since there is growing interest in policy learning in joint space (like in the ALOHA system).

[1] Wang, Dian, et al. "The Surprising Effectiveness of Equivariant Models in Domains with Latent Symmetry." The Eleventh International Conference on Learning Representations.

[2] Laskin, Misha, et al. "Reinforcement learning with augmented data." Advances in neural information processing systems 33 (2020): 19884-19895.