Thank you for your helpful comments! We have revised the paper to provide more discussion for sim-to-real transfer and downstream tasks. To address your concerns and questions:

---

Transfer to Real Humanoid

We acknowledge that Omnigrasp in its current form could not be applied to the real world. Its potential for real-world deployment lies in the scalable learning framework (first learn motion representation and then train grasping and object trajectory following). To deploy in the real-world and avoid using privileged information, the popular practice in sim-to-real is to distill a teacher policy into a student policy that does not use privileged information [1, 2, 3]. We imagine an input space similar to [3] (8 point bounding-box) could be used in the real-world for grasping. The mesh information could be replaced with vision or point cloud [1]. Similar to the transfer observed from the motion imitation task to real humanoid motion imitation [4, 5, 6], we believe that a framework similar to Omnigrasp could be deployed to the real world.

More Downstream Task Like Fine-grained Manipulation

We believe that if we train specifically for the object manipulation task for one object, our framework is capable of learning fine-grained manipulation. Our early results indicate that we can overfit to fine-grained object movement and handovers using the same motion latent space. This indicates that the motor skills learned in PULSE-X could support fine-grained manipulation. Once we add more diverse objects and trajectories, the learning problem becomes significantly harder and we observe the policy struggle with more precise trajectory tracking. While we focus on scaling the grasping task using simulated humanoids, we believe that tuning the reward for the fine-grained manipulation task on a smaller number of objects can lead to promising results. How to learn a general policy that can do both is an interesting yet challenging future direction.

References

[1] DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulation
[2] Robot parkour learning
[3] DextrAH-G: Pixels-to-Action Dexterous Arm-Hand Grasping with Geometric Fabrics
[4] Expressive whole-body control for humanoid robots
[5] HumanPlus: Humanoid Shadowing and Imitation from Humans
[6] Learning human-to-humanoid real-time whole-body teleoperation

Thank you for your insightful suggestions and feedback. We have revised the paper to provide additional baselines (AMP and PHC), add details about dexterous hands, and discuss decoupled body and hand prior. To address your questions:

---

Additional Baselines

In Table 2 of the global PDF, Row 2, we train the policy without PULSE-X to collect $10^{10}$ samples for 1 month (Omnigrasp uses $10^{9}$) and observe its performance still significantly lags behind Omnigrasp. Training with only RL (even with AMP) also leads to non-human-like motion (see supplement videos). Row 3 reports training our method with AMP but not PULSE-X, which does not lead to a high success rate. Row 4 reports results from Braun et al. [1], which uses ASE-style latent space trained specifically for the grasping task on GRAB data. Compared to Braun et al., we achieve a much higher success rate in terms of grasping and trajectory following and can follow complex trajectories. In comparison to ASE, PULSE-X's latent space is trained on AMASS and covers much broader types of human motion. ASE's discriminative latent space, while effective when learning from specialized and curated datasets, struggles to learn from large-scale unstructured datasets such as AMASS, as shown in [2].

Details about the Dexterous Hand

We will add more details about how the hands are controlled in the revised manuscript. The hands are treated the same as the rest of the body (e.g. toe or wrist) and actuated using PD controllers (Section 4.1).

Using PHC for body control and raw action space for the hand is an excellent suggestion. However, this approach requires we learn a policy to output kinematic poses as "actions" for PHC to imitate. As observed in [2], kinematic motion is a poor sampling space for RL due to its high dimension and lack of physical constraints (e.g. a small change in root position can lead to a large jump in motion). Thus, prior art that uses kinematic motion as the action space (e.g. kinematic policy [3, 4]) uses supervised learning to train the motion generator instead of RL. Supervised learning would require paired full-body human grasping data, which is scarce and limited in diversity. One of the main advantages of Omnigrasp is its ability to learn grasping policy without paired full-body grasping data, enabling it to scale to many objects and diverse trajectories.

Further, even if a policy could output ground-truth kinematic pose for PHC, small errors in imitation can lead to the hand missing the objects. To demonstrate this point, we use ground truth MoCap as input to a pretrained PHC-X policy (~30mm imitation error on average) for grasping, using sequences from GRAB. The result from Table 2 (global PDF) Row 1 indicates that the accuracy of a trained imitator does not support object grasping. To use PHC for the grasping task, we will need to fine-tune PHC with object awareness and pair it with a strong kinematic motion generator. Such an approach has been explored for box loco-manipulation [6] without hands, but it only supports moving boxes for now.

Training with Dex-AMASS

We apologize for the confusion. When comparing training with or without Dex-AMASS for PULSE-X, the only difference is the training data. In both cases, the hand joints are controlled using PD controllers to output PD targets, but one with regular AMASS and one with our Dex-AMASS as training data. We will further clarify this.

Extending to Real Robots

We will revise the sentence to "While the state has access to privileged information and the current humanoid has no real-world counterpart, the overall system design methodology has the potential to be transferred to a real humanoid robot, similar to how the motion imitation task is applied in recent humanoid work [5, 7, 8]." We believe that the framework of first learning a universal motion representation, then learning grasping policy in simulation, followed by conducting sim-to-real modifications (e.g. domain randomization, distilling into a vision-based policy), has the potential to be applied to real-world humanoids.

Train Separate Body and Hand Models

As mentioned in Section 6, while we utilize a simple yet effective unified motion latent space, separate motion representation for hands and body could lead to further improvements. We completely agree that training a separate model could be beneficial. Braun et al [1] use decoupled body and hand prior, and achieve a lower success rate. We hypothesize that since the hand is tethered to the body and performs actions based on different wrist movements (which affects gravity), training a hand-only model that performs well when combined with the body is non-trivial. Since we observe accurate hand-tracking results when training a motion imitator jointly for hands and body, we proceed with the joint latent space design and find it is sufficient for achieving good grasping results. We are actively exploring using decoupled hand and body latent space, though it has not yet shown real benefit.

References

[1] Physically plausible full-body hand-object interaction synthesis
[2] Universal humanoid motion representations for physics-based control
[3] Learning predict-and-simulate policies from unorganized human motion data
[4] Dynamics-regulated kinematic policy for egocentric pose estimation
[5] Learning human-to-humanoid real-time whole-body teleoperation
[6] Hierarchical planning and control for box loco-manipulation
[7] Expressive whole-body control for humanoid robots
[8] HumanPlus: Humanoid Shadowing and Imitation from Humans

Thank you for the discussion!

PULSE is not compatible with the body of the SMPL-X humanoid, since it is trained using the SMPL humanoid. As the body shape for SMPL and SMPL-X is not the same, SMPL-X and SMPL humanoids have slightly different bone lengths and directions. Since the GRAB dataset we are using is in SMPL-X format, we initiated this work to support the SMPL-X humanoid. As more and more dataset is captured using SMPL-X instead of SMPL-H, we hope our pipelines can be future-proof and support SMPL-X based datasets.

We would also like to note that raw hand actions are problematic as the controller can use non-human-like strategies in grasping (see Supplement Site, Training Without PULSE-X). Using raw actions leads to hands being distorted due to their high degree-of-freedom and unbounded exploration. To provide motion prior to the hand's motion, either AMP or PULSE-like motion prior will be needed. Using PULSE for the body and then the raw-action space for the hand plus some form of hand prior would also be quite close to our method in terms of methodology and compute overhead (for training PHC and PULSE), and would support our story that using a strong motion prior one can enable learning diverse grasping and object trajectory following. We are actively exploring using separate hand and body priors, though it does not seem to be the bottleneck yet.

Please feel free to ask us any additional questions!

Thank you for the helpful feedback and comments. We have revised the paper to include robustness tests, a discussion about kinematic latent space, and failure case analysis. To address your concerns:

---

Robustness

In Table 1 of the global PDF, we add uniform random noise [-0.01, 0.01] to both task observation $s_t^g$ (positions, object latent codes, etc.) and proprioception $s_t^p$. A similar scale (0.01) of random noise is used in sim-to-real RL to tackle noisy input in real-world humanoids [1]. We see that Omnigrasp is relatively robust to input noise, even though it has not been trained with noisy input. Performance drop is more prominent in the acceleration $E_{\text{acc}}$ and velocity $E_{\text{vel}}$ metrics. Adding noise during training can further improve robustness. We do not claim that Omnigrasp is currently ready for real-world deployment, but we believe that a similar system design plus sim-to-real modifications (e.g. domain randomization, distilling into a vision-based policy) has such potential.

Morphology

The AMASS [2] dataset has been successfully used in real-world humanoid control [1, 3, 4]. Retargeting techniques can be used to transfer human motion to the humanoid, even when they do not have the same morphology or joint numbers. Similarly, human hand motion in Mano [6] format can be transferred to robotic hands [7, 8].

One of the main strengths of Omnigrasp is its independence from paired full-body grasping and object trajectory data, which is expensive to capture and therefore only available in small-scale datasets. Since the retargeting process could introduce errors that cause human-object interaction to be imprecise, retargeted interaction data can be hard to use for methods that rely on demonstrations. Our two-stage process (first obtain motion prior and then learn grasping policy) can leverage imperfectly retargeted motion, since as long as the motion is human-like, we can acquire motor skills by imitating it.

Baseline Alternative

Excellent advice! Such a general-purpose kinematic latent space has been used in physics-based control for pose estimation [10] and animation [11], though few have been extended to include dexterous hands. These latent spaces, like HuMoR [9], model motion transition using an encoder $q_\phi (z_t| x_t, x_{t-1})$ and decoder $p_\theta(x_t | z_t, x_{t-1})$ where $x_t$ is the pose at time step t and $z_t$ is the latent code. $q_\phi$ and $p_\theta$ are trained using supervised learning. The issue with applying such a latent space to simulated humanoid control is twofold:

• The output $x_t$ from the VAE model, while representing natural human motion, does not model the PD-target (action) space required to maintain balance. This is shown in prior art [10, 11], where an additional motion imitator is still needed to actuate the humanoid by imitating $x_t$ instead of using $x_t$ as policy output (PD-target).
• $q_\phi$ and $p_\theta$ are optimized using MoCap data, whose $x_t$ values are computed using ground truth motion and finite difference (for velocities). As a result, $q_\phi$ and $p_\theta$ handle noisy humanoid states from simulation poorly. Thus, [10] runs the kinematic latent space in an open-loop auto-regressive fashion without feedback from physics simulation (e.g. using $x_{t-1}$ from the previous time step's output rather than from simulation). The lack of feedback from physics simulation leads to floating and unnatural artifacts [10], and the imitator heavily relies on residual force control [12] to maintain stability.

PULSE directly models the action distribution instead of the kinematic pose and does not need a motion imitator during inference. Directly learning the latent space without distillation is provided as an ablation in the PULSE paper (Section C.2 Table 6), and it shows that training using RL does not converge to good performance. Random sampling for the variational bottleneck together with random sampling for RL leads to noisy gradients, which hinders policy learning.

We will add additional discussion to elaborate on these points.

Reward Complexity

We will clarify this in the text. The “simple reward” here refers to not needing paired full-body-and-hand MoCap data in the reward, which increases complexity. Prior art often involves graph-based [13, 14] or style rewards [5] that depend on paired data.

Failure Analysis

We categorize the failures into two types: failure due to grasping and due to dropping the object after during transport. The “restabilizing the object in hand” during transport behavior might require further dexterity and training rewards to master. Appendix Section C.3 provides a per-object breakdown of the GRAB-Goal split. We can see that toothpaste and binoculars have the lowest trajectory following success rates (80.9% and 90.5%). In our experience, objects that can roll (such as toothpaste) and large objects (such as binoculars) are more difficult.

Reference

[1] Learning human-to-humanoid real-time whole-body teleoperation
[2] AMASS: Archive of motion capture as surface shapes
[3] Expressive whole-body control for humanoid robots
[4] HumanPlus: Humanoid Shadowing and Imitation from Humans
[5] Physically plausible full-body hand-object interaction synthesis
[6] Embodied hands: Modeling and capturing hands and bodies together
[7] Kinematic Motion Retargeting for Contact-Rich Anthropomorphic Manipulations
[8] Task-oriented hand motion retargeting for dexterous manipulation imitation
[9] Humor: 3d human motion model for robust pose estimation
[10] Learning human dynamics in autonomous driving scenarios
[11] Learning Physically Simulated Tennis Skills from Broadcast Videos
[12] Residual force control for agile human behavior imitation and extended motion synthesis
[13] Simulation and retargeting of complex multi-character interactions
[14] Physhoi: Physics-based imitation of dynamic human-object interaction