Rebuttal for ntYt

Thanks for your helpful and meaningful reviews! We will provide a detailed response below to the issues outlined in the Weakness and Question sections.

Weakness:

[Lack of Experimental Validation for DexGarmentLab]: ......an analysis showing that training a policy is more sample-efficient or faster in terms of wall-clock time compared to other environments or using the AttachmentBlock method would be necessary to empirically substantiate the environment's value as a standalone scientific contribution.

[Lack of Quantitative Analysis for the Data Collection Pipeline]: ......the quality of the data it produces is never directly measured or analyzed. The paper provides no evidence to show that the generated demonstrations are indeed "expert" or how they compare to demonstrations collected via other methods, such as human teleoperation.......

Answer:

Thanks a lot for pointing out these issues! They are highly valuable and constructive suggestions for improving DexGarmentLab. Below, we provide additional clarification regarding the missing empirical validation in the Environment and Data Collection Pipeline components.

1.As for Lack of Experimental Validation for DexGarmentLab:

Firstly, it is important to clarify that garment simulation in our work is built upon NVIDIA PhysX's Position-Based Dynamics (PBD) framework, which currently offers the most realistic and stable performance for garment simulation. Alternative simulators such as SofGym or PyBullet do not support PBD-based simulation and generally exhibit inferior performance in soft body dynamics. Moreover, they also lack support for fine-grained interactions between dexterous hands and garments. Therefore, in terms of garment simulation fidelity and interaction capability, Isaac Sim remains the most suitable and advanced platform available to date.

Currently, the two main open-source garment simulation platforms based on Isaac Sim are GarmentLab and DexGarmentLab. GarmentLab adopts an attach block-based approach for garment manipulation, where grasping is achieved by explicitly attaching or detaching blocks to the garment. However, this method has significant limitations. Specifically, attachment and detachment must be explicitly triggered in code, which makes it unsuitable for end-to-end or reinforcement learning-based policy training. Instead, it is more appropriate for hierarchical planning or hardcoded control, such as those guided by vision-language models (VLMs).

Moreover, as discussed in the main paper (Lines 130–132), even minimal contact—such as a single finger block touching the garment—can result in attachment and lifting, which is clearly unrealistic. In the early stages of our project, we experimented with training policies using the attach block method. However, during evaluation, the instability of the attach/detach mechanism often caused the garment to behave erratically or "fly off," leading to frequent task failures. As a result, we ultimately abandoned this approach.

To further illustrate this point, we selected two representative tasks to evaluate the performance of the attach block-based method. The results clearly show that this approach performs poorly across both tasks.

2.As for Lack of Quantitative Analysis for the Data Collection Pipeline:

This missing component indeed affects the overall structure of the paper. To address this, we have provided the relevant quantitative analysis below.

We use LeapMotion for teleoperation data collection in simulation and an exoskeleton device for teleoperation in the real world. Three representative tasks are selected for data collection, training, and evaluation. The performance of teleoperated policies is then compared with that of policies trained on autonomously collected data. The results are presented below.

Experimental results indicate that policies trained on teleoperated and automated data achieve comparable performance during evaluation. However, from the perspective of human and time cost, teleoperation-based data collection is significantly more labor-intensive and time-consuming than automated data collection. It is also important to note that our data is not synthetically generated, but collected through real executions in either simulation or the real world, which means the demonstrations are physically realistic. The entire data collection pipeline is enabled by model inference and one-shot demonstrations.

3.As for summary:

Thanks again for your valuable suggestions regarding the organization and presentation of the paper. To be honest, we did encounter considerable difficulty when structuring the overall narrative of the paper. Receiving such constructive feedback during the rebuttal stage has been truly helpful for us. We will sincerely incorporate your suggestions in the revised version, including adding the missing content (such as citations for PBD) and further refining the structure and clarity of DexGarmentLab. Thank you once again!

Question1：

The entire data collection process is bootstrapped from a single, manually defined expert demonstration. How does the system perform if this initial demonstration is varied? For instance, if you provide a slightly different (but still successful) folding sequence, how does that affect the diversity and quality of the 100 generated demonstrations and the performance of the resulting policy?

Answer:

To address this question, we conducted additional experiments on the Fold Tops task by providing two new variations of initial demonstrations:

1）First Variation: The task sequence remains unchanged (i.e., folding the left sleeve to the right collar, the right sleeve to the left collar, then folding the bottom hem to the collar), but the demo points used to locate key regions are altered.

2）Second Variation: The task sequence itself is modified—first folding the left sleeve to the right hem, then the right sleeve to the left hem, followed by folding both hems toward the collar. The demo points for keypoint localization are also different from the original.

For each case, we collected 100 demonstrations and trained the HALO model accordingly. The evaluation results are as follows:

From the evaluation results, models trained on demonstrations collected with different task sequences perform comparably during validation. This suggests that changing the task sequence does not significantly affect the quality or diversity of the demonstrations, nor does it have a notable impact on the final policy performance.

Question2-1：

Could the authors comment on the extensibility of DexGarmentLab to facilitate future research? For example, what is the workflow for tuning the simulation parameters if a new garment is introduced? The paper mentions parameters were "carefully tuned"; is there a systematic procedure for this, or does each new asset require extensive, ad-hoc manual tuning?

Answer:

Our simulation parameters are carefully tuned for garment manipulation and are suitable for most tasks without the need of further adjustment.

For researchers seeking customization, we will provide a markdown document in the GitHub repository detailing garment-related parameters, including official references and effect descriptions. Additionally, key parameters in Env_Config/Garment/Particle(Deformable)_Garment.py are marked with '# important' to help researchers focus on critical settings.

Question2-2：

How readily can the policy and data collection frameworks be adapted to different hardware? How hard would it be to map the 60-DoF action space and the LeapMotion data collection to a new bimanual setup with different kinematics?

Answer:

From an implementation perspective, extending the system to a new bimanual setup involves the following steps:

1）Motion Control in Simulation and Real World:

In simulation, arm motion control can be implemented using libraries such as Curobo, while hand control can be adapted based on our provided codebase.

In the real world, most commercially available arms and hands come with well-encapsulated motion control interfaces, which can be directly used.

2）Customizing Hand Poses:

If the hand has a low number of degrees of freedom (DoF), the joint configurations can be manually specified without relying on teleoperation.

For hands with higher DoF, teleoperation devices such as LeapMotion, VR systems, or manufacturer-provided interfaces are recommended. Several open-source repositories for hand teleoperation are also available and have reached a high level of maturity. For LeapMotion, users can refer to our codebase and modify it to accommodate different dexterous hand configurations.

3）Note:

We do not recommend directly mapping the 60-DoF action space and the LeapMotion-based data collection to a new bimanual setup with different kinematics. This is because not only the action space differs, but the visual observations also vary. To collect data and train policies under a new bimanual setup, it is necessary to rebuild the simulation and real-world environment, and adapt both joint states and action spaces accordingly, in line with the kinematic structure of the new robotic system.

Rebuttal for MdvE

Thanks for your helpful and meaningful reviews! We will provide a detailed response below to the issues in the Weakness and Question sections.

Weakness1:

The tasks are limited in difficulty, and the advantages of dexterous hands over parallel grippers are not clearly demonstrated.

Answer:

Due to current restrictions in simulation technology, certain complex tasks cannot yet be effectively implemented in simulated environments. However, across the existing tasks, dexterous hands still demonstrate clear advantages over parallel grippers in several aspects.

1.Our real-world experiments highlight the limitations of two-finger grippers in handling thin or flat garment regions(such as), where stable grasps often fail due to slipping or missed contacts. Many existing demonstrations rely on grasping the garment’s boundary, but these strategies also often leads to repeated failed attempts. In contrast, our system employing a five-finger dexterous hand demonstrates clear advantages. Owing to the hand’s larger operation range, greater surface contact, and coordinated multi-finger control, it can stably grasp various regions of a garment—not just its edges—and maintain a firm hold throughout the manipulation process. This capability greatly enhances the flexibility and robustness of robotic garment handling, freeing the system from constraints on graspable areas or predefined strategies.

2.Furthermore, the diverse hand postures offered by the dexterous hand allow us to tailor the manipulation strategy to the specific requirements of each task.

• to accomplish a fine-grained task such as putting on a glove, the hand can adopt a pinch posture to precisely grasp the glove edge.
• For stable garment pick-and-place operations, an open/close posture can be used to maximize contact and grip.
• When placing folded clothes, a cradle posture helps preserve the folded structure.
• To smooth wrinkles, the hand can switch to a smoothening pose that maximizes surface coverage.

These example poses are shown in the bottom-left corner of Figure 1 (main paper).

The dexterous hand not only replicates the functionality of two-finger grippers but also enables finer and more robust manipulation of garments.

3.Although current simulation still faces restrictions in precisely modeling highly complex interactions and multi contact between deformable objects and dexterous hands, these limitations would be gradually addressed with the continued advancement of computer graphics and robotics technologies, just like Sapien and Isaac Sim. As a result, the advantages of dexterous hands will become even more prominent in complex tasks such as assistive dressing or knot tying, which require flexible coordination among multiple fingers. We will continue to extend tasks and optimize the simulation framework based on DexGarmentLab, aiming to support a wider range of more complex dexterous-hand garment interaction tasks.

Weakness2:

The paper lacks comparisons with RL and IL methods based on teleoperated demonstrations.

Answer:

We have incorporated all reviewer suggestions related to the benchmark and additionally included four new baselines for comparison: ACT (IL), pi0 (VLA), RDT (VLA), and Eureka (RL+VLM):

Analysis for RL-Based Methods:

We selected Eureka as the representative RL-VLM-combined method and conducted preliminary experiments on several tasks in simulation. While we observed some initial successful cases on simpler tasks such as Hang Coat and Wear Bowlhat, the overall success rate remained low. Additionally, due to the lack of robust parallelization support in the current Isaac Sim environment and the limited effectiveness of reward functions in handling long-horizon tasks like Fling and Fold, this baseline did not yield valid results across all tasks. Nevertheless, we are actively developing a multi-parallel deformable object manipulation environment based on Isaac Lab, and RL-based approaches for deformable manipulation will be a key focus of our future work.

Comparison with teleoperation data:

For teleoperation, We use LeapMotion in simulation and an exoskeleton device in the real world. The results are presented below.

Experimental results indicate that policies trained on teleoperated and automated data achieve comparable performance during evaluation.

Weakness3 & Question1:

Section 3 takes too much space......

Why could the authors claim their GAM as a "generalizable policy"?......

Answer:

Thanks a lot for your quite helpful advice on paper organization and presentation! We believe these suggestions are highly valuable for DexGarmentLab and we will address these problems in camera-ready version to further enhance the quality of the paper.

Question2:

...... explain details on how LeapMotion generates grasping poses, how physical validity is ensured, whether a filtering mechanism is used, and the success rate of demonstration generation.

Answer:

For single specific task category (e.g., Hang Coat), the hand poses required to perform specific actions in specific regions, can generally be treated as consistent. Moreover, the deformable nature of garments allows them to adapt naturally to the dexterous hand's pose during manipulation. Therefore, for each specific task, we first define a set of task-specific hand poses (the number depends on the task; for example, Hang Coat requires both a closed grasping pose for the collar and an open pose for release). These poses are generated via teleoperation using LeapMotion, with physical feasibility considered during generation. In this way, the pose definition process inherently includes a manual filtering step to select the most suitable poses.

These task-specific hand poses are then transferred directly into the data collection process for that task category. That is, during data collection across different garments within the same task, the same predefined hand poses are reused.

To further validate the effectiveness of our automated data collection pipeline, we report the success rates for all tasks in the table below.

Question3:

......How to directly transfer the policy trained on shadow hands to the real robot? Would it require retraining the policy using Psi Bot in the simulator? ......the LeapMotion is designed for shadow hand -- how to generate grasping poses for Psi Bot?

Answer:

To prevent potential misunderstandings, we clarify this point in advance: the real-world experiments presented in the submitted paper are not the result of a policy-level sim-to-real transfer.

Actually, there are two ways using HALO for garment manipulation tasks:

1. transfer the data collection pipeline from simulation to real world, conduct automated data collection and policy training directly in the real world.
2. train the policy in the simulation and transfer the policy to the real world.

In our paper, we adopt way 1 for demonstrate the effectiveness of HALO. In the section Question2, we report the efficiency and success rate of real-world data collection, highlighting the strong sim-to-real performance of GAM.

Sim-to-real transfer from the simulation environment remains an important aspect of our study. To this end, we aligned the settings of both the simulation and real-world environments by using the same hardware setup—Shadow Hand and UR10e—and selected two tasks, Hang Trousers and Wear Hat, for policy-level sim-to-real transfer, which means way 2. The evaluation criteria follow those used in the previous real-world experiments.

So if you want to train the policy in simulation and use a sim-to-real method to transfer policy, you need to align the settings of both the simulation and real-world environments, in the meanwhile, re-collect demonstrations for training. As for psibot setting, because psibot G0-R dexhand only has six freedoms, we can just define each freedom by ourselves to generate target hand pose, or we can also use teleoperation device in real world to get target pose.

Thanks for the detailed rebuttal. My questions regarding the real-world experiment on PsiBot, the use of LeapMotion for data collection, and the comparison with IL/RL methods have been properly addressed. Overall, I believe this is a solid paper. With its rigorous study on dexterous garment manipulation and comprehensive benchmarking, the work makes a valuable contribution to the community. I will maintain my positive rating. The reason I am not increasing the score is that, as an experiment-oriented application paper, it does not convincingly demonstrate the ability to tackle more challenging garment manipulation tasks or show improved generalization compared to prior works. Pi series has demonstrated quite impressive garment manipulation results, though using grippers. According to the real-world comparisons presented in the rebuttal, pi0 performs worse than HALO -- but I think perhaps it is an unfair comparison. Based on the videos available on the project website, the Fold Tops task does not appear particularly difficult or long-horizon. It involves grasping the garment from a table and hanging it—something that should still be feasible with grippers. I cannot understand why pi0 shows worse performance --- they have demonstrated impressive generalization ability and the ability to tackle long-horizon tasks. If the data is enough, this task would pose a significant difficulty.

Rebuttal for b8Ya

Thanks for your helpful and meaningful reviews! We will provide a detailed response below to the issues outlined in the Weakness and Question sections.

Weakness1:

The methodological comparison is limited to Diffusion Policy (DP) and 3D Diffusion Policy (DP3). Including other relevant imitation learning or deformable object manipulation methods would provide a more comprehensive performance benchmark.

Answer：

We have incorporated all reviewer suggestions related to the benchmark and additionally included four new baselines for comparison: ACT (IL), pi0 (VLA), RDT (VLA), and Eureka (RL+VLM). We selected representative tasks for evaluation, including Fling Dress, Fold Trousers, Hang Coat, and Wear Bowlhat in simulation, as well as Fold Tops in real world. The evaluation protocol remains consistent with the original paper:

• For simulation tasks, each task is evaluated over 50 episodes using three different random seeds. We report the success rates as Mean ± Std across all trials.
• For real-world task, we evaluate using 3 distinct garments per category, each tested under 5 different initial deformations. We report the success rates as successful_trials / all_trials.

Analysis For Imitation-Learning-Based Methods:

We collect 100 demonstrations for each simulation task and collect 50 demonstrations for each real-world task, which are used for policy training (DP/DP3/ACT). A notable limitation of their performance lies in the inability to accurately grasp the target region of the garment, as well as the lack of fine-grained control during placement based on the garment’s shape(/state).

Analysis for VLA Methods:

We combine all the demonstrations of simulation and real-world tasks to fine-tune pi0 and RDT model. In terms of final performance, pretrained models such as pi0 and RDT outperform from-scratch approaches like DP, DP3, and ACT. However, compared to HALO, VLA-based models still exhibit clear limitations in accurately perceiving garment shape and state, and in executing precise grasp-and-place actions, resulting in a noticeable performance gap relative to HALO.

Analysis for RL-Based Methods:

We selected Eureka as the representative RL-VLM-combined method and conducted preliminary experiments on several tasks in simulation. While we observed some initial successful cases on simpler tasks such as Hang Coat and Wear Bowlhat, the overall success rate remained low. Additionally, due to the lack of robust parallelization support in the current Isaac Sim environment and the limited effectiveness of reward functions in handling long-horizon tasks like Fling and Fold, this baseline did not yield valid results across all tasks. Nevertheless, we are actively developing a multi-parallel deformable object manipulation environment based on Isaac Lab, and RL-based approaches for deformable manipulation will be a key focus of our future work.

Weakness3:

A more in-depth analysis of the sim-to-real transfer, including specific challenges and the impact of perception noise in the real world, would provide greater insight.

Answer：

To prevent potential misunderstandings, we clarify this point in advance: the real-world experiments presented in the submitted paper are not the result of a policy-level sim-to-real transfer. As can be observed, the real-world setup differs from the simulation environment.

Actually, there are two ways using HALO for garment manipulation tasks:

1. transfer the data collection pipeline from simulation to real world, conduct automated data collection and policy training directly in the real world.
2. train the policy in the simulation and transfer the policy to the real world.

In our paper, we adopt way 1 to demonstrate the effectiveness of HALO. In the section Weakness2, we report the efficiency and success rate of real-world data collection, highlighting the strong sim-to-real performance of GAM.

Sim-to-real transfer from the simulation environment remains an important aspect of our study. To this end, we aligned the settings of both the simulation and real-world environments by using the same hardware setup—Shadow Hand and UR10e—and selected two tasks, Hang Trousers and Wear Hat, for policy-level sim-to-real transfer, which means way 2. The evaluation criteria follow those used in the previous real-world experiments. It is worth noting that sim-to-real performance is more sensitive to point cloud noise. To address the limited precision of the Realsense D435 in this context, we employed a Kinect camera for more accurate point cloud acquisition.

Experimental results show that due to the gap between simulation and the real world, training the policy solely on simulated data leads to a drop in sim-to-real performance. Incorporating a small amount of real-world data into the training process can effectively enhance the policy’s generalization ability.

Weakness2:

The quality of the automatically generated data is not quantitatively compared against traditional data collection methods (e.g., human teleoperation), which could further validate the pipeline's effectiveness. Moreover, the paper emphasizes the efficiency of the automated data collection pipeline. What is the success rate of the pipeline in generating valid and successful task demonstrations that can be used for policy training?

Answer：

We use LeapMotion for teleoperation data collection in simulation and an exoskeleton device for teleoperation in the real world. Three representative tasks are selected for data collection, training, and evaluation. The performance of teleoperated policies is then compared with that of policies trained on autonomously collected data. The results are presented below.

Experimental results indicate that policies trained on teleoperated and automated data achieve comparable performance during evaluation. However, from the perspective of human and time cost, teleoperation-based data collection is significantly more labor-intensive and time-consuming than automated data collection. It is also important to note that our data is not synthetically generated, but collected through real executions in either simulation or the real world, which means the demonstrations are physically realistic. The entire data collection pipeline is enabled by model inference and one-shot demonstrations.

The following table reports the data collection time and success rate across all tasks, demonstrating the effectiveness and efficiency of the automated data collection pipeline.

Weakness4：

The paper would benefit from a clearer discussion on the boundaries of the policy's generalization, including an analysis of specific failure cases on more complex or unusual garments.

Answer：

Based on both simulation and real-world experimental results, we summarize the following points regarding boundary-related performance:

1.For garments with occluded or highly deformed regions, the Garment Affordance Model (GAM) does not always predict manipulation points accurately. This can lead to failed or suboptimal grasps by the dexterous hand, impacting the final performance. Nonetheless, GAM performs reliably on most garments. For challenging cases, an initial unfold or fling action is recommended to expose key regions.

2.The policy struggles with garments that deviate from standard geometries, such as asymmetrical designs (e.g., single-sleeved tops) or heavily adorned costumes, which distort the point cloud and affect both GAM and SADP performance.

Rebuttal for bHU9

Thanks for your helpful and meaningful reviews! We will provide a detailed response below to the issues outlined in the Weakness and Question sections.

Weakness1:

While the authors compare DP and DP3 methods, the study lacks benchmarking against state-of-the-art VLAs (pi0 or RDT) that demonstrate superior performance on folding tasks.

Answer：

We have incorporated all reviewer suggestions related to the benchmark and additionally included four new baselines for comparison: ACT (IL), pi0 (VLA), RDT (VLA), and Eureka (RL+VLM). We selected representative tasks for evaluation, including Fling Dress, Fold Trousers, Hang Coat, and Wear Bowlhat in simulation, as well as Fold Tops in real world. The evaluation protocol remains consistent with the original paper:

• For simulation tasks, each task is evaluated over 50 episodes using three different random seeds. We report the success rates as Mean ± Std across all trials.
• For real-world task, we evaluate using 3 distinct garments per category, each tested under 5 different initial deformations. We report the success rates as successful_trials / all_trials.

Analysis For Imitation-Learning-Based Methods:

We collect 100 demonstrations for each simulation task and collect 50 demonstrations for each real-world task, which are used for policy training (DP/DP3/ACT). A notable limitation of their performance lies in the inability to accurately grasp the target region of the garment, as well as the lack of fine-grained control during placement based on the garment’s shape(/state).

Analysis for VLA Methods:

We combine all the demonstrations of simulation and real-world tasks to fine-tune pi0 and RDT model. In terms of final performance, pretrained models such as pi0 and RDT outperform from-scratch approaches like DP, DP3, and ACT. However, compared to HALO, VLA-based models still exhibit clear limitations in accurately perceiving garment shape and state, and in executing precise grasp-and-place actions, resulting in a noticeable performance gap relative to HALO.

Analysis for RL-Based Methods:

We selected Eureka as the representative RL-VLM-combined method and conducted preliminary experiments on several tasks in simulation. While we observed some initial successful cases on simpler tasks such as Hang Coat and Wear Bowlhat, the overall success rate remained low. Additionally, due to the lack of robust parallelization support in the current Isaac Sim environment and the limited effectiveness of reward functions in handling long-horizon tasks like Fling and Fold, this baseline did not yield valid results across all tasks. Nevertheless, we are actively developing a multi-parallel deformable object manipulation environment based on Isaac Lab, and RL-based approaches for deformable manipulation will be a key focus of our future work.

Weakness2：

While demonstrating performance gains on this benchmark DexGarmentLab, the claimed generalizability of the proposed method requires validation across diverse simulation environments. Additional cross-platform testing would strengthen the evidence for its universal applicability.

Answer:

Garment simulation in our work is built upon NVIDIA PhysX's Position-Based Dynamics (PBD) and Finite-Element Method(FEM) framework, which currently offers the most realistic and stable performance for garment simulation. Alternative simulators such as SofGym or PyBullet only support PBD and generally exhibit inferior performance in soft body dynamics. Moreover, they also lack support for fine-grained interactions between dexterous hands and garments. Therefore, in terms of garment simulation fidelity and interaction capability, Isaac Sim remains the most suitable and advanced platform available to date, and it's not feasible to perform garment manipulation tasks in simulators outside of Isaac.

However, regardless of the simulation platform used, real-world validation remains essential. Therefore, we directly consider the real-world environment as the final platform. Our real-world garment manipulation experiments demonstrate the effectiveness of our policy in physical settings, further supporting its general applicability.

What's more, for cross-platform testing, we selected three rigid-object manipulation tasks—covering both single-arm and dual-arm settings—from three different benchmarks: ManiSkill, RLBench, and Robosuite, each built on a distinct simulation backend. We used DP and pi0 as baselines for comparison, and the success rates were obtained from existing publications.

The results show that HALO performs on par with, or even better than, pi0. This further supports the universal applicability of the HALO strategy across a wide range of tasks, including both rigid and deformable object manipulation.

Weakness3:

Could the authors clarify whether this is a single-task or multi-task setting?

Answer:

If you are referring to whether a single model is trained for a single task or multiple tasks, this is a single-task setting. However, there are also some plausible ways to transfer it to multi-task setting, as discussed in [1,2].

If this is not what you meant, please feel free to let us know and we can have a further discussion.

ref:

[1]Hao C, Lin K, Luo S, et al. Language-guided manipulation with diffusion policies and constrained inpainting[J]. arXiv preprint arXiv:2406.09767, 2024.

[2]Ma X, Patidar S, Haughton I, et al. Hierarchical diffusion policy for kinematics-aware multi-task robotic manipulation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024: 18081-18090.

Weakness4:

Please specify the computational costs (time overhead) for both training and inference phases of the algorithm.

Answer:

The training of GAM is performed on a single RTX 4090 GPU, requiring approximately 22 GB of GPU memory when using a batch size of 64. Depending on the dataset size, the training process takes around 12 hours to complete. GAM requires only about 1 GB of GPU memory during the inference stage.

The SADP module is trained on a single NVIDIA A800 GPU, with a memory usage of approximately 75 GB when using a batch size of 200. The training process spans approximately 16 hours for 3000 epochs. However, earlier checkpoints can be selected for validation depending on the use case.

HALO, which integrates both GAM and SADP, requires about 3 GB of GPU memory during the inference stage. When running HALO inference in the Isaac Sim environment, the total GPU memory footprint is approximately 15 GB.

Rebuttal for JjAQ

Thanks for your helpful and meaningful reviews! We will provide a detailed response below to the issues outlined in the Weakness and Question sections.

Weakness1:

In the real-world experiments, particularly on the Fold Tops and Hang Tops tasks, the garments used appear to have relatively simple and uniform colors. It is unclear how well the proposed method would generalize to garments with more complex patterns, prints, or highly varied textures, which are common in real-world scenarios.

Answer:

For our policy training and validation, we only utilized the XYZ positional coordinates of the point clouds without incorporating color attributes. This is because, for garments of the same category, color, material, and appearance can vary significantly, while their underlying geometric structure remains largely consistent. Introducing color information could potentially hinder generalization performance by introducing unnecessary variance, as garment colors in the real world are highly diverse and will further be significantly affected by variations in viewpoint and lighting. Since HALO relies solely on positional information of point cloud, it is theoretically robust to variations in color and texture.

To further validate this, we conducted real-world experiments (Fold Tops) using three distinct types of garments featuring prints, textures and patterns : (1) those with highly saturated colors, (2) those with complex textures (e.g., black-and-white patterns, color-blocking, or printed designs), and (3) those featuring raised elements (e.g., embroidery or appliqués). For each type, we performed five folding trials, resulting in a total of 15 experiments. The overall success rate remained at 85% (13/15), confirming the method's robustness.

However, we acknowledge a limitation: the policy may underperform when handling garments with atypical structures, such as asymmetrical designs (e.g., single-sleeved tops) or deviations from conventional garment geometry. While such cases are relatively rare, they represent an important direction for future research.

Weakness2:

In Table 3, please clarify what metric is being reported (e.g., is it the number of successful trials out of total attempts, or another measure?).

Answer:

For real-world evaluation, in every task, we have 3 distinct garments, each with 5 initial deformations and positions, resulting in a total of 15 experiments. The metric is the number of successful trials out of total 15 experiments.

Question1:

Your method uses five-finger dexterous hands instead of common two-finger grippers. What specific advantages do five fingers provide for garment manipulation tasks in your experiments?

Answer:

1.Our experimental results, particularly those real-world results, reveal significant limitations in two-finger grippers when handling thin or flat regions of garments. In such cases, the gripper frequently fails to achieve a stable grasp, with garments either slipping during manipulation or not being grasped at all. Many existing demonstrations involve garment folding with two-finger grippers rely on grasping the garment’s boundary, but these strategies also often leads to repeated failed attempts. In contrast, our system employing a five-finger dexterous hand demonstrates clear advantages. Owing to the hand’s larger operation range, greater surface contact, and coordinated multi-finger control, it can stably grasp various regions of a garment—not just its edges—and maintain a firm hold throughout the manipulation process. This capability greatly enhances the flexibility and robustness of robotic garment handling, freeing the system from constraints on graspable areas or predefined strategies.

2.Furthermore, the diverse hand postures offered by the dexterous hand allow us to tailor the manipulation strategy to the specific requirements of each task.

• to accomplish a fine-grained task such as putting on a glove, the hand can adopt a pinch posture to precisely grasp the glove edge.
• For stable garment pick-and-place operations, an open/close posture can be used to maximize contact and grip.
• When placing folded clothes, a cradle posture helps preserve the folded structure.
• To smooth wrinkles, the hand can switch to a smoothening pose that maximizes surface coverage.

These example poses are shown in the bottom-left corner of Figure 1 (main paper).

The dexterous hand not only replicates the functionality of two-finger grippers but also enables finer and more robust manipulation of garments.

3.Although current simulation still faces significant challenges in accurately modeling interactions between deformable objects and dexterous hands—limiting its ability to simulate complex tasks—these limitations are expected to be gradually addressed with the continued advancement of computer graphics and robotics technologies. As a result, the advantages of dexterous hands will become even more prominent in complex tasks such as assistive dressing or knot tying, which require flexible coordination among multiple fingers. We will continue to extend tasks and optimize the simulation framework based on DexGarmentLab, aiming to support a wider range of more complex dexterous-hand garment interaction tasks.

Question2:

Are the grasp points on the garment defined in 2D (on the surface) or 3D coordinates?

Answer:

The grasp points are defined directly in 3D coordinates .

Question3:

Since garments are flexible, did you consider grasping force or pressure to avoid slippage or fabric damage?

Answer:

This is a highly promising research direction. Current studies in the field are still exploring how to enable robots to grasp garments both accurately and stably. As a result, the most straightforward strategy has been to apply maximum force during grasping to ensure success. However, beyond grasp success, it is equally important to detect garment slippage in real time and to avoid potential fabric damage. This requires considering grasping force or pressure, and leveraging tactile sensing during manipulation. We view this as an important and promising direction for future work. Relevant developments are currently underway in simulation, and we will continue to update and expand this line of research in subsequent work.