CADGrasp: Learning Contact and Collision Aware General Dexterous Grasping in Cluttered Scenes

We sincerely appreciate your constructive feedback and are excited that you find our work "effective", "interpretable", and "solid"! We want to clarify your concerns and questions in the following.

Q1: As mentioned by the authors, the optimization is time-consuming. What is the inference speed of the method?

A1: We thank you for your comment. We provide the following clarifications and will include a discussion of this part in the final version.

Thank you for raising this important point regarding the inference time of our method. In our experiments, the average inference runtime per sample on a single NVIDIA RTX 4090 GPU is summarized in Table 2. While our approach currently does not achieve real-time performance, we emphasize that these results are obtained with unoptimized code and no engineering-level acceleration. There is significant potential to reduce inference time using more efficient sampling strategies [1][2] and further parallelization. Importantly, even in its current implementation, our method achieves faster inference compared to prior state-of-the-art two-stage approaches (see Table 3), while also delivering strong grasping performance in challenging real-world scenarios. We will elaborate on the potential for further speedup and optimization in the final version.

Table 1: Average runtime per stage of our method (NVIDIA RTX 4090)

Table 2: Comparison with existing two-stage methods

Q2: Voxel resolution can largely limit the precision of the IBS surface prediction. And computational cost grows cubically when we increase the resolution.

A2: We appreciate your comment and agree that voxel resolution is a key hyperparameter that impacts both the precision of IBS surface prediction and the associated computational cost. In our work, we select a voxel size of 5mm to balance accuracy and efficiency. This resolution is chosen because it is sufficient to capture contact surfaces for most objects in practical applications, while still maintaining manageable memory and runtime requirements (see Table 1 in A1 for runtime details). To further investigate this trade-off, we conducted ablation experiments on the GraspNet-1B Loose setting, as shown in Table 3 below. Finer resolutions lead to substantially higher memory consumption, while offering only marginal improvements in success rate. Conversely, coarser resolutions reduce memory usage but result in clear performance drops. We will add a more detailed discussion of this design choice in the final version of the paper.

Table 3: Ablation Study on Voxel Resolution

Q3: Do you use simple motion planning to execute the pre-grasp?

A3: We thank the reviewer for this question. In our experiments, all methods employ the same motion planning algorithm to execute both pre-grasp and grasp poses. Developing goal-conditioned reinforcement learning algorithms for planning based on grasp poses is a promising direction for future work.

Q4: The paper lacks a deeper theoretical analysis or visualization to explain why contact and collision conditioning improve grasp synthesis.

A4: We thank you for pointing out the importance of deeper theoretical analysis and visualization of our sparse IBS representation and the effect of contact and collision conditioning. We provide further clarification from two perspectives:

• Compared to one-stage direct grasp pose prediction: Directly mapping 3D point cloud observations to grasp poses is challenging due to the highly non-linear nature of the mapping from 3D point cloud space to pose space, resulting in low sample efficiency[3][4]. Furthermore, such representations are usually embodiment-specific, limiting cross-embodiment generalization. Our sparse IBS representation is spatially aligned with the 3D point cloud, improving sample efficiency and enabling our two-stage approach to achieve cross-embodiment generalization, as shown in our experiments.
• Compared to two-stage methods that optimize from predicted contact map: Approaches using contact map as intermediate representations generally assume that complete point cloud observations are available, which is unrealistic in real-world settings with partial observation. Our proposed sparse IBS representation, in contrast, does not rely on this assumption. As shown in both the quantitative experiments in the manuscript and qualitative visualizations in the supplementary material, our method achieves robust dexterous grasp pose prediction using only single-view partial point cloud inputs, making it suitable for complex, real-world tasks.

We appreciate the suggestion and agree that further theoretical insights would strengthen our work. We will include additional theoretical analysis and visualizations in the final version.

Q5: "Additionally, the current second-stage optimization is time-consuming due to the use of the DDPM for sampling IBS." In my understanding, the second stage optimizes the hand pose given a fixed IBS surface. Why is the DDPM used for sampling IBS in the second-stage optimization?

A5: We apologize for the confusion caused by this statement. This was a misstatement in the manuscript, and the intended statement should be: “The Diffusion-based IBS prediction in the first stage is time-consuming.” Thank you for catching this error. We will correct this in the final version.

Q6: Small and thin objects are particularly subject to the voxelization.

A6: We appreciate your comment. As shown in Table 3 of A2, increasing the voxel resolution yields only marginal improvements in grasp success rate. However, we agree that object size has a substantial impact on grasping performance. In Table 4 below, we report the success rate of our method on the GraspNet-1B (Loose) setting as a function of object volume, along with the proportion of objects in each size category. The results indicate that, within a reasonable range, the grasp success rate for small objects is indeed lower compared to larger objects. Combined with the findings in Table 3 of A2, we attribute this to the inherently higher demands for precision and strategy when grasping small and thin objects, rather than to insufficient voxel resolution. We will add further discussion of this point in the final version.

Table 4: Success Rate on Small and Thin Objects

[1] Salimans T, et al. Progressive distillation for fast sampling of diffusion models. ICLR 2022.

[2] Song Y, et al. Consistency models. ICML 2023.

[3] Haoming Li, et al. Contact2Grasp: 3D Grasp Synthesis via Hand-Object Contact Constraint. IJCAI 2023.

[4] Hao-Shu Fang, et al. AnyDexGrasp: Learning General Dexterous Grasping for Different Hands with Human-level Efficiency. ArXiv 2025.

We sincerely appreciate your constructive feedback and are excited that you find our work "novel", "addressing a critical limitation in existing dexterous grasping methods", and "technically sound"! We want to clarify your concerns and questions in the following.

Q1: The ablation study is limited. It only removes individual components (e.g., IBS ranking or decompose), without evaluating combinations, making it difficult to assess interaction effects between modules.

A1: We thank you for your suggestions regarding more comprehensive ablation studies. In response, we conducted additional experiments to analyze the interaction effects between different modules and the contribution of each design choice (see Table 1 below). The results show that decomposed modeling of thumb and other finger contacts with the object leads to a substantial improvement in grasp performance, confirming that the thumb plays a crucial role in dexterous grasping [1]. Additionally, both the IBS ranking and grasp pose ranking modules further enhance overall grasp performance. We will update Table 3 in the final version and provide further discussion in the manuscript.

Table 1: Ablation Study on Interaction Effects and Design Choices

Q2: Real-world evaluations in Table 3 show only marginal improvement over baselines, with inconsistent numbers of trials across methods, which affects the fairness and reliability of the comparison.

A2: We apologize for any confusion. As described in the Real-world Setup section of the manuscript, we follow the cleared-table grasping evaluation protocol used in GS-Net [2]. For each scene, the policy continues grasping until two consecutive failures occur. As some baselines tend to reach this failure criterion earlier, the total number of grasp attempts appears inconsistent across methods in Table 3. This actually reflects the greater robustness of our method. We will clarify this evaluation protocol and the reason for varying trial numbers in more detail in the final version for better clarity.

Q3: The paper lacks a deeper theoretical analysis or visualization to explain why contact and collision conditioning improve grasp synthesis.

A3: We thank you for pointing out the importance of deeper theoretical analysis and visualization of our sparse IBS representation and the effect of contact and collision conditioning. We provide further clarification from two perspectives:

• Compared to one-stage direct grasp pose prediction: Directly mapping 3D point cloud observations to grasp poses is challenging due to the highly non-linear nature of the mapping from 3D point cloud space to pose space, resulting in low sample efficiency[3][4]. Furthermore, such representations are usually embodiment-specific, limiting cross-embodiment generalization. Our sparse IBS representation is spatially aligned with the 3D point cloud, improving sample efficiency and enabling our two-stage approach to achieve cross-embodiment generalization, as shown in our experiments.
• Compared to two-stage methods that optimize from predicted contact map: Approaches using the contact map as intermediate representations generally assume that complete point cloud observations are available, which is unrealistic in real-world settings with partial observation. Our proposed sparse IBS representation, in contrast, does not rely on this assumption. As shown in both the quantitative experiments in the manuscript and qualitative visualizations in the supplementary material, our method achieves robust dexterous grasp pose prediction using only single-view partial point cloud inputs, making it suitable for complex, real-world tasks.

We appreciate the suggestion and agree that further theoretical insights would strengthen our work. We will include additional theoretical analysis and visualizations in the final version.

Q4: How robust is CADGrasp to noisy supervision during training, especially in contact or collision labels?

A4: We appreciate the reviewer’s question. In our training process, the contact and collision labels are not manually annotated but are generated automatically in simulation. Specifically, force-closure grasps are optimized in simulation, and both the IBS surface and contact ground-truth are computed programmatically. This approach ensures label consistency and reduces the likelihood of significant supervision noise.

Q5: Is CADGrasp feasible without object meshes during inference? If not, how practical is this in real-world grasping scenarios?

A5: Unlike contact map-based two-stage methods, our model does not require object CAD models or complete point cloud observations as input, thanks to our proposed sparse IBS representation. CADGrasp only relies on single-view partial point clouds, enabling dexterous grasping in complex real-world scenarios using partial observations. This design ensures that our method can be directly deployed in real-world environments without any special modifications or assumptions.

[1] Thomas Feix, et al. The grasp taxonomy of human grasp types. IEEE Transactions on human-machine systems, 2015.

[2] Chenxi Wang, et al. Graspness discovery in clutters for fast and accurate grasp detection. ICCV 2021.

[3] Haoming Li, et al. Contact2Grasp: 3D Grasp Synthesis via Hand-Object Contact Constraint. IJCAI 2023.

[4] Hao-Shu Fang, et al. AnyDexGrasp: Learning General Dexterous Grasping for Different Hands with Human-level Efficiency. ArXiv 2025.

We sincerely appreciate your constructive feedback and are excited that you find our work "well-motivated", "competitive", and "clear"! We want to clarify your concerns and questions in the following.

Q1: All success-rate tables should provide variance across random seed.

A1: We thank you for your comment, and we agree that variance is important for evaluating the robustness of our method. As suggested, we conducted each experiment with 20 different random seeds and report the success rates together with the corresponding standard deviations in Table 1 below. The observed standard deviations are consistently small across different datasets and scene types, which indicates that our method exhibits stable performance across random initializations. Note that the reported variance captures both the inherent randomness of our method and the variability introduced by the simulation environment, further demonstrating the robustness of our approach.

Table 1: Success rates of CADGrasp on different datasets and scene types

Q2: The diffusion sampling plus gradient-based optimisation could be slow; yet runtimes per grasp are absent.

A2: We thank you for your comment. We provide the following clarifications and will include a discussion of this part in the final version.

We performed 50 inference runs on a single NVIDIA RTX 4090 GPU and report the average runtime for each stage of our method in Table 2 below. Although our method does not meet real-time requirements, we believe this is reasonable given its high performance and the potential of the proposed novel representation for cross-embodiment generalization. We also note that these reported runtimes are obtained without any engineering-level optimization—real-time feasibility can be further improved by using faster samplers [1][2] and by increasing parallelization in the second stage. Furthermore, as shown in Table 3, even without engineering acceleration, our method outperforms the speed of existing SOTA two-stage methods and provides effective representations for complex real-world grasping tasks.

Table 2: Average runtime per stage of our method (NVIDIA RTX 4090)

Table 3: Comparison with existing two-stage methods

Q3: Comparison to some recent contact-based two-stage graspers in clutter may be omitted, such as UniDexGrasp++ and UniDexGrasp in cluttered settings.

A3: We thank you for your comment. We would like to clarify the following points:

• We would like to clarify that UniDexGrasp and UniDexGrasp++ focus on single-object grasping and do not support grasping in cluttered scenes. Furthermore, these approaches have not been extended to the dexterous grasping in the real-world setting that is the focus of our work.

• Existing contact-based two-stage methods, which first predict a contact map and then optimize for a grasp pose, generally rely on two strong assumptions: (1) access to full point clouds, and (2) single-object grasping. Both assumptions are unrealistic in the context of general real-world grasping in cluttered scenes. Nevertheless, following the modifications suggested by DexGraspNet 2.0, we adapted these two-stage baselines (e.g., GraspTTA in Table 1 of the manuscript) for a fair comparison under our proposed setting. Our method still outperforms these baselines in both performance and speed. Please refer to Table 2 in A2 for detailed runtime results. This advantage largely stems from our proposed sparse IBS representation, which does not require a complete scene point cloud.

Q4: Some typos are found. For example, in line 296, the "." should be ",".

A4: We appreciate your careful reading of our manuscript. We will correct the typos in the final version.

Q5: Will the implementation and trained models be open-sourced upon acceptance, and if so, under which licence?

A5: Yes, we plan to open-source the implementation and trained models upon acceptance to facilitate reproducibility and further research in this area. They will be released under the MIT License.

[1] Salimans T, et al. Progressive distillation for fast sampling of diffusion models. ICLR 2022.

[2] Song Y, et al. Consistency models. ICML 2023.

Dear Reviewer #H1AQ,

Thank you for your careful review. We sincerely appreciate your feedback and the time you dedicated to evaluating our work.

Should you have any further questions or require additional clarification, please do not hesitate to contact us.

Best regards,

The Authors

We sincerely appreciate your constructive feedback and are excited that you find our work "novel", "strong experimental validation", and "technical soundness"! We want to clarify your concerns and questions in the following.

Q1: Limited Theoretical Analysis: The paper lacks a theoretical justification for why sparse IBS should be more effective than alternative representations. While empirical results are strong, a deeper analysis of the representation's properties would strengthen the contribution.

A1: We thank you for pointing out the importance of deeper theoretical analysis. We provide further clarification from two perspectives:

• Compared to one-stage direct grasp pose prediction: Directly mapping 3D point cloud observations to grasp poses is challenging due to the highly non-linear nature of the mapping from 3D point cloud space to pose space, resulting in low sample efficiency[1][2]. Furthermore, such representations are usually embodiment-specific, limiting cross-embodiment generalization. Our sparse IBS representation is spatially aligned with the 3D point cloud, improving sample efficiency and enabling our two-stage approach to achieve cross-embodiment generalization, as shown in our experiments.
• Compared to two-stage methods that optimize from predicted contact map: Approaches using the contact map as intermediate representations generally assume that complete point cloud observations are available, which is unrealistic in real-world settings with partial observation. Our proposed sparse IBS representation, in contrast, does not rely on this assumption. As shown in both the quantitative experiments in the manuscript and qualitative visualizations in the supplementary material, our method achieves robust dexterous grasp pose prediction using only single-view partial point cloud inputs, making it suitable for complex, real-world tasks.

We appreciate your suggestions and agree that further theoretical insights would strengthen our work. We will include additional theoretical analysis and visualizations in the final version.

Q2: Computational Efficiency Concerns: The method requires DDPM sampling for IBS generation followed by gradient-based optimization, which the authors acknowledge is time-consuming. The paper doesn't provide detailed timing comparisons or discuss scalability to real-time applications.

A2: We appreciate your comment. We provide the following clarifications and will include a discussion of this in the final version.

• Runtimes: We thank the reviewers for highlighting the importance of computational efficiency. We performed 50 inference runs on a single NVIDIA RTX 4090 GPU and report the average runtime for each stage of our method in Table 1 below. Though our method does not meet real-time requirements, we believe this is reasonable given its high performance and the potential of the proposed novel representation for cross-embodiment generalization. We would also like to note that the reported runtime is without engineering-level optimization—real-time feasibility can be further improved by using faster samplers[3][4] and increasing parallelization in the second stage.

  Table 1: Average runtime per stage of our method (NVIDIA RTX 4090)

  Module	Stage	Average Runtime (seconds)
  Wrist Pose Estimation	1	1.38
  Diffusion-based IBS Generation	1	1.39
  IBS Ranking	1	0.71
  Grasp Pose Optimization & Ranking	2	3.03
  Total	-	6.51

• Comparison with Baselines: We also compare overall runtime with mainstream two-stage methods in Table 2. Our approach not only provides effective representations for complex real-world grasping but also achieves better runtime than baselines, even without inference-time optimizations.

  Table 2: Comparison with existing two-stage methods

  Method	Intermediate Representation	Partial Observation	Cluttered Scene	Runtime (seconds)
  GraspTTA	Contact Map	✗	✗	43.23
  UniGrasp	3 Contact Points	✗	✗	9.33
  GenDexGrasp	Contact Map	✗	✗	16.42
  CADGrasp (Ours)	Sparse IBS	✓	✓	6.51

Q3: Incomplete Ablation Studies: While the paper includes ablation studies, some design choices lack sufficient justification. For example, the choice of three channels for sparse IBS (surface occupancy, thumb contact, other finger contact) could benefit from more systematic analysis.

A3: We thank you for your suggestions regarding more comprehensive ablation studies. In response, we conducted additional experiments to analyze the interaction effects between different modules and the contribution of each design choice (see Table 3 below). The results show that decomposed modeling of thumb and other finger contacts with the object leads to a substantial improvement in grasp performance, confirming that the thumb plays a crucial role in dexterous grasping [5]. Additionally, both the IBS ranking and grasp pose ranking modules further enhance overall grasp performance. We will update Table 3 in the final version and provide further discussion in the manuscript.

Table 3: Ablation Study on Interaction Effects and Design Choices

Q4: Generalization Boundaries: While cross-embodiment results are impressive, what are the limits of this generalization? How sensitive is the method to significant differences in hand morphology, scale, or degrees of freedom? Could you provide an analysis of failure cases where the representation breaks down?

A4: We thank you for this insightful question. We agree that understanding the boundaries of cross-embodiment generalization is an important direction for future work. While our experiments demonstrate that the proposed sparse IBS representation enables a degree of cross-embodiment generalization, we do not over-claim this capability in the paper. As currently designed, the sparse IBS generation stage does not condition on the specific hand morphology, which means that generalization is primarily observed between dexterous hands with similar shape, scale, and degrees of freedom. For example, if the target hand is much smaller, when the fingertips satisfy the predicted contact constraints, the palm will collide with the object due to the shorter fingers. Similarly, hands with fewer degrees of freedom or very different structures cannot achieve the predicted contacts and will face kinematic infeasibility. Incorporating hand morphology and kinematics as part of the conditioning input for sparse IBS generation could potentially address these limitations and enable broader generalization—a direction we plan to explore in future work. We will discuss these boundaries and potential improvements more explicitly in the final version.

[1] Haoming Li, et al. Contact2Grasp: 3D Grasp Synthesis via Hand-Object Contact Constraint. IJCAI 2023.

[2] Hao-Shu Fang, et al. AnyDexGrasp: Learning General Dexterous Grasping for Different Hands with Human-level Efficiency. ArXiv 2025.

[3] Salimans T, et al. Progressive distillation for fast sampling of diffusion models. ICLR 2022.

[4] Song Y, et al. Consistency models. ICML 2023.

[5] Thomas Feix, et al. The grasp taxonomy of human grasp types. IEEE Transactions on human-machine systems, 2015.