Contact Map Transfer with Conditional Diffusion Model for Generalizable Dexterous Grasp Generation

Dear reviewer jrqf, thank you for your thoughtful review and constructive suggestions. In the following, we respond to your questions in detail one by one:

---

Q1&Q2: Robustness to object point cloud noise and real-world robot experiments.

A1&A2: We appreciate the reviewer’s important comments. Following your suggestion, we performed additional robustness evaluations in both simulation and real-world settings. It is worth noting that our model was already designed for robustness, having been trained with standard point cloud data augmentation techniques (e.g., random sampling and Gaussian noise). For the new simulation analysis, we followed [A] to randomly mask out 30% of the object points to simulate irregular and incomplete point cloud distributions. As evidenced by the table below, our grasp transfer model's performance remains stable, with no significant degradation in quantitative grasp quality metrics under these conditions.

Table A: Quantitative Results of Robustness to Simulated Point Cloud Noise

Although the grasp success rate saw a minor decrease from 74.14% to 69.34%, we attribute this drop primarily to missing points in critical object regions (such as a mug's handle or a bowl's rim), which hinders accurate contact computation during the final optimization stage. For our real-world experiments, object point clouds were reconstructed from a multi-view camera setup (one ZED, two RealSense). These point clouds naturally contained realistic noise and missing areas owing to partial occlusions and limited sensor coverage. Feeding these imperfect point clouds directly into our grasp transfer pipeline, our method achieved an average success rate of 70% on a dexterous hand, underscoring its strong robustness to real-world artifacts. Due to page constraints, please see our response to Q2 of reviewer UCkK for a more detailed account of these experiments.

[A] Wang, Tao, et al. Sparse convolutional networks for surface reconstruction from noisy point clouds. WACV 2024.

---

Q3: More results with generative evaluation metrics.

A3: We thank the reviewer for this valuable suggestion. Following your recommendation, we have adopted three generative evaluation metrics from SemGrasp and Afforddexgrasp to more comprehensively assess the alignment between the generated grasps and their corresponding tasks. P-FID measures the distributional similarity between generated and ground-truth grasps with the same intention. We leverage the pre-trained feature extractor from [75] for this evaluation. Chamfer Distance (CD) quantifies the average point-wise discrepancy between the generated hand point cloud and the ground-truth point cloud. R-Precision evaluates the semantic alignment between the language instructions and the generated grasps. Specifically, we render the generated hand-object interactions into images and use pre-trained image and text encoders from [75] to measure how well the visual output matches the textual description. For each generated grasp image, we create a candidate pool composed of its ground-truth instruction and 31 randomly selected samples. We then compute the cosine distances between the image features and the language features for every sample in the pool and then rank them accordingly. We report the Top-k retrieval accuracy (for k=1, 3, 5), which is the percentage of trials where the ground-truth instruction is successfully ranked among the top k candidates.

We conducted these evaluations on the unseen categories benchmark to assess generalization. As shown in the below table, our method significantly outperforms all baselines across these generative metrics, confirming its superior ability to generalize and maintain semantic consistency.

Table B: Generative Evaluation Metrics on Unseen Categories

---

Q4: How is the template hand selected, and how robust is our model to this choice?

A4: We thank the reviewer for raising this important point. We apologize for the confusion and would like to clarify that in all our main experiments, we consistently use the template hand provided by the CapGrasp dataset. Importantly, for fair comparison, the same template hand is also used by other transfer-based baselines such as Tink. We choose the CapGrasp-defined template hand because it allows us to leverage the provided point-wise ground-truth contact maps on various novel objects, enabling rigorous quantitative evaluation of the transferred contact maps. Additionally, CapGrasp offers ground-truth grasps for different tasks, which further supports task-level evaluation of the transferred grasps using metrics such as Contact Error (Cont. Err.) and Chamfer Distance (CD). However, we emphasize that our method is not limited to this specific template hand. To verify the robustness of our framework to the choice of template hand, we conducted additional experiments using alternative template hands generated by two different methods: DexGraspNet (an analytical method) and ContactGen (a generative method). We sampled a diverse set of grasps around the shape template using these methods, then executed them in simulation and selected the top-10 grasps based on success rate as template hands for grasp transfer. We then evaluated the transferred grasps using three label-free quality metrics.

Table C: Performance of Transferring Grasps from Different Sources

The results demonstrate that our model maintains strong performance across different template hand sources, indicating that our framework is robust to the selection and generation process of the template hand. This highlights the generalizability and applicability of our dexterous grasp transfer framework.

Dear reviewer EFeh, we appreciate the time and effort you dedicated to reviewing our work. Our detailed, point-by-point responses are provided below:

---

Q1: How is the shape template selected in the experiments, and how does varying the templates influence the performance?

A1: We thank the reviewer for the thoughtful question and apologize for any confusion. In our experiments, for each object category, one shape template is randomly selected from the CapGrasp dataset and used consistently across all experiments. The same template is also applied to other transfer-based baselines such as Tink to ensure a fair comparison. During training, the template-target object pairs are randomly sampled to encourage diversity, and we further employ standard point cloud augmentation techniques such as random point masking out and additive Gaussian noise to improve the model’s robustness to shape and distribution variations. To further evaluate our model’s sensitivity to the choice of shape template, we conducted additional experiments on 8 unseen object categories using varying template shapes. Specifically, for each target object during testing, we randomly selected a different shape template (from the same category but not the same instance) from CapGrasp, and directly applied our trained model for grasp transfer without any re-training. As shown in the Table, our model maintains strong performance across different template-target combinations, demonstrating that the proposed grasp transfer framework is not restricted to a fixed shape template and is robust to shape variation between the template and the target object.

Table A: Robustness to Template Selection Strategy on Unseen Categories

---

Q2: Additional comparison with generative models + templates.

A2: We sincerely thank the reviewer for the insightful suggestion to strengthen our experimental comparisons. Following your suggestion, we modified the recent generative model DexGYS [32] to incorporate template information. Specifically, we introduced a new template branch in the first-stage module of DexGYS, which takes both the template object point cloud and the corresponding hand point cloud as inputs. We extracted template-relevant features using a point cloud backbone and concatenated them with the original target object and textual task features to condition the grasp generation process. We re-trained this customized DexGYS+Template model from scratch on the same training dataset used for our method.

Table B: Comparison With Baseline With or Without Template on Unseen Categories

As shown in the table, incorporating the template grasp improves the performance of DexGYS to some extent, especially in terms of grasp quality and task relevance. However, the performance of this variant remains consistently inferior to our proposed model across all 8 unseen object categories. This result suggests that simply using the template grasp as an additional condition, without explicitly modeling the geometric similarity between the template and the target object, is insufficient for generalizing to novel categories. Furthermore, we qualitatively observe that the improvements from template conditioning in DexGYS are not always consistent, particularly when the template and target objects differ significantly in shape. These results demonstrate the effectiveness of our proposed transfer framework, which explicitly learns task-conditioned template-object geometric similarity and enables robust grasp transfer across diverse objects. Thanks again for this insightful comment to further enhance our experiments. We will include this new experimental comparison and the corresponding quantitative and qualitative results in the revised manuscript.

---

Q3: More detailed analysis on real-world noise and different hands.

A3: We thank the reviewer for this important question. In our main paper, we have already demonstrated that our model can be applied to different five-fingered dexterous hands (e.g., the ShadowHand and a human hand model Mano), which showcases the generalization capability of our method across various hand morphologies. To further evaluate the effectiveness of our method under real-world sensor noise and more types of robotic hands, we conducted real-world dexterous grasping experiments on a self-developed humanoid robotic platform equipped with an Inspire dexterous hand mounted at the end of the robot arm. Our visual perception setup used a head-mounted ZED camera and two RealSense cameras on the left and right sides of the robot to capture multi-view RGB images. Then, we leveraged Grounding-DINO to segment the target object from the captured views according to language instruction, and employed the recent 3D foundation model VGGT to reconstruct the target’s point cloud from these segmented views. Without any re-training or fine-tuning of our diffusion model, we directly used the reconstructed point cloud to predict the dexterous grasp configuration. To execute the grasp, we adopted the protocol from [32]. First, the robot arm positioned the hand's root at the predicted 6-DOF pose. The Inspire hand then actuated its joints according to the predicted grasp poses. Due to limited time, our evaluation covered four novel object categories (Bowl, Wineglass, Cup, and Headphones) across five different task specifications. We attempted each grasp five times per object to record the average success rate. A grasp was considered successful if the object was lifted over 30 cm from the table and held securely for five seconds. Our method achieved an overall success rate of 70% across these categories (Bowl: 60%, Wineglass: 80%, Cup: 80%, Headphones: 60%), which highlights its robustness to real-world sensor noise and generalizability to different types of robotic hands. As mentioned in our future work, exploring the framework’s generalization to dexterous hands with varying numbers of fingers and morphological structures is an interesting and open research direction we plan to pursue. Additionally, we will incorporate a thorough description of our robotic platform and the results from the real-world experiments into the revised manuscript. We thank you again for your constructive suggestions.

---

Q4: Possible limitation for practical application in robot platforms with limited computational resources.

A4: We thank the reviewer for this very important and practical comment. We agree that deploying computationally intensive models on resource-constrained robots is a critical challenge for real-world application, and we appreciate the constructive, application-focused feedback. Our framework, like many state-of-the-art generative models, is not yet optimized for real-time performance on edge devices, and we see this as an important direction for future engineering work. To bridge this gap for a practical deployment, we would investigate established pathways such as model quantization to reduce the memory footprint and a hybrid cloud/edge computing architecture, where the heavy grasp generation is offloaded to a remote server, leaving the local robot to handle low-latency execution. We will add a discussion of these limitations and potential solutions to the final version of our manuscript. Thank you again for your insightful suggestions.

---

Q5: More discussion on works with hand-object interaction.

A5: We thank the reviewer for this good suggestion. We were indeed inspired by prior research such as SAGA and agree that object-centric representations are a highly effective way to model hand-object interaction. Building upon these works, our key contribution is a conditional diffusion model designed to transfer a known, high-quality contact map from a template to a novel object, a task specifically aimed at enhancing generalization. We will add a more detailed discussion contextualizing our work within this line of research to the revised manuscript.

Dear reviewer 2Xnf, thanks for your time to review our submission and provide constructive comments. In the following, we respond to your questions in detail one by one:

---

Q1: Clarify how to acquire the shape template used for grasp transfer.

A1: We thank the reviewer for raising this important question. In our practical implementation, we maintain a shape template for each object category by collecting representative 3D mesh models from publicly available object datasets and asset libraries (e.g., [A, B]), thereby forming a library of shape templates across categories. During inference, we retrieve the corresponding shape template from the library based on the textual task description and the involved object category name. For object categories that are not included in the library beforehand, we follow the approach of [C] to leverage a large-scale pre-trained 3D generative model (e.g., [D]) to generate the required shape template based on the category-level text description. Since our proposed grasp transfer framework operates at the category level, it only requires a single canonical shape template to handle a variety of instances with different shapes and sizes within the same category. This makes the cost of maintaining such a template library affordable and scalable. Moreover, thanks to the expressiveness of the diffusion-based generative model and our design of language-conditioned geometric similarity modeling, our method does not rely on texture information and exhibits robustness to shape variations in the template. This enables us to easily expand the template library using 3D generative models and ensures applicability across a wide range of object categories. We will append these implementation details in our revised manuscript.

[A] Chang A. X. et al. ShapeNet: an information-rich 3D model repository. Arxiv 2015.

[B] Deitke, Matt, et al. Objaverse: A universe of annotated 3d objects. CVPR 2023.

[C] Wang, Zhenwei, et al. Phidias: A Generative Model for Creating 3D Content from Text, Image, and 3D Conditions with Reference-Augmented Diffusion. ICLR 2025.

[D] Meshy: Create stunning 3d models with ai. 2024.

---

Q2: Robustness to object point cloud artifacts in simulation and real-world settings.

A2: We appreciate the reviewer’s insightful comment. Before reporting our robustness analysis, we would like to clarify that during training, we applied standard point cloud data augmentation techniques, including random point sampling and additive Gaussian noise in the 3D coordinates, to improve the grasp transfer model’s robustness to real-world point cloud artifacts. Following the reviewer’s suggestion, we conducted robustness evaluations in both simulation and real-world settings. In simulation, we followed [A] to randomly mask out 30% of the object points to simulate irregular and incomplete point cloud distributions. Additionally, we added Gaussian noise with a standard deviation of 0.03 to the remaining points to emulate potential sensor-induced coordinate errors. As shown in the table below, our grasp transfer model remains robust under such conditions, with no significant drop in quantitative grasp quality metrics.

Table A: Quantitative Results of Robustness to Simulated Point Cloud Noise

While the grasp success rate slightly decreased from 74.14% to 69.34%, we found this was primarily due to missing points in critical object regions (e.g., the handle of a mug or the rim of a bowl) that can impair accurate contact computation during the final optimization step. In our real-world experiments, we reconstructed object point clouds from multi-view RGB images captured by one ZED camera and two RealSense cameras. Due to partial occlusion and limited sensor coverage, the reconstructed point clouds exhibited realistic noise and missing regions. We directly fed these point clouds into our grasp transfer pipeline and executed the transferred configuration on a dexterous robotic hand. The results show that our method achieves an average grasp success rate of 70% under real-world point cloud artifacts, demonstrating strong robustness. Due to the page limit, please refer to our response to Q2 of reviewer UCkK for further details on real-world experiments.

[A] Wang, Tao, et al. Sparse convolutional networks for surface reconstruction from noisy point clouds. WACV 2024.

---

Q3: More detailed analysis on the inference speed.

A3: We thank the reviewer for the valuable suggestion. To provide a detailed breakdown of the inference speed, we conducted repeated grasp transfer experiments on 10 different objects across 50 task specifications. All experiments were performed on a workstation equipped with an Intel(R) Xeon(R) Gold 6326 CPU and an NVIDIA RTX 3090 GPU (24 GB). On average, our method requires 14.5 seconds for contact map transfer, 15.8 seconds for part map transfer, and 16.3 seconds for direction map transfer. These steps are followed by a final grasp optimization stage, which takes approximately 15.6 seconds. We also compared the overall inference time of our method with two analytical approaches, two generative baselines, and one transfer-based baseline.

Table B: Quantitative Comparison of Inference Time for Grasp Generation

As shown in the table, our method generates high-quality grasps more than 10× faster than analytical methods, while achieving comparable or even superior grasping quality and success rates (i.e., 84.65 for ours vs. 78.98 for DFC and 83.64 for DexGraspNet). Compared to generative methods, our framework operates at a similar inference speed but significantly outperforms them in grasp quality and success rate (i.e., 84.65 for ours vs. 73.00 for ContactGen and 70.50 for UGG). Overall, these results indicate that our method strikes a favorable balance between inference speed and grasping performance. We believe this makes our framework a strong foundation for further optimization and deployment in practical, time-sensitive robotic applications. We will report the detailed inference speed of our method in the revised manuscript.

---

Q4: Possible failure analysis and discussion.

A4: We thank the reviewer for this important question. Due to policy constraints this year, we are unfortunately unable to provide qualitative visualizations of failure cases during the rebuttal phase. However, we will elaborate on these cases with qualitative examples and detailed discussion in the revised manuscript. Within our framework, the textual task specification is used not to directly predict the grasp pose, but rather as a task-relevant condition to guide the modeling of geometric similarity between the shape template and the target object. This design enhances the model’s robustness to variations in task instructions, especially compared to conventional generative methods that map text directly to grasp configurations (as demonstrated in Table 3 of the main paper). That said, failure cases do exist in our grasp transfer framework, particularly when object instances within the same category exhibit not only shape variations but also significant differences in topology or structural composition. In such cases, the success of grasp transfer depends not only on geometric similarity but also on whether the task-relevant affordance region lies within or outside the topologically mismatched area. For example, within the 'camera' category, transferring a grasp from a DSLR camera (with a protruding lens) to a compact camera (without a lens) works well for the task “to press the camera's button,” since the relevant affordance region is preserved. However, for the task “to hold the camera by its lens,” the transfer fails due to the absence of a corresponding semantic region on the target object. These limitations highlight a potential weakness in our current approach when handling large topological or semantic mismatches. In future work, we plan to incorporate object-centric semantic primitives to enhance the model’s ability to accommodate such structural variations. Thanks again for this constructive comment, and we will expand the discussion on failure cases in our revised manuscript.

---

Q5: Spelling typo in section header.

A5: Thanks for your careful checking. We will proofread the manuscript thoroughly and correct all spelling and grammatical issues in the revised version.

Dear Reviewer 2Xnf,

The author-rebuttal phase is now underway, and the authors have provided additional clarifications and performance results in their rebuttal. Could you please take a moment to review their response and engage in the discussion? In particular, we’d appreciate your thoughts on whether their revisions adequately address your initial concerns. Thank you for your time and valuable contributions.

Best, Your AC

Dear reviewer UCkK, we are grateful for your time and supportive comments highlighting our method as a "principled solution" that is both "conceptually sound" and "technically appealing" to a "well-motivated problem". We are also glad you recognized our "comprehensive evaluation" and the "clarity of presentation". We have provided detailed responses to each of your questions below:

---

Q1: Comparison with the additional pose transfer approach DexFuncGrasp, and contact-region representation approach SSCF.

A1: We sincerely thank the reviewer for pointing out these two relevant works. Following your suggestion, we have expanded our experiments to include comparisons with both DexFuncGrasp and SSCF. For DexFuncGrasp, which transfers grasps via shape interpolation between the template and target object, we note that its transfer strategy is conceptually similar to the transfer-based method Tink that we have evaluated in our main paper. DexFuncGrasp requires category-specific training to ensure interpolation quality, and thus cannot generalize to unseen object categories. To ensure fairness, we compared our method with DexFuncGrasp on various novel objects from the 16 seen categories and evaluated both human and robotic hand grasp transfers. As reported in Table A and B, our method consistently outperforms DexFuncGrasp in both settings. Furthermore, unlike DexFuncGrasp, our method does not require retraining and can directly transfer grasps to unseen object categories.

Table A: Human Grasp Transfer Comparison on Seen Categories

Table B: Robotic Grasp Transfer Comparison on Seen Categories

For SSCF, which models contact regions via a learned spatial and surface correspondence field, we found that it requires a separately trained model for each object category. Moreover, its official GitHub repository does not provide training code, and only a pre-trained model for the ‘mug’ category is available. Therefore, we conducted a focused comparison on the ‘mug’ category. As shown in Table C, our method can achieve superior performance when compared with SSCF, even though their model was specifically trained for that category. These results highlight the superior generalization capability and effectiveness of our proposed approach compared to both DexFuncGrasp and SSCF. We thank the reviewer again for the valuable suggestions and will include these additional experimental results and analyses in the revised manuscript.

Table C: Human Grasp Transfer Comparison on the 'Mug' Category

---

Q2: Real-world robot experiments.

A2: We thank the reviewer for the constructive suggestion. Following your advice, we conducted real-world dexterous grasping experiments to evaluate the effectiveness of our method in practical scenarios. The experiments were performed on a self-developed humanoid robotic platform equipped with an Inspire dexterous hand mounted at the end of the robot arm. For visual perception, we deployed one ZED camera on the robot’s head and placed two RealSense cameras on the left and right sides of the robot to capture multi-view RGB images. Given a textual task instruction, we used Grounding-DINO to segment the target object and leveraged the recent 3D foundation model VGGT to reconstruct the object’s point cloud from the captured views. Without any re-training or fine-tuning of our model, we directly used the reconstructed point cloud to predict the dexterous grasp configuration. For grasp execution, we followed the protocol in [32], where the robot arm first moves to the predicted 6-DOF pose of the hand’s root, and then the Inspire hand actuates its joint angles based on the predicted contact poses. Due to limited time, we tested four novel object categories (i.e., Bowl, Wineglass, Cup, and Headphones) across five different task specifications. For each object, we repeated the grasping process five times and recorded the average grasp success rate. A successful grasp is defined as lifting the object at least 30 cm from the table and holding it stably for 5 seconds. Overall, our method achieved an average success rate of 70% across the tested categories (60% for Bowl, 80% for Wineglass, 80% for Cup, and 60% for Headphones), demonstrating the practical applicability of our approach and its robustness to real-world observation noise. We also observed two typical failure patterns in our real-world experiments: (1) collisions during the arm's approach to the object, often due to motion planning limitations, and (2) unintended contact with the table or object during finger closing, especially for grasping the rim of the cup or bowl. These issues highlight the gap between dexterous grasp generation and its robust execution in practice, which we identify as a key area for future exploration. Also, we will include a detailed description of the robot hardware setup and real-world experimental results in the revised manuscript. Thanks again for your valuable feedback.

---

Q3: Only output a single grasp configuration while the dynamic manipulation may require full action sequences.

A3: We greatly appreciate the reviewer’s insightful comment, which well aligns with our observations during the real-world grasping experiments. Predicting a high-quality object-centric dexterous grasp pose is only the first step toward generalizable and stable dexterous manipulation. In real deployments, due to the high degrees of freedom and precision requirements of dexterous robotic hands, executing even a high-quality grasp can be challenging and often prone to failure. Generating full action sequences for dexterous grasp execution remains an open and highly challenging problem, receiving increasing attention in both reinforcement learning (RL) and vision-language-action (VLA) modeling communities. In future work, we are interested in exploring how the generalizable, task-conditioned object-centric representation produced by our transfer framework can be used to guide the training of RL policies or VLA models for sequential dexterous manipulation. Moreover, while our current model outputs a single grasp pose, apart from its use for object grasping, it provides a strong prior about the intended robot-object interaction and the preferred spatial hand-object relationship. This information can be valuable for many other robot-relevant scenarios, such as guiding pre-grasp planning, reachability analysis, safety-aware interaction design, and downstream policy learning. Thanks again for this insightful comment. We will further elaborate on this discussion in our revised manuscript.