Autonomous Catheterization with Open-source Simulator and Expert Trajectory

Weaknesses:

Comment 1

ENN model architecture is not state-of-the-art, e.g. no transformer

Response:

We acknowledge the absence of a transformer in the ENN model architecture. Our initial trials included experimenting with a transformer architecture, specifically based on the visual transformer framework ¹. However, this approach did not yield the desired performance. In our experiments, the transformer-based model required approximately 4.5 hours of training but failed to successfully complete the task within the allocated time frame. Therefore, we opted for the current ENN model architecture, which demonstrated superior performance and efficiency in achieving the task objectives. We now include these details in Appendix J.

---

Comment 2

Unclear how trustworth the user study is. 10 participants in user study is a small sample size, and it’s unclear how they were recruited and how trustworthy their judgements are.

Response:

The user study involved ten student volunteers as participants. We have included this information, along with details on their recruitment and the study procedures, in Appendix G. Additionally, the Discussion section has been updated to acknowledge the non-expert status of these participants. This addresses the concerns regarding the sample size and the trustworthiness of their judgements.

The study was conducted with ten volunteer participants to evaluate the effectiveness and authenticity of our endovascular simulator. These participants, who had no prior experience in endovascular navigation, provided feedback using a 5-point Likert scale.

---

Comment 3

Lack of relevant details, e.g. what is the expert policy that the ENN is trained with. Why don’t you show results for the expert policy in the table?

Response:

To clarify, the ENN in our study is a standalone system, representing a multi-modal network combined with a soft actor-critic approach. It does not utilize or train with a separate expert policy, hence there are no additional policy results to present in the table.

---

Comment 4

If an expert policy exists, why do you train an ML model (ENN) model for the expert navigation task?

Response:

As per Comment 3, the policy is part of the network.

---

Comment 5

Some figures / tables are not easy to understand on their own, e.g. Table 2, for which e.g. instead of Q1 you could write “Anatomical accuracy”.

Response:

Thank you for your feedback on the clarity of our figures and tables. As per your suggestion, we explicitly state the assessed properties in Table 2 and rewrote the description to match within Section 5.1, under the User Study paragraph.

Footnotes

1. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S. and Uszkoreit, J., 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929. ↩

Questions

Comment 10

“Both for the BCA and LCCA targets, the integration of expert trajectories results in lower force, shorter path and episode length, higher safety, and a high success rate and SPL score.” (Section 5.3)** If the goal is to create an autonomous catheterization technique, why is integration of expert trajectories important and/or beneficial? It would seem the opposite is true."

Response:

IL is used to demonstrate the downstream tasks that our simulator can be used for. In this task, we use the expert to aid the training of an IL agent. This has explicit value in sim-to-real transfer. We emphasize this in subsection 4.2, particularly in the Imitation Learning paragraph.

The use of Imitation Learning (IL) is crucial in training an IL agent for autonomous catheterization. Expert trajectories provide a foundational dataset that enables the IL agent to learn efficient, safe, and effective maneuvers. This methodology is vital for sim-to-real transfer, ensuring that the autonomous system performs reliably in real-world scenarios. Detailed evidence supporting this approach is presented in Subsection 4.2, particularly in the Imitation Learning paragraph, where we demonstrate improved performance metrics including lower force, shorter path lengths, and higher safety and success rates.

Imitation Learning.* We utilize our ENN using behavioral cloning, a form of imitation learning, to train our navigation algorithm in a supervised manner. This approach emphasizes the utility of the simulation environment in extracting meaningful representations for imitation learning purposes. Firstly, we generate expert trajectories by executing the expert policy, denoted as $\\pi\_{\\rm exp}$, within CathSim. These trajectories serve as our labeled training data, providing the desired actions for each state encountered during navigation. Secondly, to mimic the real-world observation space, we select the image as the only input modality. Thirdly, we train the behavioral cloning algorithm by learning to replicate the expert's actions given the input observations and optimizing the policy parameters to minimize the discrepancy between the expert actions and the predicted actions:*

Reviewer HLTP

Comment 1

Authors should clearly explain that the expert trajectory is creating using their own simulator, and therefore, may be biased so that the results may not be representative of practical use of their method.

Response:

We now incorporate this in the Discussion section, under the Limitations paragraph.

"Additionally, it’s pertinent to note that the expert trajectory utilized in our study is generated using CathSim. This may introduce an inherent bias, as the trajectory is specific to our simulator’s environment and might not fully replicate real-world scenarios."

---

Comment 2

Overall, I thought the idea was clever, but the implications for the ML community were not clearly described. Authors should discuss the importance of generating simulated images, relationship to other generative methods (e.g., purely DL based approaches that may not capture underlying physics), and most importantly, why is the approach important to ML.

Response:

We want to emphasize that the rich diversity of the simulated data addresses the challenge of data scarcity, enhancing the robustness and generalization ability of ML models. Moreover, our method opens up new possibilities for ML research in specialized medical fields, previously constrained due to limited data availability, thus making a significant contribution to advancing ML applications in critical healthcare domains. We highlight this in the Introduction.

---

Methodology

Comment 3

It is not clear what is a guidewire and what is its purpose.

Response:

We have included a description in Section 3, The CathSim Simulator within the Guidewire paragraph.

Guidewire.* A rope-like structure designed to direct the catheter towards its intended position. The guidewire is composed of the main body and a tip, where the tip is characterized by a lower stiffness and a specific shape (depending on the procedure)*

---

Comment 4

Not clear how force labels are extracted from CatSim outputs, which seems to be images + joint position info (as discussed in Section 4.1).

Response:

Given that we have access to the simulation environment, we extract the normal and frictional forces and compute the Euclidean norm. These are then averaged across all contacts. This has been discussed in Appendix, Section C under the Force paragraph.

---

Comment 5

In Section 4.1, authors mention that the output is a feature vector Z, but it is unclear how it is used afterward.

Response:

We clarify that in our implementation, the feature vector ($Z$) is utilized as the input for the soft actor-critic policy. This allows the policy ($\pi$) to approximate the optimal action based on the given feature vector ($Z$). We clarify this in Section 4.1 Expert Navigation Network under the second paragraph.

The feature vector (Z) serves as the input for training the soft-actor-critic (SAC) policy ($\pi$), a core component of our reinforcement learning approach.

---

Manuscript Organization

Comment 6

Statement such as “Other aspects of our simulator such as blood simulation and AR/VR are designed for surgical training and, hence are not discussed in this paper” indicate that blood simulation information is not critical to be include in the main manuscript, yet there is a section on blood simulation in Section 3.

Response:

Our manuscript primarily discusses experiments focusing on core simulation aspects. The AR/VR and blood simulation components, while crucial, were not the experiment's main focus as are not directly related to ML. These are, however, fully implemented and accessible in our GitHub repository.

The blood simulation is detailed under the fluid-branch. The blood flow simulation was executed using ANSYS, tailored to a silicone-based anthropomorphic phantom from Elastrat Sarl Ltd., Switzerland. The simulation's output is formatted into spatial $(x, y, z)$ and corresponding velocity $(v_x, v_y, v_z)$ components. This data structure allows us to implement a query function that retrieves specific velocity vectors based on the input spatial coordinates. This approach precisely models blood movement within the phantom, whilst being computationally efficient.

Dear Reviewers HLTP,

Thanks for your time and comments on our paper. Please let us know if you have any further comments on our rebuttal. We hope to receive your feedback or any questions before the authors-reviewers discussion period ends.

Best regards,

Reviewer 3umj

Weaknesses:

Comment 1:

I was very confused about the presentation. For example, I didn't find the definition of BCA, LCCA.

Response:

We defined the brachiocephalic artery (BCA) and the left common carotid artery (LCCA) in our manuscript. The initial definitions were included in Figure 8. Recognizing the need for clarity, we have now additionally incorporated these definitions into the main text. In Section 5.2, under the Experimental Setup paragraph, we now explicitly state:

"Two targets are selected for the procedures, specifically the brachiocephalic artery (BCA) and the left common carotid artery (LCCA)."

---

Comment 2:

Since this is a simulator designed specifically for endovascular robots, it will make little sense if the application of it involves modalities that are very inaccessible: for example, the image as shown in the first input to the expert navigation network. I would expect real-world images are highly diverse compared to rendering images that have a clean background, unified rendering parameters, and a fixed camera position. I don't think the proposed ENN really showed the importance of the work.

Response:

The images generated by our simulator can be adapted for realistic X-Ray imaging scenarios. This adaptation is achieved using advanced domain adaptation techniques, further elaborated in our referenced work¹. Furthermore, initial real-world tests, conducted on physical aortic arch models, have produced results closely mirroring our simulated outcomes². This underscores the practical relevance and validity of our simulation for endovascular robotic applications.

---

Comment 3:

I am not sure why the authors skipped AR/VR and blood simulation parts in the experiment section but still claim the contribution of them in the table. I have a feeling that these contributions are not grounded.

Response:

Our manuscript primarily discusses experiments focusing on core simulation aspects. The AR/VR and blood simulation components, while crucial, were not the experiment's main focus as are not directly related to ML. These are, however, fully implemented and accessible in our GitHub repository.

The blood simulation is detailed under the fluid-branch of our repo. The blood flow simulation was executed using ANSYS, tailored to a silicone-based anthropomorphic phantom from Elastrat Sarl Ltd., Switzerland. The simulation's output is formatted into spatial $(x, y, z)$ and corresponding velocity $(v_x, v_y, v_z)$ components. This data structure allows us to implement a query function that retrieves specific velocity vectors based on the input spatial coordinates. This approach precisely models blood movement within the phantom, whilst being computationally efficient.

Footnotes

1. Kang, J., Jianu, T., Huang, B., Bhattarai, B., Le, N., Coenen, F. and Nguyen, A., 2023. Translating Simulation Images to X-ray Images via Multi-Scale Semantic Matching. arXiv preprint arXiv:2304.07693. ↩

2. Kundrat, D., Dagnino, G., Kwok, T.M., Abdelaziz, M.E., Chi, W., Nguyen, A., Riga, C. and Yang, G.Z., 2021. An MR-Safe endovascular robotic platform: Design, control, and ex-vivo evaluation. IEEE transactions on biomedical engineering, 68(10), pp.3110-3121. ↩

Dear Reviewer 3umj,

Many thanks again for your time and feedback during the initial review. As the authors-reviewer's discussion will end soon, please let us know if you have any further questions or comments on our rebuttal.

Best regards,

Authors.

Dear Reviewers,

We sincerely appreciate the time and effort throughout the reviewing process of our submission. As the author-reviewer discussion is due soon, please let us know if you have further questions about our submission.

Once again, thank you in advance, and we look forward to your feedback.

Best regards, Authors.