TriSAM: Tri-Plane SAM for zero-shot cortical blood vessel segmentation in VEM images

The authors have not submitted a rebuttal addressing the concerns raised during the first round of reviews. Hence, I will maintain my original score.

We apologize for the delay in submission, as fulfilling the reviewers' request for additional experiments on our proposed large-scale dataset proved to be a time-consuming process. Nonetheless, we managed to submit our response before the rebuttal deadline. As we have clarified some misunderstandings and addressed all your questions about our paper, we sincerely hope that you can reconsider the rating. Please feel free to let us know if you have any further questions.

Thanks for the insightful feedback. We clarify the issues and address the questions accordingly as described below.

Response Highlight

We would like to point out that one of our main contributions is our constructed BvEM dataset, which is the largest-to-date public benchmark for cortical blood vessel segmentation in VEM images. It addresses a significant gap in the neuroimaging field.

Q1: Experiment

[Reply] We have conducted more experiments, such as nnUNET and MAESTER. Please refer to the answer to Q1 of General Response. We have cited and discussed the provided references [1-3] in Section 5.

We also would like to highlight that the main focus of this paper is zero-shot vessel segmentation. As annotating 3D volumes for blood vessels is not only labor-intensive but also requires well-trained domain experts, efficient deep-learning algorithms are urgently needed. End-to-end training methods require annotating data for training in any new scenario. We deeply believe that in the era of foundational models, the zero-shot setting is the correct path to solving this problem. The emergence of the SAM model and the success of our proposed TriSAM have demonstrated this point. Comparing our approach to supervised learning methods is meaningless because supervised learning methods fall far short of the generalization capacity of our approach, and the annotation-training paradigm of supervised learning is far from what we desire. Therefore, the proposed BvEM dataset is also designed for evaluating zero-shot model performances, not only for training deep learning models. We reported the performance of 3D UNet solely for the completeness of the paper.

Q2: The meaning of "Moreover, imaging the whole mouse brain using VEM technology is under planning".

[Reply] In response to the BRAIN Initiative, NIH has proposed a series of transformative projects. A major goal is scaling up the VEM technology to the whole mouse brain. A group of leading scientists calls for mapping a whole mouse brain in the next ten years. In response, a project mapping the connectome in a whole hippocampus has been granted as the first step. Considering the deep neuroscience background that is not familiar to computer scientists, we have deleted this sentence to avoid confusion.

Q3: Turning point detection

[Reply] When tracking along the z-axis, we apply SAM to segment blood vessels in the zx and zy planes. Potential turning points are identified as points in these segments with the smallest and largest z-values. The process is detailed at the end of Section 3.

Q4: SAM-based tracking

[Reply] SAM-based tracking: For the next slice, we use prompts derived from the enlarged bounding box and the segment's center in the current slice. Specifically, we pad the bounding box of the segment to create the bounding box prompt and calculate the segment's center for the point prompt. These prompts are used to segment blood vessels in the subsequent slice, as detailed in Section 3.2 "SAM-based Tracking."

Q5: Model training

[Reply] Model training: Our proposed zero-shot method eliminates the need for model training. We utilize a pre-trained SAM for segmentation, leveraging its ability to segment objects from a point or bounding box. The initial points for segmentation are generated using an unsupervised color thresholding method. While this thresholding may miss significant portions of the blood vessel, these areas are effectively detected and segmented through SAM-based tracking.

Q6: Dynamics

[Reply] Blood vessels gradually change their shape and size in 3D, but the variations can significantly differ across different 2D planes, as illustrated in Figure 10 in the revised paper. We aim to highlight that along the optimally selected plane, changes in shape and size of the blood vessel are more stable compared to other planes. This statement has been updated in the revised version. We also compare the scale variation for different methods. Please refer to the answer to Q2 of the General Response.

We hope our response clarifies your initial concerns/questions. We would be happy to provide further clarifications where necessary.

Thanks for the constructive comments. We answer your questions below:

Q1: Threshold

[Reply] The ablation study on the threshold for initial seed selection and triplane selection is detailed in Section 4.1, with results in Figure 7. We observed consistent method performance across various thresholds.

As shown in Section 4.1, the performance of the proposed method is largely insensitive to threshold variations, except at extremely high values, indicating our method's robustness to threshold settings.

Q2: Tracking Dynamics

[Reply] Please refer to the answer to Q2 of General Response.

Q3: Experiments

[Reply] We have conducted more experiments, such as nnUNET and MAESTER. Please refer to the answer to Q1 of General Response.

We also would like to highlight that the main focus of this paper is zero-shot vessel segmentation. As annotating 3D volumes for blood vessels is not only labor-intensive but also requires well-trained domain experts, efficient deep-learning algorithms are urgently needed. End-to-end training methods require annotating data for training in any new scenario. We deeply believe that in the era of foundational models, the zero-shot setting is the correct path to solving this problem. The emergence of the SAM model and the success of our proposed TriSAM have demonstrated this point. Comparing our approach to supervised learning methods is meaningless because supervised learning methods fall far short of the generalization capacity of our approach, and the annotation-training paradigm of supervised learning is far from what we desire. Therefore, the proposed BvEM dataset is also designed for evaluating zero-shot model performances, not only for training deep learning models. We reported the performance of 3D UNet solely for the completeness of the paper.

Q4: Why SAM

[Reply] We chose SAM for its efficacy in segmenting objects with varied prompts like a point, box, or mask, and its excellent generalization to unseen data. In the era of foundational models, we firmly believe that for various segmentation tasks, a foundational model is all we need, and adaptations specific to different downstream tasks can be made based on this foundational model. As a foundational model for image segmentation, SAM has already been widely demonstrated for its strong generalization capabilities. More details have been included in the revised manuscript.

We hope our response clarifies your initial concerns/questions. We would be happy to provide further clarifications where necessary.

Thanks for the constructive comments. We address the questions and clarify the issues accordingly as described below.

Q1: Novelty

[Reply] (a) Zero-shot approach: One major challenge in 3D blood vessel segmentation is the limited labeled data to train effective deep learning models, as labeling 3D volumes is extremely time-consuming. Although existing unsupervised learning approaches eliminate the need for manual labeling, their 3D segmentation performance is not adequate. In contrast, we convert the 3D segmentation task into the tracking task and propose to integrate the existing foundation model, SAM, into our zero-shot pipeline achieving SOTA performance without any manual annotation. (b) SAM Adaptation: Directly applying SAM to track blood vessel segments over slices does not achieve optimal performance (Table 2). Popular SAM-adaption methods need labeled data for finetuning. In contrast, the proposed SAM-based tracking pipeline is both effective and data efficient.

Q2: Experiment

[Reply] We further compare with nnUNET and MAESTER, please refer to the answer to Q1 of General Response.

Q3: Threshold

[Reply] The ablation study concerning the threshold is presented in Figure 7 and detailed in Section 4.1. We applied consistent hyperparameters across all species, which demonstrated effective generalization.

Q4: Discussion of Existing VEM Segmentation Methods

[Reply] Most existing VEM segmentation methods are supervised and need extensive labeled data, often limiting generalization across datasets. We summarized the existing VEM segmentation works in Section 5. Our method, requiring no labeled data, shows superior generalization to different datasets.

We hope our response clarifies your initial concerns/questions. We would be happy to provide further clarifications where necessary.