Reconstruction of Cortical Surfaces with Spherical Topology from Infant Brain MRI via Recurrent Deformation Learning

1. No methodological or implementation details provided

Response: We refer the reviewer to Figure S1 in the supplementary material for details on the common network architecture for the white and pial models. We added descriptions on data processing and more details on reproducing other baseline methods in Sections 4.1 and 4.2. Please let us know if more details are needed.

2. ODE and iterative deformation contributions presented as novel

Response: We want to address a potential misunderstanding regarding the contribution of our work. Our primary innovation lies in the integration of CSR and spherical mapping into a unified framework. Unlike existing CSR methods, our approach takes advantage of one-to-one vertex correspondence between the spherical template and cortical surfaces, eliminating the need for additional time-consuming spherical mapping and enhancing overall efficiency.

While the ODE-based problem formulation and recurrent deformation learning strategy are not novel when considered individually, we want to highlight that we are the first to apply a recurrent learning strategy for CSR. This decision serves a dual purpose. Firstly, it optimizes parameter usage and GPU memory, resulting in a streamlined design that enables a single model to predict both white and pial surfaces. This contrasts with many existing methods, such as CorticalFlow++ and SurfFlow, which employ distinct models for each surface. Secondly, the recurrent learning strategy allows for the incremental learning of large deformations through multiple smaller deformations. This is crucial for our method, given the challenges posed by using a spherical template, where the initial template is farther from the target surfaces compared to segmentation-based initial surfaces or convex hulls, commonly used in existing deep learning-based methods.

We apologize for any confusion caused by our initial description and have revised our paper to present our contributions more clearly.

3. Baseline performance does not match trends reported in previous work

Response: We attribute the observed challenges faced by baseline methods in our study to the unique characteristics of the dataset employed. The infant brain MRI images pose significant difficulties due to poor image contrast and dynamic variations in both contrast and brain volume. These challenges have not been adequately addressed by existing methods, as these methods are typically designed for adult brain MRI images where such issues are either absent or less pronounced.

4. A single dataset for evaluation

Response: Our primary goal is to solve an important but less-explored problem, which is to do CSR from infant brain MRI images. A vast majority of existing methods are developed for adult brain MRI images. Compared to adult brain MRI images, which exhibit good white matter vs. gray matter contrast, infant brain MRI images show much worse and dynamically changing image contrast and thus are significantly more challenging for CSR (see Figure S2 for some examples). In addition, the brain sizes are relatively stable for adult brains. However, the infant brain is fast growing. For example, the brain volume for a 12-month infant is approximately 2-3 times larger than a newborn (i.e., 0-month-old). Therefore, this issue is not well considered by existing methods.

5. Overstretched claims

Response:

1. We have adjusted the corresponding section in our paper to moderate this assertion, as recommended. While the authors of graph convolutional network-based works have noted that their methods do not require topology correction, it is important to clarify that these methods cannot ensure the topological correctness of the generated surfaces, even if the initial template surface is topologically correct. This is evidenced by our updated results in Table 1, which shows the self-intersection rates of the predicted surfaces by Vox2Cortex are significantly higher than flow-based methods, including ours.

2. In contrast to certain existing methods, such as CortexODE, which require the prior segmentation of the ribbon (i.e., subcortical-filled white matter), our approach directly reconstructs cortical surfaces from the image. Consequently, our model faces substantial challenges due to the poor image contrast.

3. We want to clarify that our method uses the segmentation as an auxiliary task mainly for additional regularization. During inference, the model does not require a segmentation map, thus the MR image quality will have a big impact on the results. In spite of this, our method showed decent performance even when the segmentation module is disabled, as we reported in the ablation study (see Table 2 in our paper).

1. It is thought by the reviewer that it is straightforward to use a spherical template in other baseline methods. However, this is not true for two reasons. Firstly, some methods impose stringent requirements on the initial surface. For instance, CortexODE requires the segmentation of the ribbon as a preliminary step, utilizing the segmentation mask to construct the initial surface. Altering this surface would fundamentally disrupt their pipelines, making it a nontrivial task. Secondly, opting for a spherical template amplifies the deformation required to align with the target surface. Current methods struggle with substantial deformations, hence their reliance on templates closer to the target. The recurrent deformation learning strategy we adopted is designed to tackle this issue, enabling our model to effectively handle such large deformations.

2. While we acknowledge there may still be room for improvement in the performance of competing methods, it's essential to underscore that we have exerted our utmost effort within the constraints of time and resources available to us.

3. As previously stated, our primary objective is to address a significant yet less-explored issue. Consequently, we directed our attention to CSR within what is arguably the most challenging age group—infants aged 0-12 months. It is noteworthy that, although neonatal brains undergo substantial changes in volume similar to infant brains, neonatal images from dHCP do not display the pronounced contrast changes observed in infant brains (refer to Figure 2 in the supplementary material).

1. Incremental work based on existing methods

Response: Our research builds upon established CSR methods, particularly those based on diffeomorphic flows. While we recognize the significant achievements of previous methods in reducing processing time from hours to mere seconds, our work seeks to contribute by addressing challenges that remain unresolved or insufficiently considered in existing approaches.

Our primary contribution lies in the integration of CSR and spherical mapping. To achieve this goal, we introduce a recurrent deformation learning strategy to deform a spherical template onto the white and pial surfaces based on images. Unlike existing CSR methods, our approach leverages the one-to-one vertex correspondence between the spherical template and the cortical surfaces, requiring no extra time to achieve spherical mapping.

Moreover, our method significantly enhances efficiency compared to all existing methods by eliminating the time-consuming spherical mapping step. However, it is essential to acknowledge that employing an initial template that is more distant from target surfaces, as opposed to using a convex hull or an initial surface based on cortical ribbon segmentation, significantly increases the level of difficulty. To tackle this challenge, we introduced a recurrent learning strategy to allow a large deformation to be learned incrementally with multiple smaller deformations. Notably, using ribbon segmentation to obtain an initial surface for deformation can lead to topological defects and thus requires topology correction, as no segmentation methods are perfect. In contrast, our method circumvents the error-prone segmentation by reconstructing the surfaces directly from images and requires no time-consuming topology correction as the flow-based approach can theoretically guarantee topological correctness.

In summary, we believe our method signifies a significant stride forward in the field, serving as a pioneering solution in seamlessly integrating CSR and spherical mapping.

2. Benefits of the spherical template

Response: Many downstream tasks of CSR, such as cortical surface parcellation, require spherical mapping. For example, parcellation involves surface projection onto a sphere through metric-preserving inflation (i.e., spherical mapping) first and then registration to a spherical atlas. Notably, the inflation process alone takes about half an hour to complete and is only possible if a mesh has a homotopic topology to that of a sphere. The use of a spherical template allows our approach to take the advantage of one-to-one vertex correspondence between the spherical template and cortical surfaces. It not only guarantees that the topology of the predicted surfaces is homotopic to a sphere, but also eliminates the need for additional time-consuming spherical mapping and thus enhances overall efficiency significantly.

In contrast, existing CSR methods often neglect spherical mapping in their algorithms, resulting in an inefficient pipeline. Our approach is distinct from the existing ones, addressing the inefficiencies of the traditional pipeline, and presenting a more streamlined and effective solution.

For additional information on cortical analysis, we refer the reviewer to Figure 1 in our paper and related work below:

[1] Hoopes, Andrew, et al. "TopoFit: rapid reconstruction of topologically-correct cortical surfaces." Proceedings of machine learning research 172 (2022): 508.

[2] Gopinath, Karthik, Christian Desrosiers, and Herve Lombaert. "Learning joint surface reconstruction and segmentation, from brain images to cortical surface parcellation." Medical Image Analysis 90 (2023): 102974.

3. Recurrent learning not novel

Response: We want to emphasize again that we see the overarching concept of integrating CSR and spherical mapping as our main contribution, not the specific strategies, e.g., the recurrent deformation learning.

Additionally, we want to mention that although the recurrent learning strategy is not new, we are the first to use it for CSR. Recurrent learning benefits both efficiency and effectiveness, allowing our method to handle both the white and pial surface reconstruction within a unified framework.

4. Multi-step strategy similar to existing works

Response: The multi-step strategy, progressing from the template to the white surface and culminating at the pial surface, has its roots in traditional CSR methods, such as FreeSurfer. This sequential approach is widely embraced across all CSR methods, except for those specifically tailored for exclusive white or pial surface reconstruction. Our method adopts this proven strategy in a deep learning framework.

1. Details on the proposed method are not clear

Response: We refer the reviewer to Figure S1 in the supplementary material for details on the common network architecture of the white and pial models. Please let us know if more details are needed.

2. Topology correctness is not validated

Response: To evaluate topology correctness, we reported the self-intersection (SI) rates for all methods in the table below. Our method demonstrates a relatively low SI rate (~0.3% on pial surfaces and ~0.06% on white surfaces). Notably, DeepCSR and CortexODE outperform in this metric. DeepCSR benefits from using the marching cubes algorithm for final predictions rather than deforming a template, so it is expected to have a low SI rate. CortexODE achieves a lower SI rate possibly due to the use of initial surfaces closer to target surfaces than the spherical template employed in our method. In comparison, all other methods exhibit significantly poorer performance than ours.

For reference, we calculated the SI rate on our pseudo-ground truth, revealing that the pseudo-ground truth (GT) meshes are far from perfect. It also suggests that the performance of our proposed method in terms of the SI rate may be underestimated, potentially influenced by the pseudo-ground truth. Additionally, we computed the Euler characteristic of our method's predicted surfaces, all of which equal to 2, indicating correct genus-zero topology in our predictions.

SI (%) for pial surface:

SI (%) for white surface:

3. The time consumed should be given

Response: The inference time of our method is 1.2 seconds for predicting the white and pial surfaces in the same hemisphere. It is noteworthy that our method saves a substantial amount of time for downstream tasks because no extra time is needed for spherical mapping. CorticalFlow++ and SurfFlow inference time are at the same level (~1 seconds) for CSR. However, they typically require an extra 30 minutes for spherical mapping.

The CortexODE inference process involves multiple steps, including ribbon segmentation, topology correction, SDF computation, initial surface reconstruction via marching cubes, and surface deformation. The total computing time for all key steps is 5.5 seconds, covering the prediction of white and pial surfaces in the same hemisphere.

The Vox2Cortex inference time is 17 seconds, encompassing all four surfaces in both hemispheres.

4. The segmentation performance also should be given

Response: We reported the segmentation performance and a visual comparison with ground truth of Figure 2 in the supplementary material. The average dice coefficient values of graymatter, white matter, and cerebrospinal fluid are 0.883土0.014, 0.885土0.014 and 0.826土0.028, respectively.