Diffeomorphic Mesh Deformation via Efficient Optimal Transport for Cortical Surface Reconstruction

We thank Reviewer your time and effort you dedicated to reviewing our paper. We would like to address each of your concerns below.

1. Is the initial surface in your method the same as the initial surfaces used in competing methods?

In the process of generating initial surfaces from MRI images, each of our competing methods uses a different initial surface. The initial surface in our approach, as well as in CortexODE, is derived from the segmentation of MRI images. Conversely, CFPP and Vox2Cortex commence with a pre-defined mean shape template. DeepCSR, on the other hand, employs an implicit-based methodology and thus does not rely on a template for initiation.

2. How about the topological information of the initial surface and the ground truth surfaces? Are they all genus-0 surfaces? Does the DDOT framework maintain the topological information during the deformation process?

Yes, the topological information of initial surface and the groundtruth surface is the same, which is genus-0 surfaces. The utilization of the diffeomorphic deformation framework, DDOT, ensures the preservation of this topological information throughout the deformation process.

3. It is better to give a brief introduction of oriented varifold for paper's readability.

We thank Reviewer for the valuable suggestion. In response, we have included a concise overview of oriented varifolds in Appendix A. Additionally, a detailed derivation of mesh as an oriented varifold can be found in Section 3.1. We believe that our current information about oriented varifold sufficiently conveys our concepts and remain easily comprehensible for scholars familiar with the field.

We thank Reviewer your time and effort you dedicated to reviewing our paper. We would like to address each of your concerns below.

1. The novelty of the idea is limited -- optimizing meshes with CD and/or earth mover distance (EMD) between sampled point clouds is not new (for instance [1]). Replacing EMD with another distributional distance seems more or less some pure trial-and-error endeavor.

We respectfully disagree with Reviewer in two aspects:

First of all, while acknowledging the prior exploration of optimizing meshes with Chamfer Distance (CD)/Earth Mover's Distance (EMD) between sampled point clouds, we argue that the computation of EMD as an optimization metric is not practical for high-resolution meshes like the cortical surface. We refer the reviewer to the issue: Earth Mover's Distance (EMD) raised on pythorch3D GitHub and its answer by Georgia Gkioxari (link). To the best of our knowledge, the computation of the exact transport map between two point clouds with $n$ points has a polynomial complexity and is not affordable for $n > 20,000$ [5]. The reason [1] successfully employs EMD is due to their experiments on the ShapeNet dataset, which is representable by a few thousand points. In contrast, the complex surface topology of the cortical surface requires mesh approximations with several hundred thousand points.

Secondly, we do not replace EMD with another distributional distance due to merely a result of trial-and-error. Specifically, we aim to derive a valid, fast, and scalable metric for high-resolution mesh deformation. While EMD is a valid metric, its applicability to high-resolution meshes is limited, as previously mentioned. On the other hand, Chamfer distance (CD) is faster, but it is not a valid distance since it does not satisfy the identity of indiscernibles, i.e., when CD is 0, two point clouds are not necessarily the same, thus leading to the local minimum issue presented in our paper. Furthermore, as shown in Section 4.2.2 and Figures 3 in our paper, CD (and its variants) is not as scalable as our proposed metric. To the best of our knowledge, we are the first to propose a metric that is valid, fast, and scalable in learning high resolution shape deformation.

2. The paper [2] has shown that optimization with SWD on point clouds is beneficial. It is therefore not surprising that SWD can be used for mesh optimization.

While we acknowledge the fact that [2] has shown the optimization with SWD on point cloud is beneficial, it is worth noting the significant differences between point cloud and mesh representations. Loss functions that are effective for point clouds may not directly translate to effective solutions for meshes. Specifically, unlike point cloud presented in [2], the cortical mesh surface we deal with is notably more complex, characterized by its dense connectivity. Moreover, the application of SWD in mesh deformation does not inherently ensure the preservation of the mesh's topological structure. Therefore, our approach, which combines diffeomorphic deformation with SWD loss applied to the probability measures of meshes, is specifically designed to accurately handle complex mesh structures like the cortical surface. This distinction is critical in understanding the novelty and effectiveness of our proposed framework in the context of intricate mesh optimization.

3. I feel that the math for the probabilistic interpretation, although probably not explicitly presented in the context of mesh optimization, is unnecessary at least from the practical point of view, especially given that EMD has already been used for mesh optimization.

We respectfully disagree with reviewer for two reasons.

First, our probabilistic interpretation, especially our theorem 1, is necessary in the context of this paper. This theorem lays a robust foundation, motivating the representation of meshes as probability measures and justifying the use of Sliced Wasserstein Distance (SWD) as our optimization metric. In fact, from independent point of view, Reviewer 4B9Y also acknowledges our theorem as "novel and promising" and "suitable for practical applications".

Secondly, although EMD has already used for mesh optimization, to the best of our knowledge, there is no theoretical justification for its usage. Furthermore, as presented in Q1, EMD is not a suitable metric for our use case with high-resolution mesh.

I thank the authors' for the extensive response which partially addressed my concerns, mainly those on the empirical results. However, I am still not quite convinced that the theory is that necessary --- from a typical practitioner point of view it does not provide that many additional insights (make a probably not so accurate analogy: one might feel safer to use an algorithm with a better generalization bound but eventually just pick algorithms based on validation error). And what I meant is basically that one may propose the same method by simply noticing the benefits of SWD in point cloud optimization without mathematical verification, and the tone of the paper sounds like the other way. Saying that the authors are inspired by the theoretical framework sounds more comfortable to me. Nevertheless I raise my rating for the empirical contribution of the paper and some previously unexplored theoretical point of view. I would further suggest the authors reduce the amount of math in the main paper and instead elaborate the details in the appendix (for instance, maybe provide an informal version of Theorem 1 in the main paper instead).

We thank reviewer your time and effort you dedicated to reviewing our paper. We would like to address each of your concern below.

1. The results seem limited to the brain surface. The authors do mention the limitation, which I assume would be a high requirement on the mesh quality. I would like to see more details on that, i.e. how applicable this method is on a common dataset/mesh.

Our decision to showcase the proposed method through cortical surface reconstruction is motivated by its applicability in neuroscience and neuroimaging fields. Furthermore, the fine resolution of the reconstructed mesh allows us to exhibit the effectiveness of the sliced Wasserstein distance, especially in aspects of scalability and accuracy. Regarding applicability on other dataset, our method can be applied to other organ reconstruction problems such as heart reconstruction [1] or lung reconstruction [2].

2. The genus zero requirement might be another limitation? can we extend it to shapes of other topology?

Yes, we agree that, as currently presented in this paper, our method is restricted to genus-zero shapes. The exploration of other shape topologies is a subject for our future research endeavors.

[1] Kong et al. A Deep-Learning Approach For Direct Whole-Heart Mesh Reconstruction.

[2] Wickramasinghe et al. Voxel2Mesh: 3D Mesh Model Generation from Volumetric Data.

We thank reviewer your time and effort you dedicated to reviewing our paper. We would like to address each of your concerns below.

1. What is the cost function defined on the position-orientation space?

The cost function defined on the position-orientation space is delienated at Appendix D. Specifically, the cost is computed as the Monte Carlo sampling of sliced Wasserstein distance between the predicted mesh $\mathcal{\hat{M}}$ and the target mesh ${\mathcal{M}^\ast}$ represented as probability measures $\mu^{\mathcal{\hat{M}}}$ and $\mu^{\mathcal{M}^\ast}$, respectively: $\mathcal{L}(\mathcal{\hat{M}}, \mathcal{M}^\ast) = \widehat{SW}\_p^p (\tilde{\mu}^{\mathcal{\hat{M}}},\tilde{\mu}^{\mathcal{M}^\ast}) = \frac{1}{L}\sum\_{l=1}^L \text{W}\_p^p (\theta\_l \sharp \tilde{\mu}^{\mathcal{\hat{M}}},\theta\_l \sharp \tilde{\mu}^{\mathcal{M}^\ast}),$

where $\text{W}_p^p (\theta_l \sharp \tilde{\mu}^{\mathcal{\hat{M}}},\theta_l \sharp \tilde{\mu}^{\mathcal{M}^\ast})$ is the Wasserstein-$p$ [1] distance between $\tilde{\mu}^{\mathcal{\hat{M}}}$ and $\tilde{\mu}^{\mathcal{M}^\ast}$, and $\theta\_1,\ldots,\theta\_L$ ($L$ is the number of projections) are drawn i.i.d from $\mathcal{U}(\mathbb{S}^{d-1})$. We empirically set the hyperparameter by tuning on a small subset of OASIS dataset. To be specific, we set $L=100, p=2$. In terms of oriented varifold, for each support, we concatenate the barycenter of vertices of the face and the unit normal vector as a single vector in $\mathbb{R}^6$.

2. Does the Wasserstien distance vary when one mesh is transformed under a rigid motion ?

It is possible that the Wasserstein distance changes when one mesh is transformed under rigid motion. Therefore, during training we do not apply any transformation to keep Wasserstein distance stable.

3. Why use the position-orientation space to represent the mesh measure? Why not just use position space?

By relying on position-orientation space, we can employ oriented varifold theory, which gives us a theoretical bound [2] for approximating discrete mesh. Furthermore, our experimental results demonstrate that using position-orientation space yields better performance compared to position space only (see Sec. 4.3. Ablation). Therefore, in this paper, we rely on position-orientation space to represent the mesh measure.

4. Can we say something about the regularity of the optimal transportation map? Is it diffeomorphic ?

We are assuming that you are asking whether the optimal transport map is a diffeomorphism. It is important to clarify that our approach does not seek to establish an optimal transport map between two manifolds. Instead, we utilize the sliced Wasserstein distance to determine the OT cost. This method primarily serves as a loss function, aiding in quantifying the discrepancy between the predicted and target mesh. Consequently, given the focus of our methodology on sliced Wasserstein distance for loss computation, we cannot definitively ascertain whether the OT map resulting from this process is a diffeomorphism. We are happy to discuss more if Reviewer is still not satisfied with our answer.

[1] Villani et al. Topics in optimal transportation. In American Mathematical Soc., 2003

[2] Kaltenmark et al. A general framework for curve and surface comparison and registration with oriented varifolds. In CVPR 2017.