Response To Reviewer DBfv

We sincerely appreciate your constructive feedback and meticulous evaluation of our work. Below, we provide responses to each point raised.

Q1. In Preliminaries section, there is a typo: (2,0)-tensor $u^∗⊗v^∗$ should be (0,2)-tensor.

A: Thank you for catching this typo. We will fix it in the next version of manuscript.

Q2. The motivation for using attention-based GNN to solve PDEs is unclear.

A: Reviewer y7yT raised similar concerns. While accuracy is crucial in many applications, time-sensitive scenarios prioritize speed over minor inaccuracies, motivating the development of neural solvers like Neural Operators in the ML community (see Related Work: Neural Operator in our paper).

A key example is gaming engines, where faster rendering enhances real-time FPS. To achieve realistic illumination based on blackbody radiation, the heat equation is solved to determine surface temperature distribution. In this case, the underlying geometry remains fixed, and high accuracy is less critical as long as the rendering appears plausible. Smooth scene transitions (speed) matter far more than pixel-perfect details (accuracy).

Besides, our GNN solver is SE(3)-invariant and can operate without direct coordinate input. Even if retraining is required for a new mesh, it excels in scenarios where the same mesh appears repeatedly under rotations and translations, which are common in material science and gaming engines (e.g., triply periodic minimal surfaces in crystals and block copolymers). In these cases, our method outperforms traditional solvers like FDM and FEM.

We will revise our manuscript to incorporate these examples, further reinforcing the motivation for our approach.

Q3. The subtrees partition may be very imbalanced. Will this affect the model performance? Will the random selection for the source nodes affect the results?

A: Your concern about stability makes sense. We conducted an extra experiment involving random subtree partition 100 times on a torus with 1024 nodes. It shows that a rather imbalanced partition is unlikely and the outcomes is relatively stable:

• Subtree amount: 33.51±1.61
• Mean and standard error of Subtree Scale: 30.63±1.47, 1.92±0.18
• $L^2$ & $H^1$ loss(%): 1.69±2.85, 1.93±2.82

Other settings align with the original paper.

Q4. Eq. (17) contains the parallel transports(PT) of tangent vector and covector, but the paper only gives the closed form of tangent vector parallel transport.

A: By musical isomorphism, we can identify a tangent vector $v$ as a 1-form $v^♭$, a.k.a. a cotangent vector, by $v^♭(u):=\braket{v,u}$. The motivation to introduce PT $Γ(η)$ is to use its inner-product-preserving nature. Thus the PT of a 1-form is $Γ(η)v^♭(u):=\braket{Γ(η)v,u}$ and thus is equivalent to PT a tangent vector in implementation.

Thus in your example, the PT of a (0,2)-type tensor is $Γ(η)(\tilde w_1^\*⊗\tilde w_2^\*)(u,v)=\braket{Γ(η)\tilde w_1,u}\braket{Γ(η)\tilde w_2,v}$.

Q5. For experiments, the wave equation is important for the theory of relativity, which connects tightly with the Lorentz model. Thus, I think this manifold should be considered.

A: Yes, the wave equation is crucial in relativity. It will be valuable if our model can also handle it on the Lorentz model, which is, however, not a Riemannian manifold since its metric is not positive-definite. This raises questions about whether the tools adopted currently apply to a psuedo-Riemannian manifold. This is a constructive suggestion and we will leave it for future work.

From your point of view, I understand that it will be more impactful if it can handle PDEs in mechanics. Thus, we also examine the Navier-Stokes equation for incompressible and viscous fluid with unit density on a sphere: $∂_t\mathbf u+(\mathbf u·∇)\mathbf u=-∇p+νΔ\mathbf u$. It is insightful in studying the ocean currents on Earth. The experiment setting and dataset generation align with the presented paper.

Q6. If a neighborhood has large-scale paths, the Poincaré half-plane model may not accurately reflect the parallel transmission characteristics on the original surface.

A: Perhaps our illustration on embedding is also not clear enough to you, you may refer to Q3 and Q4 in the response to Reviewer y7yT if needed. We embed the surface at a node locally into a constant-curvature surface, with parallel transport decorated edge-wise, not path-wise. Subtree partition also mitigates this issue, ensuring consistent parallel transmission unless mesh quality is rather poor.

Thanks again for your energy!

Response To Reviewer Ao56

We are sincerely grateful for the time and effort you have dedicated to reviewing our manuscript. Below, we address each of your comments in detail. Should additional revisions be necessary, we are more than willing to make further adjustments.

Q1. Time considerations in the computations.

A: Yes, solving speed is crucial, as it is one of the key advantages of neural solvers over traditional numerical solvers. We find that both the training and inference speeds of our model are acceptable.

We have compared the training times of five neural PDE solvers in Figure 14 (line 973), where our model remains faster than the GNN-based solver GINO. In practice, training time is closely tied to the maximum depth of subtrees $d$ and the number of attention heads. Notably, reducing the number of attention heads to one brings the training speed close to that of Transolver (ICML 2024) and GNOT.

Moreover, once training is complete, neural PDE solvers are much faster than traditional numerical methods in solving forward problems. When it comes to inference speed, there is no significant difference among neural solvers, even on large meshes with around 2,500 vertices. Specifically, as analyzed (line 365), its inference computational complexity is on par with GAT.

Q2. Have you tried it on more complex manifolds?

A: While our current experiments primarily focus on simple manifolds and more complex ones (wrinkled surfaces), our framework is designed to generalize to broader geometric settings. In our experiments, we primarily aim to verify the effectiveness of curvature-awareness via parallel transport in GNNs, and the results strongly support our assumptions.

Although these manifolds may be considered toy examples, in practice, many complex surfaces can be decomposed into simpler ones. These simple manifolds are able to encompass a wide range of common surfaces encountered in game engine design.

We are looking forward to your reply!

Response To Reviewer nE4H

We sincerely thank you for your insightful feedback and recognition of our work. We remain fully open to implementing any additional revisions. Below, we address each of your comments in detail.

Q1.  Such approach requires constant curvature.

A: This approach only assumes that the surface at a node has approximately constant curvature. Therefore, the entire surface is not necessary to have constant curvature, as we solve PDEs on a more complicated manifold wrinkle (in Figure 11, line 915). Due to this locality, we cannot directly parallel transport a vector at a node to those too far away since the numerical errors will accumulate. Instead, we turn to a sub-tree partition strategy to balance the trade-off between the computation accuracy and reflecting the curvature changes in a wider range.

Q2. It cannot be generalized to high-dim manifolds since in line 139, section curvature will be different that Gaussian curvatures.

A: Your observation here is actually a crucial limitation of the proposed framework. This approach cannot be directly extended to manifolds in higher dimensions since in that case, the sectional curvature is no longer a real number.

Nevertheless, a slight modification is sufficient to extend it to high-dim cases. If the manifold is compact, then we can use finite charts (each with a coordinate frame) to describe the manifold. Note that sectional curvature is the extended version of Gaussian curvature in higher-dimensional manifolds, and it is indeed a map $K:T\mathcal M\times T\mathcal M\to\mathbb R,(X,Y)\mapsto K(X,Y)$. On a 2d-manifold, it gives Gaussian curvature if the two input tangent vectors are linearly independent.

This observation sheds light on the high-dim cases. For instance, on a 3d-manifold with a coordinate frame $\{X_1,X_2,X_3\}$, there are 3 sectional curvatures, $K(X_1,X_2),K(X_2,X_3),K(X_3,X_1)$. We can treat each of them in the same way as the 2D case and combine them. But how to compute them consistently and efficiently requires further consideration. We leave it for our future work.

We will update the manuscript to incorporate the above discussion. Thanks again for your time!

Response To Reviewer y7yT

Thank you for your time and constructive feedback. We appreciate your thorough evaluation and valuable suggestions, which have helped improve our work. Below is our point-by-point response.

Q1. Why use this GNN for surface PDEs? It is less accurate than FEM. It requires training with precomputed numerical solutions and retraining for any geometry changes due to its surface-specific nature.

A: Reviewer DBfv raised similar concerns. Due to space limitations, we kindly refer you to Q2 in our response to Reviewer DBfv. We will revise the manuscript to incorporate these examples, further highlighting the motivation behind our proposed method.

Q2. Can it accomodate node features that cannot be interpreted as elements of the tangent space above a node?

A: We would like to clarify that our method can actually solve PDEs where variables are not tangent vectors. In fact, the PDEs in our experiments involve scalar fields—for instance, in the heat equation, the node feature $u(\mathbf{x})$ represents temperature. Since parallel transport acts on vectors rather than scalars, we bridge this gap by constructing a natural tangent vector field associated with the function and manifold, namely, the discrete gradient field of temperature (as noted in Remark 2, line 256). Furthermore, since the gradient of tensor products can be defined, our approach can theoretically extend to PDEs with tensor variables.

We appreciate this insightful question and will update the manuscript to make it clearer.

Q3. The embeddings themselves seemed a bit unclearly defined to me. For example, in the spherical case, you consider the two edge endpoints as vectors in $S^2$, but it was not clear how you normalize these and with respect to what center. This would seem to be crucial. (On a related note: it's surprising that one did not just scale the sphere or pseudosphere to account for fractional curvature, as would naturally arise for $K|_u$)

A: We are sorry for the confusion caused. It would be better to first clarify our embedding methods before resolving your concerns in "Claims and evidence I".

For a node $u$ on a mesh $\mathcal M$, we first compute its Gaussian curvature $K|_u$ at $u$. If $K|_u>ε>0$, then we embed the tangent space $T_u\mathcal M$ at $u$ into a sphere $S$ with curvature $K|_u$ which is tangent to $\mathcal M$ at $u$. Therefore, the edge connecting $u$ and $v$ is embedded into $S$ such that the Euclidean distance between $u,v$ equals the spherical distance (embedding in isometric sense). In this way, $\mathcal M$ is locally approximated by $S$ at $u$. Therefore, we do not need normalizations and we do scale the spheres based on the curvature estimate. It is the same for pseudospheres.

We will revise the manuscript to further clarify these in our next version.

Q4. It was based off of a curvature estimate at a vertex, but one was philisophically aimed at embedding the edge. Can you explain your reasoning here?

A: Sure. It seems embedding at a node $u$ cannot reflect the curvature of an edge $(u,v)$ at first sight. However, in the context of attention mechanism, what matters is how to distinguish the neighbors in a neighborhood. Each $(u,v)$ has a different length and thus the vector will rotate differently with $K|_u>0$. A feature tangent vector becomes different along different edges due to $K|_u$. In this sense, the net can discern edge differences based on $K|_u$.

Moreover, under a mild mesh assumption, the geodesic between $u$ and $v$ can be approximated by $K|_u$. This is because, the Jacobi field $J(t)$ has an expansion $|J(t)|^2=t^2-{1\over 3}\braket{R(X,Y)X,Y}t^4+o(t^4)$ where $R$ is the curvature tensor at $u$ and $X,Y\in T_u\mathcal M$. Thus, the edge curvature can be reflected by $K|_u$ if $(u,v)$ is short enough.

Q5.  If given an explicit triangulation, why not leverage prior works that come up with an explicit notion of discrete connection?

A: We have read the Globally Optimal Direction Fields you suggest and find it less suitable in our model. Discrete connections require geodesic estimation (Eq.(2)), but as no free lunch in discretization, discrete geodesics must lose some smooth-case properties. In our implementation, the geodesic we estimate is obedient to its Euclidean distance. Besides, triangulation is not a must in our framework. So it is just as you say, there is no right or wrong.

Q6. It might be nice to include references to the many recent papers that use a neurally-parameterized space of functions to solve PDEs in an unsupervised fashion.

A: Thanks for pointing out the useful references. We will include them in the Related Work section in the next version.

Thanks again for your time!