Towards Universal Mesh Movement Networks

We greatly appreciate your insightful comments that precisely recognize the strengths of our work! We are also very happy the Reviewer Jzyt checked our supplementary video and found it nice and useful. Below, we provide our point-to-point responses to the comments, denoted by [W] for weaknesses and [Q] for questions.

Response to Q1: The monitor values 𝑚 correspond to the original mesh. Given a time-dependent PDE, the monitor values $𝑚$ at the first time step are computed based on the initial condition on the original mesh. Then the model takes in the monitor values $𝑚$ as input (with other features) and outputs a moved mesh. The PDE is solved based on this moved mesh and the solution obtained will be used for computing monitor values for next time-step.

The target meshes only come in during the training process: the moved mesh is compared to the target mesh with element volume loss and Chamfer distance. The networks get to know the target during the training. In other words, the monitor values are part of input features and are defined in the nodes of the input meshes and not those of the target meshes. This indeed explains the notation $m_ξ$ with the monitor values being associated with the input mesh. The training process can be explained to teach the model to learn to solve an optimal transport problem according to equidistribution theory given the monitor values and other related features.

Response to Q2: Thanks for raising this question. We acknowledge that Chamfer distance-based optimization for mesh matching or point cloud registration can potentially lead to convergence at local minima. However, in our case, we did not observe significant issues of being trapped in poor local minima during the training process, though we cannot guarantee that a global minimum was reached. We believe there are two primary reasons for this. First, the initial alignment between the original uniform meshes and the target meshes in our training dataset is not significantly different. This likely increases the chances of finding well-matched vertices when searching for nearest points using Chamfer distance. Second, in addition to the Chamfer distance, we employed an element volume loss, which provides additional information beneficial for vertex matching.

Response to limitation: Thanks for the suggestions. We include an additional experiment about poor quality initial mesh as shown in Figure R4. The input mesh has highly anisotropic elements and several vertices with valence 6. This can certainly be considered a “low quality” mesh. Consider the monitor function to be the l2-norm of the recovered gradient of the solution of an anisotropic Helmholtz problem. With this monitor function, we tried applying conventional MA solvers and found that both the quasi-Newton and relaxation approaches failed to converge and/or resulted in tangled meshes, even though the elements are already aligned with the anisotropy of the Helmholtz solution. The UM2N approach, however, was able to successfully apply mesh movement without tangling. We would also like to clarify that our work targeting mesh movement that dynamically adapts to a PDE solution, which is not a replacement for mesh generation in general, so we would always expect a reasonable input mesh when applying our model.

We thank Reviewer jEN5 for acknowledging the contributions, soundness, and presentation quality of our paper. And greatly appreciate Reviewer jEN5 for proposing these insightful questions. Below, we provide our point-to-point responses to the comments, denoted by [W] for weaknesses and [Q] for questions.

Response to W1: We perform an additional ablation study of our network architecture on the Swirl case. We construct two models, which replace the GAT Decoder with one layer GAT denoted as UM2N-w/o-Decoder and remove the graph transformer denoted as UM2N-w/o-GT. The results are shown in the Table R1.It can be observed that both variants give worse results compared to the full UM2N. A visualization is also shown in Figure R2. It can be observed that without the Decoder, the model distorts the shape of the ring, i.e., missing relative information between vertices; without the graph transformer, the model fails to capture the details of the ring shape.

Response to W2: To the best of our knowledge, there are not a lot of works addressing the end-to-end learning-based r-adaptation problem. Many existing works focus on end-to-end surrogate neural PDE solvers. The models proposed in these works try to predict the PDE solutions directly, therefore it is hard to make a fair comparison as our UM2N predicts the mesh i.e., PDE discretization. To compare with them qualitatively, these methods have advantages in efficiency. On the other hand, our approach has the advantage that PDE solutions obtained are physically plausible based on guarantees from classical numerical analysis regarding the PDE discretization, whereas such guarantees are absent or enforced in a weaker sense in directly learned PDE solutions.

For the network architecture, we apply the graph transformer as encoder for its powerful expressivity, and apply the graph attention network as decoder because it can naturally help alleviate mesh tangling. There are existing works such as efficient transformers from natural language processing or computer vision community, which provide promising future directions to improve the efficiency of our model. The semantic mesh segmentation mentioned by the Reviewer jEN5, although not directly applicable to our problem, would also be an interesting direction to explore.

Response to W3/Q1: We agree that there are no guarantees for our method to produce non-tangled meshes as we have also noted in the limitations section of our paper. In Figure 6, we claim that UM2N is better at handling complex boundaries compared to the MA method. The MA method requires smooth boundaries and convex domains. Its behaviour can be observed from our additional experiments shown in Figure R1. It produces good results for the rectangle domain (shown in the first row) but outputs a highly tangled mesh given a non-convex domain (shown in the following rows).

The intuitions behind why our data-driven UM2N can handle complex boundaries can be divided into two aspects. The first is that we use Graph Attention Network (GAT) to build the mesh deformer. Therefore, the coordinate of each vertex is updated by the weighted sum of its neighbors, the weights (or coefficients) of which are determined by the GAT module. This guarantees each vertex to only move within the convex hull of its neighboring vertices, hence effectively alleviating mesh tangling issues. In addition, we utilize element volume loss penalizing negative element volumes (i.e., inverted elements), which helps reduce mesh tangling.

Response to W4/W5/Q2: As for the scale of the dataset, we consider the test case shown in the paper - Cylinder (~5k vertices, ~10k triangles), Tsunami ( ~8k vertices, ~16k triangles) - to have moderate degrees of freedom. The tsunami case is already a real-world scenario application in the ocean modelling community. To show a case with more degree of freedom, we further apply our trained model on a flow past cylinder case with ~11k vertices and ~22k triangles as shown in Figure R5. It can be shown that our UM2N works well in this case.

As for the large case such as >1 million vertices/triangles mentioned by the reviewer, the model can be naturally applied to larger scale problems as there is no limitation for the input length of graph transformer and graph neural network but the GPU memory is the limit. We performed stress tests using a time-independent Helmholtz case on our RTX 3090 24 GB GPU and observe that the maximum scale of the problem that can be run on this GPU has ~50k number of elements. The inference time is ~760 ms, the MA method on the same problem requires ~37800 ms with a residual threshold 1e-4. Therefore, given our current limited computational resource, it is hard for us to construct a case with 1 > million vertices. Considering this limitation, applying the memory efficient transformer (linear-attention transformer for further improvement of the inference efficiency) is targeted as future work.

We like to thank the reviewer for their kind words on the presentation and level of contribution of our paper. Below, we present our detailed responses to the comments, indicating [W] for weaknesses and [Q] for questions.

We appreciate the feedback regarding the amount of explanation we were able to provide regarding limitations of our approach and validation of the flow past a cylinder case. As the reviewer correctly identifies our paper focuses specifically on r-adaptation which has benefits over other mesh adaptation methods - maintaining mesh topology, and, in the Monge-Ampere (MA) approach, minimising the amount of change - but also drawbacks: solving the MA equations is very costly, and it puts limitations on the physical geometry, demanding smooth boundaries and convex domains. In our paper, we demonstrate a significant reduction in the cost through the UM2N acceleration. We also demonstrate a relaxation to some extent of the geometric limitations, but this is indeed a bit harder to outline and quantify.

Response to W1/Q1:‘tangled results of the UM2N model’: There are in fact still cases where UM2N might fail, we perform additional experiments as shown in Figure R1. Starting from a rectangle, we gradually distort the geometry to a more non-convex shape i.e., the case is more and more challenging from top to bottom. The UM2N finally fails in the case surrounding sudden jumps in the boundary accompanied with a large variation in the required resolution as shown in the bottom row of Figure R1: a sudden constriction of the flow leads to tangling with UM2N in the two left corners of the channel. In a more smoothed version UM2N does produce a valid mesh, where the original MA method still fails (second to bottom rows in Figure R1). We agree that this warrants a more extensive discussion in our paper in the revision.

Response to W2/Q2: ‘non-tangled results of other methods on the flow-past-cylinder’: We additionally perform a non-tangled flow-past-cylinder shown in Figure.R6 as mentioned by the reviewer xMNp. We simplified the setting: lower the Rrenynolds number by reducing the inflow velocity and set the top and bottom slip boundary, which finally makes it a laminar flow. As shown in Figure.R6, both the UM2N and M2N give non-tangled meshes and it can be observed that UM2N better captures the dynamics i.e., adapts to the PDE solution of stable state compared to the High Res solution. The quantitative results shown in the Tab.R3 also indicate that UM2N perform better than M2N regarding to error reduction.

The aim of the flow past a cylinder case was to demonstrate that mesh movement can now readily be applied, which is not feasible with a classical approach like MA for several reasons: the non-convexity of the domain means the equations are fundamentally ill-posed, and, even if that could be overcome, e.g. by applying mesh movement only in the right half of the domain, the prohibitive costs to apply this every timestep. This case is based on the so called DFG-benchmark 2D-2 [1], which has been modelled with a variety of other frameworks, including those that incorporate other mesh adaptation approaches [see e.g. 2-5] The results in our paper were validated by comparison with a high resolution uniform-mesh model, which corresponds well with the reference value in [2,3]. Note that convergence to the reference depends on various discretisation choices, finite-element pair, type of boundary condition, etc., which are not the focus of our paper. We therefore decided to only compare with a high resolution case, using the same discretisation details, to demonstrate the improvement in accuracy through mesh movement alone. We would like to clarify this however in our paper, and make better reference to and comparison with published results for this case.

[1] https://wwwold.mathematik.tu-dortmund.de/~featflow/en/benchmarks/cfdbenchmarking/flow/dfg_benchmark2_re100.html

[2] D. Capatina et al. ‘22 https://doi.org/10.1016/j.cma.2019.112775

[3] V. John ‘04 https://onlinelibrary.wiley.com/doi/abs/10.1002/fld.679

[4] T. Coupez et al. ‘13 https://doi.org/10.1016/j.jcp.2012.12.010

[5] Hachem et al ‘13 https://doi.org/10.1002/nme.4481

We thank Reviewer yaVX providing a detailed summary of our strengths and valuable suggestions. Below, we present our detailed responses to the comments, indicating [W] for weaknesses and [Q] for questions.

Response to W1: As the reviewer correctly identifies we do compare our method with existing methods, focussing on the M2N method. The reason for the limited number of comparisons is because there really has not been a lot of research so far on learning-based mesh-movement methods, most of the literature has focused on acceleration of other mesh adaptation strategies. We do cite the very recent flow2mesh paper by Jian Ju et al. (ref. [32] in our paper) who confirm our view: “However, these works all focused on mesh refinement rather than the mesh movement"

Response to W2: We perform an additional ablation study of our network architecture on the Swirl case, analysing the contribution of each network component. We construct two models, which replace the GAT Decoder with one layer GAT denoted as UM2N-w/o-Decoder and remove the graph transformer denoted as UM2N-w/o-GT. The results are shown in Table R1. Both variants give worse results compared to the full UM2N. A visualization is also shown in Figure R2. It can be observed that without Decoder, the model distorts the shape of the ring, i.e., missing relative information between vertices; without the GT, the model fails to capture the details of the ring shape.

Response to W3/W4/Q1: Thanks for raising this question. We include an additional experiment dealing with poor quality initial meshes as shown in Figure R4 and an additional experiment for irregular geometries. Also note the real-world Tsunami case shown in Figure 5 of our paper is also a good example for highly irregular geometries. And more cases with complex geometries are shown in our supplementary video. Figure R4 shows a low-quality input mesh with highly anisotropic elements. Using a monitor function based on the l2-norm of the gradient of the solution of an anisotropic Helmholtz problem, we tried applying conventional MA solvers and found that both the quasi-Newton and relaxation approaches failed to converge and/or resulted in tangled meshes, even though the elements are already aligned with the anisotropy of the Helmholtz solution. The UM2N approach, however, was able to successfully apply mesh movement without tangling. We would also like to clarify that our work targets mesh movement that dynamically adapts to a PDE solution, which is not a replacement for mesh generation in general, so we would always expect a reasonable input mesh when applying our model. In Figure R1, we show a stress test for both MA and UM2N. It can be observed that MA soon fails in the presence of non-convexity in the geometry while UM2N only gets tangled in the most challenging right angle case. It indicates that our method allows a relaxation of the geometric limitations compared to MA, although we admit that there is no theoretical guarantee for no mesh tangling as we mentioned in the limitation.

Response to W5/W6: We appreciate that the reviewer acknowledges that our Tsunami case is an excellent real-world scenario application. We also kindly remind that there are also more applications shown in our supplementary video. We do agree more examples for different areas would better demonstrate the versatility of UM2N. Due to the time limit in the rebuttal period, we would like to leave this to our revisions. For some limited scalability, we further apply our trained model on a flow past cylinder case with ~11k vertices and ~22k triangles as shown in Figure R5. It is shown that our UM2N works well on this case with more degrees of freedom compared to the one in the paper. For an even larger case, we perform stress tests using a time-independent Helmholtz case on our RTX 3090 24 GB GPU and observe that the maximum scale of the problem that can be run on this GPU has ~50k number of elements. The inference time is ~760 ms, the MA method on the same problem requires ~37800 ms with a residual threshold 1e-4. Considering this limitation, applying the memory efficient transformer (linear-attention transformer for further improvement of the inference efficiency) is targeted as future work.

Response to Q2/W8: There are two types of mesh continuity to consider: continuity in space (i.e. mesh smoothness) and continuity in time (avoiding large changes between timesteps). The latter is promoted by training with samples based on Monge Ampere solutions. Based on optimal transport theory, these represent the unique minimal change transformation that satisfies the equidistribution principle provided certain smoothness and convexity criteria are satisfied. Given that this is already the unique optimal mesh that with these properties, quality of elements in terms of their aspect ratio is not explicitly enforced, but the minimal change property does mean a high quality input mesh is transformed to a close-in-quality output mesh as long as the required transformation in terms of the monitor function is not too demanding (e.g. non-smooth). It should be noted it is not the case that equal-aspect triangles always provide the highest quality mesh. This in fact depends on the anisotropy of the PDE solution, and in cases where more control over the aspect ratio is required, other mesh movement methods allow for a tensorial input metric, but this necessarily means the minimal change (continuity) property is no longer satisfied in general. There is thus always a trade-off between different desired properties. Likewise, when asking for a considerable global redistribution of degrees of freedom this may come at the cost of the local mesh quality. Any mesh movement method will therefore be a compromise with UM2N focusing on maintaining minimal movement and maintaining desired output cells areas.

Please refer to Supplement Response to yaVX in general response for more responses Q3/W9, Q4, Q5.

The authors have provided a comprehensive and thoughtful rebuttal, effectively addressing the primary concerns and questions raised about the UM2N architecture. They clarified their focus on mesh movement rather than generation and highlighted the challenges of comparing their work with the limited number of learning-based mesh movement methods available. The additional experiments conducted, especially on poor-quality initial meshes and irregular geometries, demonstrate UM2N’s robustness and versatility. Their detailed responses on the integration of Graph Transformer and Graph Attention Network (GAT) components, the role of element volume loss, and handling mesh deformations provide valuable insights into the technical strengths of their approach. However, as the field evolves, ongoing comparisons with new methods, further exploration of hyperparameter sensitivities, and expansion of real-world applications will strengthen the work’s impact and applicability.

Overall, the authors have made a strong case for the effectiveness and innovation of UM2N, particularly through their emphasis on zero-shot generalization and computational efficiency. The clear explanations and additional experiments help illustrate the model's capabilities in diverse scenarios. I appreciate the thoroughness and clarity of the paper, as well as the authors' proactive engagement with the feedback. The improvements and insightful discussions provided in the rebuttal demonstrate the paper's potential to advance simulation technologies through adaptive mesh movement.