Holistic Physics Solver: Learning PDEs in a Unified Spectral-Physical Space

Thank you for your thoughtful review and insightful questions. They have significantly improved this work. We respond to them below. "#1-#4" provide complements for several important concerns. "#5-#12" provide "point-by-point responses" exactly aligned with your review.

#1 Statistical Significance Analysis

To confirm statistical significance, we perform paired t-tests across multiple runs, confirming confidence levels above 95% for all improvements.

#2 Evaluation Metrics

We test more metrics on the Navier-Stokes in the table below, demonstrating HPM's improvement across all evaluation criteria.

#3 Computational Efficiency

We provide more computational measurements, concluding that the HPM don't increase computational burden.

#4 Design of $H(x,\Phi)$

Softmax-based modulation is a favorable approach with several practical advantages. It ensures balanced scales and lets each point adaptively focus on proper spectral components, providing an effective balance of stability and flexibility.

It is typically challenging to establish theoretical optimality for neural architecture design. Nevertheless, this doesn't diminish our core contribution, a framework that bridges spectral prior with point states. We have provided two principles for $H(x,\Phi)$ in Section 4.5: point-wise processing for local adaptivity, and normalization for balanced information across physical points. They will provide guidance for future explorations.

#5 Claims And Evidence

(1) To statistically validate the improvements, we report stds across multiple runs per dataset, and add statistical significance in "#1". This fully demonstrates HPM's consistent superiority and notable gains in some problems (e.g., Navier-Stokes: 18.4%, Plasticity: 35.0%).

(2) In Table 6, HPM with 4 layers outperforms both baselines, despite having fewer parameters. This confirms our gains come from architectural design, not parameter count.

(3) Please see "#4".

(4) We agree including complex PDEs is important for comprehensive evaluation. We clarify that our benchmark have included significantly nonlinear PDE, the Navier-Stokes with very low viscosity (1e-5), exhibiting turbulence features.

(5) This work goes further by explicitly formulating physical-spectral structure. This theoretical foundation validates that HPM properly extends previous operator frameworks, complementing the empirical results.

#6 Methods And Evaluation

(1) Benchmarks: Please see our response "#2" to Reviewer bPLw, which addresses the same concern.

(2) Baselines: To avoid misleading comparison, we have compared baselines following exact protocol as previous works.

(3) Metrics: Please see "#2".

#7 Theoretical Claims

(1) Please see "(5)" in "#5".

(2) Please see "#4".

(3) Time-series results on Navier-Stokes confirm HPM's spectral stability with reduced error. Section 4.4 demonstrates consistent spectral patterns across resolutions, validating stability beyond training resolutions.

#8 Experimental Designs Or Analyses

(1) We have included state-of-the-art comparisons with advanced Transformer-based solvers (Transolver, GNOT, etc.).

(2) The datasets are standard benchmarks used across the field, ensuring no bias favoring HPM over baselines.

(3) Please see "#3".

#9 Broader Scientific Literature

(1) We will add more related works.

#10 Other Strengths And Weaknesses

(1) Please see "#4".

(2) Please see "(1)" in "#5".

(3) This may be due to the small fonts in Figure 2 and a lack of details. We will fix it.

#11 Other Comments Or Suggestions

(1) Please see "#1".

(2) Please see "#3".

(3) Yes, to isolate component contributions, we have included ablations on $k$ value (Table 9), head count (Table 10) and model depth (Table 11).

#12 Questions For Authors

Q1: HPM and PINNs solve different problems - HPM learns operator mappings across multiple PDE solutions, while PINNs encode PDEs in losses for single-instance solutions.

Q2: Please see our response "#1" to Reviewer bPLw, which addresses the same concern.

Q3: Please see our response "#2" to Reviewer bPLw.

Q4: HPM is promising for higher-dimensional PDEs due to its linear scaling with physical points and high accuracy with fewer frequencies.

Q5: The basis functions are derived directly from geometry and don't undergo training, thus initializations don't affects.

Thank you for your thoughtful review and insightful questions. They have significantly improved this work. We respond to them below.

#1 Handling triangular mesh data

I do have one question to ask, is the method applicable to triangular mesh data? Since FNO is only applicable to structured grid, I am not sure if the method is the same.

Yes, HPM is fully applicable to triangular mesh data. Unlike FNO which is limited to structured grids, our method works effectively on both structured and unstructured meshes. The following is detailed explanation:

(a) We use Laplace-Beltrami Operator (LBO) eigenfunctions [1] as our spectral basis, which can be computed on arbitrary mesh structures, including triangular meshes. This is a key advantage over traditional Fourier-based methods that require structured grids.

(b) Our experiments explicitly demonstrate this capability in Section 4.3, where we evaluate HPM on five unstructured mesh problems: Irregular Darcy, Pipe Turbulence, Heat Transfer, Composite, and Blood Flow. These problems use triangular meshes with node counts ranging from 1,656 to 8,232.

Reference

• [1] A Laplacian for Nonmanifold Triangle Meshes

Thank you for your thoughtful review and insightful questions. They have significantly improved this work. We respond to them below.

#1 Discussion about other generalization scenarios

Question 1: Beyond resolution generalization, would the design of HPM still be beneficial in other generalization scenarios, such as PDEs with different physical parameter settings?

Thank you for this insightful question. Below we provide further discussion about the generalization capability of HPM.

(a) First, we wish to clarify that the strong resolution generalization and limited-data performance naturally stem from the preservation of spectral bias. As derived in Section 3.1, HPM inherits the established inductive bias of fixed neural operators (like FNO) for learning continuous operator mappings. Beyond this, the coupling mechanism allows adaptive spectral modulation based on local physical features, improving the flexibility while maintaining such inductive bias. While not specifically optimized for other generalization scenarios, this unified approach creates fundamental advantages applicable to various generalization tasks.

(b) For specific generalization scenarios, as a versatile neural module, HPM could potentially be combined with dedicated techniques (such as hypernetworks or meta-learning approaches [1,2]) to enhance generalization across different physical parameters. This integration would be a promising direction for future, leveraging HPM's unified representation alongside specialized parameter generalization methods.

#2 More real-world scenarios beyond standard benchmarks

Experimental design has thorough considerations that include structured and unstructured mesh PDEs, generalizability assessments in zero-shot resolution, and the impact of training data. In addition, the author considers ablation studies in spectral coupling functions. One improvement could be the inclusion of some real-world data, such as climate data or smoke plume data, to further demonstrate the robustness of HPM.

Thanks for your recognition for our comprehensive experiments.

(a) First, We'd like to highlight that the standard benchmarks encompass diverse physical scenarios and have been widely adopted by the community [4,5]. They can effectively validate HPM's capabilities across various physical domains, from fluid dynamics to elasticity problems, demonstrating its general applicability and superior performance compared to existing methods.

(b) Additionally, our current experiments already include problems derived from real-world industry scenarios. For example: (1) The Composite problem simulates deformation fields of Carbon Fiber Reinforced Polymer under high-temperature conditions, directly relevant to aerospace manufacturing of jet air intake components. (2) The Blood Flow problem models hemodynamics in the human thoracic aorta, which has significant clinical applications. These real-world examples demonstrate HPM's practical utility beyond synthetic benchmarks.

We agree that expanding to additional domains like climate data or smoke plume simulations would further validate HPM's robustness. We plan to explore more real-world applications in future work to further demonstrate the practical benefits of our unified spectral-physical approach.

Reference

• [1] Meta-Auto-Decoder for Solving Parametric Partial Differential Equations
• [2] HyperDeepONet: learning operator with complex target function space using the limited resources via hypernetwork
• [3] Learning Neural Operators on Riemannian Manifolds
• [4] Fourier Neural Operator for Parametric Partial Differential Equations
• [5] Transolver: A Fast Transformer Solver for PDEs on General Geometries

Thank you for your thoughtful review and insightful questions. They have significantly improved this work. We respond to them below.

#1 Computational cost of LBO

What is the computational cost of precomputing the LBO?

The LBO eigenfunctions is computed efficiently using the robust-laplacian library, with near-linear time scaling relative to mesh nodes for both structured and unstructured meshes. Our empirical evaluation on a single Intel Xeon CPU demonstrates the favorable scaling:

Importantly, this computation is a one-time cost per physical domain, with eigenfunctions reusable across simulations in that domain. It is negligible cost for applications with fixed geometry but varying ICs.

Future work could explore more methods (e.g., learning-based approaches, and FFT-like approaches for uniform grids) to further enhance the computational efficiency.

#2 Chosen of frequency number

And how is the number of bases being chosen?

The number of frequency bases $k$ in HPM is chosen based on several considerations to balance representational capacity and fair comparison.

For most structured mesh problems, we use $k=128$, while unstructured mesh problems use $k=64$. This aligns with: (a) We align $k$ with the hidden dimension to ensure fair comparison with linear attentions, where $k$ corresponds to $Q$, $K$ dimensions. (b) We maintain consistency with spectral methods [1,2], which typically use comparable total frequency numbers. (c) Following [2,3], we make problem-specific adjustments - set $k=128$ for Navier-Stokes despite its larger hidden dimension (256) to control parameter count, and set $k=32$ for Blood Flow due to limited mesh nodes.

Table 9 confirms HPM performs well across different $k$ values, demonstrating the effectiveness of HPM regardless of basis count. While more sophisticated basis selection strategies (such as AFNO [4]) could be explored in future work, current results effectively demonstrates the benefits of spectral-physical integration.

#3 Computational complexity comparison with linear attention

As HPM can be viewed as an extension of linear attention, its computational complexity should also be similar to linear attention ...

Below we provide a concise analysis comparing HPM with linear attentions.

(a) HPM and linear attention share similar computational complexity. Linear attention operates at $O(Nd^2)$ for features $\mathbf{x} \in \mathbb{R}^{N \times d}$, while HPM requires $O(Ndk)$ operations, where $k$ represents number of frequency basis. The main computations include computing coupling function at $O(Nk)$ cost, and forward/inverse transforms at $O(Ndk)$.

Since $k$ is typically much smaller than $N$ and comparable to $d$ (e.g., $k=d=128$, $N=11271$ for Airfoil), HPM maintains efficiency like linear attentions.

(b) Our measurements below confirm this analysis. On Airfoil, Galerkin attention requires 16.5ms for inference compared to HPM's 17.4ms. This minor overhead enables HPM to capture both domain-level structure and point-wise adaptivity, delivering significant advantages.

#4 In-depth discussion about AFNO

Despite the author cited AFNO in the paper, I think it is worth more ...

Thanks for this insightful suggestion. We'll include a comprehensive discussion about AFNO in the methodology section of final version.

While AFNO [4] also incorporates data-dependent modulation, HPM differs in several aspects: (a) Objective: AFNO aims to improve "efficiency and robustness" of spectral features by sparsifying the frequency modes. In contrast, HPM focuses on "enhancing the flexibility of preset spectral features" via point-wise modulation. (b) Methodology: AFNO selects and modifies modes in the spectral domain post-transformation, while HPM introduces spatial domain modulation pre-transformation, enabling point-wise flexibility while preserving spectral structure. (c) Application: AFNO targets computer vision tasks, whereas HPM addresses PDE challenges requiring both global spectral coherence and local adaptivity.

The spectral processing technique from AFNO could complement HPM in future work, potentially combining both benefits (AFNO's efficiency, HPM's flexibility) for diverse applications.

Reference

• [1] Fourier Neural Operator for Parametric Partial Differential Equations
• [2] Learning Neural Operators on Riemannian Manifolds
• [3] Transolver: A Fast Transformer Solver for PDEs on General Geometries
• [4] Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers