Physics-Informed Deep B-Spline Networks

Thank you for your comments.

Q1: While the authors provide a good theoretical basis for B-spline based PINNs, the experimental section is quite small. They evaluate on 2 settings. However, it would be good to showcase the performance on other standard evaluation benchmarks (Navier-Stokes, Darcy Flow, Burgers, Kolmogorov flow etc.)

A1: We have added additional experiment results on Burgers' equation in Appendix D.6. The results of PI-DBSN are comparable to Fourier Neural Operators as reported in [4], in terms of prediction error.

Q2: How does the proposed approach compare against [1], [2], [3]?

[1] Wandel, Nils, et al. "Spline-pinn: Approaching pdes without data using fast, physics-informed hermite-spline cnns." Proceedings of the AAAI conference on artificial intelligence. Vol. 36. No. 8. 2022.

[2] Doległo, Kamil, et al. "Deep neural networks for smooth approximation of physics with higher order and continuity B-spline base functions." arXiv preprint arXiv:2201.00904 (2022).

[3] Zhu, Xuedong, et al. "A Best-Fitting B-Spline Neural Network Approach to the Prediction of Advection–Diffusion Physical Fields with Absorption and Source Terms." Entropy 26.7 (2024): 577.

A2: The proposed approach is different from the mentioned existing literature in the following ways.

For [1], firstly, Hermite splines and B-splines are two different methods for different problems. In general, Hermite splines are for interpolations, while B-splines are for approximations. The proposed method is different from [1] as we leverage B-spline properties to address an approximation problem. Secondly, with Hermite splines, one needs to “choose the right spline order $n$ to be able to compute the physics-informed loss for a given PDE” according to [1], while we do not have such issues. Finally, [1] proposes to use CNN to learn mappings from control points at time $t$ to control points at $t+1$, while we directly learn control points for the entire state-time domain.

For [2] and [3], both papers learn B-splines to approximate PDEs, but they only consider data fitting problems, where no physics model loss is imposed in the learning process. Instead, we propose physics-informed deep B-spline networks and show that the analytical calculation of derivatives for physics model losses and direct assignment of ICBCs are advantageous over existing physic-informed learning methods.

[4] Li, Zongyi, et al. "Fourier neural operator for parametric partial differential equations." arXiv preprint arXiv:2010.08895 (2020).

Thank you for your comments. We have revised the paper to correct the typos and ambiguous claims you raised.

Q1: Figure 3 indicates PINN attains average better performance. It is also indicated in line 387-388 that prediction MSE of PI-DBSN is worse compared with PINN. Can you provide more details of network setup, and explain "similar NN configuration" stated in line 364-365. How many parameters of NN in each setup and how computation time would change if one set up same or similar number of parameters in each experiment.

A1: The loss functions of PINN and PI-DBSN are slightly different, as PINN takes initial condition and boundary condition (ICBC) losses as well, while PI-DBSN can directly specify ICBC values. The loss plot in Figure 3 is normalized, thus PINN appears to have lower overall loss values. From the prediction results we know that the test performance of PINN and PI-DBSN are comparable, and note that PI-DBSN (MSE $3.064 \cdot 10^{-4}$) has slightly better performance than PINN (MSE $4.323 \cdot 10^{-4}$). We use exactly the same NN architecture for PI-DBSN and PINN (3-layer fully connected NN with ReLU activation functions, with $64$ neurons in each hidden layer), and the same NN for both trunk net and branch net for PI-DeepONet. We have also added NN size ablation experiments in Appendix D.5.

Q2: What is the residual error in convection-diffusion setups? All norms in loss functions $\mathcal{L}$ are of form $|\cdot|$ (line 223-229). Is this $l^2$ norm or $L^2$ norm or absolute value? Please specify.

A2: The residual error in the convection-diffusion setup is defined as the mean square error on testing data points, which are generated on the mesh grid of the domain of interest $(x, t) \in [-10, \alpha] \times [0, 10]$ with $dx = 0.1$ and $dt = 0.1$. The $|\cdot|$ used in evaluating data and physics losses denote absolute values. We have added such information in Appendix C.4 to elaborate on evaluation metrics.

Q3: How does a plot of number of NN parameters v.s. MSE. looks like? In table 2 only the relationship between number of control points v.s. MSE is reported.

A3: We have added ablation experiments of NN size in Appendix D.5. In general we do not observe much difference in performance by varying the NN size, for our case study.

Q4: Are all experiments conducted over noiseless setup? Is PI-DBSN robust against noisy observation? Is there any ablation study on how to choose $w_p$ and $w_d$ to attain better performance? In line 230, authors claim both are set to 1.

A4: We have added ablation experiments on robustness against noise and different loss function weights in Appendix D.4. We observe that when the training data is noiseless, increasing data loss weight $w_d$ helps with performance as the network better learns the ground truth data. When the training data is noisy, increasing physics loss weight $w_p$ helps with enforcing the neighboring relationships in the PDE, resulting in better prediction accuracy.

Q5: What is the performance of PI-DBSN over Burger's equation and Navier-Stokes equation using the same setup in FNO paper? Are there any specific reasons that the comparison ruled out FNO branch of method?

A5: We have added experiment results on Burgers' equation in Appendix D.6. The results of PI-DBSN are comparable to FNO as reported in [1], in terms of prediction error.

Q6: How is physics model loss being evaluated. Namely, what is the quadrature rule when computing $|\mathcal{F}_i(s, x, u)|^2$?

A6: The physics model loss is evaluated as the squared residuals of the PDE on a mesh grid of the domain of interest, similar to PINNs [2]. We have elaborated the definition in the paper to $\mathcal{L}_p = \sum_i \sum _\mathcal{P} \frac{1}{|\mathcal{P}|} |\mathcal{F}_i(s, x, u)|^2$ where $\mathcal{P}$ is the set of points sampled to evaluate the governing physics model. Specifically, for the recovery probability experiment, $\mathcal{P}$ is the mesh grid on the domain of interest with $dx = dt = 0.1$, and for the 3D heat equation is the mesh grid with $dx = dt = 0.01$. We have added such information in the revised version of the paper.

Q7: Though not required due to extensive work of experiments, I wish to, at least hear from authors, if complex geometric domain, such as $L$-shape problem introduced in Doleglo et al. (2022), or hyper-elastic problems setup in Geo-FNO can be computed via PI-DBSN.

A7: Please see response to common questions in our General Response.

[1] Li, Zongyi, et al. "Fourier neural operator for parametric partial differential equations." arXiv preprint arXiv:2010.08895 (2020).

[2] Raissi, Maziar, Paris Perdikaris, and George E. Karniadakis. "Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations." Journal of Computational physics 378 (2019): 686-707.

Thank you for your comments.

To address your concerns, we have added additional results on Burgers' equations (Appendix D.6), which is widely considered in the literature. Our proposed PI-DBSN method achieves comparable prediction results against Fourier Neural Networks (FNO) [1] in this test case. We have also added discussions on the limitations and future directions in the revised conclusion section. We point out that our proposed method has similar scalability compared with other physics-informed learning methods, but the computation complexity is drastically reduced because of the direct assignment of ICBC values and analytical calculation of derivatives. The advantageous training time is visualized in Figure 3 and quantitatively reported in Table 1.

[1] Li, Zongyi, et al. "Fourier neural operator for parametric partial differential equations." arXiv preprint arXiv:2010.08895 (2020).

Thank you for your comments.

Q1: In the experiments part, for 3D heat equation, it seems that there is no quantitative results for the accuracy. It is unclear how the proposed method perform comparing to benchmarks especially PINN on this case.

A1: For the 3D heat equation case study, the average residuals for PI-DBSN during training and testing are $0.0028$ and $0.0032$, as reported in section 5.2. We have added the results of PINN on this example in Appendix C.6. The testing residual for PINN is $0.0121$, which is higher than the reported value ($0.0032$) for PI-DBSN.

Q2: In Figure 3, though PI-DBSN converges faster than PINN, the final training loss is much higher than the averaged loss of PINN. In addition, the PI-DBSN almost converge in less than 500 epochs. Does the training process of PI-DBSN not converge properly? Or the test case is too simple so that the ability of the proposed method is not fully tested?

A2: The loss functions of PINN and PI-DBSN are slightly different, as PINN takes initial condition and boundary condition (ICBC) losses as well, while PI-DBSN can directly specify ICBC values. The loss plot in Figure 3 is normalized, thus PINN appears to have lower overall loss values. From the prediction results we know that the test performance of PINN and PI-DBSN are comparable. However, the training of PI-DBSN is much faster than PINN - we do observe that PI-DBSN converges within ~500 epochs, thanks to the compact representation and the fact that only control points are learned.

Q3: The authors claim that "we can directly specify Dirichlet initial and boundary conditions through the control points tensor without imposing loss functions, which helps with learning extreme and complex ICBCs". It seems there is no test case with such a extreme and complex ICBCs in the paper. Therefore it is unclear how the proposed method shows advantages on ICBC. Would you minding providing such a case that benchmarks such as PINN and PI-DeepONet fail while the proposed method works?

A3: We believe the convection diffusion equation example in section 5.1 serves the purpose, as the initial and boundary condition is discontinuous at $(x, T) = (\alpha, 0)$. PI-DeepONet fails to learn this PDE, and PI-DBSN is much faster than PINN for this problem.

Q4: It seems that all the performance comparisons are all run on CPU, would you minding providing some results on GPU? If it is hard to get a GPU version, how do you think the results will diff from that on CPU? As GPU might be more friendly for methods with larger NNs for example PINN.

A4: We added experiments on GPU performance in the revised version of the paper (Appendix D.3). We observe consistent results compared to CPU performance (PI-DBSN faster than two baselines), while GPU will accelerate all methods as expected.

Q5: What is the scale/problem size of the test cases? Can the proposed method scale to much larger test cases?

A5: The scale/problem size of the proposed method is comparable to other physics-informed learning methods such as PINNs, where usually PDEs with dimensions less than 4 are considered. While the naive construction of the proposed method also suffers the curse of dimensionality, we do not see any issue if any problem-specific dimensionality reduction techniques can not be applied to our method if they could be applied to other physics-informed learning methods.

Q6: The current test cases have relatively simple geometry i.e., rectangular domains, is it possible to apply the proposed method to scenarios with more complex boundary geometries (e.g., arbitrary geometry shape )? or non-convex domain (such as flow past a cylinder)

A6: Please see responses to common questions in our General Response.