PINNs with Learnable Quadrature

We thank the reviewer for the constructive and insightful feedback. We address each point below and will incorporate the relevant clarifications and improvements into the final version of the paper.

Q1. Scope of Experiments: Why are all examples 1D in space?

Thanks for bringing this up. Our primary intent was to show the effectiveness of LearnQuad on benchmark problems used in prior PINN literature. As mentioned in line 291, we evaluated our method on standard PDEs from several baselines to enable direct comparison. That said, our formulation (Remark 4.1) naturally can be extended to higher-dimensional settings via tensor-product or sparse-grid constructions. We have checked that in principle the implementation works for simple cases, but these need to be developed in a separate follow-up work. We are happy to include additional results in the final version if the reviewer recommends a specific benchmark PDE.

Q2. Notation clarity: Confusion around $\theta, \phi$, and their roles in Algorithm 1 and Figure 4.

Many thanks for pointing this out. Yes there was some notational inconsistency in the original draft. We have revised the text and figures to consistently denote:

$\theta$: parameters of the weight function $w_\theta​$

$\phi$: parameters of the solution function $u_\phi​$

In Algorithm 1, the line $w_\phi$ from $\phi(\mu)$ has a typo and was intended to show that the weight function may optionally depend on PDE parameters $\mu$. However, this caused confusion since the dependence was not consistently carried forward. We have now removed this line. Figure 4 has been revised to indicate more clearly that $\theta$ governs the quadrature module and $\phi$ the solution function.

Q3. How does the learned weight function compare to classical ones like Gauss-Jacobi? Can you show plots or examples?

This is a nice suggestion. We observe that the learned weight functions differ quite a bit from classical choices. While Gauss-Jacobi weights have fixed singularities at the endpoints, our learned modifier $h_\phi(x)$ dynamically adapts the weight distribution based on the PDE solution’s characteristics. Empirically, we find that the collocation points tend to cluster in high density regions and re-distribute them as training proceeds, just as desired. We will include representative plots comparing learned versus classical weights and their induced node distributions in the final version to illustrate these differences clearly.

Q4. Equation (20) and clarity about trainable parameters: Are the quadrature weights $w_i$ themselves learnable?

We agree that Equation (20) could be better explained. Yes your interpretation is correct, only the weight function $w_\theta(x)$(parameterized via a neural network) is learnable. The quadrature weights $w_i$ and nodes $x_i$​ are computed numerically from the learned weight function using asymptotic expansions (Section $5$), and not directly learned. We have revised the text to clearly emphasize that the model learns the modulator $h_\phi(x)$, from which quadrature rules are derived numerically.

Q5. Clarification on “interlacing of roots” and how it mitigates overfitting. What does “overfitting” mean here?

We appreciate the question in this regard and make an effort to clear the confusion. Given a weight function, it induces a family of orthogonal polynomials each of a different degree. LearnQuad utilizes the roots of these polynomials by computing them using asymptotic expansions. While the weight function is being learnt, we observed that only sampling the roots of a fixed degree lead to local minima and the solution function didn’t generalize well to unknown points in the domain. To mitigate this, we leverage a classical property of orthogonal polynomials: interlacing of roots. The roots of the degree $n+1$ polynomial interleave those of degree $n$, as shown in Figure 3. Hence, by using different degrees to sample roots in different epochs leads to a drastically better generalization of the learnt solution. We thank the reviewer for pointing this out and have updated the manuscript to clearly describe this mechanism, its theoretical motivation, and its empirical benefit to generalization.

To Summarize:

We will improve clarity throughout the manuscript, particularly in Section 5 and Algorithm 1. We will revise Equation (20) to explain more precisely how it prevents degenerate quadrature weights and enforces normalization. As suggested, we will add visualizations of the learned weights and node distributions. Code will be released to enhance reproducibility and clarity.

We thank the reviewer again for their thoughtful evaluation and helpful suggestions, which we believe will greatly improve the final version of the paper.

We thank the reviewer for the thoughtful and encouraging evaluation. We deeply appreciate the recognition of the novelty and principled nature of our approach, as well as the constructive suggestions. Below, we address each of the reviewer’s comments and questions in detail.

Q1. Clarification on parameterization and network architecture

We appreciate the reviewer’s request for a more cohesive description of the parameterization strategy and network architecture. The architecture details are currently included in Appendix §10.1.6. To summarize: for both the solution function and the weight function (i.e., the LearnQuad module), we use fully connected multilayer perceptrons with depth $3–5$ and width ranging from $50$ to $200$, depending on the PDE. We use tanh as the activation function throughout. We acknowledge the wording in line 230 (“linear layers”) is misleading and will revise this to explicitly state the use of MLPs. Additionally, we will expand Algorithm 1 in the final version to provide a more comprehensive description of how quadrature points and weights are computed and how they are integrated into the end-to-end training loop.

Q2. Benchmarking against non-PINN PDE solvers

Thank you for this suggestion. In response, we conducted a quick experiment comparing the performance of existing numerical solvers and PINN with LearnQuad on three representative PDEs with known analytic solutions (the details of which are in Appendix 10). We find that LearnQuad consistently closes the gap between existing (best adaptive) PINNs and classical solvers in terms of $L_2$ relative error, while maintaining computational efficiency. These results will be included in the final version of the paper, and we welcome further recommendations for such comparisons.

$L_2$ Relative Error Comparison: Numerical Solvers vs. Adaptive PINNs

Legend:
L-W = Lax-Wendroff; BW = Beam-Warming; FE = Forward Euler; BE = Backward Euler; CN = Crank-Nicolson; LF = Leapfrog; CFD = Centered FDM; NB = Newmark-beta; R3, RAR-G, RAD = existing adaptive PINNs; LQ = LearnQuad (ours)

Q3. Applicability beyond low-dimensional problems and standard PDEs

We agree with the reviewer that evaluating LearnQuad on more complex and real-world PDEs including irregular domains and higher dimensions is a valuable direction ripe for future work. As noted in line 291, our current PDE choices follow established baselines to enable direct empirical comparison. We also mention that extension to higher dimensions can involve the use of tensor-products or sparse grids to make it computationally efficient. If there is a specific PDE the reviewer recommends we are happy to include it in the final version.

Q4. Choice of $\alpha$, $\beta$, and expansion coefficients:

Thank you for raising this point. In Appendix §10.1.8 (Table 6), we performed an ablation study on various choices of $\alpha$ and $\beta$ in the modified Gauss-Jacobi weight function. We observed that LearnQuad's performance shows minor variations to these hyperparameters. Based on this, we chose $\alpha, \beta \in$ 1,2 $$ for all experiments reported in the paper. We emphasize that the choice was not arbitrary, but rather empirically sufficient to achieve strong performance across PDEs, without requiring exhaustive hyper-parameter sweeps. We will revise the text to reflect this clearly. As for the expansion coefficients (e.g., $c_0, d_0, \tau_0, \tau_1$​), these are not fixed but are learned from data as part of the weight function network, as described in lines 232–236. We will make this explicit in the main text to improve clarity.

Q5. Assumption of real roots in asymptotic expansion (Eq. 18)

This is an important technical point. Our derivation assumes that the domain of the PDE is real-valued, which ensures that the roots of the orthogonal polynomials are real and simple. This assumption holds for all PDEs considered in the paper and for most physical PDEs encountered in practice. However, as the reviewer rightly points out, this assumption could break down in special cases where PDEs are defined on the imaginary plane. We will include this clarification and limitation explicitly in the final version.

Q6. Impact of quadrature loss term in Equation (20)

The first term in Equation (20) enforces that the sum of quadrature weights approximates the integral of the learned weight function. This is indeed a relaxation of the exact constraint. Empirically, we observe that this term typically converges to a value of order $e^{-3}$ or lower. If we don’t include this term, the final performance is worse and also, its inclusion enforces that the quadrature weights are valid.

Q7. Adversarial training of the quadrature module

We explored the idea of learning quadrature points adversarially — where one module proposes points and the other learns under this challenging distribution. However, in our experiments, training both modules jointly (end-to-end) or in an alternating min-max fashion resulted in comparable performance. Adversarial training is known to be more unstable and harder to tune. Without careful balancing (e.g., learning rates, update schedules), it may converge to a similar fixed point as joint training, effectively reducing to the same solution. This could have been an issue specific to our implementation but we were unable to get it working. If there are specific papers we can use as guidance, we welcome suggestions.

Q8. Clarification on “linear layers” in line 230

We apologize for the inconsistent wording. As detailed in the appendix, we use multi-layer perceptrons (MLPs) with $3–4$ hidden layers for both the solution and quadrature modules. Each hidden layer consists of fully connected (linear) operations followed by tanh non-linearities. We will correct the text in line 230 to reflect this architecture accurately. Thank you for pointing this out.

We sincerely thank the reviewer for their thoughtful engagement and strong endorsement. We believe that the clarifications and additions outlined above further strengthen the paper and reinforce the rigor and applicability of LearnQuad.

Thanks for your detailed reply and for conducting the additional experiments. All my questions and concerns are well addressed, and I look forward to seeing them incorporated in your final draft.

We thank the reviewer for their thoughtful comments. We begin by summarizing our understanding of the core concern and then offer detailed responses.

Core Concern: We appreciate the pointer to McGreivy/Hakim’s important critique of weak baselines in ML-for-PDEs. We agree with the call for transparent comparisons and support benchmarking against classical solvers at equal accuracy or runtime. We already align with many of these principles—comparing all sampling/PINN methods under identical setups (same network, optimizer, and compute budget). We also clearly acknowledge that PINNs are typically more expensive and less accurate than classical solvers for forward problems. What we had not done is explicitly quantify this gap. Since this is a key factor in your assessment, we now address it directly. We have benchmarked strong non-PINN solvers on multiple problems and report these results prominently. Rather than undermining our work, we believe these comparisons reinforce our contribution: LearnQuad advances PINN performance over existing sampling methods, though we do not claim it closes the gap with classical solvers.

Q1. Other Reference:

We find the developments in the second and third papers to be complementary to us. The second introduces (a) an annealed learning rate and (b) a refined network architecture; the third proposes (a) a modified optimizer and (b) an adjusted loss. LearnQuad is orthogonal to these techniques—our focus is adaptive sampling via a principled quadrature framework using orthogonal polynomials, which precedes the solution network. In contrast, the cited methods enhance optimization and modeling. Thus, LearnQuad can be seamlessly integrated with them. To validate this, we ran quick experiments combining LearnQuad with proposed setups and observed consistently improved or comparable results. We will include these findings in the final version. This reinforces our message: LearnQuad is a modular, drop-in enhancement that boosts PINN performance across architectures and optimizers.

Q2. Comparison (Choice of PDE and baselines):

We urge the reviewer to look at Table 1 and Table 2 in the main paper which has the following PDEs: Burger’s, Allen-Cahn, Wave, Convection with 1000-200 points. The Diffusion equation with 30 points is indeed a toy setting and hence included in the Appendix. The multiple baselines are chosen following major works which introduce adaptive sampling methods as mentioned in l291.

Q3. Comparison to classical solvers:

We fully agree with the reviewer that PINNs, in their current form, often fall short of classical numerical solvers in accuracy, efficiency, and robustness—a well-known limitation in the field. Our work does not claim that this is not the case. In fact, we explicitly acknowledge that classical solvers (e.g., finite difference, finite element, spectral methods) generally outperform PINNs on well-posed PDEs.

However, our goal is not to argue for PINN superiority, but to contribute to ongoing efforts to improve PINN performance and understanding. The gap between PINNs and traditional solvers is real, and our work aims to help narrow it—not by surpassing classical methods, but by enhancing generalization and accuracy through adaptive, learnable quadrature schemes. Importantly, our contribution is orthogonal to the broader debate about whether PINNs can eventually match classical solvers. We focus on one key aspect: sampling efficiency, which directly impacts the quadrature error term in generalization bounds (Mishra & Molinaro, 2023). By improving training-time sampling distributions, LearnQuad yields significant empirical gains. In short, LearnQuad is not meant to outperform standard numerical methods like finite elements or finite differences—but to improve over standard PINNs. We believe this is a necessary step if PINNs are to become viable PDE solvers.

$L_2$ Relative Error Comparison: Numerical Solvers vs. Adaptive PINNs

Legend:
L-W = Lax-Wendroff; BW = Beam-Warming; FE = Forward Euler; BE = Backward Euler; CN = Crank-Nicolson; LF = Leapfrog; CFD = Centered FDM; NB = Newmark-beta; R3, RAR-G, RAD = existing adaptive PINNs; LQ = LearnQuad (ours)

In response to the reviewer’s suggestion, we benchmarked LearnQuad against classical solvers on three PDEs with known analytic solutions (details in Appendix §10). As expected, standard methods (e.g., Lax-Wendroff, Crank-Nicolson), still achieve the lowest relative $L_2$ errors. However, LearnQuad consistently improves PINN performance across all cases and notably outperforms the strongest adaptive PINN baselines. These results are not to suggest superiority over classical solvers, but to contextualize the scale of gains LearnQuad brings within the PINN framework. We hope this clarifies the scope of our contribution. We will highlight these results in the final version.

Q4. Theoretical Analysis:

We thank the reviewer for pushing the need for theoretical justification. While our main contribution is algorithmic, LearnQuad is grounded in recent theory on PINN generalization—most notably Mishra & Molinaro (2023). Their Theorem 2.6 decomposes the generalization error into training error $E_T$ and a quadrature term $C_{\text{pde}} C_{\text{quad}}^{1/p} N^{-\alpha/p}$, where $C_{\text{quad}}$ depends on the sampling distribution. This shows that sampling is a central factor in solution quality—not a secondary detail.

LearnQuad directly targets the quadrature term by learning a sampling distribution that concentrates collocation points where the residual is high. This lowers $C_{\text{quad}}$ without modifying the PINN architecture or PDE model. It offers a practical means to realize the theoretical insights of (Mishra & Molinaro). We will make this connection explicit in the final version. The link is direct—Theorem 2.6 gives an interpretable bound that LearnQuad implements in a scalable, theory-aligned way. LearnQuad is a principled, theoretically grounded adaptive sampling method that minimizes one of the two main sources of PINN error.

Mishra, Siddhartha, and Roberto Molinaro. "Estimates on the generalization error of physics-informed neural networks for approximating PDEs." IMA Journal of Numerical Analysis 43.1 (2023):1-43

Q5. Why are the sampled points better than Monte Carlo?

The case for adaptive sampling outperforming uniform Monte Carlo in PINNs has been well established by prior works cited as baselines. These show that allocating points in regions of high residual or solution complexity improves generalization. Our results align with this: LearnQuad consistently outperforms vanilla Monte Carlo PINNs and other adaptive methods across benchmarks. We introduce a principled approach to adaptive sampling via a learnable quadrature scheme. During training, the sampling distribution is not fixed—it evolves by learning a weight function informed by the PDE residual. This function induces a probability distribution that biases collocation points toward informative regions. A key challenge in adaptive sampling is that drawing from such tailored distributions is often costly or complex. We address this by using asymptotic expansions to generate quadrature points quickly and stably from the learned weights, making our method both scalable and easy to integrate with existing PINN models.

We believe the reviewer will agree that adaptive sampling from problem-specific distributions generally outperforms uniform sampling, especially when residuals or features are spatially non-uniform. By formulating sampling via asymptotic expansions and learnable weights, we enhance performance and offer a general-purpose tool that integrates smoothly into broader learning pipelines. We hope this is seen not just as an empirical gain, but as a step toward scalable, theory-informed adaptive sampling in physics-informed learning.

Q6. Limitations:

The reviewer is correct in pointing this out, since PINNs are a vessel for LearnQuad we had outlined the limitations when used in this setting, despite LearnQuad improving the performance of PINNs are demonstrated empirically. We are happy to expand the section to include a discussion of challenges in extension to higher dimensions which we believe is a ripe direction for future research.

Q7. Paper formatting Concern:

We have fixed typesetting errors and inserted commas after “where”. L101 we meant it is being parameterized using neural networks, the weights of which are learnt. We adjusted the abstraction level, made style consistent throughout the paper. We are happy to remove the color. We have updated the tables in the Appendix to a more standard format.

We thank the reviewer for their thoughtful critique, which helped refine our scope and positioning. We hope our clarifications—on benchmarking, theoretical grounding, and compatibility with recent PINN advances—highlight that LearnQuad is a principled, modular improvement that strengthens PINNs and moves them closer to bridging the gap with classical solvers.

We thank the reviewer for the thoughtful and constructive feedback. We address each point in detail below and will incorporate clarifications and additional results into the final version of the paper.

Q1. Performance scaling with number of collocation points: Is the proposed method still advantageous when more points are used? Is asymptotic expansion numerically stable at large scale?

Our experimental setup (Tables 1 and 2) uses $1000–2000$ collocation points only to maintain consistency with baseline works cited in line 291. However, LearnQuad, by design, scales efficiently to a much larger number of points, enabled by the use of asymptotic expansions. To empirically validate this, we reported runtime and scaling behavior in Table 7 (Appendix Section 11.1) and request the reviewer to take a look. To summarize, the wall-clock time to generate quadrature nodes and weights remains nearly flat up to $10,000$ points. Beyond that, we observe modest sub-linear growth. This can be attributed to GPU memory bandwidth bottlenecks, not to any instability or complexity of the core method. As the number of nodes reaches millions, transferring tensors associated with nodes between compute units and memory increases runtime. This memory bandwidth issue specific to our GPU can be mitigated (although not eliminated) by higher-end hardware and/or optimized implementation. The core algorithm, by design, remains fully parallel. In Table 5 and Section 10.1.7 in the Appendix we include numerical results obtained on increasing the number of collocation points. As demonstrated the results improve on using more points in general. We are happy to expand on this section and include larger scale experiments in the main paper.

Q2. Smoothness of PDE solutions: Does LearnQuad retain its advantage for more complex or chaotic PDEs?

We appreciate the reviewer’s concern regarding generalization to more complex PDEs. Our current benchmarks (Table 1 and 2) do include non-trivial settings like Burgers' and Allen-Cahn equations. As mentioned in Appendix 10.1.3, the value of $\nu=0.01/\pi$ leads results in a non-smooth solution with steep gradients. Furthermore, the solution to the wave equation used in the experiments as outlined in Appendix 10.1.5 demonstrates multi-scale behavior in space and time. We mention in l291 how existing baseline papers informed our choice of the PDEs. That said, we agree that applying PINNs (with or without LearnQuad) on much more complex PDEs remains open. We note that the adaptivity of LearnQuad driven by a learnable weight function allows collocation points to cluster dynamically/adaptively in regions of high solution complexity. As the number of collocation points increases, the method redistributes these adaptively, we show this in Appendix Table 5. We would be happy to include additional PDEs with more challenging behavior – we have covered a mostly exhaustive set covered in adaptive PINN papers (see lines 291) but welcome specific suggestions from the reviewer.

Q3. Computational overhead of LearnQuad: How does the training time compare to baselines? How does it scale with the number of points?

Thanks for bringing up this point. We compare wall-clock training time across all methods in Table 8 (Appendix §11.2). We find that our Jax implementation of LearnQuad is significantly faster than other adaptive methods. As reported, compared to vanilla PINNs LearnQuad is bit more expensive due to an additional compute operations but converges faster apart from improving the accuracy of the solution. Additionally, we analyze the scaling behavior of LearnQuad with increasing collocation points in Table 7. The time to generate quadrature nodes and weights remains nearly flat up to $10,000$ points. Beyond that, we observe modest sub-linear growth. Quadrature generation and solution training are fully parallelized and GPU-compatible.

Q4. Hyper-parameter tuning for $\alpha$ and $\beta$ in Eq. $(10)$: How sensitive is performance to these parameters?

We reported an ablation study in Table 6 (Appendix §10.1.8), which explores variations of $\alpha$ and $\beta$ in the modified Gauss-Jacobi weight function. We observe only minor variations in performance. Based on this, we fixed $\alpha$ and $\beta$ to moderate values (typically in the range 1–2) for all experiments in the main paper. This choice turned out to be sufficient to achieve the desired performance across tasks, without requiring extensive hyper-parameter sweeps. We are happy to expand this section in the final version of the paper.

Q5. Applicability to high-dimensional PDEs: Does LearnQuad remain accurate and efficient? Any training challenges?

This is a very good point. An extension of LearnQuad to higher dimensions is indeed possible and incurs some increased memory (which would be the case for PINNs in general) due to the requirement of sampling more points in multiple dimensions to reach a desired error. In higher dimensions techniques of tensor-products or sparse grids can help with reducing the computational complexity. The time complexity of LearnQuad does not explode with increasing dimensions due to parallel compute. However, as mentioned earlier (Table 7) one can expect a a moderate increase with sampling an extremely large number of collocation points. When we performed a simple experiment on a 100 dimensional Poisson equation with a closed form ground truth solution: $-\Delta u = -200,\quad x \in (0,1)^{100};\quad u(x) = \sum_{i=1}^{100} x_i^2,\quad x \in \partial(0,1)^{100}$.

We observe LearnQuad achieves a relative $L_2$ error of 0.085 which is similar to using naive Monte Carlo (with the same number of collocation points) in this setting with relative $L_2$ error of 0.09. This illustrates the viability of LearnQuad for high dimensional PDEs although this is a preliminary finding. Since the solution is smooth in this particular case, there is no substantial benefit in using a data-driven adaptive method. For this problem, our training took 18 seconds to converge in 300 epochs. After this, evaluating the trained model on any given resolution takes 0.0065 seconds. During the testing phase errors were computed with respect to the true analytical solution which is readily available in this case.

Q6. Contrast with Neural Operators: How does its performance compare?

We appreciate the reviewer’s interest in this connection. The reviewer is right that with the help of LearnQuad PINNs can be extended to provide the functionality of neural operators albeit without the use of any labelled data. We believe that this is a step towards bridging the gap between the two modelling paradigms. In the problems considered in the paper, neural operators perform equally well but as the reviewer would agree we needed to provide it with labelled data. We are happy to include these results in the final version.

We once again thank the reviewer for their detailed evaluation and helpful suggestions. We believe these clarifications and additions strengthen the contributions of LearnQuad and demonstrate its applicability across a broad range of PDE-solving tasks.

Dear Reviewer,

We sincerely appreciate the time and feedback you provided. If any part of our rebuttal needs further clarification or prompts additional questions, we’d be more than happy to elaborate. We also welcome any updates you might consider making to your recommendation in light of the new information provided.

Thank you again for your thoughtful review and for helping improve the quality of our work.