CoPINN: Cognitive Physics-Informed Neural Networks

We appreciate your detailed comments. We believe the following point-to-point response can address all the concerns:

Q1: Weaknesses (1) and Questions For Authors(4)

R1: Compared with Helmholtz and (2+1)-d Klein-Gordon datasets, (3+1)-d Klein-Gordon and Diffusion datasets are relatively less challenging. Therefore, almost all the methods achieve good results. In fact, because most of these two datasets are easy, which leads to limited mining of difficult samples by our CoPINN, and the performance advantage is relatively unobvious. In fact, CoPINN yields improvements of $50\\%$ (Klein-Gordon in (3+1)-d) and $10\\%$ (diffusion equations).

Q2: Weaknesses (2)

R2: We claim that PINN and FPINN can solve high-dimensional PDEs. However, as the number of training (collocation) points increases, especially for higher-dimensional or more complex problems, their computational burden becomes more pronounced. To solve this problem, we adopt a separable architecture to efficiently handle high-dimensional PDEs. In addition, we also compare with SPINN (which is good at solving high-dimensional PDEs), so the experimental setting is fair.

Q3: Questions For Authors(1)

R3: Separable learning and Cognitive Training Scheduler are two independent modules. To be specific, Separable Learning mitigates the computational burden in high-dimensional PDE solutions. Conversely, the Cognitive Training Scheduler assesses sample-level difficulty, enabling neural networks to fit samples from easy to hard when solving PDEs. Cognitive Training Scheduler is plug-and-play and can be used for the vanilla PINN.

Q4: Questions For Authors(2)

R4: We apologize for the lack of clarity. Following the setup of SPINN, for Helmholtz, (2+1)-d Klein-Gordon, (3+1)-d Klein-Gordon, Diffusion and flow Mixing 3D equations, we set the sampling points to $100^3, 100^3, 50^3, 101^3$, and $100^3$ during testing, respectively. We will include this in the implementation details of the next version.

Q5: Questions For Authors(3)

R5: Through parameter sensitivity analysis in Figure 5, we find that the performance is better when the $\beta$ value is $0.01$ to $0.00001$. In fact, if $\beta$ is fixed to $0.001$, CoPINN achieves SOTA on all datasets. Therefore, for other PDEs, such as the N-S equation, we recommend that the $\beta$ value be selected in $\\{0.01, 0.001, 0.0001, 0.00001\\}$.

Q6: Questions For Authors(5)

R6: In Section 3.3 of our original paper, we conduct an ablation study to analyze the effectiveness of each component in the cognitive training scheduler of our CoPINN. CoPINN-1 represents using the original loss function, i.e., removing the Cognitive Training Scheduler. The results in Table 2 show that the performance of CoPINN-1 is lower than our proposed CoPINN.

We greatly appreciate your valuable comments. Below is our point-by-point response.

Q1: Essential References Not Discussed & Questions For Authors (1)

R1:

1. We will include the discussion about adaptive sampling in the Introduction and Related Work Sections of the next version. The key to adaptive sampling is to improve the learning efficiency and accuracy of the neural network by dynamically selecting or reallocating sampling points. Specifically, for [1] you mentioned, it balances regions of high and low residuals by dynamically resampling. For [2] you mentioned, it uses resampling to increase the sampling points in high PDE residual regions to avoid so-called propagation failures. We further add a review of other literature: [3] proposes a risk min–max framework to do adaptive sampling to speed up the convergence of the loss and achieve higher accuracy.
2. From a technical perspective, adaptive sampling technology uses PDE residuals to adjust sample density. Our CoPINN uses the gradient of PDE residuals to estimate the learning difficulty of samples, thereby optimizing the learning process of the neural network. From a conceptual perspective, adaptive sampling technology increases the density of difficult samples while reducing the density of simple samples, thereby fully learning difficult samples. Unlike them, our CoPINN does not involve changes in sampling and sample density. CoPINN directly evaluates the learning difficulty of samples and imitates the human cognitive process to learn from easy to difficult, thereby improving accuracy. In general, adaptive sampling focuses on adjusting the number of difficult and easy samples, and CoPINN uses the early learning of simple samples to fully mine information from difficult samples in the later stage of learning. They are two different technical routes and there is no conflict.
3. [2]proposes that the correct solution of PINN should propagate from the boundary to the center, otherwise, it will cause trivial solutions. Therefore, PINN training should focus on the boundary. Our CoPINN doesn‘t ignore boundary conditions and initial conditions during training. Although CoPINN pays less attention to difficult samples (usually at the boundary) according to the gradient of PDE loss in the early stage of training, it does not ignore difficult samples. As learning progresses, CoPINN pays more attention to difficult samples. Therefore, CoPINN doesn't ignore difficult samples in the early stage of learning, which will not cause trivial solutions mentioned by the reviewer and is therefore reasonable.

[1]A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks.

[2]Mitigating Propagation Failures in Physics-informed Neural Networks using Retain-Resample-Release (R3) Sampling.

[3]A Gaussian mixture distribution-based adaptive sampling method for physics-informed neural networks.

Q2: Questions For Authors (2)

R2: Regarding your question, "A quantification of the randomness of the training process ", we don‘t understand what it means. Based on your comment, we guess that you are doubtful about the stability of our CoPINN. In our paper, all quantitative experiments are repeated 5 times with different random seeds, and the average values ​​are reported. To demonstrate the stability of CoPINN, below, we report the mean and standard deviation of the relative $L_2$ error on the Helmholtz dataset. The results show that CoPINN achieves both the lowest mean and standard deviation in relative $L_2$ error, and confirm that CoPINN's training process is more robust than competing methods. If we misunderstood, please clarify, and we will provide further details.

Q3: Other Comments Or Suggestions

R3: Thanks, we will increase the resolution of Figure 1 in the next version.

We sincerely thank the reviewers for their constructive feedback. Below are our responses to the questions raised:

Q1: Questions For Authors(1)

R1: Our proposed CoPINN employs a multi-faceted approach to prevent overfitting to hard samples during training. First, the cognitive training scheduler ensures a controlled and gradual transition of focus from easy to hard samples through the hyperparameter $\beta$, which is intentionally set to values less than 0.5 to avoid abrupt weight shifts and maintain a balanced emphasis across regions. Second, the physics-informed loss terms, specifically the initial condition loss $\mathcal L_{ic}$ and boundary condition loss $\mathcal L_{bc}$, act as implicit regularizers by enforcing physical constraints across the entire domain, thereby anchoring predictions to known conditions even as harder samples are prioritized. Finally, the robust generalization has been experimentally validated, with consistently low errors in boundary and smooth regions (e.g., Table 1, Figures 4, 9, and 10 in the original paper), confirming that the model does not overfit to specific challenging regions while maintaining accuracy across the entire solution space. This combination of gradual weighting, physics-based regularization, and experiment validation ensures stable and generalizable training.

Q2: Questions For Authors(2)

R2: The separable architecture of our proposed CoPINN inherently supports scalability to higher-dimensional PDEs by encoding each dimension independently through dedicated subnetworks, effectively reducing computational complexity from $O(N^d)$ to $O(dN)$, where $d$ is the number of dimensions and $N$ is the resolution per axis. While our experiments have focused on up to 4D systems (e.g., (3+1)-d Klein-Gordon) due to hardware limitations, the design principles theoretically generalize to higher dimensions. For instance, in 5D/6D scenarios, techniques like tensor decomposition (e.g., CP or Tucker formats) can optimize the aggregation of outer products in Equation 3, further enhancing efficiency. Our future work will explore these optimizations to validate scalability in extreme dimensions while maintaining the model’s accuracy and computational feasibility.

Q3: Questions For Authors(3)

R3: We agree that clarity in evaluation metrics is critical. The IMP metric in our paper is calculated as:

$IMP=\frac{|u_s-u_b|}{u_s}\times 100\\%$

where $u_s$ is the error of the suboptimal baseline and $u_b$ is the error of CoPINN. Since $u_b<u_s$ in all experiments (as CoPINN outperforms baselines), it is mathematically equivalent to the standard relative improvement formula. To eliminate ambiguity, we will revise the formula in the final version to: $IMP=\frac{u_s-u_b}{u_s} \times 100 \\%$ removing the absolute value. This aligns with standard practice and ensures no overstatement. For example: In Table 1, a baseline error of $(0.0311)$ SPINN(m) vs. CoPINN’s $(0.0006)$ gives $IMP=\frac{0.0311-0.0006}{0.0311}\times 100\\%\approx 98\\%$, which correctly reflects the relative error reduction. In addition, we have explicitly reported both relative $L_2$ errors (e.g., Helmholtz's CoPINN: $0.0006$ vs. SPINN(m): $0.0311$), $RMSE$, and $IMP$ in Table 1 of the original paper. The performance is comprehensively demonstrated through two metrics (relative $L2$​ and $RMSE$), with the improvement magnitude illustrated via $IMP$.  We will clarify this in the text.

Q4: Other Comments Or Suggestions

R4: We have carefully corrected the typos in the paper:

"Klei-Gordon" → "Klein-Gordon" (Page 14);

"$u(x,0)=x_1+x_2, x \in \Omega, x \in \Omega$"→"$u(x,0)=x_1+x_2, x \in \Omega$" (Page 12).

We sincerely thank the reviewers for their constructive feedback. Below are our responses to the questions raised:

Q1: Methods And Evaluation Criteria & Questions For Authors

R1: The reference solutions refer to the labels used to compute the relative $L_2$ error and $RMSE$ by comparing them with the model's predicted solutions. Following [1-3], for the Diffusion equation, reference solutions are obtained through the widely-used PDE solver platform FEniCS at a resolution of 101×101×101. For the Helmholtz and Klein-Gordon equations (3D and 4D), reference solutions with 100×100×100 resolution are independently designed and manufactured respectively.

Q2: Claims And Evidence & Experimental Designs Or Analyses & Essential References Not Discussed & Questions For Authors

R2: According to your suggestion, we will discuss adaptive sampling in the related work section. We acknowledge that existing work in the field of adaptive sampling, such as RAD/RAR-D [4] provides valuable insights relevant to our study. However, CoPINN fundamentally differs from these existing adaptive sampling methods:

1. Existing adaptive sampling methods like RAD/RAR-D [4] primarily focus on the global residual distribution, improving overall accuracy by resampling or adding points to the region near the high-residual points, but they ignore the Unbalanced Prediction Problem (UPP) of PINN. In contrast, the core objective of CoPINN is to resolve the UPP commonly observed in traditional PINNs near physical boundaries. We find that relying solely on absolute residual values (e.g., as in RAD) may overlook the high-gradient nature of boundary regions, leading to local overfitting or underfitting. Therefore, CoPINN introduces a residual gradient-based dynamic difficulty assessment to more accurately identify abrupt changes in boundary regions (e.g., shocks and singularities).

2. Existing adaptive methods (e.g., RAR-D) employ a greedy strategy to incrementally add high-residual points but do not consider the model’s optimization capability at different training stages. To address this, CoPINN incorporates a cognitive training scheduler, which prioritizes simpler regions (low-gradient areas) in the early stages to stabilize training, while progressively increasing the weight of more complex regions (high-gradient areas) in later stages. This prevents the model from prematurely converging to suboptimal local solutions.

Q3: Experimental Designs Or Analyses

R3: The reviewer notes that prior work [5] addresses similar issues via loss reweighting. However, our CoPINN fundamentally differs in both methodology and scope. [5] focuses on loss-term balancing (e.g., PDE residual vs. boundary loss) without considering spatial/temporal variations in sample difficulty. In contrast, inspired by the human curriculum learning strategy of progressing from easy to hard tasks, CoPINN introduces a cognitive learning approach. It evaluates sample-level difficulty (via PDE residual gradients) and progressively prioritizes harder regions during training. CoPINN is finer-grained, addressing intra-term imbalances (e.g., stubborn points vs. smooth regions).

Q4: Experimental Designs Or Analyses

R4: There are two key differences between the approach with hard constraints [6-7] and our proposed CoPINN.

1. [6-7] focus on inverse problems (e.g., geometric parameter optimization), with the core objective of simultaneously satisfying PDE constraints and optimization objectives. In contrast, CoPINN targets the Unbalanced Prediction Problem (UPP) in forward PDE solving, aiming to address error caused by varying sample difficulties in PINN training. Their application scenarios are fundamentally distinct.

2. [6-7] employ fixed hard constraints (e.g., modified network architectures) and optimization strategies to ensure strict adherence to physical properties during the prediction process, enhancing accuracy through the incorporation of additional constraints. CoPINN introduces a dynamic cognitive scheduler that evaluates sample difficulty via PDE residual gradients and progressively adjusts training weights from easy to hard samples, constituting a data-driven adaptive learning paradigm. This training strategy, inspired by human cognitive processes, remains unexplored in the PINN domain.

[1] Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.

[2] Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems.

[3] Separable physics-informed neural networks.

[4]A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks

[5]Understanding and mitigating gradient flow pathologies in physics-informed neural networks.

[6]Physics-informed neural networks with hard constraints for inverse design.

[7]Scaling physics-informed hard constraints with mixture-of-experts.