Neural Electrostatics: A 3D Physics-Informed Boundary Element Poisson Equation Solver

Questions

1. Why should PINNs be used instead of numerical solvers?

While classical numerical solvers have benefited from decades of optimization, PINNs offer distinct advantages that make them valuable for specific problem classes. PINNs compress the solution space into a small neural network, making them particularly suitable for scenarios requiring repeated evaluations or parameter sweeps. By using a boundary form, our approach eliminates the need to mesh the entire domain and enables field evaluations at arbitrary points in space, which is a significant advantage over traditional solvers. Furthermore, our single loss term formulation avoids the instability commonly associated with balancing multiple terms in traditional PINNs, resulting in a more stable and efficient training process. Through this work, we aim to advance the development of PINNs to enhance their competitiveness and utility in solving challenging problems.

2. Is training longer than the classical solve? Is the BEM always preferable?

We acknowledge that training time for PINNs can be longer compared to classical solvers, but our method introduces improvements that make PINNs more competitive. Unlike traditional BEM, which requires assembling and solving a full $N\times N$ matrix, our PINN-based approach avoids this step, offering potential scaling benefits, especially for large problems. This reduction in computational requirements aligns with the broader goal of optimizing PINNs for real-world applications where classical methods may become computationally prohibitive. We find that our training tim is on the order of 50 minutes for most problems, but we were focused mainly on correctness as opposed to optimization of training time, which is left to future work.

3. Can better evaluations be provided?

Yes, we will address this issue in the revised manuscript by including additional evaluations that consider more complex voltage functions with multiple frequencies and non-differentiable discontinuities. These expanded evaluations will demonstrate the robustness and general applicability of our method to a wider range of scenarios, beyond the simple settings presented in the current draft.

4. Can comparisons to RAD or a large number of collocation points be provided?

Adding more collocation points could yield a more favorable comparison but would place traditional collocation methods at a disadvantage relative to our approach, which achieves strong performance with far fewer test elements. We will look at adding RAD to our ablation study in the upcoming draft. Unlike RAD, which increases the training dataset size and slows down training, our variational adaptive scheme reduces the size of the training dataset, resulting in faster training and improved accuracy. This efficiency makes our method a compelling alternative for large-scale problems.

5. Does the singularity removal only hold for point sources?

Yes, the form of our singularity removal is only valid for point discontinuities as found in the Green's function in the electrostatics problem. This singularity removal is specific to the problem formulation but has broad applicability in computational electromagnetics, where $\phi$ (the boundary potential) is known, and $\rho$ (the charge density) is unknown. This targeted singularity removal significantly enhances the stability of the solution process, as demonstrated in our ablation studies, compared to simple Monte Carlo integration.

6. Isn't the dimensionality reduction from 3D to 2D only possible for point sources?

In the context of electrostatics, the boundary can be viewed as a surface composed of infinitesimal point sources, each radiating an electric field according to the Green's function and subject to zero boundary conditions at infinity. This dimensionality reduction is a natural consequence of the boundary integral formulation and is critical for many practical applications like capacitive touch screen design (Zhaou et al., 2020) or general electrostatic simulation (Gibson, 2014) in computational electromagnetics. Our approach aligns with these applications and leverages this formulation to provide efficient and accurate solutions, as discussed in the introduction.

We appreciate your thoughtful feedback and have worked to address all concerns and improve the clarity and importance of our manuscript. These updates, alongside additional evaluations and comparisons, will be reflected in the upcoming draft. Thank you again for the time and effort spent reviewing our work.

Thank you for your detailed feedback and for highlighting both the strengths and areas for improvement in our manuscript. We appreciate the opportunity to respond and have carefully considered your concerns and questions. We outline the steps we are taking to improve the work in an upcoming draft.

Weaknesses

1. The paper is limited to spherical coordinates and resampling test functions

We ask that the reviewer clarify this concern more if we do not address it adequately. While we use a local coordinate transform to remove the singularity, this choice does not inherently limit the applicability of our approach. We simply remove the singularity locally, which greatly improves the stability of training, as shown by our ablation study. The terms, away from the singularity, are integrated as expected in a Euclidean frame. The variational sampling technique is also important for stability. Unlike previous work (Lu et al., 2019; Wu et al., 2022), our technique does not increase training time because it only decreases the training dataset instead of adding new points.

2. Classical solver is preferred

We agree that classical solvers have been optimized for many problems; however, like past PINN work, we use the classical solver (Figure 1) as a baseline to validate our answer. PINNs in general struggle to beat classical solvers, but classical solvers also have decades of optimization. Our method provides improvements over previous PINN techniques (Section 5.3), on which the field can build to make the methods more competitive.

3. Missing reference to important works

We apologize for the oversight in omitting these important references. We will include these papers in our updated "Related Work" and "Results" section. These works provide valuable contributions to the field, but there are key differences compared to our approach. BINN focuses on 2D domains and does not easily generalize to arbitrary 3D shapes, employs a different singularity removal strategy, and lacks a variational form, which we found to be critical for stability in our ablation studies. Bi-GreenNet fits more within the operator learning literature, aiming to learn a class of functions rather than solve a single instance of a PDE. BiNet, by the same authors as Bi-GreenNet, is a closer comparison to our work. BiNet lacks a variational formulation, evaluates only on a 2D domain, and does not directly address the singularity removal. Lastly, PIBI-Nets use Monte Carlo evaluation, which we found to unstable in our ablation study and focuses on both $\phi$ and $\rho$ being unknown. All three of these works form their boundary integral using potential theory, whereas we focus on a form that is of more interest to the computational electromagnetics community. These works are all covered by our ablation studies, as they use the collocation method. They also only evaluate on a 2D domain, while we can handle an arbitrary 3D quadrilateral mesh.

Questions

2. "We acknowledge that training time for PINNs can be longer compared to classical solvers"

I would suggest the authors to explicitly include in the paper a discussion about computational costs of the proposed approach and a comparison with numerical solvers. Unfortunately, if the proposed method is not as accurate as numerical solvers and still takes significantly longer than numerical solvers, it is hard to think that it could be advantageous in practice. This is especially true if, compared to numerical solver, the proposed approach takes "orders of magnitude longer".

However, in some settings (e.g. when data is also available) you might be able to argue that your method is preferable to numerical solvers. Also, advancement with respect to other competitive PINN approaches should be shown explicitly in the paper and in a way that highlights limitations and advantages of the proposed approach.

3. Can better evaluations be provided?

I thank the authors for working on additional experiments, which should not be given for granted.

4. "Adding more collocation points could yield a more favorable comparison but would place traditional collocation methods at a disadvantage relative to our approach"

Then I would suggest to show the error with respect to the ground truth for fixed computational resources. If it is true that "variational adaptive scheme reduces the size of the training dataset, resulting in faster training and improved accuracy" then it should be easy to see by setting a fixed number of computational resources and use for RAD as many collocation points as to reach that value. Or do you in general achieve improved accuracy?

5./6. Isn't the dimensionality reduction from 3D to 2D only possible for point sources?

I still don't see how the dimensionality reduction is possible only for point sources but can be extended to any surface as long as it is considered as a set of point sources. Is the surface treated as a (finite) collection of points sources or is the argument made in the limit of infinitesimal point sources? in the first case I agree it would work (but then this should be mentioned explicitly in the paper) while in the second case I still don't get how it would work.

I would like to thank the authors for their answers to my concerns. I will follow-up below on a few points mentioned.

Weaknesses

1. The paper is limited to spherical coordinates and resampling test functions

I am afraid we had a misunderstanding there. I meant that the contribution of the paper (in terms of novelty) are limited to the use of spherical coordinates and to the use of resampling functions (which is a standard technique for collocation points). I still believe that these two are the main contributions of the work.

2. Classical solver is preferred

I agree that classical solver have benefited from years of research. However, PINNs are still very valuable in situations where numerical solvers struggle (which is not the case in the present work). For instance, numerical solvers struggle to integrate information about data, which in contrast can be easily integrated in PINNs. Since the proposed method is not explicitly compared to any other work it is hard to see the improvement and one still wonders why the numerical solver should not be used in practice anyways (since it is apparently even much faster). Also, other works like BINN or PIBI-Nets either show extremely accurate reconstruction or have a setting where numerical solvers cannot be used directly.

3. Missing reference to important works

I thank the authors for including the mentioned literature in the upcoming revised pdf (which I am looking forward to reading).

Weaknesses

2. "Though we are not immediately faster" sounds very different from "orders of magnitude longer". Anyways, I think that in order for the proposed method to be useful in practice it must be either more accurate or faster. Else, it might be that either the particular application setting or the method are not suitable, in my opinion. If your point is that your proposed method might be beneficial in "larger, complex domains", then this should be reflected in the experiments as well. At least showing some trend as the dimension/complexity increases.

3. In PIBI-Net it seems that the Monte Carlo integration works perfectly, so it might be that it depends on how points are sampled (particularly those close to the boundary) or by the specific setting under study.

Thank you for your thorough and constructive feedback. We appreciate the time spent to review our work and help strengthen our contributions. Below, we address your concerns and questions. We also include key additions that we will make to the revised draft.

Weaknesses

1. No architecture diagram

We apologize for the omission of a diagram illustrating the neural network architecture. We will include a clear diagram in a revised draft, along with extended tables detailing all hyperparameters in the appendix.

2. Lack of evaluations

Our ablation studies on simple domains demonstrate the robustness of our approach compared to previous PINN methods, even in simple domains. We will include results with more complex imposed voltages. One example imposes a voltage of a series of sines and cosines to show the network's ability to capture multifrequency details. The second example uses a ramp function to show results with a non-differentiable imposed voltage. The derivation is also general and supports any 3D geometry represented by a quadrilateral mesh, but our evaluations consider a rectangular domain to ease implementation of mesh free methods. Additionally, we would like to emphasize that most prior PINN methods restrict their evaluations to 2D domains, whereas we specifically tackle general 3D surfaces.

Questions

1. How does the method extend to different domains (both size and shape)?

Yes, our derivation applied to general 3D quadrilateral meshes. We restrict our domain to a rectangle to ease initial implementation; however, the sharp boundary (Figure 3) already shows how the method reacts to non-differentiable discontinuities in the domain. We will include additional evaluations on more complex forcing functions in the upcoming draft.

2. Do you have inference time recorded? Is it faster than the classical method?

We will include inference times for integrating the network across the surface, which should be on the order of about a second. To provide some context:

• Evaluations require integrating across the entire domain, which requires many evaluations of the network. This is shared by all boundary element formulations.
• The neural evaluations are trivially parallelizable, as batch process is supported by all modern machine learning frameworks.
• Classic solvers have benefited from 60 years of optimization, whereas PINN are still trying to find their footing.
• Using the neural solution and boundary element form allows for the problem out to infinity to be heavily compressed.
• Our work provides an improvement over existing PINN methods as the field seeks to improve over the existing classical methods.

We appreciate your insightful feedback and believe these updates will strengthen the manuscript. Thank you again for your valuable suggestions, and we would be happy to address any other questions or concerns. We will send a followup message once the new draft is ready.

Thank you for your thoughtful feedback and for taking the time to evaluate our work. We have carefully considered your comments and addressed the concerns and questions below. We outline our plans to address these concerns in a revised draft.

Weaknesses

1. Limited scope to the electrostatics problem.

We highlight the connection to electrostatics because we demonstrate the use of a common boundary integral from the computational electromagnetics (CEM) literature (Gibson, 2014). This approach differs from the potential theory based integral often found in other boundary element based PINN methods (Lin et al., 2021; Sun et al., 2023; Nagy-Huber and Roth, 2024). One key difference is that the forcing, $\rho$, is the unknown quantity as only $\phi$ is easily measured. As suggested, however, the analysis and techniques for singularity removal (Section 4.2) and variational adaptive sampling (Section 4.4) are not limited to electrostatics. Our hope is that our paper (esp. the title) would motivate our method for general Poisson equation solvers. We will seek to highlight these extensions as part of the future work.

2. No comparisons to previous works on boundary element based PINNs.

We apologize for missing direct comparisons to other works like BINet (Lin et al., 2021), and will include it, along with several others, in the "Related Work" section (Sun et al., 2023; Nagy-Huber and Roth, 2024). These works, however, are covered already in the ablation studies (Section 5.3), which show that our proposed boundary method with a variational loss and coordinate transform singularity removal outperforms previous methods. We will make this comparison more explicit in our results section. For BINet specifically, the main differences are:

• BINet does not use a variational form, which we found to be crucial for stability. BINet uses the collocation method without any adaptive sampling, which can be found in Figure 4a of section 5.
• BINet uses the potential theory form of the boundary integral, which is not as directly applicable to the electromagnetics community.
• BINet only performs evaluation on a 2D domain, where we provide an extension to general 3D quadrilateral surface meshes.
• BINet does directly address the singularity introduced by the Green's function. We show and compare multiple techniques for handling this common difficulty. Without care, the singularity can cause instability in training.

3. Evaluations are too simple.

Our ablations were completed on a simple domain to demonstrate that many commonly used techniques struggle even in this straightforward setting. Past work also restricted themselves to simple 2D domains for evaluation, whereas we learn a solution on an arbitrary quadrilateral 3D mesh, already a standard requirement for classic physics solvers. We also recognize that more evaluations would strengthen our results, and have included additional evaluations on complex imposed voltages to show the generality of our technique to functions with multiple frequencies and non-differentiable discontinuities.

We appreciate your constructive feedback and believe that these improvements to our results and inclusion of additional literature will enhance the relevance and impact of our work. Thank you again for your time. We would be happy to address any more questions or concerns and will send another message once the new draft is ready.

Thank you very much for your hard work on addressing the reviewers' comments. I went through all the discussions. There are a few concerns shared by all reviewers, e.g., lack of comparison to previous works, the complexity and abundance of the experimental settings, the clarification of the details, etc. With all factors considered, I tend to maintain my score.

Thank you for your thoughtful review and constructive feedback. We appreciate the time and effort taken to review our work, and have carefully considered your comments to improve the manuscript's impact. Below, we address the weaknesses and questions, and outline changes that will be present in an upcoming draft.

Weaknesses

1. Details about architecture and training are unclear.

The network takes in either uv mapped 2D or Euclidean 3D coordinates and outputs just the scalar value $\rho$. The imposed boundary voltage is given by $\phi$. This problem setup is common in electromagnetics (Gibson, 2014), where the forcing is the unknown parameter. To improve understanding, we will include an architecture diagram with network inputs, network outputs, and training loss computation.

2. Only a simple problem is considered

A simple problem was used to demonstrate that traditional PINN methods (Raissi et al., 2017; Lu et al., 2019) using collocation points and adaptive sampling struggle to obtain the solution, even in basic scenarios; however, we recognize the importance of broader evaluation. We will include results with more complex imposed voltages. One example imposes a voltage of a series of sines and cosines to show the network's ability to capture multifrequency details. The second example uses a ramp function to show results with a non-differentiable imposed voltage. We also note that our derivation is suited for arbitrary 3D meshes and many other works in the literature (Lin, et al. 2021; Raissi et al., 2017; Lu et al., 2019) have restricted their evaluations to simple boundaries in 2D domains.

3. Abbreviations Used without Explanation

We apologize for the confusion and will ensure that all abbreviations are clearly defined on first use.

4. Spelling errors

Thank you for spotting these errors. They will be corrected in the revision.

Questions

1. Specify the rectangular test function precisely.

The rectangular test function, commonly use in finite element methods (Gibson, 2014), is defined as having a single value with support over a compact element. In our case, this support is a single quadrilateral in the mesh. We will include a precise description in the paper to improve presentation.

2. Why are the errors for 225 test function higher than both 25 and 100 test functions?

The errors for 225 test functions are higher because there are more evaluations near the discontinuities along the domain edges. Capturing these discontinuities is inherently challenging for any computational method and can increase the error metric. Computational methods circumvent this by integrating farther from the discontinuity. We will add a note on this observation in the revised draft.

3. Please state the boundary conditions for the example considered in Section 5.

Our analysis assumes the common exterior electrostatics problem of an infinite domain, where the field decays to 0 along a boundary at infinity. This setup is a significant advantage of boundary element methods, as the solution can be found anywhere around the surface. We will explicitly state these assumptions in the revised draft.

5. What is the Green's function used in the example, and how does it differ from the fundamental solution?

As noted above, we focus on the infinite domain form of the problem, leveraging the benefits of the boundary element method to find solutions anywhere in space. In this setting, the Green's function is the same as the fundamental solution for the Laplace operator. The method can be adapted to interior domains with the appropriate Green's function. Our results show the charge densities plotted on a flat plate in the domain (Figure 3). By integrating these charge densities, the electric field can be found anywhere. We will discuss these adaptations and limitations in the revised future work section.

Thank you again for the detailed feedback. These revisions will strengthen our manuscript and clarify the contributions of our work. We would be happy to address any more questions or concerns that you have and will send another message once the new draft is ready.