CONFIDE: CONtextual FInite DifferencE modelling of PDEs

Thank you for the helpful feedback. We are pleased to report that we added the suggested experiments to the new version of the paper.

Regarding weaknesses:

1. While we agree that estimating PDE coefficients from observations is not a new task, the task we aim to solve is quite different. In our scenario, each signal in the datasets (both train and test) are generated from different dynamics. This means that there are dynamics in the test set never seen before. Having different coefficients per signal makes our task much more challenging than other related works, which mainly focus on the scenario where the entire dataset is created from the same dynamics but with different initial conditions. The novelty in our approach is that it is capable of creating reliable predictions even in this scenario, while also providing a specific estimation of the PDE coefficients for the given signal. This estimation holds high value in many scenarios as also discussed in many other papers in the area (for example papers [1] and [2].
2. We agree that using a PDE-solver has an expensive time cost. However, the main goal of our paper is not to create a fast replacement to a solver (such as in neural-operator related works). Our algorithm’s main goal is to utilize given mechanistic knowledge in order to estimate the coefficients for each signal. We then show at inference time how these coefficients could be used to provide reliable predictions.
3. As we show throughout our experiments section, a relatively simple PDE is already a hard task when considering every signal in the dataset comes from a different context. This task gets even harder when the test set signals are generated from coefficients that were never seen in the train set. Regarding the benchmarks suggested in the paper you provided, all datasets are generated from using the same coefficients. For example, in the Navier-Stokes experiment, the viscosity coefficient is constant (10^-5), and in the Darcy-Flow experiment, the external force equals 1 for all signals.

[1] Linial et. al. "Generative ODE modeling with known unknowns" [2] Yang et. al. "Learning Physics Constrained Dynamics Using Autoencoders"

Regarding questions:

1. For params estimation, our method provides a very fast alternative to the “integral” approach used in the adjoint method. Adding a PINN loss does not improve the time cost of this method. To demonstrate this, we added a new demonstration to the appendix (appendix E), where we demonstrate how CONFIDE works in comparison to an adjoint+PINN method. In this experiment, we evaluate these methods on a pendulum equation (which is formulated as an ODE), and demonstrate that CONFIDE’s results are not only better in terms of prediction and coefficient estimation, but also trains ~44 times faster.
2. As requested, we added a new shockwave experiment based on the Burgers’ equation in appendix C1. For this experiment, we created a new test set of signals that have a discontinuity at t=0, and compared the algorithms on this data. We observed that CONFIDE still outperformed other baselines (as can be seen in table 3). We believe that the reason is that CONFIDE evaluates the derivatives at different patches of the observed context $u^c$ at each iteration, and most of them are smooth. Note that in this case, CONFIDE could be modified to handle discontinuity, by evaluating the spatial and temporal derivatives only on the parts in the signal that are smooth. For instance, if we observe that there is a discontinuity in the initial conditions of the signal, at x=x_0, then we can evaluate finite differences for x<x0, and x>x0 separately and enforce the PDE in these regions. For the sake of evaluation we did not add this modification to our algorithm in this
3. As requested, we added a new OOD experiment based on the Burgers’ equation in appendix C2. For this experiment, we created a new test set, where in all signals the coefficient a is sampled from U[2,4] instead of U[1,2] as in the train set. Note that we did not re-train any of the algorithms, but used the ones trained on the a~U[1,2]. As can be seen in the results, CONFIDE still outperformed other baselines although it never observed coefficients as these. CONFIDE projects the observed signal to the range of coefficients in the train set, and finds the ones that best describes the observed signal.
4. In the ablation study where we change the size of the context (the amount of time steps in each signal in the train set), the only change of size is in the first layer. The network size does change but not drastically: rho=0.05: #params = 4.7M, rho=0.2: #params = 5M, rho=0.5: #params = 5.6M. As can be seen in figure 11, CONFIDE’s results are almost not impaired even when the context size is very small, and the network size is a bit smaller as a result.

Thank you for the attentive and informative feedback.

Regarding Weaknesses:

As per requested, we have added some modifications to the text (marked with red), and changed figure 3 to display the solution at particular time points.

Regarding questions:

1. The process of learning the PDE parameters does not depend on the numerical solver at all. During the training phase, our model consumes the observed signal and outputs an estimation for the coefficients. These coefficients are then optimized to best fit the PDE alongside the derivatives which are numerically computed. It is only in the inference phase where we might use a numerical solver in order to provide predictions to a given signal.
2. The main goal of our paper is not to create a new efficient PDE solver (such as in neural-operator related works), but to utilize given mechanistic knowledge in order to estimate the coefficients of the PDE (which are different for each signal). We then demonstrate how one can use the estimated coefficients to create reliable predictions, even using a rather simple solver such as PyPDE (specifically the explicit solver). Our training scheme does not use any solver, but directly infers the PDE coefficients.
3. We use finite-differences method only to obtain numerical derivatives which can be substituted into the PDE. Using other methods, as you mentioned, is completely valid as long as the derivatives can be calculated. The choice of finite-differences in our case, was enough to result in reliable and fast estimation of the coefficients. However, a user can change this easily and plug-in any choice of derivatives calculations. Regarding boundary-conditions: when using finite differences to compute the derivatives, we do not use the boundary condition (we only need them when attempting to solve the PDE). Therefore, estimating the coefficients does not depend on boundary-conditions. However, during inference time, when a user wants to use the estimated coefficients to solve the PDE, one would need to determine what boundary conditions are needed.
4. We address this question in ablation appendix C.3 (figure 10) (which in the new version became D3, and figure 12), where we compare between an architecture without a decoder part, one with a decoder part and one with a decoder part which is also “initial-conditions aware”.
5. We are sorry for the confusion. The coefficients in equation (1) are not necessarily constants, but could be functions. In the Burgers’ experiment, the coefficient b is a function of u, and in the FN2D experiment, the coefficients which are the local reaction terms depend on both solution signals u and v.
6. Our method learns a mapping between the observed signal and the coefficients. Estimating the coefficients correctly results in good long-horizon prediction as shown in the figure (and as usually desired). FNO and UNET train to predict one-step into the future, and then auto-regressively iterate to produce the next predictions. This is why these methods tend to do well in the very near future predictions, while CONFIDE tends to be more robust and reliable.
7. As can be seen in Figure 6, FNO behaves very similarly to UNET through most of the prediction time until at some point it diverges from the true solution. The prediction figures we added should help to understand how each algorithm scored at each time point.
8. CONFIDE could be used as long as numerical derivatives can be computed (e.g., by finite-differences). This notion does not change when resolution changes, therefore there is no limit to using CONFIDE on different resolutions.
9. We stress that each signal in our test set is generated not only from different initial conditions, but also from coefficients not seen in the train set. As can be seen in our experiments section, this task alone is already a hard task, as modern baselines struggle to generate reliable predictions.
10. We note that the term “explainable” here is not the same as standard “explainable-ML” methods. Our meaning was that since CONFIDE provides coefficient-estimation to a given signal, it provides the user some knowledge (or explanation) regarding the observed signal. For example, In the electric vehicle batteries case ( last paragraph of the introduction), the coefficients hold valuable information regarding the battery’s capacity, how old it is, how fast it could discharge and so on. In healthcare for instance, there has been a great effort in designing a differential system of the cardio-vascular system, and inferring the parameter of cardiac contractility, or the parameter of arterial resistance, holds great value for clinicians as they decide on their treatments.

In addition to the last comment, we have added the DINO baseline as you suggested.

Regarding weaknesses:

1. The CONFIDE algorithm is not necessarily limited to the form of equation (1). For comparison and demonstration purposes we assumed the form in (1), but in the general case CONFIDE could handle any PDE form, as long as numerical derivatives could be computed through finite-differences. For example, CONFIDE has the ability to evaluate the coefficient in the wave equation (d^2u/dt^2 = c^2 * d^2u/dx^2) by evaluating the second derivatives of time and space on the observed signal u(t=0..T, x=0…L).
2. For comparison we chose the SOTA algorithms currently existing (FNO and UNET), and added a baseline that demonstrates an integral approach vs differential approach.