Neural Time Integrator with Stage Correction

1. Note the indentation at the beginning of the paragraph.
2. Many of the quotes require parentheses: Use parentheses in citations when the author’s name is not directly mentioned in the sentence. This helps clearly separate citation information from the main content.
3. The lack of related works. There are quite a lot of other works focusing on introducing neural networks to accelerate traditional methods. [1, 2]
4. Only two time windows $L$ are considered in the Viscous Burgers Equation and Kuramoto–Sivashinsky Equation. Too few types of $L$ will make the results not convincing enough, and it is difficult to reflect the changing trend of the results of different methods as $L$ increases.
5. It will be more convincing to consider and compare the methods on different coarse time step,like multiply by 2 and divide by 2 based on the current time step.
6. Experiments on the accelerated ratio regarding the time of the traditional method and NeurTISC achieving the same accuracy can help demonstrate the effectiveness of NeurTISC. You may reduce the time step of the traditional numerical method or increase the time step of NeurTISC to achieve similar accuracy.
7. l146: the letter after commas should be lowercase: In this section, we...
8. l202: average the loss

[1] Bar-Sinai, Yohai, et al. "Learning data-driven discretizations for partial differential equations." Proceedings of the National Academy of Sciences 116.31 (2019): 15344-15349. [2] Greenfeld, Daniel, et al. "Learning to optimize multigrid PDE solvers." International Conference on Machine Learning. PMLR, 2019.

(1)The paper does not provide mathematical proof of the convergence stability of NeurTISC. (2) The numerical experiments are not sufficient to demonstrate the superiority of NeurTISC. (3) The paper is imprecise and unpolished. The format of the paper needs to be checked for better readability. Some paragraphs begin with an indentation while some do not, which makes your paper look messy. Make sure the format of the paper is aligned with the rules of ICLR. In addition, more information should be included in the figures to make them more understandable. For example, you just mark the horizontal axis as ‘x’ in figure 3, which is very ambiguous. I advise you to use a more understandable word to replace ‘x’ or explain what the meaning of ‘x’ is in the caption of the figure. (4)There is a statement in the conclusion that “therefore, it allows the use of large step sizes to accelerate the simulation of dynamical systems, including pure ODEs and ODEs converted from PDEs after spacial discretization.” However, the authors do not perform any experiment to compare the number of iteration steps or the computing times between different methods.

I believe that this paper has several weaknesses.

• In the method proposed in this paper, no constraints on neural networks are imposed. Therefore, numerical methods obtained by this approach is not guaranteed to approximate solutions of the differential equation(i.e., the order of accuracy of the numerical methods is decreased to 0.) This fact disrupts the authors' claim that their method is based on numerical integrators and, hence, highly reliable. For example, for the Heun method described on page 3, $\hat{k}_1 + \hat{k}_2$ must approximate $2F(u)$ to define a method with at least first order accuracy. To this end, a certain condition is required for $NN_1$ and $NN_2$; for example, if NN_1+NN_2=0, then $\hat{k}_1 + \hat{k}_2$ will approximate $2F(u)$.
• As far as I understand, this method requires the neural network to be re-trained whenever the step size is changed. This is not practical.
• The authors discuss the computational time below (6). It is stated that the proposed method is faster when $\varepsilon$ is small, but in what cases is this expected? Evaluating neural networks requires matrix operations, which require a certain amount of computation.

Use of Neural networks for time integrator is really now a mature field of its own. This paper does not succeed in positioning itself in the large body of work in this domain. For instance, state-of-the-art is Neural ODE (NODE) which is not mentioned in this paper. There are also other method like hierarchical methods, e.g. Hierarchical Deep Learning of Multiscale Differential Equation Time-Steppers by Liu et al, where a very different architecture is used to achieve the same results by authors. Also other advanced methods also address other issues that this method is not able to fix; for example large dimensionality of PDEs can be handled by methods like NIF (Neural Implicit Flow: a mesh-agnostic dimensionality reduction paradigm of spatio-temporal data by Pan et al) or CROM (CROM: Continuous Reduced-Order Modeling of PDEs Using Implicit Neural Representations by Chen et al) that not only can take care of large dimensionality of the dynamical system to be integrated, but also provide a continue in time manifold of solution, meaning at each arbitrary instant the solution can be evaluated by the neural network. The contribution is very incremental, even compared to NeroVec. Multi-frequency of stages can still be captured by a single NN at the last stage in theory, say by using more advanced RNN or autoregressive models. There is no theoretical analysis to show guarantees and it is mostly based on experimental results which are not the most complicated. problems arise in dynamical systems.