Sparse identification of nonlinear dynamics with Shallow Recurrent Decoder Networks

Overall, the limited originality, lack of critical experiments, and reliance on vague claims make this paper feel incomplete. Including ablation studies, clarifying comparative methods, and tightening claims around spectral bias would be essential steps to make the contribution more compelling. See the Weaknesses for detailed suggestions.

• This paper mentioned laptop-level computing. However, I didn’t find the computing configuration in the paper. How does this paper validate this statement? Also, this paper shows the training time in Table 1 without introducing the computing platforms the authors used.

• For the implementation of SINDy loss, do you train the entire measured time sequence as one batch? Suppose you have a sensor measurement s_1 with a time duration of [0,1000]. Do GRU layers take 1000 steps as input without batches? Otherwise, why do you compute the SINDy loss?

• I assume the learned latent dynamics are not sufficiently precise in terms of discovered equations. How do you measure the error propagation of latent discovered physics?

• On Page 4, why does this paper claim that the proposed method is able to avoid spectral bias issues without an encoder? The spectral bias comes from the neural network itself, not just encoders. More discussions are appreciated.

• How do you select the basis functions in the SINDy library? If the latent space is smooth, then you do not need to consider too many basis functions. What are the basis function libraries for the tested cases?

• The authors claim the GRU produces a smoother latent space than LSTM. Do you have an ablation study on that or visualizations?

• I think the superiority of the proposed method comes from the incorporation of the discovered latent physics, as shown in Figure 10. It would be good to have an ablation study to train the same pipeline without SINDy loss to give readers a better sense.

• As shown in Figure 14, how does this paper obtain the original latent space?

1. The authors claim that using SINDy the proposed method is "more likely to identify governing physics", and the authors listed some "discovered equations" in the numerical experiments. However, it is unclear how the dynamics in the latent space is related with "governing physics". If a linear sampling function is used as a decoder, as in Kalman filters, it may be possible to related the discovered dynamics to governing physics. But, using a highly flexible nonlinear mapping, correlating the dynamics in the latent state to the governing physics is unlikely.

2. There is an inconsistency in the formulation. In Eqns (1 - 2), the time evolution of $z_t$ is a function of input variable $x_t$, i.e., $\dot{z}(t) = f(x(t))$. Then, in Eqn (3), suddenly, $z_t$ becomes an autonomous system, $\dot{z}(t) = f(z(t))$. Then, for the rest of the manuscript, $z_t$ remains as an autonomous process. This treatment of $z_t$ significantly restricts the class of problems that the proposed method can be applied. The method can be applied only to a stationary process without an exogenous forcing.

3. The ensemble SINDy is a little bit strange. It would have made more sense, if $Z^{GRU}$ is regularized by an ensemble of SINDy functions, e.g, $|| Z^{GRU} - 1/B \sum_i Z^{SINDy}_i ||$. But in the manuscript the authors propose to minimize $\sum_i || Z^{GRU} -Z^{SINDy}_i ||$. Then, the minimum should be $Z^{SINDy}_1 = \cdots = Z^{SINDy}_B = \hat{Z}^{SINDy}$ that minimizes $|| Z^{GRU} -\hat{Z}^{SINDy} ||$. In other words, the ensemble SINDy converges to the single SINDy. It is mentioned that one of the purposes of the ensemble SINDy is using "varying levels of sparsity". But it is not explained how $\Xi^i$ is set to "vary the sparsity".

4. One of the purpose of the method is to use a sparse input $X^S$ to reconstruct the whole field $X$. However, to train the model, the whole field data, $X$, is required, which essentially makes a chicken-and-egg problem. The model can reconstruct the whole field, but to do so, the model requires the data for the whole field, which is very difficult to obtain in many real-world problems. Together with the issues with the stationarity of the process that can be modeled (see comment 2), it further limits the problems the problems that can be solved by the proposed method.

5. Algorithm 1 is not very clear to understand. Does line 8 indicates the optimization problem is completely solved for every $i$ and fixed $X_{t-L:t}$? How is the optimization in Line 8 solved? SGD? Then, it means SGD is run until convergence for every $i$ and every possible $t$? Lines 3 - 7 should be in a subroutine to solve Line 8. What is $n$ in Line 2 by the way?

6. Once trained, SINDy can be used to compute $Z$, instead of GRU? If not, what's the purpose of the sparse discovery? $Z$ does not follow the sparse dynamics anyway.

Questions are just more concrete illustration of the weakness session.

1. Could you justify why SINDy but not other symbolic regression models. Some experiment results would be more convincing in addition to the words

2. Could you show the impact of data noise to the final identified latent space dynamics. To what degree your model could give roughly the same form. Could you also show the impact of the library terms.

3. Could you compare with all your cases with baseline models? My concern is that in Figure 4 and 6 the prediction results don't seem very close to the ground truth data.

4. Could you provide the training cost for all your models?

5. How is your result compared with other ROM model? Could they lead the same/similar form? Is your model more accurate?

6. How sensible your model is respect to the ensemble size?