The experiments make an unfair comparison to the ClimODE baseline, where ClimODE uses 5 data variables and weatherODE 48. Results lack standard deviations. Some results show that weatherODE beats IFS, and this is not elaborated further. This is a major achievement, and is now provided a bit too casually. There should be some discussion on how the results relate to the larger model (panguweather, gencast, graphcast, etc).

In response, we add experiments where WeatherODE uses only 5 input variables in Table B below. The results continue to outperform ClimODE. The standard deviation in ClimODE's results primarily arises from the use of a source network that outputs a standard deviation term as a loss function regularization, which we have omitted in our design.

Regarding WeatherODE beating IFS, this was observed for the $t2m$ variable. For other variables, the performance is slightly below that of IFS. It's important to note that IFS results were trained on data at a 0.25° resolution, while WeatherODE was trained at 5.625° resolution, yet it still provides comparable 24-hour forecasts. This demonstrates the potential of Neural ODEs in weather prediction, even at coarser resolutions.

As for models like Pangu, GenCast, and GraphCast, they are developed by large organizations (e.g., Google, Huawei) with access to hundreds or thousands of GPUs, and trained on 0.25° data, which presents a resource challenge for us. However, it's worth noting that scaling up has been widely validated in fields like CV and NLP. We believe that WeatherODE, once scaled up, can achieve results comparable to these models even at 0.25° resolution.

Table B. Experiment of estimation of WeatherODE with only 5 variables

The clarity and text needs improvements.

Thanks for this feedback; we will work on improving the clarity and text.

Questions

I have hard time understanding the v0 estimation. First, a wave equation is introduced in general terms, and then a CNN appears. These two are not connected together, so I’m left wondering what do we do with the waves, and how do they relate to the CNN. Furthermore, the wave equation is poorly motivated, and it seems disconnected from the advection equation 3. To me this looks like a discrepancy: the system follows advection, but initial state assumes different kind of physics (ie. laplacians appear out of nowhere). Surely the initial state estimation needs to respect the chosen ODE model, and not introduce some physics effects that are not part of the ODE.

Apologies for any confusion. To train the neural ODE effectively, it is crucial to accurately estimate the initial state, $v_0$. We believe that the inaccurate estimation of $v_0$ using $\frac{\Delta u}{\Delta t}$ in ClimODE leads to suboptimal results. To address this, we introduce the wave equation under the assumption of incompressible flow, which relates $\frac{\partial u}{\partial t}$ to the spatial derivatives $(\frac{\partial u}{\partial x}, \frac{\partial u}{\partial y})$. Although the wave equation involves second-order derivatives, neural networks are capable of approximating both differentiation and integration processes. Thus, by feeding $(\frac{\partial u}{\partial x}, \frac{\partial u}{\partial y})$ into the network, we can better model $\frac{\partial u}{\partial t}$. Regarding the CNN structure, it was chosen due to its rapid convergence properties (will be clarified in later response), which help stabilize the training of the ODE and prevent numerical blow-up.

We greatly appreciate Reviewer o4jc's thoughtful feedback and their recognition of the meaningful extensions we’ve made to ODEs for weather forcasting. Below, we address each point raised, offering clarifications and detailed responses.

Weakness

The motivations behind the model choices seem a bit weak.

The motivation behind our model choices stems from the core idea of the Physical-informed Neural ODE framework, particularly the advection function proposed in ClimODE. To solve this ODE, we need to accurately estimate the initial velocity, solve the advection equation, and finally account for the error term arising from deviations in real-world conditions. These three tasks are inherently different due to their distinct physical natures. Thus, we aim to design an end-to-end architecture that can model this entire process seamlessly. WeatherODE, as proposed, provides an integrated end-to-end solution that handles all three tasks, while ensuring numerical stability during training.

The ablations are very interesting, but have some issues. In the v0 ablation there is no ClimODE baseline, so it’s difficult to say if any improvement was actually gained here. The Fig4 is superficial and difficult to interpret. The stability analysis seems interesting, but I don’t think it provides much insights into why things sometimes fail. One would not really expect such a simple ODE system to fail in the first place.

• Regarding the v0 ablation, we add the missing results in Table A below. In this ablation, we modify only the velocity network in ClimODE to handle different inputs and used the full ERA5 dataset for training. Everything else remains identical to ClimODE. The results are consistent with those in Figure 3 of our paper, showing that denser spatial information helps reduce discretization errors in velocity prediction.
• As for Figure 4, we aim to visualize the evolution of the weather variable $u$ from the ERA5 dataset over time, alongside the estimated velocity $v$ using ClimODE (approximated by $\frac{\Delta u}{\Delta t}$) and WeatherODE (derived from wave equation theory as $\frac{\partial u}{\partial t}$). The purpose is to highlight the continuous nature of $u$ over time, with WeatherODE providing a much smoother and more consistent derivative compared to ClimODE.
• Apologies for the lack of clarity in the stability analysis; it doesn’t fully explain why the system fails in some cases. The primary issue is sensitivity to initial conditions. If the initial velocity $v_0$ is poorly estimated, errors accumulate quickly, leading to blow-up (NaN values), where the values become either excessively large or small, causing numerical instability. ClimODE mitigates this by pre-training the velocity network for better $v_0$ estimation and reducing ODE solver precision. WeatherODE, on the other hand, is an end-to-end training framework, where the velocity network must converge faster than the neural ODE to avoid instability, as discussed in Section 5.3. This ensures the network provides accurate initial velocity estimates from the start, preventing numerical blow-up.

Table A. Experiment of estimation of $v_0$ of ClimODE

If the refined horizontal resolution is the key here (as it allows computing spatial gradients accurately), they can simply using higher spatial resolution training data (like 25 km x 25 km ERA5 data) lead to similar improvements in performance?

We agree with this observation. Increasing the resolution to 1.4° or even 0.25° would likely improve performance. Higher resolution not only allows for more accurate computation of spatial gradients, as you noted, but also benefits from the performance gains associated with the deep learning scaling laws. We hope that our work can be further validated in the future by organizations with greater resources on larger-scale and finer-resolution datasets.

One worry is that the achitecture is becoming too combersome to be practical: a unified model structure for past AI NWP models is one of the key cornerstones of its appeal. Traditional NWP model provide forecasts at a 9 km resolution. If the same models were to be run at 5.625 deg resolution, one can expect similar order run times between AI weather emulators and NWP models, ultimately leading one to question the central point of these models. So, having multiple ML models in sequence, such as those proposed in this study, can increase the complexity of the problem to the point that one begins to question the novelty of this approach. What would have been great could be to train one single neural net on atmospheric data sampled at a high frquency and use that for more accurate initializations.

Thank you for the excellent question. Our work explores a different perspective, leaning towards data-driven learning rather than adhering strictly to the constraints of traditional NWP models. While NWP provides solutions based on human-defined ODEs, we believe these models may not fully capture the complexities or imperfections of the real world. Simply solving these equations faster and more accurately may not necessarily lead to better predictions.

Completely data-driven learning, however, poses its own challenges. It requires enormous amounts of data, far beyond what is currently available, to replicate centuries of accumulated physical knowledge. Rare weather phenomena, for instance, are difficult for ML models to learn from limited examples.

We aim to strike a balance by integrating minimal but critical physical knowledge, guided by Occam’s Razor: "We consider it a good principle to explain the phenomena by the simplest hypothesis possible." This approach maintains enough flexibility for the model to learn autonomously from data while incorporating essential constraints from physics. While the path you suggest—relying on NWP with sufficient computational resources and ideal initial conditions—might work if our understanding of weather dynamics were complete, we believe a hybrid approach, where ML leads with NWP offering key guidance, provides a more robust path to accurate forecasting.

If i undestand correctly, the study employs a two dimensional equation. Have the authors considered using a three dimensional equation instead which considers the vertical verlocity into account as well? This could be important especially for tropical predictability ad thermodynamical processes like dry and moist convection evolve over sub-hourly timescales and can introduce notable errors into the equation. This could also affect longer lead time rollouts of the model. Or, if I undertand it correctly, the vertical velocity is simply treated within the source term (which might not be the best approach).

Thank you for raising this excellent point. This study primarily focuses on a two-dimensional equation. We acknowledge the importance of incorporating vertical velocity, especially for capturing tropical predictability and thermodynamic processes.

We are actively exploring extensions to three-dimensional modeling. For example, ERA5 data includes variables across multiple pressure levels, which could serve as a basis for expanding the advection equation to three dimensions by introducing a vertical velocity component $v_z$. This is a promising direction for future research, and we anticipate that incorporating 3D dynamics could further enhance the model’s accuracy and generalizability.

We sincerely thank Reviewer dcuz for thoughtful insights and recognition of our effort to integrate classical PDE theory with AI to enhance weather forecasting. Their championing of our work truly inspires us to pursue this research direction. Below, we address each point raised, offering clarifications and detailed responses.

I don't think ClimaX is a good baseline for such comparisons, as it was itself trained on a wide range of CMIP6 models which are (a) not really tuned for weather forecasting and (b) due to model design have multiple biases. Finetuning can only fix so many of them. Thus, I would not put too much weight into how the WeatherODE performs against ClimaX as it is a weak baseline to begin with. Similarly, I would recommend comparing the method with the newer version of FourCastNet based on the SFNO architecture which is more stable for short and long term rollouts and is more topology-aware.

While it is true that ClimaX is pretrained on CMIP6 models and finetuned on ERA5, it has demonstrated strong results at both 5.625° and 1.4° resolutions. We compared WeatherODE with ClimaX primarily because ClimODE's comparison to ClimaX involved a non-pretrained ClimaX trained on a subset of ERA5 data. Thus, we felt it was appropriate to compare our method with the original ClimaX.

As for FourCastNet based on the SFNO architecture, their model was trained on 0.25° data, and the specific training process is not publicly available. However, from Figure 6 in their paper [1], their 0–24h performance is slightly below IFS, similar to the gap we observe between WeatherODE and IFS. This suggests that WeatherODE is comparable in performance to FourCastNet based on SFNO.

Our decision to train on 5.625° data was constrained by computational resources, as training on 0.25° data would require thousands of GPUs, which we do not have access to, unlike larger companies such as Google or NVIDIA. Nevertheless, the scaling laws in deep learning—where larger datasets and models yield better results—have been widely validated in CV and NLP. We anticipate that WeatherODE, with access to larger data and model sizes, could surpass IFS in performance.

[1] Spherical Fourier Neural Operators: Learning Stable Dynamics on the Sphere

A minor point but the paper tends to over-cite at places: the Evans (2022) reference, the Vaswani et al reference and the Biswas et al. (2013) reference are totally unnecessary. Vaswani et al. has not even been cited in the correct context as it has not realtion to weather forecasting. Similarly on Line 37-38.

Thank you for pointing this out. The citations were intended to reference the original works where PDEs, the Euler method, and Transformers were first introduced. We agree that they are not directly relevant to weather forecasting. We will revise these citations in the updated version of the paper.

The figures' text could be increased. It is tough to interpret Figures 1 and 4 in print.

Thank you for pointing this out. In the revised version, we will carefully review the text size of all figures and adjust them to ensure they are easily readable in print.

It would have been interesting to see the time complexity of the sandwich model proposed in the study and how it compares with other models. Current AI weather forecasting models have been a bit over-glorified in a sense that they ignore baroclinic motions in the atmosphere and use data on very few vertical levels, and then claim to be as good as traditional NWP models - which are way more versatile thab AI NWPs in the problems they can be used to solve. Understandably, the only advantage then is the computational speed and the reduction in operational cost that they offer. Therefore, as more traditional numerical are embedded into pure data-driven architectures, it would be interesting to see the effect on more complicated hybrid architectures on the run speed.

Models that combine deep learning with physical constraints, such as WeatherODE, ClimODE, and NeuralGCM, generally have longer inference times per step compared to purely data-driven models like FourCastNet and Pangu. However, physics-constrained models often offer greater flexibility, as they can predict multiple time steps in a single inference (as WeatherODE* does).

In contrast, purely data-driven models typically train a separate model for each lead time and rely on rolling strategies for multi-step forecasts (e.g., using a 1-hour model twice to predict 2 hours ahead). When predicting multiple consecutive lead times, physics-constrained models can thus have an advantage in efficiency, as they avoid the repeated inference required by rolling strategies in purely deep learning approaches.

Continued from above ...

5. "While NWP provides solutions based on human-defined ODEs, we believe these models may not fully capture the complexities or imperfections of the real world."

I disagree with the authors here when they claim that NWP models can not capture the complexities of the real world but the data-driven methods can. I must remind them that the data-driven models (a) are trained on datasets created by these equations-driven traditional NWP models (along with observations) and (b) employ a simplified form of same equations for hybrid modeling. The only benefit AI-driven weather emulators have delivered on till date is speed. None of the models have systematically beaten IFS. Moreover, when integrated at a high-enough resolution, NWP models (without assimilation) can still be maintained physics-fidelity over longer periods of time than the state-of-the-art data driven models. It has become a fashion to bash traditional NWP models these days withouth appreciating the heavy amount of physics that goes into their design. Until these data-driven models show significant improvements over IFS after being trained on IFS input itself, alas, such claims about imperfections of the NWP models will remain baseless.

This tradeoff between model complexity and performance and how it contributes towards interpretability is argued in more detail in [1] which provides a comprehensive discussion of the different approaches - purely data-driven to hybrid to pure-physics based, and, simple models to intermediate complexity models to comprehensive weather forecasting and climate prediction model. I highly recommend the authors to atleast acknowledge such tradeoffs between model complexity, realism, and accuracy in the final manuscript as limitations of their approach and connect it to [1] (and other studies).

Also, a complete data-driven approach also leads to reduced interpretability of the model, as is again argued in [1] and multiple other studies. How do you think your approach embeds increased interpretability?

References: [1] Laura A. Mansfield, Aman Gupta, Adam C. Burnett, Brian Green, Catherine Wilka, and Aditi Sheshadri. Updates on Model Hierarchies for Understanding and Simulating the Climate System: A Focus on Data-Informed Methods and Climate Change Impacts. Journal of Advances in Modeling Earth Systems, 15(10):e2023MS003715, 2023. ISSN 1942-2466. doi: 10.1029/2023MS003715.

6. Regarding ensemble averaging: no offense but a 0.01 or a 0.02 decrease in RMSE does not equate to much and should not be read much into. A better test would be around case-specific events. If you can show that ensemble methods lead to better predictability around, say, a cyclogenesis events, that would be something. Sorry if this sounds a bit rough, but, I think the AI weather prediction community needs to be reminded every now and then that this is not merely an engineering problem where comparing RMSEs will suffice. What is needed is a set of physically more rigrous tests that demonstrate that these models are physically robust and are capable of being used for meaningful physical analysis.

I appreciate the authors' effort in preparing the response, but having had more time to think about the paper and digesting the underlying methodology better now, I reduce my score a bit.

1. ClimaX operating at 5.625 degree and 1.4 degree resolution is akin to a toy model for weather prediction at best. I reemphasize that predicting the large-scale features at 5.625 degrees with reasonable accuracy is a much easier problem than learning the actual multi-scale dyanamics from data - simplified models like Quasi-geostrophic equations can be used at these resolutions to get reasonbale forecasts as well over 12-24 hr timescales, while taking a fraction of the compute cost at NWP.

2. Sure, the neural scaling laws can be invoked to surmise that the model will continue to scale with increased resolution, but I still do not see it being independent of the underlying reanalysis. The model will continue to be (a) more computationally intensive, (b) require newer and newer reanalysis datasets, and (c) based on current equations, still not have limited value for long term forecasts. Moreover, increasingly complex equations will be needed to ensure stable rollouts over longer periods - which connects to my next point.

3. Coriolis term? What about other physics on longer time scales? A heavy limitation of your approach is that you will eventually have to embed more complex equations into the hybrid architecture - at which point the architecture might converge to existing models like NeuralGCMs.

4. I am not convinced by the authors' response on the computational complexity. Of course, auto regressive rollouts will be more expensive than single-shot predictions, but if all one is getting from the (arguably more-complex) hybrid approach is marginal improvements over 6-24 hour timescales, that barely has any significant utility because pure-data driven models are relatively easier to train, and the auto-regressive rollout issue can be circumvented for medium-range timescales, as has recently been done with the weather and climate foundation model Prithvi wxc (if i understand correctly).

We sincerely thank Reviewer g7RP for their detailed feedback and thoughtful comments, which demonstrate a deep understanding of our work. Below, we address each point raised with clarifications and detailed responses.

Weakness

First, the authors claim that all physical variables follow an advection equation, which is learned. This assumption, initially made in the ClimODE model, forms the foundation of the WeatherODE architecture and is said to provide a strong physical bias. It would be beneficial if the paper provided more physical context regarding the advection equation and why it serves as a good prior for learning atmospheric dynamics. From a physical perspective, what are the consequences of assuming that all variables follow this equation with a learned velocity field? Can the authors connect this to known atmospheric processes?

From a physical perspective, the foundation of our approach lies in fundamental conservation laws: conservation of mass, energy, and momentum. The advection equation, specifically, is derived from the principle of mass conservation, which asserts that the inflow and outflow of mass or substances within a given spatial region remain balanced. This makes it one of the most fundamental conservation laws, as energy and momentum conservation often involve more complex dissipative processes, whereas mass is relatively stable.

While not all atmospheric variables are perfectly suited to be described by mass conservation, it provides a general and reasonable physical bias. Importantly, the velocity field in our model is variable-specific, meaning each variable follows its own dynamics rather than adhering to a single universal motion. The shared assumption is that these dynamics align with the essential process of mass conservation, particularly at a large regional scale.

Although the atmosphere is compressible on micro scales, our model operates over larger patches, where the assumption of approximate mass conservation within a region is reasonable. This provides a flexible yet grounded physical context for the advection-based modeling framework, balancing simplicity with general applicability.

There is some lack of clarity in the discussion of discretization between lines 192 and 204, particularly around Equation (3). It is unclear at which stage the ODE is being discretized. The problem of learning an ODE's flow using neural ODEs should, in theory, be independent of the discretization, as neural ODEs are designed to differentiate through the ODE in a solver-agnostic way. Second, indices t0 and tn seem to play the same role. It would be clearer if a single notation were used for the initial condition, even if this involves only one discretization step, with the methodology then being extended to subsequent steps.

Allow me to clarify. In the context of Neural ODEs, the discretization referenced in our paper corresponds to the series of intermediate "adjoint states" [1] computed between the initial input and the final output. Specifically, the input to our ODE is $u \in [K, H, W]$ and $v \in [2K, H, W]$, where $K$ represents the number of input variables, and $v$ has $2K$ dimensions as it includes velocity components in two directions. The ODE outputs $\hat{u} \in [N, K, H, W]$ and $\hat{v} \in [N, 2K, H, W]$, where $N$ corresponds to the number of predefined ODE steps ($t_1, \dots, t_N$), which we set to align with the lead time in hours.

At each step of the ODE, the next state of $u$ is calculated using the current state of $u$ and $v$ via the advection equation, while the next state of $v$ is predicted by the neural network. This design allows the ODE to incorporate supervision from ground truth at every intermediate time step, not just the final state.

Regarding the indices $t_0$ and $t_n$, apologies for the confusion. In Equation (3), the underbraces correspond to the components being modeled at different stages.

1. Initial velocity ($v(t_0)$): The initial velocity $v_0$, fed into the ODE, is modeled by the velocity network. In the term $\begin{bmatrix} u(t_n) \\ v(t_n) \end{bmatrix}$, $u(t_0)$ is known, while $v(t_0)$ needs to be inferred by the network.
2. Advection ODE ($\dot{v}(t_n)$): The Neural ODE models the advection dynamics.
3. Source term ($s(t_n)$): The source model predicts the source term $s(t_n)$.

We hope this clarifies your concerns. In the revised version, we will elaborate on these points to ensure greater clarity.

[1] Neural Ordinary Differential Equations

This concern is compounded by the results of Figure 3, where the performance gap between the proposed method using the wave equation and other approximators is quite small. Couldn't it be that the performance gap comes from learning the initial velocity from a neural network conditioned on non-discretized state values instead of solving an inverse problem with discretized state derivatives, rather than from the wave equation-informed predictive structure of the neural network?

The results in Figure 3 show that $f_v(\nabla u)$ achieves over a 10% improvement compared to $f_v\left(\frac{\Delta u}{\Delta t}\right)$, demonstrating that estimating $v$ using $\nabla u$ is more accurate than using $\frac{\Delta u}{\Delta t}$. This validates the effectiveness of the wave equation-based approach.

To further address your concern, we conducted similar experiments on ClimODE, as shown in Table A. The results reinforce the advantages of the wave equation-informed predictive structure, providing additional evidence of its efficacy.

Table A. Experiment of estimation of $v_0$ of ClimODE.

Finally, the qualitative argument about spatial resolution at the end of this section is not particularly convincing, as it compares spatial resolution to time resolution in a way that seems dimensionally inconsistent. The statement on line 243, the spatial domain is nearly 100 times denser than the temporal domain. needs further clarification, as its implications are unclear.

Thank you for pointing this out. We acknowledge that the comparison between spatial and temporal resolution might appear dimensionally inconsistent. However, the key lies in their underlying periodicity. The temporal domain has an implicit period of 24 hours, while the spatial domain spans the entire Earth. This leads us to compare 1/24 (temporal) with 1/(32×64) (spatial).

Since the units of time (hours) and space (kilometers) are not directly comparable, it is difficult to determine the "ideal" resolution between the two purely from theoretical arguments. To address this, we rely on experiments. In our Neural ODE, the velocity can be estimated temporally (using $\frac{\Delta u}{\Delta t}$) or spatially (using the wave function). Before conducting the experiments, we were uncertain which approach would work better. However, we observed a clear improvement with spatial estimation, while adding temporal information often led to negative results. This suggests that the temporal sampling frequency is severely insufficient, making time-based velocity estimation much less effective.

To further support this point, we supplement our experiments with precipitation forecasting tasks using the SEVIR dataset [5], which features a temporal resolution of 5 minutes (significantly shorter than the 1-hour intervals in our primary experiments). Table B below demonstrates that when the temporal sampling is sufficiently dense, using $\frac{\partial u}{\partial t}$ instead of $\nabla u$ for velocity estimation becomes viable. The results indicate that dense temporal resolution mitigates the disadvantages of temporal derivative approximation.

Table B. WeatherODE result on SEVIR dataset

(CSI is Critical Success Index)

[5] SEVIR: A storm event imagery dataset for deep learning applications in radar and satellite meteorology.

Neural ODEs are known to be computationally demanding, yet the paper does not address computational resources or the back-propagation method used to train the neural ODE. Since computational power can often be a limiting factor, it would be valuable to compare the computational times across the different architectures, particularly for the neural ODE versus other feedforward models.

Deep learning with physics-informed models like WeatherODE, ClimODE, and NeuralGCM often have higher computational costs compared to purely data-driven models such as FourCastNet and Pangu. However, these models benefit from the ability to predict multiple time steps in a single inference pass, which makes them more versatile for extended forecasts. (As WeatherODE$^{*}$ can generate predictions for all points within a 24-hour window in a single inference pass.)

In contrast, purely data-driven models typically require separate models for each lead time or rely on iterative rolling strategies, where outputs from one model serve as inputs for the next. This approach leads to repeated computations for multi-step forecasts. As a result, for scenarios involving multiple consecutive lead times, physics-informed models can offer better computational efficiency by eliminating the overhead of repeated inference.

Questions

The WeatherODE* model appears to be an interesting research direction, offering flexibility in generating forecasts at different lead times. However, there is limited explanation as to why WeatherODE is capable of such flexibility. Could the authors clarify the connection between the following sentence: by modeling the atmosphere as a physics-driven continuous process and designing a time-dependent source network to account for errors at each time step, WeatherODE can capture information across all intermediate time points and the success of the 24-hour model WeatherODE*?

Thank you for your question. Let me clarify. As mentioned earlier, the Neural ODE produces outputs $\hat{u} \in [N, K, H, W]$, where $N$ represents the desired lead time length. Each step along the $N$-dimension corresponds to predictions at specific future time points. For example, with $N = 24$, $\hat{u}[5,:,:,:]$ represents the forecast for the 6th hour, while $\hat{u}[14,:,:,:]$ corresponds to the 15th hour. These outputs are generated in a single pass through the Neural ODE. We believe this design inherently captures temporal dependencies across adjacent time points during the ODE iterations of the "adjoint states", offering a significant advantage over purely data-driven models.

In contrast, models like Pangu or FourCastNet typically learn direct mappings between two discrete time points, such as data at $t$ and $t+6$ for a 6-hour forecast. As a result, to predict different lead times, these models either require training separate models for each lead time or rely on rolling strategies with multiple models. This makes Neural ODE-based methods more naturally suited for capturing the continuous dynamics of the atmosphere.

Regarding the time-dependent source network, we apply it to the ODE outputs $\hat{u} \in [N, K, H, W]$ to model the potential effects of the source term. Instead of embedding the source term directly within the ODE iterations—which could amplify biases inherent to the physical process—we model the source influence separately using a 3D CNN that processes all $N$-dimensional outputs simultaneously. The "time-dependent" aspect refers to our explicit consideration of temporal correlations across the $N$-dimension, leveraging the 3D CNN's ability to learn dependencies across time steps effectively.

We hope Reviewer g7RP finds these clarifications helpful and appreciates the reasoning behind our design choices. Thanks again for valuable feedback.

The limited number of variables used (5), coarse spatial resolution (5.625-degree vs 0.25; ~400x smaller horizontal resolution), and small forecasting lead-time (72-hour) are too unconvincing to test this Neural ODE formulation for real weather application.

This is a fair concern, and we acknowledge the limitations of using only five variables and a 5.625° resolution. However, these choices were driven by the computational constraints faced by our research team. Training a neural ODE model at 1.5° or 0.25° resolution requires substantial resources—potentially hundreds or thousands of GPUs—making it feasible only for major tech companies like Google, Meta, or NVIDIA. For instance, Google's GraphCast achieves 0.25° resolution without neural ODEs, and even their neuralGCM model is limited to 1.4° resolution due to the computational demands.

While our current setup is indeed coarse, we see this as a necessary first step for demonstrating the feasibility of neural ODEs in this context. The positive results at 5.625° resolution suggest the potential for improved performance as we scale to finer resolutions, as supported by established scaling laws in machine learning.

To further support this point, we supplemente our experiments with precipitation forecasting tasks using the SEVIR dataset [1], which features a temporal resolution of 5 minutes (significantly shorter than the 1-hour intervals in our primary experiments). Table A below demonstrates that when the temporal sampling is sufficiently dense, using $\frac{\partial u}{\partial t}$ instead of $\nabla u$ for velocity estimation becomes viable. The results indicate that dense temporal resolution mitigates the disadvantages of temporal derivative approximation.

Table A. WeatherODE result on SEVIR dataset

(CSI is Critical Success Index)

[1] SEVIR: A storm event imagery dataset for deep learning applications in radar and satellite meteorology

This is echoed by the result in Appendix E Table 9, which is inferior to ClimaX at 3-day lead time, with error growth much larger than the other model (also lacking IFS baseline here to see how the rate of error propagation). The ACC for u10, for instance, deteriorates from 0.93 to 0.80 even at 36-hour difference! The fundamental error in the assumptions (which ClimODE is too, and the authors should therefore use stronger baselines such as GraphCast) might contribute to this result.

We appreciate this observation and agree that our model shows limitations over longer lead times, with notable error growth. The absence of an IFS baseline in Table 9 was an oversight that we will address in future revisions. Comparing our model to stronger baselines like GraphCast would be ideal, but doing so at 5.625° resolution is challenging given the significantly finer resolution (0.25°) of GraphCast.

While our results highlight some limitations, they also point to the potential of neural ODEs for capturing atmospheric dynamics. For example, our model's RMSE for 24-hour forecasts is comparable to that of downsampled EC-IFS results at 5.625°. These findings suggest that our approximation may not be as superficial as it seems and could achieve competitive results at finer resolutions.

If the goal is to model a simplified flow, such as tracer dynamics where their evolution is passive e.g., does not have any feedback loop with other quantities, and local, then advection might be sufficient. But the atmosphere is definitely not the case.

We agree that the atmosphere involves complex feedback loops and interactions that go beyond what the advection equation can capture. However, our goal is not to create a comprehensive physical model but to explore how simplified physical principles can guide data-driven learning. While advection provides only a partial description of atmospheric processes, its simplicity offers neural networks the flexibility to learn additional patterns from data.

As noted earlier, our approach reflects a tradeoff: we incorporate minimal physical bias to guide learning while maintaining enough flexibility to adapt to data-driven insights. This balance is critical for handling the inherent uncertainties in atmospheric dynamics, particularly given the chaotic nature of the weather system.

We hope Reviewer tB5a appreciates the reasoning behind these design choices and acknowledges the constraints faced by researchers outside major institutions. We value the feedback and believe it will help refine and advance our work in future iterations. Thanks again for thoughtful comments.

We deeply appreciate Reviewer tB5a’s insightful and detailed feedback. The points raised demonstrate an exceptional understanding of the intricacies of weather forecasting and atmospheric modeling. Below, we provide detailed responses to each point and aim to clarify our approach further.

The paper assumes incompressible fluid, where fluid density remains static along the pressure level. This is a gross simplification and is unphysical as the weather dynamics is completely dependent on the fluid being compressible, to allow interesting processes to take place such as convection through buoyancy (cloud formation, precipitation, energy-water cycling), large-scale fluid motion (teleconnection), turbulence (boundary-layer interaction), etc.

We acknowledge that assuming incompressibility is a simplification. However, our intention is not to fully replicate the complexity of weather dynamics but to explore a tractable approach for introducing physical insights into data-driven models. Our assumption of mass conservation at a regional scale (approximately incompressible within our spatial patches) serves as a generalizable physical bias rather than a strict representation of atmospheric processes.

We aim for a minimal yet effective incorporation of physical constraints, following Occam's Razor: "We consider it a good principle to explain the phenomena by the simplest hypothesis possible." By simplifying the dynamics, we provide neural networks with just enough guidance to learn from the data while maintaining flexibility for capturing more nuanced, emergent patterns.

We recognize the value of incorporating vertical dynamics and more complex interactions (e.g., buoyancy, turbulence). However, doing so requires significantly more computational resources, as further detailed below.

The paper also ignores many important conservation constraints found in the classical dynamical core, such as the conservation of mass, energy, and momentum.

This is a valid point. While we agree that conservation laws such as energy and momentum are crucial for accurately modeling atmospheric dynamics, we opt to focus on the simpler mass conservation principle as an initial step. Mass conservation is one of the most fundamental constraints and provides a tractable starting point for neural ODE-based modeling, particularly given the resource constraints discussed below.

In future works, we aim to explore methods for integrating additional conservation constraints into our framework while balancing computational feasibility and model flexibility.

As such, there are better approximations to the full Navier-Stokes equation, such as Shallow Water Equation or Quasi-Geostrophic flow, that attempts to capture some of the realism of the atmosphere, and is therefore a much better differential than the advection equation.

We agree that models like the Shallow Water Equation or Quasi-Geostrophic flow offer better approximations of atmospheric dynamics compared to the advection equation. However, our primary goal is to explore the potential of neural ODEs in weather forecasting, leveraging a simplified equation to provide flexibility for data-driven learning. The advection equation, derived from mass conservation, offers a reasonable starting point as it captures essential transport dynamics while introducing minimal physical bias.

Our choice is not to suggest that the advection equation is a complete representation of atmospheric dynamics but rather to create a framework that can be scaled and adapted as computational resources and data availability improve. As resources permit, incorporating more sophisticated equations like the Shallow Water Equation is a natural next step.

I appreciate the authors' response to my feedback. However, the inclusion of precipitation highlights a fundamental flaw in the approach—compressible fluid dynamics are essential for forecasting precipitation, as changes in fluid density across vertical levels (i.e., adiabatic processes) cannot be ignored. The assumption that the system is "approximately incompressible within spatial patches" is even more problematic, as precipitation and other weather phenomena rely on localized vertical motion, including microphysical processes. Resolving these small-scale processes remains a major challenge in climate modeling, often necessitating parameterization [1] (clarification: this is still an extremely challenging task even with the full primitive equation as the dynamical core; and adding more uncertainty to this already uncertain task through the use of overly simplified dynamics to the point of unphysical, and forcing the model to somehow learn this in a fully data-driven way is less helpful and may impede progress). While I understand the intent to simplify dynamics, the advection equation is inappropriate for this case, and I recommend the authors to consider NeuralGCM [2], which employs a more appropriate simplified dynamical core (e.g., the Shallow Water Equation) that better approximates the Navier-Stokes equations. In short, the use of advection equation is fatally incorrect, and in light of the rebuttal and to avoid this error from appearing in future works, I reduce my score to reflect this.

References:

[1] Rasp, Stephan, Michael S. Pritchard, and Pierre Gentine. "Deep learning to represent subgrid processes in climate models." Proceedings of the national academy of sciences 115.39 (2018): 9684-9689.

[2] Kochkov, Dmitrii, et al. "Neural general circulation models for weather and climate." Nature 632.8027 (2024): 1060-1066.

We sincerely thank Reviewer wKzX for their thoughtful feedback. The questions posed are both intriguing and align with previous studies we've conducted but did not include in the paper. Below, we address each point raised in detail.

Weakness

The CNN used in variable space scale loses part of its physical meaning.

We acknowledge this point, and this question is closely related to what we believe is the path to pursue in weather forecasting. Our approach aims to strike a balance between incorporating minimal physical biases and leveraging the flexibility of machine learning. But our balance leans more toward a data-driven direction.Ideally, a completely data-driven model could fully capture atmospheric dynamics without introducing any biases. However, this would require vast amounts of data and computational resources, especially to model rare weather events.

To address these limitations, we incorporate simple and generalizable physical principles, such as advection, which introduce weak biases to guide the neural network. This also adheres to Occam’s Razor: "We consider it a good principle to explain the phenomena by the simplest hypothesis possible." By combining these weak biases with data-driven learning, we aim to provide enough guidance to learn robust representations while maintaining flexibility to model complex patterns beyond the physical assumptions.

The wave equation doesn’t describe the atmospheric dynamics really well so the derivative approximation is not governed by physics

We agree that the wave equation is a simplification and does not fully describe atmospheric dynamics. However, its high-level relationship to the advection equation inspired our approach. Specifically, the wave equation highlights how spatial gradients can inform temporal derivatives, enabling us to overcome limitations posed by the low temporal resolution (1-hour intervals) of our input data.

Our ablation studies demonstrate the effectiveness of incorporating spatial information, with results showing a significant improvement (10%) when modeling velocity using spatial derivatives rather than temporal differences. While the wave equation itself is not directly integrated into the model, it provides a conceptual foundation for improving derivative approximation.

To further support this point, we supplement our experiments with precipitation forecasting tasks using the SEVIR dataset [1], which features a temporal resolution of 5 minutes (significantly shorter than the 1-hour intervals in our primary experiments). Table A below demonstrates that when the temporal sampling is sufficiently dense, using $\frac{\partial u}{\partial t}$ instead of $\nabla u$ for velocity estimation becomes viable. The results indicate that dense temporal resolution mitigates the disadvantages of temporal derivative approximation.

Table A. WeatherODE result on SEVIR dataset

[1] SEVIR: A storm event imagery dataset for deep learning applications in radar and satellite meteorology.

Additionally, we conducted experiments on ERA5 data to analyze the impact of increasing the temporal interval $\Delta t$ when estimating $\frac{\Delta u}{\Delta t}$ for initial velocity $v(t_0)$. Table B demonstrates that as $\Delta t$ increases, the accuracy of $v(t_0)$ estimation decreases significantly, sometimes leading to numerical instability in the ODE. These results highlight the challenges of relying on temporal differences with coarse sampling intervals.

Table B. z500 result of the time interval $\Delta t$ for estimating $\frac{\Delta u}{\Delta t}$ to calculate $v(t_0)$. NaN indicates numerical instability. Full results in Table 12 of the paper.

I sympathize with the authors regarding the unfortunate state of peer-review conducted at top ML conferences such as ICLR. I too have had a not-so-ideal experience this year where reviewers have been harsh towards my paper for no good reason. It is unfortunate that the reviews from non-serious reviewers (who barely read or try to understand the work) carries the same weight as serious reviewers (who put geniune effort in reading the paper).

Regarding reducing the score, yes I am one of the reviewers who reduced the score. But I maintain my stance. I have been objective about my evaluation. I gave it a solid 8 after first reading. But, as is the case usually - doing good science takes time - time to understand new ideas and appreciate them - time to connect them with the big picture, and so on. The review format of these conferences does not allow enough time to read and understand these paper thoroughly - more so when one's schedule is already packed. After understanding your work more properly the second time, more limitations of the work popped up to me. Yet, because I like the overall idea, I have maintained a final decision to 'accept' the manuscript so as to not be unfair.

You have a nice manuscript and a nice idea which has the potential to be a good contributions to the field. However, there are mutiple deficiencies in your approach as well which would prevent longer-term rollouts while being efficient - this ultimately conflicts with the usefulness of the approach. I think my concerns are grounded in scientific principles which, at the end of the day, are needed for a robust experimental design. My background is both in weather&climate dynamics and machine learning so it is important that the work makes sense to me both from the physical science and data perspective.

If you can find solid grounding to nullify the concerns I make, I would be happy to reconsider my score. Until then, it is my responsibility to provide as fair of an evaluation as I can to the manuscripts assigned to me. At this point I don't see any. Conducting ML experiments is relatively easy, providing rigorous justification as to why they work is tough. Best of luck for the final decision.