Physics-Guided Learning of Meteorological Dynamics for Weather Forecasting and Downscaling

We thank the reviewer for their time and constructive comments. We have updated the manuscript to improve organization, results interpretation. We thoroughly address your concerns below. We hope the reviewer’s score can be updated to reflect the significance, novelty, and timeliness of our study.

1. Authors should discuss difference to existing works, epsecially to works in similar topics.

Thank you for the constructive feedback! We have made substantial revision. To clarify, our novel model design is partially based on the physical constraint loss used by PINN, but also expands and improves the PINN to address the problem of weather downscaling and forecasting, collaborating with meteorology experts and physicists. The weather prediction task is highly complicated, with many weather factors and physical processes in the play, easily influenced by local variations like change of boundary conditions, small-scale phenomena like microclimates and external forces like heat from the sun. Many of these vital factors, which have huge impacts in the first-principle equations, are missing in the data due to difficulties to measure and quantify. For example, as Eq. 14 in Page 9 shows, the effect of friction characterizing air viscous resistance cannot be represented by any of the existing data elements. The nuances make the physical equation incomplete. Therefore, it is intractable to rely on a fixed PDE, while PINN directly uses a known PDE to guide optimization under all circumstances, which could be hard to adapt to the real world.

To address this concern, we do not rely on any known physical equations with PINN to guide prediction, but propose a data-driven approach to adaptively consummate the first-principle physical equation to explain the physical mechanism that drives the weather prediction. Our approach has the potential of discovering the intricate interplay between various weather factors that is previously ignored, as an adaption to various conditions in different areas. We not only complete the tasks of weather downscaling and forecasting, but also provide insights of the nuances between different climates at any continuous spacetime. Our model is of great value to the meteorology community, as verified by the physicists we are collaborating with.

Furthermore, we propose a latent force term $Q_\pi$ as a parametrization term in the equation, following the parametrization strategy [1] widely adopted by meteorology experts to supplement the forces that cannot be represented by the selected explicit PDE terms. All of these novel model designs differ from existing works in the PINN family. There is almost no work that successfully applies PINN for weather prediction. We believe that our approach fills this gap in time and provides a feasible way to improve our understanding of the physical mechanism of climate and improve the deep learning models’ performances, which is well-supported by our promising experimental results. We wish to highlight that our PhyDL-NWP framework only contains 55 thousand parameters, as shown in Table 7 in the appendix, which is about 1000 times lighter than some large models and extremely efficient to train. In our experiments, the training of deep learning model usually takes 20~50x more time than obtaining the PDE we need. Our contribution is not trivial and is of great value to not only the meteorology community but also the representation learning community. When tackling similar scientific tasks that also involve complex interplay between variables and insufficient data measurement of the nuances, our work will provide a valuable reference.

2. Authors should discuss difference to existing works, especially to the downscaling problem.

Our work is inherently different from most of the existing downscaling works, as our approach directly models the continuous dynamics, instead of performing discrete superresolution. Our approach is much more elegant, adaptive and accurate. In our paper, we use a paragraph to highlight this difference in Sec. 3.2, "With the trained $f_\theta$, we can easily obtain the weather factors at any given input coordinates $(x, y, t)$, which can be continuous over the spacetime. This property of modeling the meteorology dynamics is naturally suitable for weather downscaling: unlike the previous methods powered by the discrete encoding and decoding of neural networks on finer-grained data as labels, the $f_\theta$ in PhyDL-NWP can perform weather downscaling with unlimited granularity without labels given a continuous coordinate, as described by Fig. 1. Therefore, as long as we obtain the dynamics of meteorology and the solver of weather factors through updating $\Xi$ and $\theta$, the downscaling task is solved automatically.".

[1] Warner, Thomas Tomkins. Numerical weather and climate prediction. Cambridge University Press, 2010.

7. Non-SOTA baselines.

Thank you for the comment. The downscaling baselines we use are FSRCNN (2022), ResDeepD (2022), EDSR (2022), RCAN Yu et al. (2021). On the contrary, the baselines suggested by the review are YNet (2020) and DeepSD (2017). We believe that we have compared with the strong recent baselines. However, we are glad to provide results of YNet and DeepSD.

As suggested, the two baselines seem not very strong, and our model clearly outperforms them. We hope this could dispel the reviewer’s concern.

2. PDEs are parameterized as NN and are learned from data. It's not guaranteed that they'll agree with physics laws.

Thanks for your raising this point. To clarify, not all components of PDE are parameterized as a neural network. $\phi(u)$ means the explicit equation, and only $Q_\pi$ serves as the parametrization term. It is true that our PDEs are learned from data, therefore there is no 100% guarantee that it will agree with physics law. However, we explicitly initialize the PDE with a priori partial derivative terms aligning with related physical equations, as explained in Sec. 4.3. This greatly improves the alignment with existing physics; and from experimental results, we do find that the resultant PDE aligns well with physics. Therefore, we expect our results to be at least as good as the original physical equations, and will improve upon the existing equation to include expression ability for nuances to fit the data even better.

3. Scalability.

Thank you for the comment. We wish to highlight that our PhyDL-NWP framework only contains 55 thousand parameters, as shown in Table 7 in the appendix, which is about 1000 times lighter than some large models and extremely efficient to train. Our model is in fact very efficient and easy to train, and our data setting is exactly the high resolution setting the reviewer suggests. Our data resolution is 0.25 degree, which is almost the highest resolution we can obtain from ERA5 and the same setting most large models are using. In our experiments, the training of deep learning model usually takes 20~50x more time than obtaining the PDE we need.

4. Ablation study.

Thank you for your comment. We do perform an ablation study to directly use PINN instead of combining a deep learning model, shown as “PINN” in Tables 2-3. This is the core ablation study we can think of, since it would be hard to directly find any equations to use with PINN and discard our PDE discovery part of the model. The weather prediction can be easily influenced by local variations like change of boundary conditions, small-scale phenomena like microclimates and external forces like heat from the sun. Many of these vital factors, which have huge impacts in the first-principle equations, are missing in the data due to difficulties to measure and quantify. For example, as Eq. 14 in Page 9 shows, the effect of friction characterizing air viscous resistance cannot be represented by any of the existing data elements. The nuances make the physical equation incomplete. Therefore, it is intractable to rely on a fixed PDE, while PINN directly uses a known PDE.

5. n' and m' (coordinates of the high-resolution data) for Eq.4.

Thanks for asking. Since n’ and m’ are resulting from the allocation, it requires differentiation, while Eq.4 is about data loss that requires actual data points. We have not thought about a strategy to sample high-resolution points not in data.

6. How to make prediction for downscaling.

Thanks for asking. It is extremely easy: we simply input the spatiotemporal coordinates into the approximate function $f$, which is a continuous function of input coordinates achieved by a dense neural network. Given any coordinate to predict, we directly get the value of that point.

We thank the reviewer for their time and constructive comments. We have updated the manuscript to improve organization, results interpretation, and add a new experiment in Section 4.2. In addition, we have cited all relevant papers and think they are useful. We thoroughly address your concerns below. We hope the reviewer’s score can be updated to reflect the significance, novelty, and timeliness of our study. Due to the space limit, we separate our rebuttal into several parts.

1. Small-scale datasets targeting on regions of China.

While it is true that our selected datasets are only subsets of ERA5 dataset, the original ERA5 dataset is enormously large. Only very few works, like those with large models, tend to use the full ERA5 datasets or datasets of similar scale. Limited to computational resources, it is common practice to use subsets of the large data. We want to highlight that we use fine-granular hourly data with 0.25 degrees of spatial resolution and diverse continental climate in arid areas as represented by Ningxia dataset and mild humid oceanic climate as represented by Ningbo dataset. The datasets we select are generally representative and comprehensive. We select these datasets in specific regions of China because we are in collaboration with institutions within mainland China, and we have the accessibility of exclusive datasets such as the Ningbo Meteorological Bureau dataset. This dataset is of very high quality as it is not a reanalysis data and is closer to the actual measurement data. It is undoubtedly better if we test models on actual weather data without post-processing/reanalysis.

Despite the above points, we appreciate the suggestion of the reviewer and include an additional experiment with WeatherBench. To have comparable results, we use the 5.625 degree of spatial resolution and 6 hours of time resolution setting to preprocess the dataset. We use ten years of training data from 2006 to 2015, validation data in 2016 and test data in 2017-2018. We select ground temperature (t2m), atmospheric temperature (t), geopotential (z) and ground wind vector (u10, v10) as the weather factors. We apply the normalization to these weather factors and used their recommended training regime and hyperparameters. We use this data to perform weather forecasting and compare with baselines. The results are as follows.

7. Non-SOTA baselines (continued)

The reviewer also suggested that we compare with large foundation models in recent years for weather forecasting. However, there are many constraints. First, there is no/incomplete public code for Pangu-Weather and GraphCast. Second, we have indeed implemented FourCastNet, which is shown by “AFNO” in our experiment. Third, we have implemented very comprehensive baselines covering meteorological models, spatio-temporal graph models (The GNN models as suggested by the reviewer) and vision models. In fact, many large models are built upon backbones following the three categories. We believe our evaluation is quite adequate. We hope the reviewer might consider our efforts: we compare with, additionally, ClimaX for forecasting on the WeatherBench ERA5 dataset as suggested by the reviewer.

We thank the reviewer for their time and constructive comments. The philosophical debate is very useful for improving my paper, while we update our paper with major revision and hope the comprehensive experimental results may convince the reviewer of our motivation. We have updated the manuscript to improve organization, results interpretation, and add a new experiment in Section 4.2. We thoroughly address your concerns below. We hope the reviewer’s score can be updated to reflect the significance, novelty, and timeliness of our study. Due to the space limit, we will address the rebuttal in 2 parts.

1. The training details, especially the hyperparameter tuning.

Thank you for raising this point. We are glad to provide more details on the hyperparameter setting. $\sigma_1$ is set as 0.0001, $\sigma_2$ is set as 0.001, $\sigma_3$ is set as 0.01, $\alpha$ is set as 1, $\beta$ is set as 1, $\gamma$ is set as 0.3. These coefficient hyperparameters are tuned based on grid search. $\sigma_1$, $\sigma_2$, $\sigma_3$ are tuned within [0.01, 0.001, 0.0001], $\alpha$, $\beta$, $\gamma$ are tuned within [3, 1, 0.3]. Our $f$ model is a standard dense neural network consisting of eight layers, activated by Tanh functions. The dimensions of each layer are [3, 60, 60, 60, 60, 60, 60, 60, 60, 8]. Our $Q$ model is of a similar architecture of [3, 20, 20, 20, 20, 20, 20, 20, 20, 8]. $\phi(\text{u})$ is determined by the sparse regression of derivatives of all weather factors to space and time up to the second-order, with certain terms that appear in traditional NWP and physical equations fixed as a priori.

2. Disagreement to the motivation of PhyDL-NWP.

Thank you for the comment. Respectfully, we do not agree with the critic that ML can generalize well in weather forecasting tasks. While we believe that philosophical debate is useful, whether our model actually works is most important. Supported with extensive experiments, and additional experiments provided in response to Reviewer HSXq and Reviewer W7oo, we have undoubtedly demonstrated the effectiveness and efficiency of our model. In fact, based on extensive studies of deep learning models in the field, even including large models such as Pangu-Weather, GraphCastNet, etc., one would not be certain that ML can generalize well in real-world situations. For example, Pangu-Weather failed to predict the trajectory of Typhoon Doksuri in 2023, which landed in mainland China and caused incalculable losses. In fact, people find that Pangu's forecast of the landing site is still less accurate than the numerical model, i.e., NWP. If ML models can indeed generalize well, this would not happen, and we could have saved billions of dollars and many lives. Moreover, as an interdisciplinary team focusing on weather prediction research, we have done many simulations but unfortunately found that deep learning models are generally unreliable, especially for long-term prediction. On the other hand, NWP results are relatively better. It is a hard truth, but the status quo of the field. Based on this, we strongly urge the need to incorporate physical insights and combine NWP into deep learning models.

3. Complicated "state-of-the-art" architectures are not important.

Thank you for the comment. Respectfully, based on our experiments, we do find that SOTA architectures are important. Moreover, the development of large models (such as Pangu-Weather, Graphcast, ClimaX, FourCastNet, NowCastNet, etc.) and their amazing results also suggest that model architectures and capacities do matter. For example, AFNO, the backbone of FourCastNet, performs much better than ConvLSTM. Before 2022 when there is no research on large models for weather prediction, there is almost no report that any deep learning model has ever had a comparable result to NWP, even though global weather data is available decades ago. If input and output are the only important things, we should have seen remarkable results long before 2022, maybe even before transformers are invented. Therefore, we respectfully disagree with the reviewers’ critics.

In addition, our point of view regarding the reason PINN won’t work may have been misunderstood. As explained in Sec. 3.3, “In particular, $f_\theta$ is only trained on historical coordinates, while anywhere outside the bounds of where the model was trained is completely unknown to $f_\theta$.” PINN itself also faces the problem of generalization, since it is a deep learning model itself. In contrast, the setting of leveraging historical data to predict future data and the SOTA architectures are much more favorable, which are strong inductive bias and help improve the model accuracy. We hope to combine the advantages of both physics and SOTA deep learning formulation, which motivates our work.

4. Natural images have more non-local spatio-tepmoral dependencies.

Thank you for the comment. Respectfully, we partially disagree with your argument. First, weather prediction generally has multiple channels, if not hundreds of channels in theory, while natural images only have 3 dimensions. It might be correct to say that Computer Vision models can process spatio-temporal dependencies as well, with advanced architecture design. In the paper, we only wish to highlight that, weather factors represent spatio-temporally correlated physical process, sometimes explained by chaos theory. For example, the famous Butterfly effect explained that even with very long distance and time span, we may find causation between two distanced coordinates on earth. This degree of non-local dependency, is hardly, at least not usually, the case that can be considered in natural images. To avoid controversy or misunderstanding, however, we are willing to give up that statement in the paper. We thank the reviewer for their input and encourage the reviewer to take a new review of our updated manuscript.

5. Deep learning model is not accurate enough in this paper.

Thank you for the comment. As Tables 2 and 3 have shown, deep learning models (Bi-LSTM-T, Hybrid-CBA, ConvLSTM, AFNO, MTGNN, and MegaCRN) clearly have lower RMSE than NWP. It is true that on the metric of ACC, NWP is the best when we consider the 7-day weather forecasting. However, note that, as visualized in Figure 3 in the main paper and Figures 5-8 in the Appendix, when we consider the short-term weather forecasting, especially within 48 hours, the AFNO’s ACC curve is clearly above 0.526, showing superior performance against non-deep learning NWP. NWP is based on numerical simulation, therefore has a fixed value at any certain time regardless of the prediction time and the length of historical values used. Furthermore, as we explained in response to Q4, when we use coarse resolution of WeatherBench data, the ACC metrics are much higher than the ones reported with our original two datasets. This is because a coarse resolution dataset is relatively not likely to be severely affected by local variations and noises. In conclusion, deep learning models in our paper are in fact quite accurate. The seemingly lower performance is only due to the challenging data we use and the long-term prediction setting we apply.

6. The setting of train, validation, and test. The code.

Thanks for the comment. Due to the limited space to present our paper, we explained in Sec. 4.2 on Page 7 that “More details of the datasets are given in Appendix A.1”. We are glad to provide the core code in an anonymous link. https://anonymous.4open.science/r/123-C702/README.md

4. Comparison to baselines with ERA5 or WeatherBench dataset.

Our Ningxia and Huazhong datasets are from ERA5 as explained in Appendix A.1, just like WeatherBench. We select the current three local datasets to highlight our model’s ability to provide supplement to existing physical equations even in a high-resolution local area. Especially, we test our approach on challenging and fine-granular hourly data with 0.25 degrees of spatial resolution and diverse continental climate in arid areas as represented by Ningxia dataset and mild humid oceanic climate as represented by Ningbo dataset. At this scale, weather factors are more likely to be disturbed by local variations like change of boundary conditions and small-scale phenomena like microclimates. WeatherBench, regridded the original 0.25 degree of resolution to the degrees of 5.625, 2.8125, 1.40625, which are much coarser than the data we use. The global weather forecasting based on low-resolution data is of less relevant to our work. We want to highlight our model’s supplement to physical first-principle mechanism and combine such an updated mechanism to guide deep learning models. Therefore, experiments on high-resolution datasets are necessary to test whether our PhyDL-NWP can work even in a relatively noisy environment and can work for long-term medium-range forecasting. We follow a similar setting as in NowCastNet, which also incorporates physical mechanism and focuses on the weather prediction of local regions.

We appreciate the suggestion and include an additional experiment with WeatherBench. To make fair comparison, we use the 5.625 degree of spatial resolution and 6 hours setting to preprocess the data. We use 10 years of training data from 2006 to 2015, validation data in 2016 and test data in 2017-2018. We select ground temperature (t2m), atmospheric temperature (t), geopotential (z) and ground wind vector (u10, v10) as the weather factors. We apply the normalization to these weather factors and used their recommended training regime and hyperparameters. We use this data to perform weather forecasting and compare with baselines.

It shows our model’s performance is more significant on coarser-grained weather data. Since we use coarse resolution of WeatherBench data with prediction in 48 hours, it is normal that the ACC metrics are higher than the ones reported with our original two datasets. This might answer the reviewer’s other question regarding the accuracy of deep learning models in our paper in general. In contrast, our paper considers more challenging datasets with higher resolution and long-term prediction setting.

We thank the reviewer for their time and constructive comments. We have updated the manuscript to improve organization, results interpretation, and add a new experiment in Section 4.2. We thoroughly address your concerns below. We hope the reviewer’s score can be updated to reflect the significance, novelty, and timeliness of our study. Due to the space limit, we will address the rebuttal in several parts.

1. The writing of the paper and especially the description of symbols.

Thank you for raising this point. We have made substantial revisions. We encourage the reviewer to look at the updated manuscript for further evaluation.

2. The measurement of physical consistency.

Thank you for the comment. In our paper, we consider “physical consistency” as the model’s alignment with existing physical principles. Therefore, we use Sec 4.3 to analyze how well our data-driven discovery of physical mechanism can align with the existing equations. We exclusively encode a priori PDE terms into our PhyDL-NWP framework, which ensures that relevant terms present as a constraint to improve physical consistency, as explained in Sec. 4 and Sec. 4.3. As shown by the empirical results, not only the a priori terms are discovered, but also some new terms are discovered as a supplement to the existing equations.

3. Related works of recent progress and core contributions to the broad PINN family.

Thanks for the comment. We have expanded and reorganized the related work section. Especially, we directly name certain models to be more specific about each baseline’s setting and its difference to our approach. In addition, we improve the presentation to indicate our relationships to PINN family.

Our novel model design is partially based on the physical constraint loss used by PINN, but also expands and improves the PINN to address the problem of weather downscaling and forecasting, collaborating with meteorology experts and physicists. The weather prediction task is highly complicated, with many weather factors and physical processes in the play, easily influenced by local variations like change of boundary conditions, small-scale phenomena like microclimates and external forces like heat from the sun. Many of these vital factors, which have huge impacts in the first-principle equations, are missing in the data due to difficulties to measure and quantify. For example, as Eq. 14 in Page 9 shows, the effect of friction characterizing air viscous resistance cannot be represented by any of the existing data elements. The nuances make the physical equation incomplete. Therefore, it is intractable to rely on a fixed PDE, while PINN directly uses a known PDE to guide optimization under all circumstances, which could be hard to adapt to the real world.

To address this concern, we do not rely on any known physical equations with PINN to guide prediction, but propose a data-driven approach to adaptively consummate the first-principle physical equation to explain the physical mechanism that drives the weather prediction. Our approach has the potential of discovering the intricate interplay between various weather factors that is previously ignored, as an adaption to various conditions in different areas. We not only complete the tasks of weather downscaling and forecasting, but also provide insights of the nuances between different climates at any continuous spacetime. Our model is of great value to the meteorology community, as verified by the physicists we are collaborating with.

Furthermore, we propose a latent force term $Q_\pi$ as a parametrization term in the equation, following the parametrization strategy [1] widely adopted by meteorology experts to supplement the forces that cannot be represented by the selected explicit PDE terms. All of these novel model designs differ from existing works in the PINN family. There is almost no work that successfully applies PINN for weather prediction. We believe that our approach fills this gap in time and provides a feasible way to improve our understanding of the physical mechanism of climate and improve the deep learning models’ performances, which is well-supported by our promising experimental results. We wish to highlight that our PhyDL-NWP framework only contains 55 thousand parameters, as shown in Table 7 in the appendix, which is about 1000 times lighter than some large models and extremely efficient to train. In our experiments, the training of deep learning model usually takes 20~50x more time than obtaining the PDE we need. Our contribution is not trivial and is of great value to not only the meteorology community but also the representation learning community. When tackling similar scientific tasks that also involve complex interplay between variables and insufficient data measurement of the nuances, our work will provide a valuable reference.

[1] Warner, Thomas Tomkins. Numerical weather and climate prediction. Cambridge University Press, 2010.