PostCast: Generalizable Postprocessing for Precipitation Nowcasting via Unsupervised Blurriness Modeling

W2 & Q1: Additional metrics for a more comprehensive evaluation.

A2: Thanks for your valuable suggestion. As you suggested, we have added more evaluation metrics to achieve a more comprehensive evaluation, including meteorological metrics HSS, POD(Recall), CSI-avg. and image metrics PSNR and SSIM in the revised paper (see Table 1, Table 3, and Table 5 in the revised paper). These additional metrics further validate the effectiveness of our method. Part of the results are directly presented in the tables below:

SEVIR

HKO7

TAASRAD19

Shanghai

SRAD2018

Q1: The alignment between the nowcasting outputs after deblurring and the ground truth.

A3: Thanks for your valuable question. The degree of match between the nowcasting outputs, after deblurring with this methodology, and the ground truth can be assessed both qualitatively and quantitatively. Qualitatively, Figure 11-18 in Appendix.A.10 shows that our PostCast provides the prediction with small weather patterns that better match the ground truth, and aligns the pixel value distribution of predictions closer to the ground truth. Quantitatively, we opt to evaluate our PostCase on meteological metrics such as CSI, HSS, POD rather than image metrics such as MSE or PSNR. The reason is that MSE and PSNR are limited to handling small-scale weather patterns with high-frequency details. For instance, the pixel-level offsets of the predictions in heavy precipitation significantly increase the MSE value, which implies MSE prefers smooth predictions. For PSNR, it often penalizes high-frequency details as demonstrated in Google SR3 [Ref.1], such as for the small-weather systems in precipitation. Instead, we choose other methods to evaluate the degree of matching. In methodology, the degree of matching is evaluated by the average of meteological metrics such as CSI, HSS, POD across different threshold intervals, which are added to the revised paper (see Table 1, Table 3, and Table 5 in the revised manuscript). These scores demonstrate that our PostCast aligns the predictions well with the ground truth.

[Ref.1] Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4):4713–4726, 2022b.

SCWDS CAP30

SCWDS CR

MeteoNet

Additionally, SSIM and PSNR often penalize high-frequency details as demonstrated in Google SR3 [Ref.1], such as for the small-weather systems in precipitation recovered by our PostCast. As a method for postprocessing for precipitation nowcasting, our PostCast mainly focuses on increasing the quality of predictions in a meteorological sense.

[Ref.1] Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4):4713–4726, 2022b.

Firstly, we want to sincerely thank the reviewer for the feedback and for acknowledging the strengths of this paper.

We address each weakness and question below:

W1: The organization of "method" section.

A1: Thank you for your valuable feedback. In order to improve the organization of the "method" section, following your advice, we have created subsections for the “zero-shot kernel estimation mechanism” and “auto-scale denoise guidance strategy”. Moreover, to improve the clarity, we have also added more details into Appendix A.4 and carefully created links in the "method" section. The revised sections are highlighted in blue.

The authors would like to express their heartfelt appreciation to the reviewer for the valuable and encouraging feedback.

W1 & Q1: Limited Evaluation Metrics.

A1: We thank you for the invaluable suggestion. In order to provide a more comprehensive evaluation of PostCast, we have added three meteorological metrics including CSI-avg., HSS-avg., POD-avg., as well as the two image metrics, SSIM and PSNR. We replace FAR-avg. with HSS-avg., as FAR only considers TP and FP, without accounting for FN which is quite important for precipitation nowcasting because the underreporting of extreme precipitation can lead to significant losses. Instead, HSS is a more comprehensive metric taking account into all the TP, FP, and FN. The formulations are: $FAR = \frac{FP}{TP+FP}$, $HSS = \frac{2\times(TP\times TN - FN \times FP)}{(TP + FN)\times (FN+TN)+(TP+FP)\times (FP+TN)}$. The POD-avg. and HSS-avg. are added to Table 1 and Table 3 in the revised paper with blue text. Meanwhile, CSI-avg., SSIM and PSNR scores are recorded in Table 5 in Appendix.9. We also collect a part of these metrics in the table below:

SEVIR

HKO7

TAASRAD19

Shanghai

SRAD2018

SCWDS CAP30

SCWDS CR

MeteoNet

On meteorological metrics CSI-avg., HSS-avg., and FAR-avg., our PostCast significantly surpasses DiffCast and CasCast. After applying our PostCast, these metrics are improved notably on most datasets. On image quality scores, SSIM and PSNR, our PostCast improved SSIM in some settings, such as HKO7 and TAU-SRAD2018, while the PSNR scores decreased. Such a phenomenon also appears when applying DiffCast and CasCast. It can be attributed to that the PSNR and SSIM scores often penalize synthetic high-frequency details as demonstrated in Google SR3 [Ref.1], such as for the small-weather systems in weather prediction, the shape and position of local patterns may slightly deviate from the ground truth which may also result in inferior PSNR and SSIM scores but could be tolerated in applications. Moreover, these scores are not strongly related to nowcasting skills evaluated by the meteorological metrics CSI-avg. , HSS-avg. , and FAR-avg. As a postprocessing method of precipitation nowcasting, our PostCast mainly focuses on improving meteorological metrics

[Ref.1 ] Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4):4713–4726, 2022b.

W2 & Q2: Enrich the Related Work.

A2: Thanks for your valuable suggestions. Indeed, discussing GAN-based and Transformer-based precipitation nowcasting methods could definitely better contextualize the background and contributions of PostCast. Therefore, we have discussed EarthFormer and DGMR in the related work, which could be directly found as follows:

Notable progress has been achieved by applying deep learning in precipitation nowcasting [Refs.1-5]. The initial attempts are deterministic methods focusing on spatiotemporal modeling. Researchers explore different spatiotemporal modeling structures such as RNN [Refs.6,7], CNN [Refs.8,9], and Transformer [Refs.10]. Specifically, EarthFormer [Refs.10] is proposed to capture spatial-temporal features in earth system evolution by cuboid attention. However, these methods have a shortage of blurry predictions, which hamper the nowcasting of extreme events. Probabilistic methods are proposed to alleviate blurriness [Refs.11,12,13]. DGMR [Refs.12] applies GAN to produce realistic and spatio-temporally consistent predictions to reduce blurriness. To further enhance precipitation nowcasting with accurate global movements, later methods combine blurry predictions with probabilistic models. DiffCast [Refs.14]), and CasCast [Refs.15] exploit how to generate small-scale weather pattern conditioning on blurry predictions. Although these deterministic-probabilistic coupling methods achieve both global accuracy and local details, they suffer from repeating training for different contexts and require blurry predictions as data for training. From the perspective of deblur, Our method is proposed to simplify the complex training process and enhance the generality for wide use in different contexts.

[Ref.1] Kang Chen, et al. Fengwu: Pushing the skillful global medium-range weather forecast beyond 10 days lead. arXiv preprint arXiv:2304.02948, 2023.

[Ref.2] Tao Han, et al. Fengwu-ghr: Learning the kilometer-scale medium-range global weather forecasting. arXiv preprint arXiv:2402.00059, 2024b.

[Ref.3] Wanghan Xu, et al. Extremecast: Boosting extreme value prediction for global weather forecast. arXiv preprint arXiv:2402.01295, 2024.

[Ref.4] Tao Han, et al. Weather-5k: A large-scale global station weather dataset towards comprehensive time-series forecasting benchmark. arXiv preprint arXiv:2406.14399, 2024a.

[Ref.5] Junchao Gong, et al. Weatherformer: Empowering global numerical weather forecasting with space-time transformer, 2024. arXiv preprint arXiv:2409.16321, 2024.

[Ref.6] Xingjian Shi, et al. Convolutional lstm network: A machine learning approach for precipitation nowcasting. Advances in Neural Information Processing Systems, 28, 2015.

[Ref.7] Yunbo Wang, et al. Predrnn: A recurrent neural network for spatiotemporal predictive learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(2):2208–2225, 2022.

[Ref.8] Zhangyang Gao, et al. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3170–3180, 2022a.

[Ref.9] Cheng Tan, et al. Temporal attention unit: Towards efficient spatiotemporal predictive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18770–18782, 2023.

[Ref.10] Zhihan Gao, et al. Earthformer: Exploring space-time transformers for earth system forecasting. Advances in Neural Information Processing Systems, 35:25390–25403, 2022b.

[Ref.11] Suman Ravuri, et al. Skilful precipitation nowcasting using deep generative models of radar. Nature, 597(7878):672–677, 2021.

[Ref.12] Zhihan Gao, et al. Prediff: Precipitation nowcasting with latent diffusion models. Advances in Neural Information Processing Systems, 36, 2024.

[Ref.13] Zewei Zhao, et al. Advancing realistic precipitation nowcasting with a spatiotemporal transformer-based denoising diffusion model. IEEE Transactions on Geoscience and Remote Sensing, 2024.

[Ref.14] Demin Yu, et al. Diffcast: A unified framework via residual diffusion for precipitation nowcasting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 27758–27767, 2024.

[Ref.15] Junchao Gong, et al. Cascast: Skillful high-resolution precipitation nowcasting via cascaded modelling. In Forty-first International Conference on Machine Learning.

W3 & Q3: Comparison with GAN Methods:

A3: Many thanks for your valuable suggestion. To further validate PostCast's effectiveness, we include comparisons with GAN-based methods in precipitation nowcasting including DGMR[Ref.1] and STRPM[Ref.2]. The results are added to Table 1 and Table 3 in our revised paper. We also select the comparisons of our PostCast with diffusion-based methods and GAN-based methods in the table below:

SEVIR

HKO7

TAASRAD19

Shanghai

SRAD2018

SCWDS CAP30

SCWDS CR

MeteoNet

As shown in these tables, our PostCast outperforms all GAN-based models in both in-domain and out-of-domain settings on extreme precipitation nowcasting evaluated by P1, P4, P16, and precipitation with different intensities by CSI, HSS and POD scores. These comprehensive evaluations could further validate the effectiveness of our PostCast.

[Ref.1] Ravuri, Suman, et al. "Skilful precipitation nowcasting using deep generative models of radar." Nature 597.7878 (2021): 672-677.

[Ref.2] Chang, Zheng, et al. "Strpm: A spatiotemporal residual predictive model for high-resolution video prediction." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.

Thank you to the authors for addressing my concerns regarding W1 and W3. I truly appreciate the detailed experiments provided. I have adjusted my scores accordingly. However, the issue with W2 has not been adequately resolved. If the authors could address this, I would be willing to further increase my score.

I would like to clarify that my main concern of W2 is that the related work section's discussion on PRECIPITATION NOWCASTING remains overly brief.

For example, the discussion on Transformer-based methods is limited to EarthFormer, with no further expansion. Key works such as TCTN [Ref.20] and ConvTransformer [Ref.21], which are closely related to ConvLSTM, should be cited to provide a more comprehensive overview.

In contrast, the discussion on GAN-based methods requires more depth, including an analysis of their strengths and limitations. Works like Two-stage UA-GAN [Ref.16], CLGAN [Refs.17,19], and Luo et al. [Ref.18] should be discussed in greater detail to highlight their relevance and provide critical insights into how they address precipitation nowcasting challenges.

[Ref.16] Liujia Xu, Dan Niu, Tianbao Zhang, Pengju Chen, Xunlai Chen, Yinghao Li. Two-stage UA-GAN for precipitation nowcasting. Remote Sensing, 14(23):5948, 2022.

[Ref.17] Yan Ji, Bing Gong, Michael Langguth, Amirpasha Mozaffari, Xiefei Zhi. CLGAN: a generative adversarial network (GAN)-based video prediction model for precipitation nowcasting. Geoscientific Model Development, 16(10):2737–2752, 2023.

[Ref.18] Chuyao Luo, Xutao Li, Yunming Ye, Shanshan Feng, Michael K. Ng. Experimental study on generative adversarial network for precipitation nowcasting. IEEE Transactions on Geoscience and Remote Sensing, 60:1–20, 2022.

[Ref.19] Yan Ji, Bing Gong, Michael Langguth, Amirpasha Mozaffari, Xiefei Zhi. CLGAN: A GAN-based video prediction model for precipitation nowcasting. EGUsphere, 2022:1–23, 2022.

[Ref.20] Ziao Yang, Xiangrui Yang, Qifeng Lin. TCTN: A 3D-temporal convolutional transformer network for spatiotemporal predictive learning. arXiv preprint arXiv:2112.01085, 2021.

[Ref.21] Zhouyong Liu, Shun Luo, Wubin Li, Jingben Lu, Yufan Wu, Shilei Sun, Chunguo Li, Luxi Yang. ConvTransformer: A convolutional transformer network for video frame synthesis. arXiv preprint arXiv:2011.10185, 2020.

Dear reviewer iPfR,

We sincerely thank you for acknowledging our first revision addressed the concerns regarding W1 and W3. We also would like to express our heartfelt appreciation to the reviewer for raising the score. Indeed, as you mentioned, the discussions about Transformer-based methods and GAN-based methods are still insufficient due to the page limitation. We appreciate your valuable suggestions for adding these insightful methods, including ConvTransformer [Ref.10], TCTN [Ref.11], CLGAN [Refs.14,15], UA-GAN [Ref.20], and the method proposed by Luo et al. [Ref.16], into our related work. These methods could definitely enrich the related work section and the background of our manuscript. Therefore, in response to your main concern of W2, we have further expanded our related work on PRECIPITATION NOWCASTING, which could be directly found as follows. Moreover, the revised related work will also be added to the future version of our paper:

Notable progress has been achieved by applying deep learning in precipitation nowcasting [Refs.1-5]. The initial attempts are deterministic methods focusing on spatiotemporal modeling. Researchers explore different spatiotemporal modeling structures such as RNN [Refs.6,7], CNN [Refs.8,9], and Transformer [Refs.10,11,12]. Specifically, ConvTransformer [Refs.10] integrates convolution layers into the Transformer for better spatiotemporal modeling. TCTN [Refs.11] utilizes 3D-temporal convolutions to improve short-term dependencies in Transformer-based spatiotemporal learning. EarthFormer [Refs.12] is proposed to capture spatial-temporal features in earth system evolution by cuboid attention. However, these methods have a shortage of blurry predictions, which hamper the nowcasting of extreme events. Probabilistic methods are proposed to alleviate blurriness [Refs.13,14,15,16,17,18]. DGMR [Refs.13] applies GAN to produce realistic and spatio-temporally consistent predictions to reduce blurriness. CLGAN [Refs.14, 15] introduces an adversarial network with a novel long short-term memory to improve the nowcasting skills of heavy precipitation events. Luo et al. [Ref.16] conducted a comprehensive experimental study on kinds of GAN models in precipitation nowcasting. These GAN-based methods significantly increase the similarity between the predictions and ground truth, but the unstable training, and inaccuracy of position and shape [Refs.19] hinder the further improvement on short-term forecasting. To further enhance precipitation nowcasting with accurate global movements, later methods combine blurry predictions with probabilistic models. Two-stage UA-GAN[Refs.20], DiffCast [Refs.19], and CasCast [Refs.21] exploit how to generate small-scale weather pattern conditioning on the blurry predictions. Although these deterministic-probabilistic coupling methods achieve both global accuracy and local details, they suffer from repeating training for different contexts and require blurry predictions as data for training. From the perspective of deblur, Our method is proposed to simplify the complex training process and enhance the generality for wide use in different contexts.

Thank you for the thoughtful revision and for addressing my comments regarding W2. I have raised my score accordingly.

Based on my understanding, these GAN-based methods, such as DGMR, CLGAN, and Two-stage UA-GAN, are also addressing the challenge of improving nowcasting clarity. In the related work section, it would be helpful to discuss how your method compares to GAN-based approaches under the context of nowcasting clarity, specifically highlighting the advantages of your approach.

Additionally, I suggest that the Introduction briefly mention this comparison as well, to provide a clearer context for your contributions right from the beginning. I hope the authors can consider these points in future revisions to further strengthen the manuscript.

We appreciate the reviewer’s constructive and detailed feedback. We try to address the concerns and questions below.

W1: Comparisons with other generative models for postprocessing tasks.

A1: We thank you for the insightful comment. We agree that comparing our PostCast with other generative models such as GAN is crucial. Therefore, following your advice, we have compared our PostCast with a GAN-based postprocessing method (DGP[Ref.1]) and a diffusion-based postprocessing method (GDP[Ref.2]) in our revised manuscript. The results are presented in Table 1 and Table 3 in our revised paper. We also show a part of the results in the tables below for convenience:

SEVIR

HKO7

TAASRAD19

Shanghai

SRAD2018

SCWDS CAP30

SCWDS CR

MeteoNet

As shown in these comprehensive comparisons, our PostCast has superior performance. Specifically, the predictions are generated by EarthFormer for these three postprocessing methods. Our PostCast significantly surpasses the GDP and GDP in the meteorological metrics CSI-P1, CSI-P4, CSI-P16, CSI-avg, HSS-avg, and POD-avg, which explains the effectiveness of specifical designs in our PostCast for the postprocessing in precipitation nowcasting.

[Ref.1] Xingang Pan, Xiaohang Zhan, Bo Dai, Dahua Lin, Chen Change Loy, and Ping Luo. Exploiting deep generative prior for versatile image restoration and manipulation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(11):7474–7489, 2021.

[Ref.2] Ben Fei, Zhaoyang Lyu, Liang Pan, Junzhe Zhang, Weidong Yang, Tianyue Luo, Bo Zhang, and Bo Dai. Generative diffusion prior for unified image restoration and enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9935–9946, 2023.

W2: Comparisons with CasCast and DiffCast on in-domain datasets.

A2: Thanks for your valuable suggestion. To fulfill the comprehensive evaluation of our PostCast, we have added comparisons with CasCast and DiffCast in Table 1 (in-domain datasets) in our revised paper. A part of the results is also presented below:

SEVIR

HKO7

TAASRAD19

Shanghai

SRAD2018

The predictions to be post-processed are generated by EarthFormer. When the test datasets are included in the training domain, our PostCast still has significant advantages over the CasCast and DiffCast, demonstrating the strong generalization capability of PostCast across different datasets.

W3: The underlying reasons for CSI improvement.

SEVIR-POINT-DISTRIBUTION

| model | R $0,16) | R\[16,74) | R\[74,133) | R\[133,160) | R\[160,181) | R\[181,219)| R\[219, 255$ | |-------|-------|-------|-------|-------|-------|-------|-------| | GroundTruth | 78.43% | 11.03% | 8.03% | 1.64% | 0.47% | 0.33% | 0.07%| | EathFormer | 74.71% | 16.17% | 7.62% | 1.22% | 0.20% | 0.08% | 0.00% | +ours | 74.58% | 14.98% | 7.77% | 1.90% | 0.49% | 0.24% | 0.05% |

SEVIR-CSI

Thank you for raising this insightful point. First, we chose the CSI score to evaluate our method because it is the most widely referenced by meteorological bureaus and precipitation nowcasting papers [Ref.1,Ref.2]. Additionally, to give a more sufficient evaluation, we have also evaluated all methods in terms of CSI-avg, HSS-avg, and POD-avg in Table 1, Table 3, and Table 5 in our revised paper. All these metrics are improved after applying our PostCast, indicating the increased quality of precipitation prediction. Then, to mine the underlying reasons for CSI improvement, we have analyzed the distribution of points. The table named SEVIR-POINT-DISTRIBUTION shows the proportion of points in SEVIR distributed across the ranges $0, 16), \[16, 74), \[74, 133), \[133, 160), \[160, 181), \[181, 219), \[219, 255$ . Our post-processing method improves the alignment of EarthFormer's predictions with the actual distribution, particularly in high-threshold intervals such as $160, 181), \[181, 219), and \[219, 255$ . Correspondingly, the CSI values in these intervals have shown significant improvements (CSI-160, CSI-181, CSI-219) as shown in the table named SEVIR-CSI. These results demonstrated that the CSI improvement comes from the better alignments with the distributions of ground truth and our predictions, rather than the increased intensity.

[Ref.1] Zhang, Yuchen, et al. "Skilful nowcasting of extreme precipitation with NowcastNet." Nature 619.7970 (2023): 526-532.

[Ref.2] Ravuri, Suman, et al. "Skilful precipitation nowcasting using deep generative models of radar." Nature 597.7878 (2021): 672-677.

Dear reviewer SepJ,

Thank you for the valuable time and feedback you have invested in this review. We sincerely hope our further clarification could address your concerns:

W1: Comparison with other generative models

Since our method is the first postprocessing method for recovering local details and aligning distribution in this domain as the Reviewer iPfR mentioned in Strengths 2 "the first of its kind in the domain of precipitation nowcasting", we are only able to compare with most widely used GAN-based and Diffusion-based deblurring methods from Computer Vision. The results demonstrated the specific design of our method for tackling blurry predictions in precipitation nowcasting, since traditional CV methods cannot be directly applied in precipitation nowcasting.

W2: In-domain dataset comparison

Thank you for recognizing our additional results for evaluation on in-domain datasets.

W3: Further experiments

Thanks for your insightful suggestion. We would like to re-clarify the motivation that our method aims to provide an independent framework of the spatiotemporal prediction model and can be added on top of any prediction model with a unified model. Our PostCast aligns the blurry predictions better with the distribution of ground truth in a generalizable way. This improvement in precipitation nowcasting quality could also be reflected in the newly added HSS, POD, SSIM and PSNR. Besides increasing the intensities, our PostCast also refines the quality of details and shapes which could be referred to in Fig.14-17 in Appendix.10. Moreover, we also have evaluated FID[Ref.1] to further demonstrate quality improvement:

Additionally, as observed in Fig.11-13 in Appendix.10, both CasCast(ICML'24) and DiffCast(CVPR'24) have the same phenomenon of increased intensity, while our PostCast demonstrates stronger generalizations for improving the quality of precipitation nowcasting, which are evaluated by CSI and other newly added metrics. To prove this, we have conducted additional experiments on CasCast and DiffCast, shown as follows:

SEVIR-POINT-DISTRIBUTION

| model | R $0,16) | R\[16,74) | R\[74,133) | R\[133,160) | R\[160,181) | R\[181,219)| R\[219, 255$ | |-------|-------|-------|-------|-------|-------|-------|-------| | GroundTruth | 78.43% | 11.03% | 8.03% | 1.64% | 0.47% | 0.33% | 0.07% | | EathFormer | 74.71% | 16.17% | 7.62% | 1.22% | 0.20% | 0.08% | 0.00% | | +CasCast | 81.31% | 8.82% | 6.69% | 1.67% | 0.61% | 0.49% | 0.40% | | +DiffCast | 77.19% | 12.54% | 7.93% | 1.59% | 0.44% | 0.26% | 0.06% | | +ours | 74.58% | 14.98% | 7.77% | 1.90% | 0.49% | 0.24% | 0.05% |

COMPARISON on SEVIR

Specifically, CasCast increases the most intensities in high thresholds among these three methods, and DiffCast best matches the distribution of intensity with ground truth. However, our PostCast significantly surpasses DiffCast and CasCast in CSI, HSS and POD, indicating the improvement of quality not only comes from increasing the intensity. We will carefully discuss this common phenomenon in Discussion section.

Thank you once again for your helpful constructions. We would greatly appreciate it if you could consider improving the evaluation after reviewing our responses. Thank you very much for your consideration!

Best regards,

Sincerely yours,

Authors.

[Ref.1 Heusel, Martin, et al. "Gans trained by a two time-scale update rule converge to a local nash equilibrium." Advances in neural information processing systems 30 (2017).]