OmniCast: A Masked Latent Diffusion Model for Weather Forecasting Across Time Scales

We thank the reviewer for the constructive feedback and for recognizing the design, rigorous evaluation, and strong performance of SeasonCast. We hope to address the reviewer’s main questions and concerns below.

Looking at Figure 4, the model often performs on par with—or even worse than—climatology across most variables and metrics, with the exception of absolute bias for Z500 and T850.

Theoretical limits in deterministic forecasting state that no model can surpass climatology beyond 14 days. However, S2S prediction is not purely about deterministic skill; robust probabilistic uncertainty estimation is crucial.

• Climatology inherently lacks uncertainty estimates, rendering it impractical for real-world forecasting applications where uncertainty is important for informed decision-making.
• SeasonCast demonstrably improves upon climatology by providing reliable probabilistic forecasts, which are far more valuable for decision-making in risk-sensitive applications.

Across all benchmarks and metrics, SeasonCast consistently produces stable, physically consistent, and well-calibrated forecasts for S2S. While achieving comparable RMSE due to theoretical limits, SeasonCast offers significant benefits over climatology through its superior probabilistic forecasting capabilities.

Additionally, the paper only presents SeasonCast’s performance on a subset of atmospheric variables, leaving out key surface-level indicators like 2-meter temperature and 10-meter wind components (U and V).

We evaluated SeasonCast on a total of 69 different weather variables, but were not able to show all of them due to the page limits. Please see below the performance of SeasonCast across different key surface-level variables and various metrics. We will also update the Appendix to include this result.

I recommend that the authors provide a brief explanation or context when referring to initial conditions and boundary conditions

We fully agree with the reviewer and will update the paper to discuss the initial conditions and boundary conditions problems in the future version of the paper.

We thank the reviewer for the constructive feedback and for recognizing the contributions, novel architecture design, presentation, and strong performance of SeasonCast. We hope to address the reviewer’s main questions and concerns below.

While the paper includes extensive quantitative benchmarks, it would benefit from more qualitative analyses or illustrative case studies.

We agree with the reviewer that additional illustrative studies would enhance our understanding of SeasonCast's performance. As the rebuttal phase prohibits uploading PDF files or linking to external pages, we will incorporate VAE reconstructions, sample forecasts, and extreme weather events into a future version of the paper.

Although the paper uses several meaningful metrics, it does not report widely adopted indicators such as the Anomaly Correlation Coefficient (ACC) or Forecast Activity (ACT), which are standard in meteorological skill assessments.

We further evaluated SeasonCast’s performance on the Anomaly Correlation Coefficient (ACC) metric, which we summarized in the table below. We left out Z500 because there was a bug in the climatology data provided by ChaosBench. We have contacted the authors to have this fixed and will report the results on more variables in the updated version of the paper.

This result demonstrates that SeasonCast performs competitively with ECMWF-ENS at S2S timescales in terms of the ACC metric and also maintains stable performance even up to day 40, reinforcing the conclusion observed across other metrics presented in our paper.

We thank the reviewer for the constructive feedback and for appreciating the design of SeasonCast and its strong performance in S2S prediction. We hope to address the reviewer’s questions and concerns below.

Aggressive compression (100×) may lose fine-scale features critical for regional forecasts (e.g., extreme events). No analysis of reconstruction error per variable.

While we significantly compressed the spatial dimensions of the data to reduce the training sequence length for the transformer model, we actually increased the latent channel dimension. Specifically, each raw frame has a shape of 69x128x256, where 69 represents the number of weather variables. Each latent frame has a shape of 1024x8x16, resulting in an effective compression ratio of (69x128x256) / (1024x8x16) = 17.25. Please refer to the tables below for the RMSE of the reconstruction across some key variables.

Performance and efficiency actually depend a lot on the VAE’s reconstruction performance, compression ratio, and network structure. VAE has no statement in this regard, especially in experiments.

We fully agree with the reviewer's observation that the VAE's performance is important to the overall efficacy of SeasonCast. Indeed, we dedicated substantial effort to developing a robust VAE model. Our primary goal was to achieve a high compression ratio along the spatial dimensions, as this directly reduces the number of training tokens required for the subsequent transformer model. Therefore, we employed 16x spatial reduction across all experiments. We then incrementally increased the latent dimension until we obtained an acceptable reconstruction error. Please refer to the table below for a detailed breakdown of performance across various latent dimensions.

Before selecting the specific VAE model presented in our paper, we also tried two alternative VAE architectures:

• VQ-VAE: This approach compresses data into a discrete latent space. However, we were unable to achieve satisfactory reconstruction errors; they were consistently 2 to 3 times higher than those obtained with a continuous VAE using the same spatial downsampling factor.
• Video VAE: Similar to Cosmos [1], this variant compresses data across both spatial and temporal dimensions. Our early experiments showed that for an equivalent effective compression ratio, a per-frame VAE consistently outperformed a video VAE. This led us to opt for the per-frame VAE model.

We thank the reviewer again for bringing this up and we will update the paper to discuss the VAE component in detail.

[1] Agarwal, Niket, et al. "Cosmos world foundation model platform for physical ai." arXiv preprint arXiv:2501.03575 (2025).

Trained on fixed 44-day sequences; performance degrades beyond this window (Appendix B.3). Unable to dynamically extend forecasts without retraining.

There appears to be a misunderstanding here. Figure 11b demonstrates that SeasonCast models trained on short windows exhibit performance degradation beyond their training window, whereas the SeasonCast model trained on the full 44-day window maintains consistent stability across all lead times. This finding is consistent with previous research indicating that autoregressive models typically struggle beyond the medium range. This also supports our assertion that training on the complete sequence of future frames is crucial for stabilizing the model in S2S prediction.

To further substantiate SeasonCast's ability to produce stable and well-calibrated predictions beyond its training window, we extended the model's rollout to 100 days ahead. To achieve predictions beyond the initial training window, we iteratively employed the latest predicted frame (day 44) as the initial condition for predicting the subsequent window (day 45 to day 88), continuing this process until the desired lead time was reached. The table below presents SeasonCast's performance across key variables and various metrics at lead times exceeding 44 days. SeasonCast remains exceptionally stable and well-calibrated, showing no signs of degradation even at day 100.

The scope of the comparative papers is too small. There are many excellent works now, but the authors did not compare.

Could the reviewer please elaborate on the specific baselines they have in mind? We have made every effort to compare SeasonCast with state-of-the-art methods, including GC, PW, leading numerical models for the S2S setting, and GenCast and IFS-ENS for the medium-range settings, and SeasonCast consistently demonstrates strong performance across benchmarks. We further compared it with Fuxi-S2S, another prominent method in S2S prediction. Please refer to our response to Reviewer mH3R for details on this experiment.

How does SeasonCast enforce physical laws (e.g., mass conservation) in generated forecasts? Are there constraints in the loss function or latent space?

SeasonCast employs a purely learning-based approach, without imposing any physical constraints in the loss function for either the VAE or the transformer model. Incorporating physical laws is an orthogonal and promising direction that we aim to explore in future work.

Can SeasonCast predict rare events (e.g., heatwaves)? Please provide case studies on ChaosBench extreme indices.

ChaosBench does not provide extreme case studies or indices, but instead provides a comprehensive evaluation framework across a wide range of metrics. SeasonCast consistently demonstrated strong performance within this benchmark, excelling across key variables, diverse metrics, and various lead times. Furthermore, its robust performance in the medium-range setting further highlights SeasonCast's versatile capabilities.

Dear Reviewer SUwF,

We are writing to follow up on our rebuttal, in which we have done our best to address your concerns with additional experiments and detailed clarifications. As the discussion period is coming to an end soon, we sincerely hope you have a moment to review our response. We are eager to address any remaining questions you might have and would be happy to discuss further. Thank you again for your valuable feedback.

Best regards,

The Authors

We thank the reviewer for the constructive feedback, and for recognizing the significant performance and efficiency gains of SeasonCast. We hope to address the reviewer’s remaining concerns and questions below.

The manuscript is verbose in parts and under-explained in others. Background and methodology blur together; the structure lacks discipline.

Since our methodology is built on top of existing techniques, i.e., masked generative modeling and diffusion models, we presented the background of these techniques in the Background section, and how we adapted them to SeasonCast in the Methodology section. Can the reviewer point out which specific part of the manuscript is verbose and which is under-explained? We’re happy to adjust the writing to make the paper clearer.

Notations appear before being defined (e.g., pUpU above Eq. (4)).

$p_{\mathcal{U}}$ is the distribution of the binary mask, which we explain in more detail in Section 4.3 (lines 221-222). We will edit the paper to explain this clearly.

Figure 1 omits key components and does not reflect the sampling pipeline; details are scattered rather than integrated.

We believe Figure 1 includes all key components of SeasonCast: the transformer backbone, the masking mechanism, the diffusion head, and the auxiliary deterministic head. We will further include an inference diagram in the next version of the manuscript.

The core method, VAE + masked generative transformer, is standard in adjacent fields.

While our work adapts known techniques from other fields, we firmly believe that a well-executed adaptation can yield significant impact when applied to a new domain such as weather and climate. We dedicated substantial effort to the design of SeasonCast to achieve state-of-the-art performance, specifically through the full-sequence training and the integration of the proposed auxiliary loss in the masked generative model training. Furthermore, the most impactful works in this field are themselves adaptation-focused, including MetNet (which adapted a Video GAN model), GraphCast (which adapted GNN), and GenCast (which adapted GNN + a diffusion model).

The loss function blends diffusion loss with exponentially-weighted MSE for the mean prediction. This coupling feels ad hoc and risks undermining the benefits of probabilistic output that diffusion model provides.

We respectfully disagree with the reviewer's assessment. The deterministic and diffusion components operate as separate prediction heads atop the transformer backbone, thus preventing direct interference. Instead, the deterministic loss actively aids the training of the diffusion head. This is because the deterministic loss provides an easy-to-optimize signal during training, enabling the model to learn meaningful $z_i$ that the diffusion head can effectively utilize for the probabilistic loss. Our ablation experiment in Figure 11a empirically demonstrates the positive impact of incorporating the deterministic loss across various metrics. Furthermore, our main experiments consistently show that SeasonCast remains well-calibrated across diverse scenarios, which further supports our argument.

The evaluation section reads more like a figure dump than an analysis. No statistical testing, no confidence intervals, no analysis of forecast reliability.

Due to strict page limits, we focused our main comparisons on the most crucial baselines in the experiment section, which effectively highlighted SeasonCast's advantages. For a more comprehensive understanding of SeasonCast's design, we included detailed ablation studies in Appendix B.3. These studies thoroughly investigate the impact of various design choices, such as the deterministic loss, training sequence length, unmasking strategies, and diffusion sampling temperature.

Regarding statistical testing and confidence intervals, these are not standard practices in the field. Training weather models on ERA5 is a computationally intensive process, especially given academic compute constraints. Leading papers like GraphCast, PanguWeather, Stormer, and GenCast have also omitted statistical testing and confidence intervals, making it unfair to penalize our paper on this basis.

The source of SeasonCast’s advantage over GraphCast or PanguWeather is left unexplored. If autoregressive error is to blame, it should be demonstrated explicitly in a controlled setting.

Figure 4 clearly illustrates SeasonCast's superior performance beyond 10-day lead times, where GC and PW degrade due to autoregressive error accumulation. SeasonCast, however, maintains stability up to day 44. Further evidence from Figure 11b demonstrates that training SeasonCast with shorter future sequence lengths leads to increased error accumulation and poorer long-term performance. These experiments definitively show that SeasonCast's advantage in the S2S setting stems from its non-autoregressive nature and its ability to learn from extended sequences of future weather states, unlike other methods' one-step prediction approaches.

How do you ensure the auxiliary MSE term does not bias the diffusion posterior, causing the ensemble mean to drift from the true predictive mean?

As we discussed above, the experiments in Figure 11a showed that the deterministic loss helps improve the performance of SeasonCast significantly across different metrics and lead times.

Do you need to adjust the relative weights between the two loss terms, if so, how? by calibration, ablation, or trial-and-error?

We simply used equal weighting between the two terms without any tuning because we found that to work sufficiently well.

Could you average SeasonCast outputs and evaluate against FuXi-S2S to give a rough comparison?

We compared the daily forecasts from SeasonCast with Fuxi-S2S. As shown in the table below, SeasonCast demonstrates strong performance for 2-meter temperature and geopotential at 500hPa across four different lead times. Notably, Figure 8 in the Fuxi-S2S paper (https://arxiv.org/pdf/2312.09926) illustrates that the RMSE of Fuxi-S2S exceeds 2.45 for T2M and 800 for Z500 at lead times beyond 15 days. While their SSR scores are comparable, these results clearly indicate that SeasonCast either matches or surpasses Fuxi-S2S in both RMSE and SSR metrics. This further underscores the significant capabilities of SeasonCast, as it outperforms Fuxi-S2S even without being specifically trained for daily average prediction, unlike Fuxi-S2S.

For GenCast and NeuralGCM, could you run comparisons on scaled-down domains or resolutions to offer partial but informative benchmarks?

As Figure 8 illustrates, both GenCast and NeuralGCM demand substantial computational resources. Even at a reduced resolution, generating a single 15-day forecast consumes several minutes. To evaluate these models on ChaosBench, we would need to run them for over a thousand initial conditions in the test set, producing 50 ensemble predictions for each, with a 44-day lead time. This extensive experiment is simply not feasible within the limited rebuttal period and the immense computational power it would necessitate.

We sincerely thank you for the responsive feedback. We are glad our previous response was clarifying, and we will make sure to move the ablation study on the autoregressive regime into the main paper to improve visibility, as you recommended.

Regarding your follow-up questions on Figure 11, we believe we can offer a clear explanation.

• On the similarity of Autoreg and Random curves in Figure 11(c): The key is to distinguish the experiment in 11(c) from the one in 11(b). The experiment in Figure 11(c) analyzes the impact of different unmasking strategies during inference on the same trained model, specifically, the 44f-24hr model that was trained to forecast the full sequence. Because all three sampling strategies in 11(c) use the same full-sequence model, their overall performance is similarly strong. However, the Random curve is consistently better because using a different random unmasking order for each ensemble member introduces more forecast diversity, which improves probabilistic scores like CRPS and SSR. We discussed this point in Lines 584-591 of the Appendix.
• On the SSR plot in Figure 11 vs. Figure 6: Your observation of the discrepancy is correct, and it is intentional. The difference is due to the sampling temperature hyperparameter $\rho$. For all ablation studies in Figure 11, we used a default temperature of $\rho=1.0$ for a controlled analysis. For the main experiment in Figure 6, we used a temperature of $\rho=1.3$, which was tuned to optimize the overall performance on validation data.
• This explains why the SSR in Figure 6 is higher (i.e., has more spread) than the SSR for the Random curve in Figure 11(c). This also provides a consistency check, as the MSE-10 curve in Figure 11(a), 44f-24hr in 11(b), Random in 11(c), and $\rho=1.0$ in 11(d) are all identical, as they represent the same model with the same default inference settings.

We hope this explanation fully clarifies these details. Thank you again for your diligence and for helping us strengthen the paper.