DAWP: A framework for global observation forecasting via Data Assimilation and Weather Prediction in satellite observation space

We sincerely thank you for your insightful and encouraging feedback. We greatly appreciate your recognition of the novelty and significance of our proposed DAWP framework, particularly the integration of the AIDA and AIWP modules to enable forecasting directly in satellite observation space. We are particularly encouraged by your assessment that our contribution is not merely incremental but genuinely innovative, as this reinforces the impact of our work.

---

Response to Q1: Incorporation with Physical Constraints

We sincerely appreciate your valuable suggestion and fully agree that incorporating physical information into the system could enhance performance—particularly in data-limited scenarios—by helping mitigate overfitting.

However, in our current experimental framework, DAWP is trained exclusively on satellite-observed variables—such as microwave radiance, brightness temperature, and infrared radiance—rather than on derived physical quantities like temperature or wind. Because no well‑established physical forward model adequately describes the dynamics of these observed variables, this limitation underscores the need for purely data‑driven approaches in satellite‑observation forecasting.

Moving forward, we see promising opportunities to introduce physical insights gradually. For example, we can incorporate directly measured physical quantities (e.g. wind, atmospheric temperature profiles) into our pipeline and apply Physics-Informed Neural Networks (PINNs) or similar techniques to learn physical relationships, linking these profiles with satellite-observed radiance.

We will make these future directions explicit in the revised manuscript to clarify both the current limitations and the potential for integrating physical priors into DAWP.

Response to Q2: Trustworthiness and Explainability

Thank you for your valuable feedback regarding the credibility and interpretability of data-driven forecasting methods.

While interpretability and trustworthiness remain fundamental challenges for data-driven approaches, DAWP provides an additional perspective on model trustworthiness compared to existing methods such as [1–3], where the completion of sparse observations cannot be supervised. Specifically, during the AIDA initialization stage, DAWP evaluates input completion quality by reconstruction loss over the masked regions, thus enabling a supplementary mechanism for evaluating the credibility of model predictions.

We have also conducted an initial investigation into explainability of forecasting results. In our Modality Sensitivity Analysis, we examine how different input modalities influence the model’s outputs. We observe that using only ATMS data retains most of the model’s predictive power across other modalities, indicating that Brightness Temperature inherently contains substantial information from Microwave and Infrared radiance.

These experiments demonstrate that Brightness Temperature carries rich cross-modal signals, supporting the model’s robustness even when trained with limited modalities.

We believe that interpretability and credibility in direct-observation forecasting remain rich areas for further exploration.

---

References:

[1] McNally, Anthony, et al. "Data driven weather forecasts trained and initialised directly from observations." arXiv preprint arXiv:2407.15586 (2024).

[2] Alexe, Mihai, et al. "GraphDOP: Towards skilful data-driven medium-range weather forecasts learnt and initialised directly from observations." arXiv preprint arXiv:2412.15687 (2024).

[3] Vandal, Thomas J., et al. "Global atmospheric data assimilation with multi-modal masked autoencoders." arXiv preprint arXiv:2407.11696 (2024).

We sincerely thank you for your thoughtful and encouraging feedback. We greatly appreciate your recognition of the importance of observation-based forecasting as a promising direction beyond reanalysis data, as well as your positive assessment of our AIDA module and cross-regional boundary conditioning (CBC) approach. Your comments on the robustness of our experimental design, especially the detailed ablation studies and cross-community comparisons, are highly motivating. We are particularly grateful for your acknowledgment of our efforts to address real-world challenges such as missing data and boundary artifacts through well-justified architectural choices.

OUR CORE CONTRIBUTION:

We identify a key challenge in observation-based forecasting (L55): the mismatch between sparse observations and dense prediction. Our proposed DAWP framework effectively mitigates the input–output mismatch by introducing a two-stage solution—AIDA for assimilation-based initialization, followed by an AIWP forecasting model specifically adapted to AIDA outputs. As shown in Figure 6, our results provide strong empirical evidence supporting this claim.

---

Response to Q1: Discussion about LT3P

Thank you for drawing our attention to LT3P as a relevant real‑time reference. We will include LT3P in our “Related Work” section. As you point out, LT3P leverages reanalysis data for pre‑training and combines NWP forecasts with historical typhoon trajectories to make real‑time predictions.

In contrast, we would like to clarify that our DAWP framework intentionally avoids reanalysis data and NWP systems, instead focusing exclusively on observation-driven methods. The primary contribution of DAWP is to demonstrate that the combined AIDA–AIWP framework offers an effective solution to the mismatch between observational input space and the forecasting objective.

We still believe that the approach adopted in LT3P—pretraining on ERA5 reanalysis data followed by fine-tuning on real-time data—also provides valuable inspiration for future extensions of our framework.

Response to Q2: Relationship of AIWP and Physical Solver

Thank you for bringing ClimODE to our attention and enabling a more focused discussion.

Network: ClimODE leverages the advection equation to design a prediction network trained on reanalysis data. While it does incorporate physical priors via a physics‑informed Neural ODE, its forecasting performance remains well below that of the numerical weather prediction system IFS (see Figure 4 of ClimODE). By contrast, methods such as Pangu‑Weather[1], GraphCast[2], and GenCast[3] are also trained on reanalysis datasets but do not incorporate any explicit physical solver. Despite the lack of physical constraints, they consistently outperform IFS across a wide range of performance metrics and lead times. This suggests that including a physics-based solver in network design is not strictly necessary for achieving superior forecast skill.

Data: Moreover, the use of reanalysis data itself implicitly introduces physical solver dependence, since such data are generated using NWP systems. In fact, purely observation‑driven models also show remarkable performance in some domains—for example, precipitation nowcasting[4]. Thus, from both network design and data selection perspectives, AIWP models operate independently of physical solvers.

Nevertheless, we fully agree with you that complementary integration of physical priors could be beneficial especially in scenes without enough data. Such priors can be incorporated either by training with reanalysis data or by applying Physics‑Informed Neural Networks (PINNs) to observational physical variables. In future versions of our manuscript, we will expand on these points and provide clearer justification, thereby avoiding any potential overclaim.

Response to Q3: More References

Thank you very much for your kind suggestion regarding WeatherBench. Since the ERA5 reanalysis dataset serves as the foundation for WeatherBench, it is an excellent reference to raise the concern of reanalysis. Additionally, beyond the references already included namely EarthNet[5], Transformer-DOP[6], and GraphDOP[7]—we will also incorporate references of OMG-HD[8] and LT3P, which further support the concerns raised.

Response to Q4: VideoMAE

Thank you for suggesting VideoMAE—this enables a more precise discussion.

VideoMAE’s tube masking strategy creates spatiotemporal tokens by masking the same spatial location across multiple frames to exploit temporal redundancy. However, due to the polar-orbiting satellite motion characteristic, consecutive frames often lack overlapping observations at the same location. If we employed VideoMAE’s tube masking, the resulting tokens would contain empty values, forcing the model to learn from missing data—an undesirable artifact in our context. Consequently, we did not adopt this type of MAE design that relies on temporal redundancy between neighboring frames.

Response to Q5: Evaluating POD

Thank you for your suggestion. We have evaluated the POD metric, and the conclusions are consistent with those presented in Table 3—our proposed DAWP framework significantly outperforms the other methods.

Response to W1: Motivations and In-depth Analysis about Challenges

We would like to re-clarify OUR CORE CONTRIBUTION, as presented at the beginning of the rebuttal.

We agree with you that the limitations of reanalysis data are a consensus within the community. As a result, Direct Observation-based Prediction have recently gained significant attention, given their potential to overcome those limitations.

Our main contribution lies in first identifying the mismatch between sparse inputs and dense predictions as a key challenge within DOPs. To mitigate such mismatch, we propose a novel AIDA–AIWP framework. Our experimental results provide strong support for the effectiveness of this framework in resolving the mismatch issue as shown in figure 5, highlighting its value as a step forward in DOP research.

---

We sincerely appreciate your feedback, which has significantly improved the paper’s clarity and completeness. Should further clarification be required, please do not hesitate to reach out, and we will address your concerns promptly. We look forward to your continued input.

References:

[1] Bi, Kaifeng, et al. "Accurate medium-range global weather forecasting with 3D neural networks." Nature 619.7970 (2023): 533-538.

[2] Lam, Remi, et al. "Learning skillful medium-range global weather forecasting." Science 382.6677 (2023): 1416-1421.

[3] Price, Ilan, et al. "Probabilistic weather forecasting with machine learning." Nature 637.8044 (2025): 84-90.

[4] Gao, Zhihan, et al. "Earthformer: Exploring space-time transformers for earth system forecasting." Advances in Neural Information Processing Systems 35 (2022): 25390-25403.

[5] Vandal, Thomas J., et al. "Global atmospheric data assimilation with multi-modal masked autoencoders." arXiv preprint arXiv:2407.11696 (2024).

[6] McNally, Anthony, et al. "Data driven weather forecasts trained and initialised directly from observations." arXiv preprint arXiv:2407.15586 (2024).

[7] Alexe, Mihai, et al. "GraphDOP: Towards skilful data-driven medium-range weather forecasts learnt and initialised directly from observations." arXiv preprint arXiv:2412.15687 (2024).

[8] Zhao, Pengcheng, et al. "OMG-HD: A high-resolution AI weather model for end-to-end forecasts from observations." arXiv preprint arXiv:2412.18239 (2024).

Dear Reviewer RPch,

Thank you once again for your thoughtful and constructive feedback.

In response to your comments, we first clarify our main contribution—identifying the mismatch between sparse observations and dense predictions in Direct Observation-based Prediction (DOP) and proposing a novel solution through the AIDA–AIWP framework. This two-stage approach effectively mitigates the mismatch issue and is supported by strong experimental evidence. We also evaluate POD metrics to better assess the precipitation forecasting performance of DOP methods. Additionally, we have included discussions on LT3P and ClimODE to provide a more accurate characterization of DOP methods. Furthermore, we explain why VideoMAE is not suitable for satellite observations, due to the unique challenges posed by polar-orbiting satellite motion. Finally, we express our willingness to explore the integration of physical priors in future work.

We are actively participating in the Author-Reviewer Discussion phase and would be happy to provide further clarification if needed! We are committed to addressing all reviewer concerns and ensuring timely responses. Thank you once again for your contributions to our work.

Best regards, The Authors of Submission 1085

We sincerely appreciate your thoughtful and constructive feedback. Thank you for recognizing the novelty and clarity of our proposed framework.

OUR CORE CONTRIBUTION:

We identify a key challenge in observation-based forecasting: the mismatch between sparse observations and dense prediction. DAWP mitigates this mismatch via a two-stage solution: AIDA for initialization and AIWP for forecasting. Figure 6 provides empirical evidence.

---

Response to Q1: Main claim of DAWP and Relationship between Reanalysis or Observation

Thank you for the thoughtful question. Our main claim is that DOPs trained in the space initialized by AIDA improve prediction, as stated in Figure 1, lines L54–56 and OUR CORE CONTRIBUTION. Table 2 and Figure 6 support this claim.

We agree that Reanalysis models benefit from human priors in NWP (physical laws, bias correction, assimilation). However, exploring data‑driven forecasting remains scientifically valuable and timely. DOP extends the traditional reanalysis-based forecasting paradigm into a heterogeneous, spatiotemporal observational space, encompassing multiple data modalities—an area that remains underexplored to date.

Response to Q2: Results without AIDA.

Thank you for raising this important point. Results without AIDA are shown in Figures 5 and 6.

Learning dynamics: When AIDA is not applied, learning dynamics are unstable for our spatial-temporal prediction model, producing severe spikes in learning phase (Fig.5) due to the irregular and sparse nature of input. AIDA stabilizes learning by mapping sparse inputs into a dense, structured latent space.

Roll-out: Without AIDA, roll-out degrades because sparse training inputs mismatch dense inference inputs in roll-out (Fig. 6).

Response to Q3: Window Length and Sub-image Size

A 144×144 sub-image with patch size 16 over 12 steps and 4 domains yields 3884 tokens. This length fits our transformer and computational budget, ensuring efficient attention.

The 12-step window matches the satellite revisit cycle, ensuring global spatial coverage.

Response to Q4: Experiments to Support ViT-VAE's Capability

Thank you for the suggestion. The comparisons are presented in Appendix C. When compared against SD‑VAE, our MaskViTVAE model achieves approximately 12 % relative improvement in reconstruction loss.

Response to Q5: GraphDOP

GraphDOP is an excellent project. With ECMWF's data infrastructure and preprocessing capabilities, it was trained on 32 observation types, broader than DAWP’s 4 types. Additionally, pecific implementation details are not clearly disclosed. Without access to the codes, the direct comparison is currently infeasible. We are willing to evaluate it using data available to us when GraphDOP is open-sourced.

Response to Q6: Reasons for Different Improvments and VAEs with Random Seeds

In Table 5, performance variation is due to channel count differences, not data scale. All datasets are normalized, so value ranges do not affect results. Gains follow channel counts (HIRS 20 > AMSU-A 15 > ATMS 9 > MHS 5). Our method excels on high-channel data, while SD-VAE is more suited to low-channel cases. Thus, DAWP provides stronger compression for high-dimensional satellite observations.

We also tested four seeds (0–3), retraining models and reporting mean ± std. The results show our method’s gains are robust to random seed choice.

Response to Q7: Near-realtime Availability

We use satellite Level‑1 products, available immediately after transmission; for instance, EUMETSAT[1] provides MHS with <60 min latency.

Response to W1: Main Idea and Baselines on Reanalysis

We thank the reviewers and clarify our main points. The core contribution of DAWP is the design of a satellite‑observation prediction framework—comprising AIDA and AIWP—to directly predict satellite observational quantities, rather than comparing with reanalysis-based forecasts.

Data: DAWP predicts satellite radiances (microwave, brightness temperature, infrared). In contrast, the reanalysis‑based baselines forecast physical variables like wind and temperature. As these predictions involve different modalities, direct comparison is infeasible.

Architecture: Reanalysis-based models lack designs to process sparse, irregular satellite inputs, hence not directly applicable for predicting satellite observables.

Response to W2&W3: Dataset Split and Results with Random Seeds

Thank you for your suggestions. Originally: Training: 2012.1 – 2022.6, Validation: 2022.7 – 2023.4, Testing: 2023.5 – 2023.7. All experiments strictly followed this training–validation–testing partitioning protocol. This 3-month test follows GraphDOP (which used only 1 month in Section 5.2) and EarthNet (which used 2 months in Section E.2). Following your suggestion, we revised to: Train 2012.1–2021.7, Val 2021.8–2022.7, Test 2022.8–2023.7.

We retrained all models on the revised split with seeds {0,1,2,3}; results (mean±std) confirm statistical significance. We thank you again and will include this extended evaluation.

Response to W4&W5: Baselines Implementations

Thank you for your constructive feedback. There is no open‑sourced code for EarthNet[2] or Transformer‑DOP[3]. For EarthNet, it follows the implementation of MultiMAE[4] as detailed in EarthNet’s Appendix C and D. Therefore, we reproduce it on our datasets following MultiMAE. As for Transformer‑DOP, since the original paper presents only a sketch without details, we implemented it according to our best available understanding. Specifically, EarthNet is reproduced as a 12‑layer encoder (hidden dimension 768) paired with an 8‑layer decoder (hidden dimension 512), and Transformer‑DOP is implemented as a transformer consisting of 18 layers (hidden dimension 1024). We employs sub‑images because the full 12‑hour global observation sequence would result in 124k-token sequence, which is computationally infeasible.

Artifacts (Fig. 4) arise from (1) mismatch between dense autoregressive inference and sparse training and (2) lack of cross‑region info in sub‑images. Our AIDA initialization and CBC module resolve these problems.

Response to W6: Computer resources

Thank you for your suggestion. In our experiments, we used 4 A100 80G GPUs for parallel training. In the table below, we present other details, which would be added in the future version.

Response to M1: 35TB Observations

Each observation is a 12×1152×2304 grid with spectral channels (ATMS‑9, AMSU‑A‑15, MHS‑5, HIRS‑20). With two observations per day over 11 years, the dataset totals ~35 TB.

Response to M2: Method Details

In the Methods section, we emphasize the conceptual pipeline combining AIDA and AIWP to address prediction based on irregular and sparse satellite observations. Model architecture details (layers, dimensions, attention, tokenization) are in Tables 9–11. We will expand methodological descriptions and design rationale in future work.

Response to M3: Uncommon Abbreviation

We would like to clarify that Artificial Intelligence Data Assimilation (AIDA) and Artificial Intelligence Weather Prediction (AIWP) are both expanded and carefully defined in L2 and L11, respectively. Transformer‑DOP denotes a Transformer backbone for the DOP task (Ref. 16). We are open to clearer alternative naming suggestions

Response to M4: Explanations about Transformer–DOP

Transformer‑DOP refers to Ref. 16. As the paper provides only a high-level description, we reproduced it based on its conceptual design.

Response to M5: Visualization

Thank you for the suggestion. Our aim is to emphasize the complementation capability of DAWP for handling irregular and sparse satellite observations, especially in comparison to other methods that suffer from artifacts arising from sub-image tiling and mismatched input-output density. In future versions, we will incorporate visualizations of prediction–truth differences.

---

Thank you for your thorough review, which improved our paper. We believe that some minor points can be better addressed during the discussion phase. We welcome any further questions and will respond promptly.

References:

[1] https://data.eumetsat.int/extended?query=MHS

[2] arXiv:2407.11696

[3] arXiv:2407.15586

[4] MultiMAE (ECCV'2023)

Dear Reviewer Ht96,

Thank you very much for your valuable comments and for taking the time to review our work.

To address your concerns comprehensively, we are more than happy to further clarify our evaluation methodology. In order to ensure that the test set covers a full year, we partitioned the data as follows: data from January 2012 to July 2021 was used for training, data from August 2021 to July 2022 for validation, and data from August 2022 to July 2023 for testing. The samples in our dataset are organized in 12-hour intervals. Thus, a lead time of 0–12h corresponds to a single-step prediction, 12–24h to a two-step prediction, and 24–36h to a three-step prediction. We report the mean absolute error (MAE) averaged over all observation sites for each of these lead time intervals.

While we cited the official reference document from the European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT) to accurately indicate the near-real-time availability of the MHS data we used, we appreciate your kind reminder and sincerely apologize for any potential concerns this may have caused. We confirm that this citation does not involve any identity leakage.

If you have any further questions or concerns, please feel free to raise them. We remain open and ready for further discussion.

Best regards, The Authors

We sincerely appreciate your constructive and detailed feedback. Thank you for your positive evaluation of our work’s novelty, methodological rigor, performance, and overall contribution. Your recognition that our paper addresses an interesting problem and proposes a practical approach is highly encouraging.

---

Response to Q1: More Types of Data

Thank you for your helpful suggestion. We are currently exploring the possibility of extending DAWP to incorporate point-cloud MAE pretraining[1] combined with multi-modal MAE (MMAE)[2] for integrating weather station data. However, due to constraints in data collection, preprocessing, and ttraining ime, we were unable to include any experimental results in our rebuttal. We will add these experiments in the future version of our DAWP and include discussions into the revised manuscript.

Response to W2&Q2: Comparisons with More Baselines

Thank you for your suggestion. Due to length constraints in the main manuscript, we have moved the comparisons with additional baselines to the Supplementary Material. There, we include quantitative results comparing DAWP against six classical spatio-temporal forecasting models including RNN‑based, CNN‑based and Transformer‑based ones. We will ensure that future versions of our paper prominently present this baseline comparison in the main text, to better contextualize the strengths of our DAWP framework.

Response to W1: Design Choice of Forecasting Model

Thank you for recognizing our key contributions of a data assimilation stage and a forecasting stage for direct observation–based prediction, as highlighted in the review strengths. Although the forecasting model is simple, it still effectively supports the point that data assimilation initialization can aid direct observation forecasting. Moreover, our forecasting model efficiently resolve conflicts of input areas between regional data assimilation and global forecasting through the cross-region boundary conditioning.

In addition, within the DAWP framework, the forecasting algorithm is flexible and has the potential to incorporate with more specialized forecasting methods in future work.

Response to S2: Highlight DA's Role

Thank you for your valuable suggestion. The general role of Data Assimilation (DA) is to integrate observational data. Specifically, in physics-based numerical weather prediction systems, DA incorporates real-time observations into the model state to provide more accurate initialization for forecasting. In the context of Direct Observation-based Prediction, we leverage DA to fuse sparse observational inputs into dense, forecast-compatible representations. This helps mitigate key challenges such as the input–output mismatch, which often leads to inefficient roll-outs and unstable training dynamics. Following your suggestion, we will highlight the general role of DA more clearly in revised versions of the paper to better clarify its key contribution to our framework.

---

We appreciate your feedback once again, which has helped us strengthen the overall presentation and completeness of the paper. If further clarification is needed, please do not hesitate to mention it, and we will promptly address your inquiries. We look forward to receiving your feedback.

References:

[1] Hess, Georg, et al. "Masked autoencoder for self-supervised pre-training on lidar point clouds." Proceedings of the IEEE/CVF winter conference on applications of computer vision. 2023.

[2] Mizrahi, David, et al. "4m: Massively multimodal masked modeling." Advances in Neural Information Processing Systems 36 (2023): 58363-58408.

[3] Gao, Zhihan, et al. "Earthformer: Exploring space-time transformers for earth system forecasting." Advances in Neural Information Processing Systems 35 (2022): 25390-25403.

[4] Shi, Xingjian, et al. "Convolutional LSTM network: A machine learning approach for precipitation nowcasting." Advances in neural information processing systems 28 (2015).

[5] Wang, Yunbo, et al. "Predrnn: A recurrent neural network for spatiotemporal predictive learning." IEEE Transactions on Pattern Analysis and Machine Intelligence 45.2 (2022): 2208-2225.

[6] Bai, Cong, et al. "Rainformer: Features extraction balanced network for radar-based precipitation nowcasting." IEEE Geoscience and Remote Sensing Letters 19 (2022): 1-5.

[7] Gao, Zhangyang, et al. "Simvp: Simpler yet better video prediction." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.

[8] Tan, Cheng, et al. "Temporal attention unit: Towards efficient spatiotemporal predictive learning." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023.

[9] Vandal, Thomas J., et al. "Global atmospheric data assimilation with multi-modal masked autoencoders." arXiv preprint arXiv:2407.11696 (2024).

[10] McNally, Anthony, et al. "Data driven weather forecasts trained and initialised directly from observations." arXiv preprint arXiv:2407.15586 (2024).

Dear Reviewer HaE5,

Thank you once again for your thoughtful and constructive feedback.

In response to your comments, we have added quantitative comparisons with six additional classical spatio-temporal baselines. We have also clarified the design choices for our forecasting model, including the role of Data Assimilation (DA) and the potential application of DAWP to station data.

As we are now in the Author-Reviewer Discussion phase, please feel free to raise any further questions you may have. We are committed to addressing all reviewer concerns and ensuring timely responses. Thank you once again for your valuable contributions to our work.

Best regards,

The Authors of Submission 1085