Long-Term Typhoon Trajectory Prediction: A Physics-Conditioned Approach Without Reanalysis Data

We are pleased that the reviewer acknowledges LT3P has achieved state-of-the-art in the field of typhoon trajectory prediction. We are also thankful for recognizing the effectiveness of the LT3P training pipeline. All of your feedback has been thoroughly reviewed and incorporated into our text. For ease of reference, we've highlighted the updated sections in cyan.

[W1 - Application to other regions]. The study is limited to one region, as it looks that the UN map has a fixed latitude and longitude values. How was it chosen, what if typhoons are in the borders, or even going outside? It also means that your model can't easily be applied on other regions?

The regions we selected are cropped areas where typhoons, among the numerous types of tropical cyclones, are defined. We have added this information and references on page 7 of the main text. Additionally, in Appendix A.3, we have included inference results as zero-shot validations and analyses for regions defined for other types of tropical cyclones.

[W2 - Computation time]. Since real-time computation is the goal, it would be important to give computation times values.

All deep learning methodologies can infer within 3 seconds with A100 single gpu. Indeed, additional time is required to obtain input data; however, compared to other methods, it is sufficiently close to real-time. Other models using ERA5 have more than 3 days of data acquisition delays, while our model has only a 3-hour delay. Therefore, it takes about 3 hours to calculate the typhoon's trajectory 72 hours ahead.

[W3 - Definition of ‘Probability cones’] It is not clear how the 'probability cones' are obtained: stochasticity is mentioned only in the result part, not in the method.

The probability cones are obtained through kernel density estimation (KDE). We have added this information to Figure 3.

[W4 - Details of the Dataset]. I understand that the number of data is limited, but I would like to know if a validation set was used to fine-tune the hyper-parameters or if it was done using the 2019-2021 years. Please explain better.

We have included the following content in the main text to address this: The training data spans the years 1950-2018 and includes 1,334 typhoons, while the test dataset covers the years 2019-2021 and comprises 90 typhoons. Note that for hyperparameter tuning, we train on data from 1950 to 2016 and use the data from 2017 to 2018 as the validation set.

[W5 - Worst case from our prediction]. Finally, it would be interesting to see one of the 'worst' cases also in Figure 3, with a comment on it.

We have added an analysis of a failure case in Appendix A.4.

[W6 - Typos]. Many typos are present, see below.

Thank you very much. We have corrected all typos and revised the sentences with a native speaker.

We are pleased with the reviewer’s recognition of LT3P’s novelty in Climate AI and their acknowledgment of our pipeline. All comments have been addressed in our revised text, with changes marked in blue.

[W1 - Missing implementation details] If I read correctly, several explanations about network architecture, losses, ... are missing. This would prevent the re-production by fellow researchers.

We have utilized three different baselines: GAN, CVAE, and Diffusion. Due to the extensive nature of the contents, we have added detailed explanations in Appendix A.1.

[W2 - Comparison with MGTCF]. Inconsistent trends of the scores of existing (compared) models. As I read the MGTCF paper, the authors reported that the MGTCF performs clearly better than SGAN, which is different from Tables 2, and 3.

We trained our models using the official repository of MGTCF. In our experiments, SGAN performed better than MGTCF. We suspect this discrepancy may be due to differences in the training and test data used for MGTCF. Although MGTCF utilized ERA5 data, it only trained on cropped areas around the center coordinates of typhoons. We hypothesize that MGTCF might not synergize well with the best track data. In contrast, SGAN, which relies solely on coordinate information, demonstrated better performance than MGTCF.

[https://github.com/Zjut-MultimediaPlus/MGTCF]

[W2 - Comparison with Sotas]. The scores (Distance) of the proposed LT3P update the current SotA by order of magnitude. I checked the most recent works such as (Bi+ 2023, Lam+ 2022) but the distance scores are roughly in the same order as the existing methods. I feel the current manuscript does not sufficiently explain why this huge jump happens, although the ablation study tells that it seems the joint training with reanalysis is a key factor.

Our LT3P model, which utilizes a 20-sample average ensemble, exhibits performance comparable to deterministic models such as Bi+2023 and Lam+2022, which generate only a single path, as illustrated in Table 2. (For example, Bi+2023 reports a 120km error at a +72-hour prediction.) Table 3, on the other hand, demonstrates our model's superior performance with the lowest error path selected from 20 inferences, showcasing the benefits of LT3P's stochastic approach. Consequently, this stochastic ability to infer multiple paths allows us to significantly outperform deterministic models like Bi+2023 and Lam+2022 in Table 3. Moreover, Bi+2023 and Lam+2022 rely on ERA5 data, which can not infer real-time predictions, and they produce climate variables, not actual typhoon paths, which are subsequently used to identify the typhoon center using a rule-based method. It is important to highlight that LT3P functions as a real-time stochastic model.

[Q1 - Definition of the “lead time”]. I'm a little bit confused about how to handle the "Lead time" in evaluations in a fair way. The manuscript says that one needs to wait for three hours to obtain the UM reanalysis data. This means the proposed method loses the lead time of three hours. So, the score of "6-hour leadtime" should be understood as "3-hour leadtime"?

Considering a real-world, real-time inference scenario, UM data has a 3-hour delay at data acquisition, the first lead time is +3 hours, and the rest are at +6-hour intervals. For example, if we assume that we request the UM prediction dataset of 00:00, we can access the data at 03:00. This data represents intervals of UM = {00, +06, +12, +18 … +72}, and though it can be accessed 3 hours later, all the data are at 6-hour intervals. Therefore, while there is a 3-hour delay for inference, from the dataset's perspective, the intervals are consistently every 6 hours

Thank you once again for your feedback. We hope our detailed clarification on the network architecture, model comparison, and significant performance leap of LT3P clearly conveys the novelty of LT3P in Climate AI.

[Q1 - Notation Issues]. Confusing notation.

Thank you for pointing out our mistakes. We have specified the index 'i' on page 4 and corrected the duplicated notation 'p.' To avoid confusion, we have changed all the small ‘p’ to ‘o’ which represents observations in descriptions and figures. “”

[Q2 - Timestamps of UM and ERA5]. Why are geopotential height and wind vectors inputs taken from the output timesteps.

The output time steps represent the intervals at which we aim to predict the typhoon paths, and leveraging atmospheric information for these time frames is crucial for accurate prediction. This goal is achieved by utilizing the Numerical Weather Prediction (NWP) forecast fields, especially the UM dataset. We use physics-based predicted values from UM as inputs, offering a significant advantage over using ERA5. Unlike ERA5, which not only experiences delays but also lacks access to future data, the UM forecast incorporates future atmospheric conditions through physics-based modeling. This makes it a more suitable choice for our input values.

[Q3 - Grammar error] . 3.1 Preliminaries (not preliminarily)

As the reviewer pointed out, we have corrected the typo. Do you think any further modifications are needed for the content of this section? We believe explaining how our dataset is derived from the NWP model is essential. Therefore, we have composed this section to convey that information.

[Q4 - Tone]. Tone down "is too large scale to train a model from scratch". There exist models trained from scratch on ERA5…

We have toned down the sentence as “The ERA5 dataset, accumulated at 6-hour intervals from 1950 to the present, requires a significant amount of time to train a model from scratch. To address this challenge, we have adopted a strategy of constructing a foundational model for typhoon trajectory prediction”. This is reflected in Section 3.2 on page 5.

[Q5. - Timestamp of bias-correction output] Timestamp of \tilde{X}

The output time frame of $\tilde{x}$ is from $t_{o}+1$ to $t_{o} + t_{f}$.

[Q6 - Timestamp of bias-correction output]. I don't completely understand how the bias correction phase works, is the UM data matched with ERA5 based on the timestamp?

Yes, you are correct. In our bias correction phase, we train the model on ERA5 data for the period from 1950 to 2010, ensuring it outputs ERA5 data. Simultaneously, for the period from 2010 to 2021, UM data is used as an input, and the bias correction guides the model to produce ERA5-like output from UM data.

Thank you once again for reviewing our work. We hope our comments have addressed any questions and concerns and clarified things. We look forward to your further feedback on our revised work.

We thank the reviewer for the insightful feedback concerning clarity and details. Your comment makes our paper polish up. Based on comments, we have revised it, marked in orange. Below, we address each of your questions with corresponding answers.

[S1 - Dataset Release]. Open-sourcing the data will be valuable. When doing so, the authors should put care into making it easily accessible and well-documented.

Thank you for acknowledging our Physics Track dataset. As shown in Appendix A.2, we have already prepared the project page interface and details for public release. Currently, due to the large size of our dataset, we are unable to upload an anonymized link. We plan to make the dataset publicly available upon the acceptance of our paper.

[W1 - Implementation Details] The method seems to have been tested using a GAN, a CVAE, and a diffusion model, but implementation details are largely lacking.

We have included detailed information about the implementation in Appendix A.1 section to provide clarity for readers. In particular, we have mainly described a loss function, embedding and component of each baseline.

[W2 - Detail of implementation] Unclear meaning and implementation details of "so it is matched through the latitude and longitude mapping and the interpolation method.

The original meaning of the sentence is that we just utilize a bilinear interpolation to adjust a resolution of UM data and ERA5. Instead of the expression that the reviewer point out, we have changed it to “Note that the resolution has been adjusted using bi-linear interpolation”. Please check it on page 7 of the main text.

[W3 - Definition of ‘ensemble method’] Unclear what exactly the "ensemble method" refers to and how exactly it is performed in practice for all the models.

We have employed the ensemble average method for comparison, similar to the approach used by meteorological offices. Since each inference from the GAN, CVAE, and Diffusion models produces different outputs due to random noise, we average the generated 20 trajectory samples from our model for better probabilistic forecasting. On page 7 of the main text, this method is clearly referred to as the ‘ensemble average '.

[W4 - More experiments and details on ablation study]. Please include more ablations, e.g. with UM and joint training and bias-correction but without pre-training. The whole ablation study section is a bit hard to understand and should be expanded (in the appendix if needed).

Unfortunately, due to the inherent characteristics of our model structure, conducting bias-correction independently without the pre-training phase is not feasible. The ERA5 dataset has been accumulating data from 1950 to the present, whereas the UM dataset has been available since 2010. Therefore, if only the UM dataset is used, the best track typhoon dataset is also only available from 2010. The number of typhoons since 2010 is approximately 200, which is a small fraction compared to the total of 1,424 typhoons that have occurred since 1950. Consequently, the performance of UM-only training is significantly limited. In contrast, a joint training setting, which uses ERA5 data from 1950 to 2009 and the UM dataset from 2010 to 2021, can utilize the entire typhoon dataset.This dataset is about six times larger than UM-only dataset, which seems to significantly improve the performance of data-driven models due to the increased amount of data Lastly, we have added more content on the analysis of ablation studies in Appendix A.6 section.

[W5 - Implementation details]. Are the baselines in the first set of rows of Table 2 (e.g SocialGAN) all off-the-shelf pre-trained human trajectory prediction models? If so, it is not surprising at all that they perform so badly, and it would be good to retrain them on your data.

We note all baselines in Table 2 are trained and evaluated using the best track dataset for strictly fair comparison. To avoid misunderstanding, we have specified the training dataset of the baselines on page 7.

[W5 - Selection of comparison models]. Additionally, why is Ruttgers et al. not included in the benchmarking?

We did not include Ruttgers et al.'s work in our benchmarking efforts due to differences in both the training dataset and methodology. Ruttgers et al. track typhoon paths by reconstructing them after marking the typhoon center coordinates on satellite images. In contrast, we do not use satellite images.

However, it would be valuable to conduct an experiment based on their work. We have now requested the authors for their codes and datasets. Should these sources become available, we will carry out an additional comparison with Ruttgers et al.'s work, and include the result in our paper.

We sincerely appreciate the reviewer's insightful feedback and acknowledge the highlighted weaknesses. We are truly delighted that the reviewer acknowledges LT3P as one of the first data-driven models for real-time typhoon trajectory prediction. Additionally, their belief in LT3P's value in Climate AI has been a great source of motivation for us. We have addressed all the issues pointed out by the reviewer in our text, marked in olive color.

[W1 - Application on other types of tropical cyclones]. Limited Application: The model has only been applied to typhoons and has not been tested on other types of tropical cyclones, limiting its current applicability.

From a meteorological perspective, 'typhoons' refer to tropical cyclones that occur in the North Western Pacific region. While the meteorological mechanisms of tropical cyclones may be similar, the movement patterns, such as their trajectories, can differ by region and thus should be treated differently. Additionally, creating a generalized model for global application could lead to computational resource wastage due to the inclusion of excessive irrelevant data from regions not affected by typhoons. Ideally, a single trained model that generalizes well to all tropical cyclones would be beneficial. However, our goal has been to achieve higher accuracy by training models specifically for each region.

Nevertheless, as seen from the experimental results in Appendix A.3, our LT3P shows the potential for application in diverse regions. We perform the zero-shot validation of our model for cyclones in both the North Indian Ocean and the South Indian Ocean. We note that the prediction results from their meteorological agencies are not available. As expected, our method outperforms the comparison methods. However, the FDE is larger than the test results in the North Western Pacific.

[W2 - Dependence on NWP Dataset]. Dependence on Real-Time NWP Dataset: The model's performance is dependent on the availability and accuracy of the real-time NWP dataset, which could be a potential limitation.

Our model focuses on leveraging the advantages of real-time NWP datasets, and therefore, it does not function properly in situations where NWP datasets are entirely unavailable. As the reviewer point out, this can be seen as a potential limitation of our study. However, our model requires only three basic types of meteorological variables, which in most cases can be easily obtained from freely available outputs of NWP models. A prime example is the use of the publicly available dataset from ECMWF's IFS forecasts. Known for their high accuracy, these IFS forecasts are a reliable source. To account for the practical scenario of using NWP model outputs with varying accuracies, we opt for the UM forecasts, known to have a larger bias. This decision is made to demonstrate that our model can yield impressive results even with NWP datasets of somewhat lower accuracy.

[W3 - Future work]. Need for Future Work: While the paper outlines plans for future work, including extending the application to all kinds of tropical cyclones, these aspects have not yet been addressed or tested.

In our future work, we will validate LT3P on all types of tropical cyclones occurring globally. We have added future work in Appendix A.7.