SIFusion: A Unified Fusion Framework for Multi-granularity Arctic Sea Ice Forecasting

Thanks for acknowledging the value of our work and our effort to explore effective neural network architecture for polar scientific research.

---

Response to Q1: Training on a mixture of temporal scales.

Thanks for your question. Indeed, training prior U-Net-based approaches with a dataset that contains a mixture of time scales is an intriguing idea. Considering that the temporal distribution of different time scales is not identical, this could lead to inferior performance of those models. It could be elaborated by a succinct example. If we set the input length of the model to 7, i.e., the temporal step equals 7. The model will simultaneously learn the evolving Arctic sea ice pattern from 7 days, 7 weeks’ average, and 7 months’ average values. The mixture of mean and variance derived from the above three different time scales would generate a misleading data distribution that causes forecasting error.

In contrast, our proposed architecture mitigates this non-trivial issue by explicitly mapping encoded input sea ice data sequences from different temporal scales into independent sequential tokens. Therefore, the unique temporal variation pattern of a specific time scale is preserved rather than mixed with patterns from other time scales. Additionally, by adopting the proposed approach, our model could exploit inter-granularity correlations to improve the overall performance.

Response to Q2: Embedding vectors of different time scales.

Many thanks for raising this point. Based on the analysis from Q1, we know that the temporal distributions of different time scales are distinctive from each other. On the contrary, the spatial structure across different time scales is similar. For instance, given a monthly average of sea ice maps and a sampled daily sea ice map from the same month, it would be difficult to tell which one stands for the averaged value and which one represents the daily value. This also applies to the comparison between daily value and weekly average, or between monthly average and weekly average. It is because all time scales spatially represent the identical pan-Arctic area. The variation caused by averaging along the time axis will not drastically change the spatial structure and impede the learning of spatial embedding. Therefore, we adopt the shared spatial encoder approach, which is sufficient for compressing sea ice data derived from different time scales into a latent feature space. Additional benefits would be the simpler architecture and a reduction in overall parameters.

Response to Q3: Complement of different time scales.

We appreciate your insightful question. To further validate the importance of a specific time scale, we conduct experiments under the following settings:  1. Training set period: from the year 2000 to 2013; validation and test period remain unchanged.  2. We trained baselines of three single granularities and SIFusion with all three time scales on the training dataset.  3. We perform model training of three combinations: Daily-Weekly (Model-1), Weekly-Monthly (Model-2), and Daily-Monthly (Model-3). The overall architecture is similar to SIFusion, only that one granularity is removed from the modeling process.

The experiment results are organized as following:

Performance at Daily Scale:

Adding long-term predictions to the daily time scale could improve the performance of daily forecasts. Since both weekly and monthly data contain previous long-term trends, it is expected to be useful and to facilitate initialization of the Arctic sea ice state. Considering the lower SIE error and higher R Squared for Model-1, it could suggest that the 7-day weekly data aligns more naturally with weekly granularity.

Performance at Weekly Scale:

We could find that predicting at a weekly time scale, along with shorter or longer time scales, could both be effective for improving the overall performance. This could be the benefit of additional training data. By combining weekly and monthly granularity together, Model-2 could offer RMSE and MAE performance close to the SIFusion model, and it could achieve more accurate sea ice extent identification.

Performance at Monthly Scale

For the forecasting at monthly scale, we could find that adding daily granularity was slightly less significant than adding weekly averages prediction to the model.

Overall, from the above experiments, we could find that longer-term temporal granularity tends to provide useful information to help improve the overall performance. Adding an extra time scale to the model could be rewarding and verify our proposed approach.

Response to the Increased Training Time

Based on the above experiments, we collect the training time of all models to assess the increased training time by introducing extra temporal scales:

Training Time of Experiments

From the above table, we can see that, as temporal granularity increases, the training time almost linearly increases, which could be reasonable since the models are calculating with more data.

Response to Minor Comments

Thanks for your kind suggestion. We will revise the italic fonts in equations to accurately convey their meanings.

---

Once more, thank you for acknowledging the contribution of our work. We look forward to your active participation in the discussion. We will respond promptly.

Thanks for valuing our work and acknowledging the effectiveness of the proposed approach.

---

Response to Q1: The main contributing time scale.

We totally agree with your opinion that longer-term predictions contribute more to the improvement of overall performance. For instance, assuming that our model forecasts at the beginning of a melting season, on the one hand, the prediction of the monthly average sea ice could provide critical hints that the extent of sea ice is about to drop drastically, which would facilitate the short-term predictions. On the other hand, the input of historical monthly average also provides the model with information about the amount and variation of sea ice and during the winter season, which is correlated to the strength of albedo and the energy exchange in the pan-Arctic region. This helps the short-term prediction to set up a correct initial state.

Response to Q2: Exploring pre-training and fine-tuning.

We appreciate your insightful question, and we fully agree that the pre-train and fine-tune approach is definitely worth exploring. Hence, we set up the following experiment for further investigation:  1. Training set period: from the year 2000 to 2013; validation and test periods remain unchanged.  2. Decreased model parameters for a shorter training set period due to the limited computational resources and time.  3. Baselines of three single granularities and SIFusion with all three time scales.  4. Pre-train and fine-tune experiments design:       - a. Pre-train a 7 days single granularity model, then fine-tune on 6 months' average data.       - b. Pre-train a 8 weeks' average model, then fine-tune on 6 months' average data.  5. We additionally trained two models for comparison: the SIFusion and the single-granularity model that performs at a monthly scale.

We report our experiment as follows: Performance at Monthly Scale

We could find that pre-training a 7 days forecasting model then fine-tuning on monthly data, suffers a performance drop in all metrics. This could be caused by the temporal distribution of daily and monthly scales, which are distinct from each other; hence fine-tuning the pre-trained model on monthly data could be difficult to converge. For the other setting, pre-training a 7 days forecasting model then fine-tuning for predicting 6 months' averages could improve the performance on most evaluation metrics, except that the identification of sea ice extent is inferior. The success of fine-tuning the model in this setting could indicate that the temporal distribution of medium range and seasonal time scales is closer than the distribution between daily and monthly scales, hence the forecasting skill could be further improved. We will continue investigating this approach and fully exploit the potential of the proposed pretrain-finetune architecture.

Response Weaknesses: Description of proposed architecture and performance of physical models.

Sincere thanks for your kind suggestion.

1. Improve the description of the model:

We agree that explaining our proposed architecture in plain text is hard to get through. We think the description of the model could be improved by providing an equation or example of the manipulation of the input data dimension. This could facilitate the understanding of the difference between our multi-granularity fusion module and the previously used vanilla Transformer.

2. Regarding the physical model:

Considering the sea ice forecasting physical model SEAS5 [1] only provides prediction of the binary index, which indicates the sea ice extent, from the first date of each month, it could only provide limited evalation metrics (for reference, we applied for the SEAS5 to forecast the first 216 days of 2016, the calculated SIE difference is 0.3359 millions of square killometers). We choose a surrogate model [2] that is usually utilized to benchmark dynamical sea ice forecasting models to provide more evaluation metrics. We train the model from the dynamical model benchmark under our experimental setting and compare it with our model. From the following table, we can find that the dynamical model benchmark performs well at the recent time scale, and the performance drops as the prediction horizon extends to the seasonal scale.

Comparison with Dynamical Model Benchmark Performance at Daily Scale:

Performance at Weekly Scale:

Performance at Monthly Scale:

3. Indeed, our initial intention in the limitation section is to discuss the potential of leveraging sea ice correlated meteorological and oceanic variables to facilitate forecasting. We will revise the expression to a more precise version.

Response to Minor Comments

Thanks you for kindly bringing these issues to our attention. We will accordingly revise the expression in line 84, 89, 122, 215, 236 and the use of term “variate”.

Reference

[1] Johnson, S. J.; Stockdale, T. N.; Ferranti, L.; Balmaseda, M. A.; Molteni, F.; Magnusson, L.; Tietsche, S.; Decremer, D.; Weisheimer, A.; Balsamo, G.; et al. 2019. SEAS5: the new ECMWF seasonal forecast system. Geoscientific Model Development, 12(3): 1087–1117.

[2] Niraula, B.; and Goessling, H. F. 2021. Spatial damped anomaly persistence of the sea ice edge as a benchmark for dynamical forecast systems. Journal of Geophysical Re- search: Oceans, 126(12): e2021JC017784.

---

We thank you once again and warmly invite you to engage in the ongoing discussion. We will response promtly.

Thank you once again for dedicating your valuable time to the review. We sincerely appreciate your positive assessment of our work and the insightful suggestions that could help us further refine our paper.

Best regards,

Sincerely yours,

Authors.

Thanks for your valuable review and comments of our work.

---

Response to Q1: Additional climatic variables.

Thanks for your insightful question. Adding multiple climatic variables could be effective and beneficial, as you suggested, but it also poses a non-trivial challenge for the model to learn multiple variables. We implement the integration of variable correlation to the variation of Arctic sea ice for verification based on the SIFusion architecture. We choose three atmospheric ERA5 variables, i.e., air temperature of 2 meters (t2m), u component of 10 meter wind (u10), and v component of 10 meter wind (v10), to investigate the impact of integrating them into our proposed architecture.

We conduct additional experiments under the following settings:

1. Training set period: from the year 2000 to 2013; validation and test period remain unchanged.
2. We download hourly global ERA5 data (with resolution of 128 x 256) and average 24-hour files into daily values. To geophysically align global ERA5 with pan-Arctic SIC data, we regrid and interpolate those variables to the same Arctic grid.
3. We trained baselines of three single granularities and SIFusion with all three time scales on the training dataset.
4. We utilize two sets of spatial encoder-decoders, one for SIC data and the other for encoding three ERA5 variables. The sequential backbone is the same as SIFusion. The input and prediction length of all ERA5 data and SIC are identical, i.e., 7 days. The additional variables are normalized to [0,1]. We denote this model as Model-1.

Experiment Results:

Comparing the performance of Model-1 with 7 Days daily baseline, we find that adding ERA5 variables could lead to a slightly lower RMSE but a much higher MAE. Considering the lower R Squared, NSE, and larger SIE difference, the prediction error could be around the 15% sea ice concentration threshold for determining sea ice extent. Adding additional variables to the model could be challenging, especially when the spatio-temporal distribution of the variables is unique from each other. We will further investigate the effective integration of extra climatic variables.

Response to Q2: Dataset pre-processing.

Thanks for your kind suggestion. We provide a brief summary of our pre-processing procedure to curate the AI-ready dataset and will open-source the data pre-processing code to facilitate reproduction.

1. We download the NOAA/NSIDC Climate Data Record of Passive Microwave Sea Ice Concentration G02202 Version 4 dataset from the official website.
2. The official data was preserved in the NetCDF format. We use the open-sourced python package “netCDF4” to read data from the file with “.nc” suffix.
3. The “netCDF4” package provides us an API to extract sea ice concentration data by providing the official variable name, i.e., “cdr_seaice_conc”.
4. Since the acquired SIC data contains mask values that stand for land, ocean, and lake, we filter out these values with special meanings.
5. We calculate the weekly and monthly average of SIC based on daily values to generate the AI-ready dataset. We will add a brief summary of data curation to the dataset section.

Response to Q3: Intra and inter-gruanularity.

Thanks for raising this point. In our SIFusion architecture, the modeling of intra-granularity is naturally achieved by two steps:

1. Spatial patterns of sea ice are tokenized for each time step within a specific granularity.
2. Temporally concatenated spatial tokens are further embedded as one holistic token that represents the temporal pattern of the sea ice sequence.

For inter-granularity modeling, sea ice variations of each temporal granularity are first independently embedded as a single token. Then the multi-head self-attention is applied across the granularity dimension to explicitly capture pairwise correlations.

Response to Q4: Change of the temporal granularities.

Thanks for your intriguing question. Firstly, we choose to investigate the fusion of daily, weekly, and monthly granularity based on the literature of sea ice research. Since these three granularities are commonly studied scales, our motivation is to further exploit the inter-granularity between those time scales and to achieve simultaneous forecasting at all scales. Secondly, to further investigate the sensitivity of different granularity and quantitatively enhance the understanding of adding or removing specific granularity, we perform model training of three combinations: Daily-Weekly (Model-2), Weekly-Monthly (Model-3), and Daily-Monthly (Model-4). The overall architecture is similar to SIFusion, only that one granularity is removed from the modeling process. The experiment results are organized as follows:

Performance at Daily Scale:

Adding long-term predictions to the daily time scale could improve the performance of daily forecasts. Since both weekly and monthly data contain previous long-term trends, it is expected to be useful and to facilitate initialization of the Arctic sea ice state. Considering the lower SIE error and higher R Squared for Model-2, it could suggest that the 7-day weekly data aligns more naturally to weekly granularity.

Performance at Weekly Scale:

We could find that predicting at a weekly time scale, along with shorter or longer time scales, could both be effective for improving the overall performance. This could be the benefit of additional training data. By combining weekly and monthly granularity together, Model-3 could offer RMSE and MAE performance close to the SIFusion model and could achieve more accurate sea ice extent identification.

** Performance at Monthly Scale:**

For the forecasting at the monthly scale, we could find that adding daily granularity was slightly less significant than adding weekly averages prediction to the model. Lastly, from the above experiments, we could find that longer-term temporal granularity tends to provide useful information to help improve the overall performance. Adding an extra time scale to the model could be rewarding and verify our proposed approach.

Response to Q5: Using pre-trained weather model.

We sincerely appreciate your insightful inquiry. Indeed, it is an innovative and quite promising idea to leverage pre-trained weather and climate foundation models to boost the overall performance. Specifically, the hidden atmospheric and oceanic dynamics learned by these foundation models are coupled with sea ice changes, and they could be leveraged to improve the forecasting skill. We will definitely look into the transfer of knowledge embedded in climate foundation models to SIC forecasting. We will add this to the discussion of future work.

Response to Q6: Independent spatial tokenization.

Thanks for your question. Most of prior works that leverage the U-Net architecture will concatenate variables from different time step to construct input batch with a dimension of [B,C,H,W], where B is the batch size, C stands for the total number of concatenated daily/weekly/monthly SIC data, H and W represents the shape of SIC data. There are no independent spatial and temporal encoding processes, since the data are concatenated along the time axis, and it will be transformed altogether by U-Net encoder. Different from prior approach, the shape of our proposed SIFusion has a [B,C,1,H,W]. This means that C stays unchanged during spatial feature extraction, i.e., "independent spatial tokenization", and we will get spatial tokens with dimension of [B,C,Feats]. This allows us to explicitly extract useful intra-granularity information that would benefit forecasting changes of sea ice. Moreover, we could further model inter-granularity correlations.

Response to weakness.

Thanks for your question. Since prior sea ice frameworks mainly focus on forecasting at single granularity, we evaluate the proposed framework by comparing their reported results.

Response to Formatting Issues.

We sincerely thank you for bringing these issues to our attention.

1. We will revise the heading to 'Introduction'.
2. We will adjust the spacing of Figure 4 to produce a consistent line of text on page 5.

---

Once more, sincere thanks for your review, and grateful for your insightful query. We look forward to your active participation in the discussion and we will respond promptly.

Thanks for your dedication to reviewing our work.

---

Response to Q1: Choice of Swin Transformer.

Thanks for your question. We select Swin transformer as our spatial encoder mainly due to the following reasons: a. Swin transformer uses a shifted window strategy that helps the model to effectively capture spatial correlations and features between the neighboring local Arctic regions. b. Swin transformer builds a hierarchical feature map by merging neighboring sea ice map patches. This design could preserve local detail while effectively extracting the global feature of the pan-Arctic region, which would facilitate the model to more accurately capture the overall distribution of sea ice extent.

Response to Q2: Application for missing time series.

Thanks for your insightful question.To verify whether our method is applicable to missing time series, we conduct additional experiments under the following settings:  1. Training set period: from the year 2000 to 2013; validation and test period remain unchanged.  2. Decreased model parameters for a shorter training set period due to the limited computational resources and time.  3. We trained baselines of three single granularities and SIFusion with all three time scales on the training dataset without masking.  4. We randomly mask the input of SIFusion at a daily scale, and the number of applied daily masks N in [1,3].

We report the results as follows.

Performance at Daily Scale:

We could find that when training SIFusion with randomly masked daily values, the performance slightly drops, but still outperforms the daily baseline.

Performance at Weekly Scale:

Performance at Monthly Scale:

Similarly, the performance at the weekly and monthly scales also drops slightly, but they still perform better than the baselines. This could indicate that our SIFusion is still applicable to missing time series.

Response to Q3: Change of temporal scale.

We appreciate your insightful inquiry. Incorporating a more temporal scale contributes to the improvement of SIFusion's forecasting skills. We agree that incorporating weekly/monthly scales is a natural choice. The main reason why we specifically use 7 days on a daily scale is that the 7 days time naturally aligns with one week's time. This alignment with tokens in the weekly scale could facilitate the learning of the model. We perform a revised version of SIFusion with 10 days daily forecast time.

Performance at Daily Scale:

Performance at Weekly Scale:

Performance at Monthly Scale:

By comparing the variance of performance between SIFusion and SIFusion trained with 10 10-day leads at the daily scale, we could find that the daily and monthly figures are close, but the identification of sea ice extent at the weekly scale suffers a larger performance drop. This could indicate that keeping this alignment between the daily scale and a one-week token is beneficial. As to the 8 weeks' average at the weekly scale, the time span of 8 weeks is still within a month, so that one one-month token could represent the information of the weekly scale.

Response to Weaknesses: Valuable application beyond this problem.

Many thanks for raising this point. Our model is capable of capturing the variation of sea ice across multiple temporal scales, thereby enhancing overall forecasting skill. Specifically, the monthly scale incorporates important seasonal trends for predicting future sea ice. At the onset of the melt season, for example, monthly prediction of SIFusion provides guidance that could subsequently steer shorter-range forecasts based on the expected long-term trend. Moreover, the historical monthly means fed into the model carry the imprint of winter ice volume and its variability information intimately linked to pan-Arctic albedo strength and energy exchange, thereby facilitating short-term predictions with a more accurate initial state.

Considering the surface ocean directly contacts with the sea ice and the oceanic variable, e.g., sea surface salinity, which impacts the concentration of sea ice, our model could be extended to the prediction of oceanic variables and the joint modeling of multiple sea-ice-related variables.

Response to Formatting Issues.

Thanks for you for bringing formatting issues to our attention. We will revise Figure 5 and the caption to make a clearer description that it represents the comparison between single daily time scale and our proposed SIFusion.

---

Once again, sincere thanks for your review and insightful questions. We look forward to your active participation in the discussion and we will respond promptly.