SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data via Scalable Conditioned Neural Fields

We sincerely thank the reviewer for the insightful discussions, suggested papers, and constructive comments. It was a pleasant surprise to find substantial similarities as well as subtle yet important distinctions between SCENT, STFNN, and the referenced works. We found STFNN's inference mechanism and gradient-based formulation particularly innovative, and have therefore reached out to the authors for further exploration. As suggested, we will move the related work section to the main manuscript and provide more in-depth discussions on spatiotemporal forecasting methods, such as AutoST and ST-ResNet.

$\$

1. Comparing SCENT and STFNN

Motivation and Similarities. SCENT aims to develop a flexible model capable of generating continuous spatiotemporal fields from sparse observations, addressing the common scenario in scientific domains where sensor coverage is limited. Similarly, STFNN effectively infers unobserved regions using its sophisticated Pyramidal Inference. Both methods assume continuous fields and handle irregular, sparse, and noisy observations robustly.

Fundamental Differences in Generalization. However, SCENT explicitly models a family of functions, acting as a generalizable implicit neural representation (GINR). It can represent various spatiotemporal scenarios conditioned on input data from different contexts. In contrast, STFNN models one specific spatiotemporal field (e.g., PM2.5 over China), allowing interpolation and extrapolation strictly within that field, rather than generalizing across distinct fields.

Architectural Design. Both models utilize INRs but differ in structure and intent. SCENT employs an encoder–decoder architecture with attention-based querying to generalize across tasks. STFNN is more akin to a per-task INR, enhanced with gradient-based modeling and local graph-based correction to capture detailed local variations within a single domain.

$\$

2. RainNet: Rainfall Nowcasting

We appreciate for the suggestion. We have strengthened our experimental evaluation by incorporating RainNet and the new nowcasting dataset.

Dataset & Task. We use the RY product from the German Weather Service (DWD), a quality-controlled rainfall composite at 1km~$\times$1km spatial and 5-minute temporal resolution. Data from 2012-2016 are used for training, and 2017 for testing. The task is to predict rainfall fields for future timestamps $t \in [5, 10, \dots, 60]$ minutes, given four historical fields. Following the official preprocessing, we use 173,345 / 43,456 training / test instances, and downsample the original 900$\times$900 resolution to 64$\times$~64 for faster training during the rebuttal period. SCENT is trained with a forecast horizon ($t_h$ = 60).

Results. We report root mean-squared error (RMSE [mm$~h^{-1}$]) in the table below. SCENT consistently outperforms RainNet across all lead times, reducing MSE by approximately 30%. We attribute this improvement in part to SCENT's ability to train with variable target times $t_o$, which serves as a form of data augmentation.

$\$

3. Choosing M

We agree with the reviewer that selecting the OPTIMAL number of latent tokens $M$ is important. While performance generally improves with larger $M$, we observe diminishing returns beyond $M=192$ on the S5 dataset—a saturation pattern also seen in Perceiver IO. This suggests that excessively large $M$ offers limited benefit while incurring higher computational cost, and a moderate $M$ can strike a better balance between performance and efficiency.

$\$

4. On statistical testing

We appreciate the reviewer’s suggestion. While full significance testing is infeasible due to time constraints, we provide evidence of robustness through multiple runs: for example, our small model (Rel-MSE = 0.467) shows a standard deviation of only 0.0042 across five seeds. Our evaluation also spans multiple benchmarks and a range of lead times, demonstrating consistent performance across diverse tasks and forecasting horizons. We acknowledge the value of statistical testing and will consider incorporating it in future revision.

$\$

We hope our responses address your concerns, and we’d be grateful if you’d consider updating your score accordingly.

1. Hyperparameters
We appreciate the reviewer’s thoughtful questions regarding the extent of hyperparameter tuning conducted for SCENT in comparison to the baseline methods. This is indeed a crucial aspect when evaluating model performance fairly across methods. We included detailed hyperparameters in the appendix for transparency and reproducibility, and will release the code upon acceptance.

Inherited from Literature. We clarify that we did not extensively tune hyperparameters for baseline methods, but instead adopted configurations from prior work. Many baselines, including IPOT and Perceiver-IO, share architectural similarities with our encoder-processor-decoder design, allowing us to inherit most of their recommended settings. For instance, Appendix Table C largely follows IPOT, with only minor changes such as warmup and total training steps.

Dataset Dependence. Dataset-specific parameters (e.g., optimizer profiles, embedding configurations) reflect widely accepted strategies from prior literature, especially for common datasets like NS-3–5. These datasets exhibit varied dynamics but benefit from established training practices—such as long schedules for PDE dynamics and lower sensitivity to learning rate. As shown in Appendix Table D, most hyperparameters are consistent across datasets. We deliberately avoided aggressive tuning to promote generalizability. For example, embedding dimensions result from simple design choices—e.g., combining linear projections and Fourier features—rather than extensive tuning.

Baseline Hyperparameters. When implementing the baselines, we made every effort to adhere to published or well-established hyperparameters. We first ensured each baseline was reproducible on its original dataset, then applied either the reported settings or those that worked well for SCENT—whichever yielded better performance. Given the complexity of our proposed data, which reflects realistic and large-scale scientific scenarios, we devoted substantial effort to adapting the baselines accordingly.

2. Newer FNO approaches
We thank the reviewer for highlighting recent FNO variants, including Geo-FNO and F-FNO. These works offer notable improvements, particularly in handling irregular geometries and sampling patterns. While we were not able to implement both approaches in full due to time constraints, we successfully implemented F-FNO on the S5 dataset and included those results in our evaluation. We evaluated two configurations of F-FNO with different parameter sizes (m1, m2). Notably, even the smaller variant (1M parameters) performs similarly to the standard FNO, demonstrating the efficiency of the architecture. When scaled to match the parameter count of SCENT and FNO (7.4M), F-FNO achieves significantly improved performance, approaching that of SCENT. We believe this provides meaningful insight into how newer FNO variants perform in challenging, non-grid-based scenarios.

3. On Vaughan et al. (2024) [1]
We appreciate the reviewer’s reference to the Aardvark Weather system. The idea of sequentially trained processors extending the recursive rollout strategy is particularly interesting. By allowing each processor to specialize in longer lead times, the approach may indeed improve inference stability and long-range forecasting performance. While the sequential training of multiple models could introduce computational overhead, the modular nature of the design offers flexibility that might benefit certain applications. Additionally, although the current formulation appears to target fixed-step forecasting, integrating mechanisms for continuous-time inference could be an exciting direction for future work. We see this as a promising and complementary line of research and welcome further discussion on how such ideas might be integrated with or compared to SCENT’s joint modeling capabilities.

4. On CORAL
As the reviewer correctly notes, OFormer, CORAL, and AROMA are all trained using recursive forecasting with a fixed time step. However, CORAL distinguishes itself by introducing a Neural ODE solver, $g_{\phi}$, as the autoregressive processor operating in the latent z-code space. As a result, CORAL adopts a two-stage training process: the first trains the autoencoder, and the second trains the Neural ODE for forecasting. The key advantage of this approach is that the learned ODE can be evaluated at any arbitrary time point, enabling CORAL to capture both spatial and temporal continuity.

We appreciate your thoughtful review and hope our clarifications address your questions.

[1] Allen, Anna, et al. "End-to-end data-driven weather prediction." Nature (2025): 1–3.

We truly appreciate your valuable comments. We agree on your concerns and suggestions, hence provide below our thoughts and additional experimental results for each of the questions.

$\$

1. On Fourier feature

Although Fourier features are well established, their formulation can vary. Here, we describe our approach and present additional ablation studies. Let $R$ denote the maximum frequency resolution and $L$ the number of frequency bands. For the $i$th band, we define the frequency as $f_i = \frac{iR}{L}, \quad i=1,2,\ldots, L.$ Then, the positional encoding for a scalar $x$ is given by the concatenation of sine and cosine functions: $\gamma(x)=\operatorname{concat}_{i=1}^L \Big[\sin\big(2\pi f_i x\big), \cos\big(2\pi f_i x\big)\Big].$

We tune $R$ and $L$ to match the data’s inherent frequency characteristics: $R$ is set high enough to capture rapid variations, while $L$ is chosen to balance detail with computational efficiency. The following ablation experiments were conducted using a baseline SCENT model on dataset S5 with $M=32$ learnable queries, latent dimension $l=128$, and a batch size of 128.

The results show that while the number of frequency bands $L$ has little impact on performance, a higher maximum resolution $R$ clearly improves it. Values were not explored beyond the Nyquist criterion (i.e., $R=32$).

$\$

2. SCENT’s sparse attention mechanism

We appreciate the reviewer's insightful question. Since scientific data typically appear as smooth and continuous signals, attending to all tokens may become redundant and inefficient for high sampling rates. In our approach, we employ random sparse attention for both the Context Embedding Network and Calibration Network (see Fig. 2), where each token attends to a random subset of $p$ tokens, reducing the complexity from $O(n^2)$ to $O(pn)$ with $p \ll n$. This simple random sparse attention mechanism has demonstrated strong empirical performance. We also acknowledge the extensive literature on efficient attention mechanisms – such as Longformer and Linformer – which may work equally well; indeed, additional experiments with these methods (with roughly matched parameter sizes for fair comparison) reveal that the Longformer (replacing sparse attentions within SCENT) outperforms the others, suggesting promising avenues for future research.

$\$

3. Additional real-world datasets

Thank you for raising concerns over limited real dataset employed. Here we report explore additional spatiotemporal datasets.

(i) Rainfall Nowcasting: Using DWD radar rainfall data, SCENT nowcasts lead times from 5 to 60 minutes based on four consecutive 5-minute intervals. Compared to the CNN-based RainNet baseline, SCENT reduces the root-mean-squared error (RMSE) by approximately 30%. The table summarizes the performance metrics (RMSE in mm$~h^{-1}$) and the number of parameters for each method.

(ii) Kuroshio Path Prediction: Using 50 years of CORA [4] reanalysis data, SCENT predicts the Kuroshio current's latitude over a 120-day horizon (with data from 1958–1997 for training and 1998–2007 for testing). SCENT demonstrates stable performance beyond a 50-day lead time. Table below shows that SCENT achieves lower RMSE (in degrees) compared to an LSTM baseline, particularly for longer forecast horizons.

We hope our responses have addressed your key concerns and would be grateful if you would consider a higher score in light of these clarifications.

[1] Wang, S, et al. "Linformer: Self-attention with linear complexity", arXiv 2020.

[2] Beltagy, I, et al. "Longformer: The long-document transformer", arXiv 2020.

[3] Ayzel, G, et al. "RainNet v1.0: a convolutional neural network for radar-based precipitation nowcasting", GMD (2020).

[4] Han, G. et al. "A new version of regional ocean reanalysis for coastal waters of China and adjacent seas", Adv. Atmos. Sci. (2013).

We appreciate the reviewer's suggestion. We agree that the Kuroshio current provides an excellent yet challenging testbed for evaluating SCENT. We use 50‐year records from the China Ocean Reanalysis (CORA) [1] as our benchmark and follow the data processing guidelines established by Wu et al. (2023) [2].

Background. The Kuroshio current, originating from the North Equatorial Current (NEC) and flowing northward along the eastern side of the Philippine Islands, is the world's second-largest warm current. Accurately predicting its path is crucial because its variations significantly affect the exchange of water masses and heat between the North Pacific subtropical and subarctic circulations. CORA provides daily oceanographic reanalysis data for the Kuroshio current spanning 50 years (January 1958-December 2007).

Implementation. We adopt the four baseline methods from Wu et al. (2023) for comparison. We perform a 120-day prediction experiment, using data from the first 40 years (1958-1997) for training and the final 10 years (1998-2007) for testing. During training, SCENT is used to predict the Kuroshio path in terms of latitude (ranging from 29°N to 36°N) for various forecast horizons ($t_o$). In the test phase, we measure the root mean-squared error (RMSE, in degrees) against the true latitude for every $t_o \in [1, 120]$ days, and we present comparisons at 10-day intervals.

Results. Comparison analyses are shown below. We categorize baseline methods into those employing feature engineering (FE) and those that are purely neural network–based. LSTM is used as the baseline, and we also compare FE-enhanced LSTM variants, where FE includes empirical orthogonal functions (EOF) and complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN). The CEEMDAN-based variants perform competitively across all lead times, while SCENT exhibits second-best performance at and beyond a 50-day lead time. The relative strength of the FE methods may be due to the limited data scale—with 14,610 instances in the training split and 3,652 in the test split. However, our scalability study (Fig. 4) suggests that SCENT could outperform the FE methods when larger datasets become available. Additionally, SCENT maintains stable performance even as the lead time increases. This is in stark contrast to the other baselines, which experience significantly sharper performance degradation with longer lead times.

Thank you again for your time and feedback. We respectfully ask you to consider a revised score if you feel our responses address your concerns.

[1] Han, G. et al. "A new version of regional ocean reanalysis for coastal waters of China and adjacent seas", Adv. Atmos. Sci. 30, 974–982 (2013).

[2] Wu, X. et al. "Deep Learning–Based Prediction of Kuroshio Path South of Japan", J. Atmos. Oceanic Tech. 40.2 (2023).