Learning with Calibration: Exploring Test-Time Computing of Spatio-Temporal Forecasting

Dear Reviewer T8C2,

We are very sorry for the slight delay in our response due to a fire at our server room yesterday.

We sincerely thank you for taking the time and effort to carefully review our work. We are honored by your recognition of our hard work and have meticulously considered each of your comments, addressing them one by one.

---

Method-related

• Comparison with Other OOD Solutions:
  • We apologize for any misunderstanding regarding our ST-TTC method. Our approach is designed as a general plug-in and is orthogonal to existing OOD methods, including continual (CT) learning approaches. This means our method doesn't compete with them; rather, it can be applied on top of these models to further enhance their performance. Our goal is to maintain the inherent ability of these models to handle spatio-temporal shifts while simultaneously improving their generalization to new distribution shifts during actual deployment.
• The absolute improvement does not seem significant enough:
  • We agree that our method may not show significant improvements on small-scale spatio-temporal benchmarks. We clearly acknowledged this limitation in Appendix F.1. We must point out that the test sets for these smaller benchmarks are typically too short (spanning only days or weeks) compared to larger datasets or continual learning benchmarks (which can span months or years). This limited duration often doesn't lead to severe out-of-distribution shifts, which our method is designed to address. However, our method still provides a certain level of performance improvement, and this gain can be combined with other complex model designs to further enhance overall performance.
  • Furthermore, in scenarios with severe spatio-temporal distribution shifts (e.g., OOD and continual spatio-temporal forecasting), the absolute improvements from our method are quite significant. For instance, we achieved an absolute MAPE improvement of around 10% in both OOD and continual learning settings.

---

Experiments-related

• Effectiveness on Video Prediction Tasks?:
  • Good question! Video prediction can be considered a special form of spatio-temporal forecasting on a grid (i.e., Euclidean vs. non-Euclidean spatial structures). However, the research community typically treats these two tasks separately. Video prediction models often use vision backbones to capture inductive biases like translation invariance, while spatio-temporal forecasting models use graph backbones to capture locality and permutation invariance. From a data perspective, there is some overlap, as traffic and meteorological data can be represented as grids similar to video inputs.
  • To test the effectiveness of our method on this type of data, we selected two common datasets for such studies, NYC-Taxi and T-Drive, and their corresponding spatio-temporal backbone model, PDFormer [1]. The results are as follows:

• Handling Other OOD Challenges (input data corruption, events/corner cases) ?:
  • We are happy to clarify this point. Our core objective is to focus on calibrating gradual, systematic periodic shifts while avoiding the fitting of anomalous noise from data corruption or abrupt structural changes from extreme events. Recall that these kinds of anomalous changes are typically accidental, temporary, and non-repeatable. If our calibrator tries to adapt to past anomalies and use them to calibrate current forecasts, this will lead to negative effects. Our proposed phase-amplitude modulation is specifically designed to prevent this exact situation.

---

References:

[1] PDFormer: Propagation Delay-Aware Dynamic Long-Range Transformer for Traffic Flow Prediction. AAAI, 2023.

---

Thank you again for your valuable feedback. We will appropriately incorporate these detailed discussions into the final revised version. We are grateful for your guidance and suggestions!

Best regards,

Authors.

Dear Reviewer xiA6,

We are very sorry for the slight delay in our response due to a fire at our server room yesterday.

Thank you for your hard work and valuable feedback. We understand your concerns and believe they stem from unnecessary misunderstandings. We are happy to clarify and address each of your questions.

---

Method-related

• Generalizability to Non-Periodic or Abrupt Structural Changes:
  • We are glad to clarify this point. Our core objective is to focus on calibrating gradual, systematic periodic shifts while avoiding the fitting of anomalous noise and abrupt structural changes. Recall that these kinds of anomalous changes are typically accidental, temporary, and non-repeatable. If our calibrator tries to adapt to past anomalies and use them to calibrate current forecasts, this will lead to negative effects. Our proposed phase-amplitude modulation is specifically designed to prevent this exact situation.

---

Experiments-related

• Formalization of OOD Scenarios:
  • We apologize for this misunderstanding. Our method is designed to be orthogonal to existing OOD methods, meaning we can achieve significant performance gains on top of their results. Regarding the formal definition of ST-OOD, we actually provided a general formal definition in Table 1, which is widely used and consistent with the ST-OOD literature [1, 2, 3].
• Quantifying Distribution Shifts:
  • We followed the setup of previous ST-OOD experiments [1] to simulate a significant distribution shift along the temporal dimension. Specifically, we used traffic flow data from 1 to 8, 2019 (spring and summer) as the training set and traffic flow from 11 to 12, 2020 (winter) as the test set. This setup exacerbates the temporal distribution shift across both years and seasons. Due to the limitations of the rebuttal phase, we can’t provide visual quantifications of the distribution, but metrics for this can be found in the LargeST paper [4]. We appreciate your suggestion and will consider adding more visualizations to quantify this shift in the revised version.
• Adaptability to Structural Changes:
  • Our method is explicitly designed to handle changes in graph structure. We addressed this issue on line 139, where we mentioned our use of a spatial-aware decomposition scheme to create node-level parameter calibrators. This allows our method to handle changes in graph topology by selectively loading parameters. When new nodes are added, we simply supplement the corresponding node-level calibration parameters. When nodes are removed, we don't load their parameters.
  • Furthermore, since our method is orthogonal to existing OOD models, we can preserve their inherent ability to handle topological changes while simultaneously improving the model's generalization capabilities to unknown topologies.
  • We evaluated performance under varying degrees of topology changes (10%, 15%, 20% node changes) on the test set in Table 3. We also specifically evaluated performance on new nodes in the test graph, as shown in Table 8, demonstrating that we significantly improve generalization on top of existing OOD models. To save you time, here is a summary of the results from Table 8:

• Support for Dynamic Graphs:
  • Our method is also designed to adapt to dynamic graph structures that evolve over time. This is consistent with the experimental settings of continual spatio-temporal learning [5]. In Section 5.4, we specifically investigated the performance improvements of our method on existing continual ST learning approaches, as detailed in Table 4. Our method effectively adapts to ST forecasting on dynamic graphs that evolve over time, even across periods as long as seven years.

---

References:

[1] STONE: A Spatio-temporal OOD Learning Framework Kills Both Spatial and Temporal Shifts. KDD, 2024.

[2] Robust Spatio-Temporal Centralized Interaction for OOD Learning. ICML, 2025.

[3] Zhang, et al. STRAP: Spatio-Temporal Pattern Retrieval for Out-of-Distribution Generalization. arXiv, 2025.

[4] LargeST: A Benchmark Dataset for Large-Scale Traffic Forecasting. NeurIPS, 2023.

[5] Expand and Compress: Exploring Tuning Principles for Continual Spatio-Temporal Graph Forecasting. ICLR, 2025.

---

We believe that your concerns may stem from a potential misunderstanding of our method. We apologize for any confusion and will revise the details in the appendix more carefully. We appreciate your guidance and suggestions and welcome further discussions!

Best regards,

Authors.

Dear Reviewer eeBu,

We sincerely appreciate the time and effort you dedicated to providing insightful feedback on our manuscript. We are honored that you recognized our hard work. We have carefully considered each of your comments and addressed them point by point, categorized by different type.

---

A. Method-related

• A.1: Initialization of Offset Parameter:
  • We acknowledge the lack of explicit detail regarding the initialization of the offset parameter in the main text. As you correctly pointed out, in Algorithm on page 21, both $\lambda_\alpha$ and $\lambda_\phi$ are clearly initialized to 0. The underlying motivation for this is to unify the code design of Algorithm 3, thereby preventing erroneous calibration of predictions before the calibration parameters are learned (i.e., during a very short warm-up period before acquiring the first sample-label pair). We apologize for any potential confusion and will incorporate these details into the main text.
• A.2: Distinction between Long-Term Time Series and Spatio-Temporal Forecasting
  • As stated in the paper, our work adheres to the common settings of previous short-term and long-term spatio-temporal forecasting studies (e.g., 12 → 12, 24 → 24) [1, 2]. However, we are fully aware of the 96 → 96~720 settings prevalent in current long-term time series forecasting. We wish to share our insights regarding this:
    • There has been a long-standing debate concerning the long-term predictability of time series [3], which we do not intend to overly critique here. Nevertheless, it is important to note that in spatio-temporal forecasting, specifically in traffic flow theory research, previous studies [4] utilizing residual analysis have indicated that, after minimizing periodicity, the correlation between traffic data beyond one hour and the past hour's observations is significantly limited for most sensors. Furthermore, typical traffic sensor data collection frequency is approximately 5 minutes. Therefore, for practical traffic forecasting and decision-making scenarios, which usually focus on the next 1-2 hours (i.e., max 24 steps), our settings are more aligned with real-world applications.
    • Given that spatio-temporal forecasting can be considered a complex extension of time series forecasting, the additional dimension of sensor count (up to hundreds or thousands) results in an order of magnitude higher data training cost. This is a secondary reason why existing spatio-temporal forecasting often considers 12-step settings.
    • Despite this, we have included an experimental performance comparison for 96 → 96 prediction on the PeMS08 dataset, using STID as the backbone network, as shown below:
    • The results show that our approach can also achieve certain performance improvements in long-term forecasting. We sincerely hope that these insights can help you better correct misunderstandings.

---

B. Presentation-related

• B.1: Figure 3 (Left) Visualization:
  • You are correct that a bar chart is more appropriate for Figure 3 (Left). We apologize for this oversight; the revised version will feature a bar chart.
• B.2: Discussion of Limitations in main text:
  • We will adopt your suggestion and briefly outline potential limitations in the conclusion section.
• B.3: Typo Errors in Line 336 and Figure 1b:
  • Thank you for your meticulous review. We will conduct a thorough check for all potential typo errors, and rectify them accordingly.

---

C. Experiments-related

• C.1: The statement "fewer groups generally lead to poorer performance" on line 351 caused confusion. Figure 7 (middle) has a constant behavior similar to Figure 7 (right).
  • We apologize for the confusion caused. This visual discrepancy is due to the differing metric scales between Figure 7 (Middle) and Figure 7 (Right). In actuality, the significant changes in learning rate overshadowed the impact of varying group numbers. However, performance indeed deteriorates as the number of groups decreases. Below we provide a specific example with a learning rate of 1e-3. We will avoid the above misunderstandings in the revised version.

• C.2: Statistical Significance Testing
  • Thank you for your rigorousness. We plan to include statistical significance test results in the revised version.

---

D. Theoretical-related

• D.1: Theorem 1 should be stated as an approximate result and is, strictly speaking, sub-linear.
  • We appreciate your precision. We will add more detailed supplements and discussions in the revised version.

---

References:

[1] Zezhi Shao, et al. Exploring Progress in Multivariate Time Series Forecasting: Comprehensive Benchmarking and Heterogeneity Analysis. TKDE, 2024.

[2] Wei Shao, et al. Long-term spatio-temporal forecasting via dynamic multiple-graph attention. IJCAI, 2022.

[3] Christoph Bergmeir. Fundamental limitations of foundational forecasting models. NeurIPS Workshop InvTalk, 2024.

[4] Lorenzo Brigato, et al. Position: There are no Champions in Long-Term Time Series Forecasting. arXiv, 2025.

[5] Vincent Zhihao Zheng, et al. Probabilistic Traffic Forecasting with Dynamic Regression. Transportation Science, 2025.

---

Thank you for your invaluable feedback. We will appropriately incorporate these detailed discussions into the final revised version. Once again, we are grateful for your insightful guidance!

Best regards,

Authors.

Dear Reviewer N98K,

We are very sorry for the slight delay in our response due to a fire at our server room yesterday.

We sincerely thank you for taking the time and effort to carefully review our work. We are honored by your recognition of our hard work and have meticulously considered each of your comments, addressing them one by one.

---

Method-related

• Performance on Small-Scale Spatio-Temporal Benchmarks:
  • We agree with your assessment. As we clearly acknowledged in the limitations section (Appendix F.1), our method's effectiveness is constrained on small benchmarks. We also outlined a potential future direction: enhancing performance on these smaller datasets by incorporating our method with retrieval-augmented models for arbitrary exogenous data or variables.
  • However, we must also point out that the test sets of small benchmarks [1] are often too short (spanning only days or weeks) compared to those of large-scale [2,3] or continual learning benchmarks [4] (which can span months or years). This limited span does not typically lead to severe out-of-distribution shifts, which our method is designed to address. Nevertheless, we still achieved a certain performance improvement. This gain, though modest, can be combined with other complex model designs to further enhance overall performance.
  • Furthermore, in scenarios with severe spatio-temporal distribution shifts (e.g., OOD and continual spatio-temporal forecasting), the absolute improvements from our method are quite significant. For instance, we achieved an absolute MAPE improvement of around 10% in both OOD and continual learning settings.

---

Presentation-related

• Lack of Method Overview Diagram:
  • Good suggestion! We agree with your point. Due to the limitations of the rebuttal period, we are unable to upload images. We plan to incorporate a visual representation of our method into the final revised version.

---

Experiments-related

• Lack of Testing on Spatio-Temporal Foundation Models:
  • We clearly discussed our thoughts on this issue in the limitations section (Appendix F.1). However, we also agree with your feedback and have supplemented our experiments with the advanced ST-foundation model, Opencity [5]. We used the official OpenCity-mini weights and tested it on the PEMS04, PEMS07M, and PEMS08 datasets. The results are shown in the following table. Although the improvements are limited (a fact we attribute to the restricted time span of the provided benchmarks), we still achieved a noticeable gain.

---

References:

[1] Spatial-Temporal Synchronous Graph Convolutional Networks: A New Framework for Spatial-Temporal Network Data Forecasting. AAAI, 2020.

[2] LargeST: A Benchmark Dataset for Large-Scale Traffic Forecasting. NeurIPS, 2023.

[3] Efficient Large-Scale Traffic Forecasting with Transformers: A Spatial Data Management Perspective. KDD, 2025.

[4] TrafficStream: A Streaming Traffic Flow Forecasting Framework Based on Graph Neural Networks and Continual Learning. IJCAI, 2021

[5] Opencity: Open spatio-temporal foundation models for traffic prediction. ArXiv, 2024.

---

Thank you again for your valuable feedback. We will appropriately incorporate these detailed discussions into the final revised version. We are humbled and grateful for your recognition of our work!

Best regards,

Authors.