Spatiotemporal Backward Inconsistency Learning Gives STGNNs Icing on the Cake

(1) ' Adding just an inconsistency loss should not bring any new information for such impressive performance gains'

In our method, we did not add any additional losses except the regression loss.

(2). ‘Those gains should be achievable through traditional architectures as well’

Unfortunately, even if an STGNN is designed with a more sophisticated structure, they do not emphasize modeling the inconsistency between input and labels. STBIM, on the other hand, addresses this issue and achieves performance improvements by focusing on it. Through extensive experimental cases (approximately 15 spatiotemporal prediction models), we demonstrate that STGNNs can benefit from STBIM to gain additional potential performance improvements, with performance increases of up to 18.74%, even for advanced models. As mentioned earlier, this gain cannot be achieved through simply expanding the parameter scale.

(3) ' Conducting an ablation study comparing STBIM to equivalent increases in model complexity for traditional architectures. ’

Increasing model parameters and computational complexity is not equivalent to the benefits brought by STBIM, as discussed above.

(4) What information traditional architectures may miss or No additional information?

This is where our design gets magical.

These traditional architectures cannot effectively model label features, leading to confused predictions for samples with inconsistent input labels, as discussed in our introduction. This is where our clever design comes in - we noticed this key information about label representation, which contains abundant table features, enabling the model to explicitly utilize label features during training to enhance learning from inconsistent samples without additional information, thereby improving the accuracy of the inference process. We validated this motivation through extensive experiments in the Experiment Section 5.3 and Appendix Section B.6 of the manuscript.

Dear Reviewer tHSS,

Thank you very much for your valuable comments; we will respond to your concerns point by point.

(1) Simply expanding the parameters cannot achieve the same gains as STBIM.

We have completely refuted this point in the original manuscript, involving the line 451 in Experiment Section 5.1. Subsequently, we have demonstrated this point in Appendix B.7.2 in detail. Taking the spatiotemporal backbones STID and D2STGNN as examples, we introduced variants: STID-Plus and D2STGNN-Plus, which stacked two backbones to expand the parameter scale, while also increasing the maximum training epochs. The experimental results are presented in Table 14 of the manuscript, which we have replicated below for convenience.

We can observe that simply increasing the parameter size does not lead to performance enhancement, For the complex model D2STGNN, simply stacking models may actually decrease model performance, as an excessively large parameter size can lead to overftting of the model to the data. Our performance improvement originates from the effective modeling of inconsistent features.

Please refer to the experimental analysis section of the main body and Appendix B.7.1 in the submitted manuscript, we have discussed this when we submitted the paper.

---

"The method for measuring the inconsistency between the input data and labels seems redundant?"

We argue that the method of modeling the inconsistency between input data and labels is not redundant ! On the contrary, it is a crucial factor contributing to our performance. In Experiment Section 5.3 and Appendix Section B.6 of the manuscript, we evaluated the performance of various spatiotemporal models on spatiotemporal inconsistency samples. Clearly, STBIM can effectively enhance these models to better handle such inconsistencies. For example, as shown in Table 4, taking STID as an example, STBIM can improve its performance on handling inconsistent samples by 24.05%, significantly boosting the predictive performance of STID.

Q1. Computational complexity.

Please refer to Weakness 3.

Q2. how much of the above performance can be recovered by using additional spatio-temporal layers?

Simply expanding the parameters cannot achieve the same gains as STBIM.

We have completely refuted this point in the original manuscript, involving the line 451 in Experiment Section 5.1. Subsequently, we have demonstrated this point in Appendix B.7.2 in detail. Taking the spatiotemporal backbones STID and D2STGNN as examples, we introduced variants: STID-Plus and D2STGNN-Plus, which stacked two backbones to expand the parameter scale, while also increasing the maximum training epochs. The experimental results are presented in Table 14 of the manuscript, which we have replicated below for convenience.

We can observe that simply increasing the parameter size does not lead to performance enhancement, For the complex model D2STGNN, simply stacking models may actually decrease model performance, as an excessively large parameter size can lead to overftting of the model to the data. Our performance improvement originates from the effective modeling of inconsistent features.

Q3. How will the test-time labels be used if available?

According to the general paradigm of machine learning, labels are not accessible during testing. While labels are available during training, they cannot be directly used as inputs to the model. Typically, we optimize the model by utilizing backpropagation with label information loss.

Q4. What extra information is the inconsistency loss bringing/recovering from the input data apart from just maintaining a spatial smoothness？

Our inconsistency modeling (not a loss function) can incorporate label features into the model without using additional inputs, improving the model's prediction performance on inconsistent samples. In addition to maintaining spatial smoothness, it also preserves temporal smoothness.

We evaluated the effectiveness of STBIM in helping models to eliminate inconsistency between input and label dimensions in the temporal dimension in Section 5.3 and Appendix B.5, thereby ensuring temporal smoothness of the backbone, and the results prove that our proposed module STBIM comprehensively deals with the inconsistencies of temporal and spatial dimensions.

We have reported the computational overhead introduced by STBIM in Appendix B.7 (as indicated in the main manuscript at line 347). In the paper, we present the parameter scale, training time, and performance improvements achieved. To address your concerns further, we additionally report the computational complexity on CA dataset with 8600 nodes to demonstrate the scalability of STBIM on the large-scale scenario. For your convenience, we have replicated the results for the CA dataset in the table below as an example:

Our analysis shows that STBIM achieves significant performance gains with minimal computational complexity. Fine-tuning training methods involve less computational time by directly fine-tuning pre-trained STNN with STBIM. Both approaches balance performance and efficiency, offering flexibility in selection. In conclusion, given the notable performance enhancement, the complexity burden introduced by STBIM is considered acceptable. Please note that we have analyzed that traditional architectures cannot simply directly increase the computational complexity to achieve similar performance gains.

Q1. The paper introduces the unique concept of "input-label inconsistencies" in spatiotemporal prediction. Could you clarify how this concept differs from other types of spatiotemporal inconsistency, such as spatiotemporal out-of-distribution (OOD) or distribution shift? Please refer to W1.

Q2. The current claim regarding differences between input and label features (Line 269) seems ambiguous. Please refer to W2.

Q3. Similar input data resulting in different labels.

As emphasized in the introduction, this dilemma is a critical challenge faced by traditional spatiotemporal learning models. In cases where two nodes exhibit similar input distributions but have different labels, the model struggles to effectively differentiate between these nodes, thus resulting in similar predictions for both and leading to substantial errors between predictions and labels. This observation serves as our motivation: to explicitly model label features in order to enhance the ability of spatiotemporal learning models to address samples with similar inputs but differing labels, as well as samples with different inputs but similar labels. We will revise the corresponding content in the introduction to further emphasize this motivation.

Q4. What is STRIP in Figure 4(b)?

Sorry for the error, it should be corrected to STBIM here. And we have corrected this in the revised version.

Q5. Computational stduy

We have reported the computational overhead introduced by STBIM in Appendix B.7 (as indicated in the main manuscript at line 347) when we submitted the paper. To address your further concerns, we additionally report the computational complexity on LargeST-CA dataset with 8600 nodes to demonstrate the scalability of STBIM on the large-scale scenario. For your convenience, we have replicated the results for the LargeST-CA dataset in the table below as an example:

Our analysis shows that STBIM achieves significant performance gains with minimal computational complexity. Fine-tuning training methods involve less computational time by directly fine-tuning pre-trained STNN with STBIM. Both approaches balance performance and efficiency, offering flexibility in selection. In conclusion, given the notable performance enhancement, the complexity burden introduced by STBIM is considered acceptable. Please note that in Appendix B.7.2, we also analyzed that traditional architectures cannot simply directly increase the computational complexity to achieve similar performance gains.

Dear Reviewer FGB6,

Thank you very much for your valuable review; it is crucial for improving the quality of our manuscript.

W1. Discussion of related works addressing spatiotemporal inconsistency

The inconsistency we define differs fundamentally from the concept of shift in spatiotemporal out-of-distribution (OOD) learning. The latter refers to the overall differences in data distribution between training and testing datasets, which can lead to OOD challenges. In contrast, our focus is on a more granular aspect: the inconsistency between input values and future values. This inconsistency can occur not only in OOD scenarios but also in independent and identically distributed (IID) situations.

To further assess the impact of the proposed Spatiotemporal Bidirectional Interpolation Model (STBIM), we investigate whether spatiotemporal shift learning models can benefit from its implementation. Following the OOD setting in the open-source code of STONE [1], we use their STSD dataset as example and select two spatiotemporal OOD learning models as backbones, and STBIM is integrated with joint-training manner. The results are as follows:

We find that STBIM can also improve the predictive performance of spatiotemporal shift learning models because it effectively enhances the model's accuracy on samples with historical-future inconsistency in OOD scenario. Following your suggestion, we have deleted the introduction about "Label propagation in GNN” and included this discussion in Section Related Work in the revised version.

---

W2. The details of spatiotemporal residual theory.

Yes, if the labels are available, the difference between the input and the label is used as residual input to the model for learning. However, this assumption is never true because, according to the general paradigm of machine learning, labels cannot be directly used as model inputs for regression prediction tasks. Therefore, we propose an effective solution using label hidden embeddings, and these high-dimensional representations contain richer feature information to comprehensively learn the residual information.

Equation 8 in the paper defines the residual as the difference between the predicted expectation and the label expectation. To analyze the internal patterns of spatiotemporal data, we employ the Gaussian Markov Random Field (GMRF) model. In this framework, as discussed in lines 268 to 276 of our manuscript, the prediction expectation is determined by the high-dimensional representation of the input, which arises from the spatiotemporal learning model's abstraction of input features, as illustrated in Equation 6. Conversely, the label expectation is governed by its intrinsic characteristics. Thus, at a fundamental level, the difference between these two expectations approximates the disparity between input features and label features. We will polish this discussion for clarity.

---

W3.Ablation study

Sorry for the confusion. We conduct ablation experiments on the Large-SD dataset using the STGCN. We created two variants: "w/o MLP" means we remove the retrospect MLP, and "w/o kernel" means that we remove the propagation kernel for smoothing. The experimental results are as follows:

"w/o MLP" showed significantly higher errors, as this retrospect MLP is used to map label features to the same hidden space as input features, leading to smoother training. The experiment without the kernel resulted in poor prediction performance because residual smoothing benefits the model's learning process. We would include the ablation experiments with more backbones in the revised version.

Q1 The third subplot in Figure 1 is not mentioned in the text, and the "abnormal signal" it shows is not referenced in the document. Is it necessary to keep it? What is its significance?

We apologize for any confusion. Abnormal signals are defined as samples in which traffic flow experiences a rapid increase or decrease, highlighting a significant example of inconsistencies in the temporal dimension between input and future sequences. To elucidate this point, we would revise the original text to: "As illustrated in Figure 1 (c), inconsistencies in temporal features exist between historical and future values. Typical examples of this phenomenon include abnormal signals characterized by a rapid increase or decrease in traffic flow."

---

Q2 The text highlights the inconsistency between inputs and outputs, using the prediction label to represent the time series of a later time window. Can this be expressed more clearly? If a label is used, could the specific cases be clarified? Typically, a label is a distinct value, not a continuous range of floating-point numbers.

Thank you very much for your suggestion. In this paper, we use 'label' to represent the time series for future time windows. To avoid further confusion, we will replace "input" and "label" with "historical value" and "future value," respectively. Consequently, the term "input-label inconsistency" will be revised to "historical-future inconsistency." To prevent other reviewers from being confused by the renaming of key concepts, we promise that this change will be consistently reflected in the revised version of the manuscript.

---

Q3. If all results are accurate, why do many fine-tuned results exceed those of joint tuning? Can this be analyzed further?

The fine-tuning method entails adjusting the STBIM and the pre-trained backbone, which serves to establish a robust optimization starting point while simplifying the optimization process. This approach is beneficial for complex models like DGCRN and DDGCRN. As a result, it can outperform joint training methods in specific scenarios. Nevertheless, in the majority of cases, joint training—where STBIM and the backbone are trained simultaneously—facilitates more flexible tuning and adaptation to the task.

Dear Reviewer RjnU,

Thank you very much for your valuable review; it is crucial for improving the quality of our manuscript.

• W1 Reproducibility

We sincerely apologize for the oversight in reporting the performance of BigST, and the actual average Mean Absolute Error (MAE) performance of BigST, which was mistakenly recorded as 24.12, is accurately represented as 21.12. This error have occurred due to the proximity of the digits "4" and "1" on the numeric keypad of the Chinese (US) standard keyboard layout😭, which is illustrated below.

Below, we present the average performance metrics for BigST over 12 time steps. The table above contains the incorrectly reported performance values, while the table below provides the corrected figures.

The corrected results continue to illustrate the effectiveness of STBIM, as evidenced by a 15.50% improvement in the average MAPE performance of BigST. To address any further concerns, we will provide access to all training logs for the models via an anonymous code link for your review. We sincerely apologize for the error caused by our oversight.

---

• W2 Shift learning.

The inconsistency we define differs fundamentally from the concept of shift in spatiotemporal out-of-distribution (OOD) learning. The latter refers to the overall differences in data distribution between training and testing datasets, which can lead to OOD challenges. In contrast, our focus is on a more granular aspect: the inconsistency between input values and future values. This inconsistency can occur not only in OOD scenarios but also in independent and identically distributed (IID) situations.

To further assess the impact of the proposed Spatiotemporal Bidirectional Interpolation Model (STBIM), we investigate whether spatiotemporal shift learning models can benefit from its implementation. Following the OOD setting in the open-source code of STONE [1], we use their STSD dataset as example and select two spatiotemporal OOD learning models as backbones, and STBIM is integrated with joint-training manner. The results are as follows:

We find that STBIM can also improve the predictive performance of spatiotemporal shift learning models because it effectively enhances the model's accuracy on samples with historical-future inconsistency in OOD scenario. We w included the discussion of spatiotemporal shift learning in Section Related Work of the revised version.

---

• W3 Writing quality.

Thank you very much for your suggestion. We have enlisted the assistance of advanced language models skilled in language refinement, as well as collaborators who are native English speakers, to thoroughly proofread the manuscript and correct any grammatical errors. We appreciate your guidance in enhancing the quality of our work.