Graph Neural Networks for Multivariate Time-Series Forecasting via Learning Hierarchical Spatiotemporal Dependencies

The authors would also like to thank you for your recognition of our work as well as the time and effort you have put into carefully reviewing our paper. In what follows, you can find point-by-point responses to your comments. Please feel free to contact us if there is any further problem.

---

Q1. Novelty of the proposed method.

The novelty of this paper mainly lies in that we proposed a framework for learning dynamic dependencies across different dimensions, i.e., temporal, spatial, and intra-node. Works such as [1] try to learn a fixed attention matrix from data, which is equivalent to executing the same transformation operation at each timestamp after the matrix is learned. However, following the same transformation operation can not fully exploit the dependencies at different levels, as they can all vary with respect to time and location. On the other hand, works such as [3] and DDGCRN (SOTA) integrate the idea of learning dynamic dependencies, but they only consider dependency modeling along the temporal and spatial dimensions. We for the first time proposed a framework to model the dynamic hierarchical spatiotemporal dependencies in an effective and efficient manner.

---

Q2. Complexity.

In the proposed method, we introduce a novel framework that integrates several modules for multivariate time-series forecasting. Although the combination of these modules increases the complexity of the model, we prove in the experiments that such a combination can lead to better prediction performance. Moreover, we can see from Table 2 and Table 8 that the proposed method can achieve a higher prediction accuracy without compromising model scalability. The actual running time of our method is lower than that of DDGCRN. In summary, we can conclude that the proposed model can lead to better model performance without sacrificing time and space complexity.

---

Q3. The considered datasets have at most 3 channels which does not justify modeling intra-channel dependencies with a graph. As shown by the ablation study in Tab. 2, removing the intra-dependency learning module results in a decrease in performance that is not statistically significant by considering the reported standard deviations.

We adopt those widely used datasets to ensure fair comparisons with state-of-the-art works. These cases can be regarded as illustrations to directly prove the effectiveness of learning a hierarchical graph. Besides, the advantage of introducing the Intra-Dependency Learning Module can vary on different datasets. The absence of the IDLM can cause a performance decrease of up to 2.9% compared to the full model. Considering that the overall improvement of the proposed method over the state-of-the-art method DDGCRN is approximately 10-15%, the IDLM contributes to a relatively large portion of this improvement.

---

Q4. Empirical results on the existing traffic datasets are worse than those of much simpler baselines such as [2], which consists of a simple MLP with spatial and temporal embeddings.

MLP-based baselines such as STID [2] provide an efficient way for multivariate time-series modeling and prediction. However, the performance of such methods is inferior to the state-of-the-art GNN-based and Transformer-based methods, such as DDGCRN and PatchTST. In this regard, we further compare the proposed method with STID on the PEMSD4 and PEMSD8 datasets, which have also been used in the original paper. We train the STID model 10 times based on the code released by the author and report the average performance with standard deviations in Table r1. We can see from the results that HSDGNN still outperforms STID in most cases.

Table r1. Comparisons of the performance of HSDGNN and STID on PEMSD4 and PEMSD8 datasets.

---

---

Q5. Several claims are not supported by appropriate evidence.

We have made the following changes to our previous claims and provided references to support the claims.

Original 1: These methods can provide acceptable results depending on circumstances, but they may also introduce a higher degree of uncertainty, which can impact the reliability of predictions.
New 1: However, the future values of the target attribute (e.g. traffic flow) not only depend on its historical records, but also on the influences of other related attributes. Neglecting such influences can significantly lower the prediction accuracy [Han 2021].

References: [1] Han L, Du B, Sun L, et al. Dynamic and multi-faceted spatio-temporal deep learning for traffic speed forecasting[C]//Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining. 2021.

The authors would also like to thank you for your recognition of our work as well as the time and effort you have put into carefully reviewing our paper. In what follows, you can find point-by-point responses to your comments. Please feel free to contact us if there is any further problem.

---

Q1. Several claims appear unsubstantiated, especially in the fourth paragraph of the introduction. References and/or pieces of evidence to support such claims should be provided.

We have made the following changes to our previous claims and provided references to support the claims.

Original 1: These methods can provide acceptable results depending on circumstances, but they may also introduce a higher degree of uncertainty, which can impact the reliability of predictions. New 1: However, the future values of the target attribute (e.g. traffic flow) not only depend on its historical records, but also on the influences of other related attributes. Neglecting such influences can significantly lower the prediction accuracy [Han 2021].

Original 2: Even so, the temporal correlations regarding the changing spatial dependencies are not well-considered, which makes these methods ineffective in learning from dynamic graph topologies. New 2: Although these methods construct a dynamic graph at each timestamp, the temporal correlations among these dynamic graphs are not well-considered, which makes these methods ineffective in learning from dynamic graph topologies.

References: [1] Han L, Du B, Sun L, et al. Dynamic and multi-faceted spatio-temporal deep learning for traffic speed forecasting[C]//Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining. 2021: 547-555

---

Q2. The technical contribution is limited. The proposed method appears as a minor modification of existing works (e.g., Bai et al., 2020, Han et al., 2021, Weng et al., 2023).

The proposed method benefits from these works, however, it is not a simple stack of them. We adopt a similar backbone as that in [Weng, 2023], but on top of that, we introduce the modeling of dependency and dynamics at different levels in a unified framework, especially the modeling of intra-dependences among attributes. As we have demonstrated through the experiment, these designs can improve prediction accuracy by a great margin. Besides, compared to the work [Han 2021] which also considers the effects of additional attributes, our method can maintain the model scalability given a consistent model size regarding the number of variables.

---

Q3. I am not fully convinced by the experimental setting. What is the relevance of learning a hierarchical graph in the considered setting where each location (variable in the paper terminology) has only three components? Fig 9a shows that the graph of attributes is a fully connected one, with all edge weights ~1.

It depends on the datasets. We adopt those widely used datasets to ensure fair comparisons with state-of-the-art works. These cases can be regarded as illustrations to directly prove the effectiveness of learning a hierarchical graph. Besides, we did not normalize the attention matrix such that the values can range from 0 to approximately 3.8 after the training of the model (considering the PEMSD8 dataset). In the case shown in Fig 9, the edge weights corresponding to that specific period are within the range of 0.8-1.3.

---

Q4. Could you clarify what is used to replace the IDLM in HSDGNN_w/o_IDLM and the dynamic graph in HSDGNN_w/o_DG?

For HSDGNN_w/o_IDLM, we exclude the intra-dependency learning process in the original design by using the raw input X as the input for the Temporal-dependency Learning module, i.e., X -> Attribute Fusion F, as shown in Figure 2.

For HSDGNN_w/o_DG, we adopt a consistent graph to model the spatial dependency among different variables. To achieve this, we multiply the Static Node Embedding (M) by its transpose to obtain an attention matrix, which is then broadcast to all timestamps. In other words, we do not consider the dynamic dependencies among multiple variables in this case.

---

Q5. Confusing notation

We have made some modifications in the manuscript to clarify these confusing terms.

The authors would also like to thank you for your recognition of our work as well as the time and effort you have put into carefully reviewing our paper. In what follows, you can find point-by-point responses to your comments. Please feel free to contact us if there is any further problem.

---

Q1. In this paper, only the main attribute (traffic flow) is predicted, and the other attributes serve as auxiliary attributes for the prediction of the main attribute. If only the main attribute is predicted, the other attributes can be considered as providing additional useful information for the prediction of the main attribute. However, this does not necessarily reflect the presence of inherent dependencies between attributes, as each attribute can be treated as the main attribute. Although the ablation experiments suggest that considering only the performance of the main attribute can surpass that of DDGCRN, this only demonstrates the superiority of the model architecture. Therefore, I hope the authors can provide the prediction results and ablation experiments for the other attributes (occupancy, speed).

Thanks for the suggestion. Following this idea, we modify the output layers of both DDGCRN and the proposed HSDGNN to enable them to generate predictions for all the attributes at the same time. We re-train the new models on the PEMSD8 dataset, and calculate the three evaluation metrics (MAE/RMSE/MAPE) for each attribute separately. In this way, we can quantify the influence of intra-dependency modeling when predicting all attributes. The results can be seen in Table 1, where we also include the ablation study results.

Table 1. Prediction results on traffic flow (S), occupancy (O) and speed (S)

---

We can draw several conclusions from the results: 1) Compared to DDGCRN, the proposed method can improve the prediction accuracy of all attributes; 2) The absence of any component will decrease model performance, either regarding the main attribute or the auxiliary attributes; 3) The results also prove the necessity of modeling the inherent dependencies between attributes.

---

Q2. In long-term prediction, is the horizon of input also 12? If the horizon of input is 12, why are the values of PEMSD8 in Figure7 and Figure6 slightly different under the first 12 timestamps? If the horizon of input is not 12, can author indicate horizon of input of HSDGNN and DDGCRN for long-term prediction performance in the paper?

The horizon of the input is also 12 for long-term prediction. The results are slightly different because we have to modify the size of the output layer of both DDGCRN and HSDGNN to generate 72 steps of predictions. In other words, we re-train two new models for this task, so the results can be slightly different.

---

Q3. I am not sure whether the horizon of input of DDGCRN and HSDGNN is the same in long-term prediction. If not, can the author modify the experiment to compare the performance of HSDGNN and DDGCRN under the same horizon of input?

We adopt the same input length of 12 for DDGCRN and HSDGNN when performing long-term prediction.

The authors would also like to thank you for your recognition of our work as well as the time and effort you have put into carefully reviewing our paper. In what follows, you can find point-by-point responses to your comments. Please feel free to contact us if there is any further problem.

---

Q1.The contribution of this paper is somewhat incremental. The proposed method is kind of simple combination of existing methods.

The proposed method benefits from these works, however, it is not a simple stack of them. We adopt a similar backbone as that in [1], but on top of that, we introduce the modeling of dependency and dynamics at different levels in a unified framework, especially the modeling of intra-dependences among attributes. As we have demonstrated through the experiment, these designs can improve prediction accuracy by a great margin. Besides, compared to the work [2] which also considers the effects of additional attributes, our method can maintain the model scalability given a consistent model size regarding the number of variables.

References [1] Weng W, Fan J, Wu H, et al. A Decomposition Dynamic graph convolutional recurrent network for traffic forecasting[J]. Pattern Recognition, 2023, 142: 109670. [2] Han L, Du B, Sun L, et al. Dynamic and multi-faceted spatio-temporal deep learning for traffic speed forecasting[C]//Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining. 2021: 547-555.

---

Q2. According to the results of ablation study (Table 3), it seems that IDLM (the intra graph) is less effective. The performance of the full model HSDGNN and the ablated version HSDGNN_w/o_IDLM does not have a significant difference.

The advantage of introducing the Intra-Dependency Learning Module can vary on different datasets. The absence of the IDLM can cause a performance decrease of up to 2.9% compared to the full model. Considering that the overall improvement of the proposed method over the state-of-the-art method DDGCRN is approximately 10-15%, the IDLM contributes to a relatively large portion of this improvement.

---

Q3. How will your model perform if you simply replace IDML with some other simple static graphs (e.g., the graph constructed by Pearson Correlation or other prior knowledge)?

The IDLM was designed to capture time-varying dependencies among different attributes of a variable. To further demonstrate the necessity of IDLM, we replace it with a static graph which is learned from data for comparison. Specifically, we initialize the intra-dependency values among attributes by a 3*3 random matrix, which is also trained in an end-to-end manner. After training, we can obtain the static graph that reflects the fixed correlations among attributes. We train the new model (HSDGNN_w_s) on each dataset 10 times and report the average results and deviations in Table r1.

Table r1. Comparisons between HSDGNN and HSDGNN_w_s on all datasets

---

We can see from the results that HSDGNN_w_s performs worse than the original HSDGNN method in all cases. It proves that the intra-dependencies among attributes also vary with time, and our model can benefit from modeling such dynamic dependencies using IDLM.