$\triangleright$Differences with FourierGNN:

Although both are pure graph methods, Graph Neural Networks (GNNs) are a very broad concept. Being graph-based does not make our work incremental.

1. Dependency Modeling Approach: FourierGNN represents a trend toward modeling three kinds of dependencies within a unified framework, as shown in Figure 2 on Page 2. GraphSTAGE, however, focuses on decoupling and modeling two types of dependencies.

2. Research Focus: The drawbacks of using hypervariate graphs to unify multi-dependency modeling have not been studied in previous research. There are two main drawbacks: High Memory Consumption: This issue has also been reported in other studies [4]. Potential Noise: Modeling extensive cross-dependencies may lead the model to learn many pseudo connections (as we explained in our response to Q1). These pseudo connections can weaken the influence of genuine connections (inter-series and intra-series dependencies), potentially diminishing the model's predictive performance.

   Additionally, instead of focusing solely on FourierGNN, we aimed to broadly address hypervariate graph-based multi-dependency modeling methods. In the Variants Comparison section, we included a variant called VarC (shown in Figure 7 on Page 9) to encompass these methods and compared it with our channel-preserving GraphSTAGE. Our findings indicate that modeling only two types of dependencies can achieve excellent predictive performance while also reducing memory usage.

---

$\triangleright$Addition of Graph-Based Baselines:

To provide a more direct and comprehensive comparison with FourierGNN and other graph-based models, we have included four additional graph-based baselines in the revised paper. The detailed results are presented in Table 9 on Page 18 of the revised paper. For your convenience, we provide a summary of the relevant averaged results below:

The results indicate that GraphSTAGE achieved the top-1 rank in terms of MSE and MAE metrics across four datasets compared with four graph-based models. Although CrossGNN performs comparably to our GraphSTAGE on smaller datasets (e.g., ETT and Weather), GraphSTAGE notably outperforms CrossGNN on large-scale datasets (e.g., ECL with 321 nodes). These findings further validate the effectiveness of GraphSTAGE in capturing both intra-series and inter-series dependencies, leading to superior forecasting accuracy across datasets of varying scales.

We believe these clarifications address your concerns regarding the novelty and contributions of our work. Current models primarily focus on the advantages of channel-mixing methods for extracting multiple dependencies, often neglecting the noise these approaches can introduce. We are the first to directly address this issue. Thank you again for your valuable feedback, which has helped us improve our paper.

---

[1] iTransformer: Inverted Transformers Are Effective for Time Series Forecasting. ICLR, 2024

[2] FourierGNN: Rethinking multivariate time series forecasting from a pure graph perspective. NeurIPS, 2024

[3] UniTST: Effectively Modeling Inter-Series and Intra-Series Dependencies for Multivariate Time Series Forecasting. arXiv

[4] ForecastGrapher: Redefining Multivariate Time Series Forecasting with Graph Neural Networks. arXiv

[5] Crossgnn: Confronting noisy multivariate time series via cross interaction refinement. NeurIPS, 2023

[6] Spectral temporal graph neural network for multivariate time-series forecasting. NeurIPS, 2020

[7] Connecting the dots: Multivariate time series forecasting with graph neural networks. KDD, 2020

Response to Reviewer RvJQ (1/1)

W1 ＆ Q1 : Incremental idea & Lack of novelty: I think the core idea of decoupling the learning of intra-series and inter-series instead of channel mixing is incremental and not significant.

Thank you for pointing out this. We would like to highlight the unique aspects of our approach and clarify the distinctions between our work and the recent models (iTransformer [1] and FourierGNN [2]).

1. Novelty in Research Motivation: We are the first to study the trade-off between the variety of dependencies extracted and the noise that may be introduced by channel-mixing. As we stated in our introduction: "Is it truly necessary to model all these dependencies?" Previous works [2] [3] often adopt a "brute-force modeling" approach, stacking parameters to capture as many dependencies as possible. While this might seem effective, it overlooks a critical issue—the potential noise introduced by this process.
2. Innovation in Model Framework: We propose a novel channel-preserving framework: GraphSTAGE. Through fair model variant experiments in Table 4 on Page 10, we validate the presence of such noise, underscoring the limitations of excessive dependency extraction. GraphSTAGE is a pure graph paradigm that decouples the extraction of global temporal dependencies and global spatial dependencies, rather than being limited to local neighbor information.
3. Advancements in Model Performance: Despite its structural simplicity, our model performs comparably to or surpasses state-of-the-art models across 13 MTSF benchmark datasets. It ranks first among 8 advanced models in 22 out of 30 comparisons in Table 1 on Page 7. Specifically, on the PEMS07 dataset—which has the largest number of nodes—GraphSTAGE outperforms the recent SOTA iTransformer by 20.8%, indicating its potential for application to larger-scale MTSF tasks, such as extensive grid management.

---

$\triangleright$Differences with iTransformer:

Conceptually, iTransformer is not a channel-preserving model; it overlooks temporal channel information. iTransformer projects the original time series data $X_{\text{in}} \in \mathbb{R}^{N \times T}$ (where $N$ is the number of nodes and $T$ is the length of the time series) into $H_S \in \mathbb{R}^{N \times D}$ to capture spatial dependencies among nodes. However, this transformation ignores temporal dependencies and fails to learn the underlying temporal graph structures.

In contrast, our GraphSTAGE embeds the input data $X_{\text{in}} \in \mathbb{R}^{N \times T}$ into $H \in \mathbb{R}^{N \times T \times D}$, where $D$ is the embedding dimension. The original node and time dimensions are preserved. This channel-preserving framework enables the model to incorporate both spatial (inter-series) and temporal (intra-series) dependencies by decoupling them. This separation allows GraphSTAGE to capture temporal dynamics more effectively, which is a significant extension beyond what iTransformer offers.

Thank you for your valuable and constructive feedback, which has inspired further improvements to our paper. As a gentle reminder, it has been more than 2 days since we submitted our rebuttal. We would like to know whether our response addressed your concerns.

Following your comments and suggestions, we have answered your concerns and made the following revisions:

• To address your concerns, we have clarified the unique aspects of our approach, particularly highlighting its motivation and framework. Additionally, we included a detailed comparison with recent models. To further enhance the validity of our work, we added four graph-based baselines.
• We have expanded our experiments to include additional four graph-based baseline methods for multivariate time series forecasting tasks: (1) FourierGNN, (2) CrossGNN, (3) StemGNN, and (4) MTGNN. For complete results, refer to Table 9 in Appendix D on Page 18.

Once again, we sincerely thank you for your insightful review. We look forward to your feedback and are prepared to address any further questions or concerns you may have.

Dear Reviewer RvJQ,

We sincerely thank you for your valuable and constructive feedback. Since the End of author/reviewer discussions is coming soon, may we know if our response addresses your main concerns? If so, we kindly ask for your reconsideration of the score. Should you have any further advice on the revised paper and/or our rebuttal, please let us know and we will be more than happy to engage in more discussion and paper improvements.

Thank you so much for devoting time to improving our paper!

Response to Reviewer 7NU8 (1/4)

W1 ＆ Q1 (1/3) : Chosen baseline methods for comparisons are limited to non-graph-based models. In contrast, several methods leveraging GNNs for time series forecasting can be found in FourierGNN.

Thank you for pointing this out, we have added four graph-based models as baseline methods for comparison: FourierGNN [1], CrossGNN [2], StemGNN [3], and MTGNN [4]. The detailed results are presented in Table 9 on Page 18 of the revised paper.

Following the experimental setup of TimesNet [5], we use a fixed lookback length $T = 96$ and set the prediction lengths $K \in \{96, 192, 336, 720\}$. We reproduced the results for FourierGNN and StemGNN, running each experiment five times. For CrossGNN and MTGNN, we collected the reported results from [3].

For your convenience, a summary of the relevant (averaged) results is provided below:

The results indicate that GraphSTAGE achieved the top-1 rank in terms of MSE and MAE metrics across four datasets compared with four graph-based models. Although CrossGNN performs comparably to our GraphSTAGE on smaller datasets (e.g., ETT and Weather), GraphSTAGE notably outperforms CrossGNN on large-scale datasets (e.g., ECL with 321 nodes). These findings further validate the effectiveness of GraphSTAGE in capturing both intra-series and inter-series dependencies, leading to superior forecasting accuracy across datasets of varying scales.

[1] FourierGNN: Rethinking multivariate time series forecasting from a pure graph perspective. NeurIPS, 2024

[2] Crossgnn: Confronting noisy multivariate time series via cross interaction refinement. NeurIPS, 2023

[3] Spectral temporal graph neural network for multivariate time-series forecasting. NeurIPS, 2020

[4] Connecting the dots: Multivariate time series forecasting with graph neural networks. KDD, 2020

[5] TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis. ICLR, 2023

W2 ＆ Q1 (2/3) : Additionally, the performance improvements achieved by the proposed method are, in most cases, very small compared to the best competitor (for instance, the biggest difference achieved by GraphStage from its best competitor in Table 1 is for Solar-Energy: from MSE equal to 0.233 to MSE 0.192, yet GraphStage is outcompeted in terms of MAE).

Thank you for pointing this out, GraphSTAGE significantly outperforms recent SOTA models on datasets with evident periodicity. For example, on the four subsets of the PEMS datasets, GraphSTAGE achieves an average improvement of 14.3%. Conversely, on datasets with less clear periodicity (e.g., ETT, Exchange), the performance improvement is comparatively smaller.

This phenomenon is commonly observed and has been discussed in previous studies such as BasicTS+ [1]. On datasets where the periodic patterns are prominent, complex models with larger capacities—like GraphSTAGE and iTransformer—tend to capture these periodicity more effectively, leading to superior predictive performance. In contrast, on datasets with unclear or weak periodicity, simpler MLP-based models like DLinear and RLinear often perform better. This is because complex models may overfit the noise in datasets lacking strong periodic signals, whereas simpler models are less prone to overfitting and may generalize better under these conditions.

[1] Exploring progress in multivariate time series forecasting: Comprehensive benchmarking and heterogeneity analysis. IEEE Transactions on Knowledge and Data Engineering, 2024

---

W3 ＆ Q1 (3/3) : It is unclear if the authors have performed multiple runs with random seeds to capture the model’s performance variability. The lack of standard deviations makes it impossible to assess whether the results' difference is statistically significant.

Thank you for pointing this out, Yes, the results of GraphSTAGE's performance are obtained from five random seeds. To provide a clearer picture of the model's performance stability, we have added the standard deviations for each task of all time series datasets in Table 10 and Table 11 on Page 22 of the revised manuscript.

For your convenience, we summarize the standard deviations of our approach and the second-best method (iTransformer) on four selected datasets below:

Response to Reviewer 7NU8 (2/4)

W4 ＆ Q2 (1/2) :Graph structure learning for time series forecasting is a long-term studied problem in the literature, entirely before the pure graph paradigm (Yi et al., 2024). Several methods that capture dependencies in the spatial and temporal domain jointly (Shang et al., 2021; Wu et al., 2020) or separately (Kipf et al., 2018; Xu et al., 2023) have been proposed, but are rather not mentioned in the paper and not considered in the experiments.

Thank you for pointing this out, the works you mentioned are indeed solid contributions in the fields of spatio-temporal forecasting. We have revised the Related Works to include these references and have highlighted the additions in orange in the revised paper.

Furthermore, we have selected two of the works you mentioned (FourierGNN and MTGNN), as additional baselines in our experiments. We tried our best effort to reproduce TimeGNN, however, we were unable to obtain reasonable results. Detailed results of the can be found in Table 9 on Page 18 of the revised paper. For your convenience, a summary of the relevant (averaged) results is provided below:

---

W5 ＆ Q2 (2/2) :Similarly the graph learning module (softmax on dot products between pairs) is a standard choice in relevant works (Wu et al., 2020), yet not correctly cited in this work.

Thank you for pointing this out. Yes, applying softmax on dot products between pairs is a standard approach for aggregating time features to extract spatial similarities between nodes. We did not emphasize this as our contribution, and we have added the citation in the revised paper.

Our main contribution lies in adopting a channel-preserving framework to decouple the learning of inter-series and intra-series dependencies, which not only outperforms the channel-mixing framework but also reduces memory usage. Employing a widely adopted method in [1] for information aggregation also demonstrates the generality of our channel-preserving framework.

However, to the best of our knowledge, we are the first to extend this method to aggregate all node features at each time step to extract similarities specifically between time steps. For example, in the Intra-GrAG module, the input $H_{\text{in}} \in \mathbb{R}^{P \times N \times D}$ is aggregated along the node dimension $N$ to obtain $E_{\text{src}} \in \mathbb{R}^{P \times c}$ and $E_{\text{tgt}} \in \mathbb{R}^{P \times c}$. Softmax is then applied to the dot products between these pairs to compute the similarities between time steps, resulting in the Temporal Learnable Graph $A_T \in \mathbb{R}^{P \times P}$. The effectiveness of this approach in extracting intra-dependencies has been validated through the ablation study in Table 2 and the visualization in Figure 8.

[1] Connecting the dots: Multivariate time series forecasting with graph neural networks. KDD, 2020

Response to Reviewer 7NU8 (3/4)

W6 ＆ Q3 (1/3): how the two graph learning blocks interact?

The interaction between the two graph learning blocks, Intra-GrAG and Inter-GrAG, follows a sequential structure within each STAGE block. We have provided the pseudo-code of GraphSTAGE in Algorithm 1 on Page 15 in the revised paper. For your convenience, a brief introduction to the STAGE block is provided below:

Let $L$ denote the number of STAGE blocks.

• Use $H^{l-1} \in \mathbb{R}^{N \times P \times D}$ to represent the input tensor to the $l$-th STAGE block.
• Use $H_{\text{tem}}^{l-1} \in \mathbb{R}^{P \times N \times D}$ to represent the output of the Intra-GrAG module in the $l$-th STAGE block.
• Use $H^{l} \in \mathbb{R}^{N \times P \times D}$ to represent the output of the Inter-GrAG module in the $l$​​-th STAGE block.
• $H^{0} \in \mathbb{R}^{N \times P \times D}$ is the output of the Embedding&Patching layer.

The whole process of stacked STAGE blocks is as follows:

1. For $l=1$ to $L$:
2. ​ // Intra-GrAG module
3. ​ $H_{tem}^{l-1} = IntraGrAG(H^{l-1})$
4. ​ // Inter-GrAG module
5. ​ $H^{l} = InterGrAG(H_{tem}^{l-1})$
6. End loop.

The final output $H^{L}$​ is the output of the stacked STAGE blocks.

W6 ＆ Q3 (2/3): how the final output is derived and optimized?

We apologize for neglecting to address this point. For the prediction output, we have provided the pseudo-code of GraphSTAGE in Algorithm 1 on Page 15 of the revised paper.

Regarding model optimization, the parameters are iteratively updated until convergence by minimizing the prediction loss $\mathcal{L} \leftarrow \mathcal{L}(\hat{Y}, Y)$, which is the Mean Squared Error (MSE) loss used in our experiments. $\hat{Y}$ are the predictions corresponding to the ground truth $Y$.

W6 ＆ Q3 (3/3): how the combination of spatial and temporal graphs is achieved?

There is no combination between the spatial and temporal graphs because we follow a sequential structure that decouples the extraction of temporal and spatial dependencies. This process is illustrated in the Orig of Figure 7 on Page 9. Specifically:

• Temporal graphs, denoted as $A_T \in \mathbb{R}^{P \times P}$, are used to capture the learned temporal dependencies.
• Spatial graphs, denoted as $A_S \in \mathbb{R}^{N \times N}$, are used to capture the learned spatial dependencies.

The two types of graphs are not combined in our approach.

Response to Reviewer 7NU8 (4/4)

W7 ＆ Q4 (1/2) : Performance improvements are, in the average case, around 6% (and no improvement for four datasets), thus the statement “with improvements up to 20.8%” is slightly misleading.

Thank you for pointing out this issue. We apologize for confusion caused by the statement "with improvements up to 20.8%." We have revised our manuscript accordingly and highlighted the modifications.

The statement "improvements up to 20.8%" specifically refers to the performance on the PEMS07 dataset, which has the largest number of nodes (883 nodes) among all the datasets we evaluated. In the original manuscript, we used this result to emphasize GraphSTAGE's potential for larger-scale MTSF applications.

W7 ＆ Q4 (2/2) : It is unclear how interpretability is achieved. Since patching is performed first, followed by temporal graph learning and then spatial graph learning, seems a bit exaggerated that interpretability is achieved, e.g., what can someone tell from the spatial graph concerning inter-series dependencies?

Thank you for pointing this out. Although we use patching in the Embedding&Patching layer, GraphSTAGE remains a channel-preserving framework. The patching operation merely increases the receptive field of each temporal channel and does not weaken the interpretability of the model. For example, regarding the temporal learnable graph $A_T^{(2)}$ obtained on the ECL dataset in Figure 8 on Page 10, when the lookback length $L = 96$ (the sampling frequency of the ECL dataset is 1 hour, so $96 \times 1\,\text{hour} = 4\,\text{days}$, meaning each sample contains 4 days of data). Since we use a patch stride $s = 2$, the total number of patches is $P = L / s = 96 / 2 = 48$. From $A_T^{(2)}$, we observe that similar temporal patterns appear every 12 patches (corresponding to 24 hours). This indicates that GraphSTAGE effectively captures the intrinsic periodicity of the data, which matches the daily periodicity of the ECL dataset.

Regarding the spatial graph, we apologize for the confusion caused. We have revised the colors in Figure 15 on Page 25 and have updated the relevant text in the revised paper, As shown in Figure 15, we can observe:

Firstly, the spatial learnable graph $A_S$ learned by GraphSTAGE is sparser, indicating that the model can identify the most important nodes in space and requires fewer inter-series correlations for predictions.

Secondly, the spatial learnable graph $A_S$ captures connections between time series that exhibit strong similarities. For example, according to the randomly selected case in Figure 15(left), the $A_S$ considers that the similarity between nodes 282 and 184 is high (close to 1), while the similarity between nodes 83 and 184 is low (close to 0). We plotted the ground truth of these three nodes in Figure 15(right), which shows that the trends of nodes 282 and 184 are consistent, matching the correlation coefficients learned in $A_S$. This match confirms the effectiveness of GraphSTAGE in capturing inter-series dependencies and enhances the interpretability of the model.

Thank you for your valuable and constructive feedback, which has inspired further improvements to our paper. As a gentle reminder, it has been more than 2 days since we submitted our rebuttal. We would like to know whether our response addressed your concerns.

Following your comments and suggestions, we have answered your concerns and made the following revisions:

• We have expanded our experiments to include additional four graph-based baseline methods for multivariate time series forecasting tasks: (1) FourierGNN, (2) CrossGNN, (3) StemGNN, and (4) MTGNN. For complete results, refer to Table 9 in Appendix D on Page 18.

• We have added the standard deviations of our model across all tasks to highlight the model’s strong robustness. For complete results, refer to Table 10 and Table 11 in Appendix F on Page 22.

• Additionally, we provided the pseudo-code of our model to enhance the clarity in methodological presentation. For detailed results, please see Algorithm 1 in Appendix A on Page 15.

Additionally, we have updated the related works to more clearly define the scope and background of this study. We have revised statements that could be slightly misleading, such as “improvements up to 20.8%.” We have also revised the colors in Figure 15 and have updated the relevant text in the revised paper to make the interpretability more clearly.

We sincerely thank you for your insightful review. We eagerly await your feedback and are ready to respond to any further questions you may have.

Dear Reviewer 7NU8,

We sincerely thank you for your valuable and constructive feedback. Since the End of author/reviewer discussions is coming soon, may we know if our response addresses your main concerns? If so, we kindly ask for your reconsideration of the score. Should you have any further advice on the revised paper or our rebuttal, please let us know and we will be more than happy to engage in more discussion and paper improvements.

Thank you so much for devoting time to improving our paper!

I appreciate the author's effort in addressing my concerns and clarifying potential misunderstandings. The additional results can potentially improve the paper's presentation and contribution. However, several of the justifications provided are unclear.

More specifically:

• Significance of results: I thank the authors for adding the standard deviations, but this was done only for their method. I remain unconvinced if all models are run 5 times with random seeds. Some seem to be directly taken from relevant papers (e.g., TimesNet) where results are, in several cases, mentioned for a fixed seed (CrossGNN). I highlight this since, for point-wise metrics, differences in terms of 2nd-3rd decimals might be minor at the prediction level.

• Comparison with GNN-based methods: I thank the authors for incorporating more experimental results concerning relevant GNN-based forecasting methods. The additional sentence about the GNN-based forecasting methods provides minimal explanations in the related section. It does not adequately support the central novelty justification in the graph-learning part of the method. It seems that using two decoupled temporal and spatial graphs in a sequential manner is a choice rather experimentally than conceptually motivated.

• Novelty of Contribution: I remain unconvinced about the significance of the methodological contribution. The idea is positioned in between GNN-based forecasting and leveraging embedding mechanisms for forecasting in terms of contribution and experiments. Patching and channel independence are not novel when targeting a contribution in sequential models for forecasting. Similarly, spatial-temporal graphs and GNN-based forecasting have been well-studied but remain hard to interpret and computationally expensive. Finally, the ablation of the model components proves the significance of patching and input embeddings rather than graph structure learning (where replacing with attention gives similar results). In the second case of positioning their work among sequential models forecasting, more recent sota (e.g., FITS) could have been used for more meaningful comparisons.

I once again sincerely thank the authors for their inputs. However, based on the above I prefer for the moment to maintain my scores.

Response to Reviewer 7NU8: Additional Concern (3/3)

Additional Concern 3 (1/5): The idea is positioned in between GNN-based forecasting and leveraging embedding mechanisms for forecasting in terms of contribution and experiments.

Thank you for pointing out this. Embedding mechanisms are used to inject relative positional information into inputs. While we have indeed proposed a refined time embedding in our paper (detailed in paragraph 2 of Section 3.1), which allows for more granular and sampling-frequency-adaptive relative positioning and significantly enhances performance on high-frequency datasets, we did not list embedding mechanisms as our core contribution.

As outlined in Section 1 of the revised paper , our main contributions are the novelty in research motivation and innovation in model framework. Specifically, we are the first to study the trade-off between the variety of dependencies extracted and the noise that may be introduced by channel-mixing. Additionally, we propose a pure graph paradigm that decouples the extraction of global temporal dependencies and global spatial dependencies.

We believe that these contributions position our work beyond being merely between GNN-based forecasting and leveraging embedding mechanisms. Our focus is on addressing previously unexplored issues in dependency modeling and proposing a novel framework to tackle these challenges.

We hope this clarifies the core contributions of our work.

---

Additional Concern 3 (2/5): Patching and channel independence are not novel when targeting a contribution in sequential models for forecasting.

Thank you for pointing this out. Patching is a common modeling method, and we do not claim it as our contribution.

We also did not claim in our paper that we are a channel independence model. Since our model extracts temporal and spatial dependencies, we do not strictly enforce channel independence.

---

Additional Concern 3 (3/5): Similarly, spatial-temporal graphs and GNN-based forecasting have been well-studied but remain hard to interpret and computationally expensive.

Thank you for pointing out this. We believe that the general statement about spatial-temporal graphs and GNN-based forecasting methods being hard to interpret and computationally expensive may not directly apply to our work.

1. Interpretability: We have addressed the interpretability of our model in the Response to Reviewer 7NU8 (4/4), specifically in W7 & Q4 (2/2). In summary, the temporal and spatial dependencies learned by our model match the inherent patterns of the dataset and match the ground truth. This demonstrates that our model provides meaningful and interpretable insights.

2. Computational Cost: As shown in Figure 5 on page 7, GraphSTAGE does not exhibit significantly higher computational costs compared to other methods. Our model achieves superior performance with acceptable computational overhead.

---

Additional Concern 3 (4/5): The ablation of the model components proves the significance of patching and input embeddings rather than graph structure learning (where replacing with attention gives similar results).

Thank you for pointing out this. We greatly appreciate your thorough review and attention to the details of our ablation study. While in a small number of tests replacing the STAGE block with attention yields comparable results, closeness does not imply complete equivalence. Overall, the STAGE block outperforms the attention mechanism.

Moreover, even when using attention, it operates within our proposed channel-preserving framework. The experimental results further highlight the superiority of our framework, which is precisely the focus of our core contributions.

Dear Reviewer 7NU8,

Hope this message finds you well.

We appreciate the diligent efforts of you in evaluating our paper. We have responded in detail to your additional concerns. As the discussion period will end soon, we would like to kindly ask whether there are any remaining concerns or questions we might be able to address. If our current response satisfactorily resolves your main concerns, we kindly ask for your reconsideration of the score.

Once more, we appreciate the time and effort you've dedicated to our paper.

Best regards,

Authors

Response to Reviewer 9agk (1/2)

W1: My primary concern is regarding the term "noise" (see line 16). The authors claim that previous methods introduce additional noise, but it remains unclear what this "noise" refers to. How is it defined, and what impact does it have on the effectiveness of the methods?

Thank you for pointing out this. The "noise" refers to the potential learning of pseudo connections when using a Hypervariate Graph to capture dependencies in channel-mixing methods.

Specifically, the size of the Hypervariate Graph ($\mathbb{R}^{NT \times NT}$) is significantly larger than the learnable graphs we propose: the Temporal Learnable Graph $A_T \in \mathbb{R}^{T \times T}$ and the Spatial Learnable Graph $A_S \in \mathbb{R}^{N \times N}$. Consequently, channel-mixing models like FourierGNN [1] and UniTST [2] have a higher degree of freedom compared to our channel-preserving model (GraphSTAGE). This increased degree of freedom can lead the model to learn many pseudo connections (connections between different nodes at different time steps), which weakens the weights of real connections (inter-series and intra-series dependencies). This may result in overfitting and a decrease in predictive performance.

To test our hypothesis that the channel-mixing strategy leads to overfitting and reduced predictive performance, we conducted experiments on two datasets. Since the dependency learning components of FourierGNN [1] and UniTST [2] differ from those in GraphSTAGE, we used the model variant VarC in Figure 7 on Page 9 as a baseline. This variant 'VarC' changes only the channel-preserving strategy of the original GraphSTAGE to a channel-mixing strategy while keeping the dependency learning components consistent.

To demonstrate the existence of overfitting in the channel-mixing strategy, we set the training epochs to 30 and used a fixed learning rate of 1e-3. We tested on the ECL and PEMS03 datasets with input length equal to prediction length (24 time steps). All experiments have been conducted five times and hyperparameters remained the same. The results are shown in the table below:

As shown, the channel-mixing model VarC has lower training MSE but higher test MSE. The results indicate that VarC (channel-mixing model) exhibits overfitting compared to Orig (channel-preserving model).

---

To further demonstrate the decrease in predictive performance on the test set caused by the channel-mixing strategy, we refer to the results provided in Table 4 on Page 10 of our paper. For your convenience, we summarize the relevant (averaged) results below:

The channel-preserving model Orig outperforms the channel-mixing variant VarC, supporting our hypothesis that channel-mixing leads to decreased predictive performance.

---

[1] FourierGNN: Rethinking multivariate time series forecasting from a pure graph perspective. NeurIPS, 2024

[2] UniTST: Effectively Modeling Inter-Series and Intra-Series Dependencies for Multivariate Time Series Forecasting. arXiv

Response to Reviewer 9agk (2/2)

W2: This paper appears to be an incremental contribution, with its components largely derived from previous works. Channel-preserving Strategy is inspired by iTransformer, and the fully GNN perspective is inspired by FourierGNN.

Thank you for pointing out this. We would like to highlight the unique aspects of our approach and clarify the distinctions between our work and the recent models you mentioned.

1. Novelty in Research Motivation: We are the first to study the trade-off between the variety of dependencies extracted and the noise that may be introduced by channel-mixing. As we stated in our introduction: "Is it truly necessary to model all these dependencies?" Previous works [1] [2] often adopt a "brute-force modeling" approach, stacking parameters to capture as many dependencies as possible. While this might seem effective, it overlooks a critical issue—the potential noise introduced by this process.
2. Innovation in Model Framework: We propose a novel channel-preserving framework: GraphSTAGE. Through fair model variant experiments in Table 4 on Page 10, we validate the presence of such noise, underscoring the limitations of excessive dependency extraction. GraphSTAGE is a pure graph paradigm that decouples the extraction of global temporal dependencies and global spatial dependencies, rather than being limited to local neighbor information.
3. Advancements in Model Performance: Despite its structural simplicity, our model performs comparably to or surpasses state-of-the-art models across 13 MTSF benchmark datasets. It ranks first among 8 advanced models in 22 out of 30 comparisons in Table 1 on Page 7. Specifically, on the PEMS07 dataset—which has the largest number of nodes—GraphSTAGE outperforms the recent SOTA iTransformer by 20.8%, indicating its potential for application to larger-scale MTSF tasks, such as extensive grid management.

---

$\triangleright$Differences with iTransformer:

Conceptually, iTransformer is not a channel-preserving model; it overlooks temporal channel information. iTransformer projects the original time series data $X_{\text{in}} \in \mathbb{R}^{N \times T}$ (where $N$ is the number of nodes and $T$ is the length of the time series) into $H_S \in \mathbb{R}^{N \times D}$ to capture spatial dependencies among nodes. However, this transformation ignores temporal dependencies and fails to learn the underlying temporal graph structures.

In contrast, our GraphSTAGE embeds the input data $X_{\text{in}} \in \mathbb{R}^{N \times T}$ into $H \in \mathbb{R}^{N \times T \times D}$, where $D$ is the embedding dimension. The original node and time dimensions are preserved. This channel-preserving framework enables the model to incorporate both spatial (inter-series) and temporal (intra-series) dependencies by decoupling them. This separation allows GraphSTAGE to capture temporal dynamics more effectively, which is a significant extension beyond what iTransformer offers.

$\triangleright$Differences with FourierGNN:

Although both are pure graph methods, Graph Neural Networks (GNNs) are a very broad concept. Being graph-based does not make our work incremental.

1. Dependency Modeling Approach: FourierGNN represents a trend toward modeling three kinds of dependencies within a unified framework, as shown in Figure 2 on Page 2. GraphSTAGE, however, focuses on decoupling and modeling two types of dependencies.

2. Research Focus: The drawbacks of using hypervariate graphs to unify multi-dependency modeling have not been studied in previous research. There are two main drawbacks: High Memory Consumption: This issue has also been reported in other studies [3]. Potential Noise: Modeling extensive cross-dependencies may lead the model to learn many pseudo connections (as we explained in our response to Q1). These pseudo connections can weaken the influence of genuine connections (inter-series and intra-series dependencies), potentially diminishing the model's predictive performance.

   Additionally, instead of focusing solely on FourierGNN, we aimed to broadly address hypervariate graph-based multi-dependency modeling methods. In the Variants Comparison section, we included a variant called VarC (shown in Figure 7 on Page 9) to encompass these methods and compared it with our channel-preserving GraphSTAGE. Our findings indicate that modeling only two types of dependencies can achieve excellent predictive performance while also reducing memory usage.

---

$\triangleright$Addition of Graph-Based Baselines:

To provide a more direct and comprehensive comparison with FourierGNN and other graph-based models, we have included four additional graph-based baselines in the revised paper. The detailed results are presented in Table 9 on Page 18 of the revised paper. For your convenience, we provide a summary of the relevant averaged results below:

The results indicate that GraphSTAGE achieved the top-1 rank in terms of MSE and MAE metrics across four datasets compared with four graph-based models. Although CrossGNN performs comparably to our GraphSTAGE on smaller datasets (e.g., ETT and Weather), GraphSTAGE notably outperforms CrossGNN on large-scale datasets (e.g., ECL with 321 nodes). These findings further validate the effectiveness of GraphSTAGE in capturing both intra-series and inter-series dependencies, leading to superior forecasting accuracy across datasets of varying scales.

We believe these clarifications address your concerns regarding the novelty and contributions of our work. Current models primarily focus on the advantages of channel-mixing methods for extracting multiple dependencies, often neglecting the noise these approaches can introduce. We are the first to directly address this issue. Thank you again for your valuable feedback, which has helped us improve our paper.

---

[1] FourierGNN: Rethinking multivariate time series forecasting from a pure graph perspective. NeurIPS, 2024

[2] UniTST: Effectively Modeling Inter-Series and Intra-Series Dependencies for Multivariate Time Series Forecasting. arXiv

[3] ForecastGrapher: Redefining Multivariate Time Series Forecasting with Graph Neural Networks. arXiv

[4] Crossgnn: Confronting noisy multivariate time series via cross interaction refinement. NeurIPS, 2023

[5] Spectral temporal graph neural network for multivariate time-series forecasting. NeurIPS, 2020

[6] Connecting the dots: Multivariate time series forecasting with graph neural networks. KDD, 2020

Thank you for your prompt feedback. I agree with your point, especially the statement, “Even if some noise is introduced, the overall benefits outweigh the drawbacks.” I cannot agree more.

Taking the example you mentioned, consider a node $x$. After patching, we obtain $x_{1:p}$, $x_{p:2p}$, $x_{2p:3p}$, $x_{3p:4p}$. If we assume that $x_{1:p}$ and $x_{3p:4p}$ are unrelated. However, when GraphSTAGE learns the temporal dependencies within the patches of the same node, it's hard to ensure that the model doesn't assign weights to the "pseudo connections" between $x_{1:p}$ and $x_{3p:4p}$. Therefore, ablation studies can only demonstrate that the model achieves overall benefits outweigh the drawbacks, but it is challenging to specifically assess whether noise is introduced in the learning of each individual connection.

However, extensive experimental results demonstrate that GraphSTAGE achieves results that are comparable to or even surpass recent SOTA methods. This demonstrates that the overall benefits outweigh the drawbacks in GraphSTAGE.

Therefore, I agree with your point that regardless of the type of dependency being modeled (whether inter-series, intra-series, or cross-series), noise might inevitably be introduced. The key is that the model should ensure the overall benefits outweigh the drawbacks. Clearly, the current hypervariate graph-based modeling methods remain inadequate. GraphSTAGE addresses this challenge by decoupling the learning of the two types of dependencies, which helps to minimize the introduction of noise as much as possible.

In the Abstract on Page 1, We stated: “The trade-off between the variety of dependencies extracted and the potential interference has not yet been fully explored.” This actually conveys a similar idea with you that better models need to strike a balance between improving performance and suppressing noise. Simply increasing the variety of dependencies extracted could not only degrade model performance but also increase memory usage.

If you have any other concerns, please don’t hesitate to share them with us. Thank you again for your valuable insights.

Dear Reviewer 9agk,

We sincerely thank you for your valuable and constructive feedback. Since the End of author/reviewer discussions is coming soon, may we know if our response addresses your main concerns? If so, we kindly ask for your reconsideration of the score. Should you have any further advice on the revised paper and/or our rebuttal, please let us know and we will be more than happy to engage in more discussion and paper improvements.

Thank you so much for devoting time to improving our paper!

Dear Reviewer 9agk,

We sincerely thank you for your valuable and constructive feedback. Since the Discussion Period Extension provides us with additional time, we are eager to address any further concerns you may have. If our current response satisfactorily resolves your main concerns, we kindly ask for your reconsideration of the score. Should you have any further advice on our rebuttal, please let us know, and we will be more than happy to engage in further discussion.

Thank you so much for devoting time to improving our paper!

Response to Reviewer 9agk: Additional Concern (1/1)

Additional Concern1: While the experimental results show some improvement in the forecasting performance of your proposed model, the underlying source of this improvement remains unclear. Therefore, I prefer to maintain my original score.

Thank you for your feedback and for acknowledging the performance improvements achieved by our model. We have addressed your concern in the revised paper through a combination of model variants comparisons, ablation studies, and visualization of learned dependencies. Below, we summarize these efforts to clarify the contributions of our design to the observed performance improvements:

1. Model Framework and Variants Comparisons We conducted extensive experiments comparing different model variants to demonstrate the advantages of our decoupled sequential learning framework. Specifically, we evaluated alternative designs that modify the sequence of dependencies learning or use parallel and channel-mixing structures (VarA, VarB, VarC). For reference, please see Table 4 on Page 10 in the revised paper. For your convenience, we provide a summary of the relevant averaged results below:

   Method	GraphSTAGE		VarA		VarB		VarC
   Metric	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
   ECL	0.166	0.263	0.192	0.282	0.184	0.276	0.192	0.284

   Our sequential structure (GraphSTAGE) outperforms variants, especially on large-scale dataset (e.g., ECL with 321 nodes), by effectively preserving the original data structure and decoupling the learning of inter-series and intra-series dependencies. These results underscore the importance of our architectural design.

---

2. Ablation Studies on Model Design We performed detailed ablation studies to assess the contribution of key components in our model. For example:

   • Embedding Layer: Results in Table 3 on Page 8 show that removing components such as time embedding or adaptive embedding leads to significant performance drops. For your convenience, we provide a summary of the relevant averaged results below:

     Dataset	PEMS03		PEMS08
     Metric	MSE	MAE	MSE	MAE
     GraphSTAGE	0.097	0.210	0.139	0.220
     w/o Patching	0.110	0.222	0.176	0.253
     w/o Time Emb.	0.114	0.223	0.199	0.264
     w/o Adaptive Emb.	0.121	0.257	0.203	0.260

     Additionally, we introduced a refined time embedding, refining 'Hour of Day' (e.g., 24 hours/day for PEMS datasets) to 'Timestamp of Day' (e.g., 288 steps/day for PEMS datasets), as detailed in paragraph 2 of Section 3.1. This allows for more granular and sampling-frequency-adaptive relative positioning and significantly enhances performance on high-frequency datasets.

   • STAGE Module: As shown in Table 2 on Page 8, the removal of key graph learning components, such as the Intra-GrAG or Inter-GrAG modules, results in notable performance degradation. This highlights their critical role in capturing both intra-series and inter-series dependencies effectively. For your convenience, we provide a summary of the relevant averaged results below:

     Dataset	ECL		Solar-Energy
     Metric	MSE	MAE	MSE	MAE
     GraphSTAGE	0.166	0.263	0.192	0.267
     w/o Intra-GrAG	0.185	0.277	0.225	0.292
     w/o Inter-GrAG	0.186	0.276	0.239	0.294

Responses to Reviewer yAwh (1/3)

W1: I don't quite understand why a purely graph-based structure can introduce noise and interference through channel mixing methods.

Thank you for pointing out this. The noise and interference in channel-mixing methods, stem from the potential learning of pseudo connections when using a Hypervariate Graph to capture dependencies.

Specifically, the size of the Hypervariate Graph ($\mathbb{R}^{NT \times NT}$) is significantly larger than the learnable graphs we propose: the Temporal Learnable Graph $A_T \in \mathbb{R}^{T \times T}$ and the Spatial Learnable Graph $A_S \in \mathbb{R}^{N \times N}$. Consequently, channel-mixing models like FourierGNN [1] and UniTST [2] have a higher degree of freedom compared to our channel-preserving model (GraphSTAGE). This increased degree of freedom can lead the model to learn many pseudo connections (connections between different nodes at different time steps), which weakens the weights of real connections (inter-series and intra-series dependencies). This may result in overfitting and a decrease in predictive performance.

To test our hypothesis that the channel-mixing strategy leads to overfitting and reduced predictive performance, we conducted experiments on two datasets. Since the dependency learning components of FourierGNN [1] and UniTST [2] differ from those in GraphSTAGE, we used the model variant VarC in Figure 7 on Page 9 as a baseline. This variant 'VarC' changes only the channel-preserving strategy of the original GraphSTAGE to a channel-mixing strategy while keeping the dependency learning components consistent.

To demonstrate the existence of overfitting in the channel-mixing strategy, we set the training epochs to 30 and used a fixed learning rate of 1e-3. We tested on the ECL and PEMS03 datasets with input length equal to prediction length (24 time steps). All experiments have been conducted five times and hyperparameters remained the same. The results are shown in the table below:

As shown, the channel-mixing model VarC has lower training MSE but higher test MSE. The results indicate that VarC (channel-mixing model) exhibits overfitting compared to Orig (channel-preserving model).

---

To further demonstrate the decrease in predictive performance on the test set caused by the channel-mixing strategy, we refer to the results provided in Table 4 on Page 10 of our paper. For your convenience, we summarize the relevant (averaged) results below:

The channel-preserving model Orig outperforms the channel-mixing variant VarC, supporting our hypothesis that channel-mixing leads to decreased predictive performance.

---

[1] FourierGNN: Rethinking multivariate time series forecasting from a pure graph perspective. NeurIPS, 2024

[2] UniTST: Effectively Modeling Inter-Series and Intra-Series Dependencies for Multivariate Time Series Forecasting. arXiv

Responses to Reviewer yAwh (2/3)

W2(1/2): I have doubts about the effectiveness of purely using GNNs for time series analysis. The performance of GNNs heavily depends on the rationality of the graph structure. However, in certain datasets or tasks, constructing a reasonable graph structure is itself a challenge. If the graph structure fails to accurately capture the dependencies in the data, even the most advanced GNNs may struggle to achieve optimal performance.

Thanks for pointing out this. Yes, the rationality of the graph structure is crucial for the effectiveness of GNNs in time series analysis. In our proposed GraphSTAGE model, we have minimized the risk that the graph structure fails to accurately capture the dependencies in the data.

If we had used pre-defined graphs—as employed by models like DCRNN [1] and STGCN [2]—there could indeed be concerns about the rationality of the graph structure. Pre-defined graphs have been argued in many studies to be biased, incorrect, or even unavailable in many cases [3-6].

Numerous studies have demonstrated that learning-based graphs can effectively enable GNNs to capture complex dependencies in time series forecasting tasks [3-6]. Therefore, our GraphSTAGE model utilizes a learning-based graph structure ($A_T$ and $A_S$) to allow the model adaptively learn temporal and spatial dependencies.

---

[1] Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting. ICLR, 2018

[2] STGCN: a spatial-temporal aware graph learning method for POI recommendation. ICDM, 2020

[3] Graph wavenet for deep spatial-temporal graph modeling. IJCAI, 2019

[4] Connecting the dots: Multivariate time series forecasting with graph neural networks. KDD, 2020

[5] Pre-training enhanced spatial-temporal graph neural network for multivariate time series forecasting. KDD, 2022

[6] Dynamic graph convolutional recurrent network for traffic prediction: Benchmark and solution. Transactions on Knowledge Discovery from Data, 2023

W2(2/2): Additionally, the representational capacity of graphs may be limited in high-noise or irregular data, leading to a decline in the model's predictive capabilities. I hope the authors can provide a more comprehensive explanation.

Thank you for pointing this out. Yes, we agree that high-noise or irregular data can pose significant challenges for predictive models. However, we believe that the decline in predictive capabilities on high-noise or irregular datasets is primarily due to the quality of the data rather than the representational capacity of graphs.

For example, the Exchange dataset used in our experiments is reported to be a high-noise dataset with unclear periodicity [1], where exchange rates are influenced by many unpredictable factors such as economic policies and geopolitical events.

To provide a more clear and intuitive evaluation, we followed the recent benchmark BasicTS+ [1] and used the Mean Absolute Percentage Error (MAPE) metric. MAPE offers insight into how far predictions deviate from actual values relative to the actual values themselves. The formula for MAPE is:

$\text{MAPE} = \frac{1}{n} \sum_{i=1}^{n} \left| \frac{y_i - \hat{y}_i}{y_i} \right| \times 100$

The performance of GraphSTAGE and the recent SOTA iTransformer on Exchange dataset are listed as follows:

As shown in the above table, even the recent SOTA model (iTransformer) exhibits extremely high MAPE values on the Exchange dataset, in contrast to the seemingly low MAE and MSE values. This indicates that the prediction of high-noise data is actually quite challenging and represents a bottleneck for all deep network methods [2].

However, our comprehensive forecasting results (Table 1 on Page 7 of the revised paper) demonstrate that GraphSTAGE significantly outperforms recent SOTA models on datasets with evident periodicity. For example, on the four subsets of the PEMS datasets, GraphSTAGE achieves an average improvement of 14.3%. Conversely, for datasets with unclear periodicity (e.g., ETT, Exchange), the performance gaps among models are comparatively smaller.

Regarding irregular data, where different nodes have distinct original scales or sampling rates, this is an important direction for our future research. In this work, we focus exclusively on regular data, where the scales and sampling rates of all nodes are fixed.

---

[1] Exploring progress in multivariate time series forecasting: Comprehensive benchmarking and heterogeneity analysis. IEEE Transactions on Knowledge and Data Engineering, 2024

[2] iTransformer: Inverted Transformers Are Effective for Time Series Forecasting. ICLR, 2024

Responses to Reviewer yAwh (3/3)

W3: FourierGNN also performs time series prediction, so why didn't the authors conduct a comprehensive comparison?

Thank you for pointing this out. There are two main reasons why we did not include FourierGNN in our initial submission:

1. Different Focus of Tasks: FourierGNN and GraphSTAGE focus on different forecasting tasks. FourierGNN primarily targets spatio-temporal forecasting tasks, with experiments concentrated on short-term spatio-temporal forecasting where the maximum prediction length is 12. In contrast, GraphSTAGE focuses on long-term time series forecasting, with prediction lengths extending up to 720.
2. Performance: Existing research suggests that FourierGNN may not serve as a strong baseline [1].

To address the your concerns, we have added four graph-based models as baseline methods for comparison: FourierGNN [2], CrossGNN [3], StemGNN [4], and MTGNN [5]. Following the experimental setup of TimesNet [6], we use a fixed lookback length $T = 96$ and set the prediction lengths $K \in \{96, 192, 336, 720\}$.

We reproduced the results for FourierGNN and StemGNN, running each experiment five times. For CrossGNN and MTGNN, we collected the reported results from [3]. The detailed results are presented in Table 9 on Page 18 of the revised paper. For your convenience, a summary of the relevant (averaged) results is provided below:

The above results show that GraphSTAGE consistently achieved the top-1 rank in terms of MSE and MAE across four datasets. Although CrossGNN exhibits performance comparable to our GraphSTAGE model on smaller-scale datasets ( e.g., ETT and Weather), GraphSTAGE notably outperforms CrossGNN on large-scale datasets (e.g., ECL with 321 nodes). These results further validate the effectiveness of GraphSTAGE in capturing both intra-series and inter-series dependencies, leading to superior forecasting accuracy across datasets of varying scales.

---

[1] MambaTS: Improved Selective State Space Models for Long-term Time Series Forecasting. arXiv

[2] FourierGNN: Rethinking multivariate time series forecasting from a pure graph perspective. NeurIPS, 2024

[3] Crossgnn: Confronting noisy multivariate time series via cross interaction refinement. NeurIPS, 2023

[4] Spectral temporal graph neural network for multivariate time-series forecasting. NeurIPS, 2020

[5] Connecting the dots: Multivariate time series forecasting with graph neural networks. KDD, 2020

[6] TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis. ICLR, 2023

Thank you for your valuable and constructive feedback, which has inspired further improvements to our paper. As a gentle reminder, it has been more than 2 days since we submitted our rebuttal. We would like to know whether our response has addressed your concerns.

Following your comments and suggestions, we have answered your concerns and made the following revisions:

• To address your concerns, we have clarified the noise and interference caused by channel-mixing methods and validated the existence and impact of this noise through experiments.
• Your concern regarding the representational capacity of graphs under high-noise data is highly valuable and insightful. We addressed this concern through experiments on Exchange dataset.
• We have expanded our experiments to include four graph-based baselines for multivariate time series forecasting tasks: (1) FourierGNN, (2) CrossGNN, (3) StemGNN, and (4) MTGNN. For complete results, refer to Table 9 in Appendix D on Page 18.

We sincerely thank you for your insightful review. We look forward to your feedback and are prepared to address any further questions or concerns you may have.

Dear Reviewer yAwh,

We sincerely thank you for your valuable and constructive feedback. Since the End of author/reviewer discussions is coming soon, may we know if our response addresses your main concerns? If so, we kindly ask for your reconsideration of the score. Should you have any further advice on the revised paper or our rebuttal, please let us know and we will be more than happy to engage in more discussion and paper improvements.

Thank you so much for devoting time to improving our paper!