Biased Temporal Convolution Graph Network for Time Series Forecasting with Missing Values

Q1

Limited methodological contribution: the paper merely combines two existing methods.

A1

We would like to highlight the innovations of our proposed model from three aspects.

• For the PartialTCN module, we introduce the instance PartialTCN that focuses on capturing the temporal dependencies whereas it leaves the instance correlation for Biased GCN, this stands in contrast to the PartialCNN which relies on the convolution to capture the correlation from the two dimensions, and thus allowing for the exploration of temporal patterns that are invariant to instances.
• For the Biased GCN module, we innovate the GCN by adding a bias term (prior knowledge) to account for different missing patterns (illustrated by Equation-9 in the revised manuscript), which will then be used to correct the message passing strength in the graph information diffusion. Our ablation study shows that such innovation is critical to the forecasting performance.
• We stack multiple blocks of the two modules and design a hierarchical architecture, and propose a new mask updating mechanism in both modules (as shown in Equation-6 and Equation-10) to ensure a corrected mask when information flows from bottom to up.

We have updated both the content and Figure-1 (b) in the revised manuscript to highlight these innovations.

Q2

missing principled baseline: to impute a zero and to add a channel carrying the imputation mask (i.e., as "1" if the value was observed and a "0" if it originally was missing and now has been imputed). This way, the information about missingness is not lost.

A2

Thanks for your suggestion. We have changed the experimental setting to make the imputation mask available to all baseline methods. This can indeed improve the performance of most methods, especially for those Transformer-based ones. We have revised the manuscript to include the corresponding analysis and discussion.

Q1

I would encourage the authors to further clarify and/or elaborate on the novelty of the two modules within BiaTCGNet.

A1

We would like to highlight the innovations of our proposed model from three aspects.

• For the PartialTCN module, we introduce the instance PartialTCN that focuses on capturing the temporal dependencies whereas it leaves the instance correlation for Biased GCN, this stands in contrast to the PartialCNN which relies on the convolution to capture the correlation from the two dimensions, and thus allowing for the exploration of temporal patterns that are invariant to instances.
• For the Biased GCN module, we innovate the GCN by adding a bias term (prior knowledge) to account for different missing patterns (illustrated by Equation-9 in the revised manuscript), which will then be used to correct the message passing strength in the graph information diffusion. Our ablation study shows that such innovation is critical to the forecasting performance.
• We stack multiple blocks of the two modules and design a hierarchical architecture, and propose a new mask updating mechanism in both modules (as shown in Equation-6 and Equation-10) to ensure a corrected mask when information flows from bottom to up.

We have updated both the content and Figure-1 (b) in the revised manuscript to highlight these innovations.

Q2

The role of $\beta$ has not been discussed further. If possible, I would also suggest that the authors consider including a brief discussion on the interpretation of those values.

A2

Thanks for your suggestion. The following table shows the change of learnable $\beta$ against missing rates on three representative datasets (traffic, solar energy, and air quality). We found that $\beta$ demonstrates similar values with different missing rates, indicating that $\beta$ is primarily responsible for adjusting the strength of correctness and is independent of missing rates. Additionally, the possible reason for the value variation of $\beta$ across different datasets may caused by the unique characteristics of different datasets. The discussion can be found in Section E of the Appendix.

Table R2-1(The values of $\beta$)

Q3

There seems to be a fairly recent work [C1] on attention-based memory networks for joint modeling of local spatio-temporal features and global historical patterns in multivariate time series with missing values. Moreover, the forecasting problem formulation in [C1] seems to be consistent with the one considered in this work. Therefore, I would suggest that the authors consider comparing the GCN-M method from [C1] with BiaTCGNet. Alternatively, I would ask the authors to provide the specific reason as to why this method has not been included among the baselines?

A3

Thanks for pointing us the missing work. The GCN-M [1] is now included as a baseline method in our revised manuscript and its results are available in Section 5.3 and Sections B, C, and G. Note that as GCN-M requires location information, it is only applicable to the Metr-LA, PEMS, and Beijing Air datasets.

References:

[1] Jingwei Zuo, Karine Zeitouni, Yehia Taher, and Sandra Garcia-Rodriguez. Graph convolutional networks for traffic forecasting with missing values. Data Mining and Knowledge Discovery, 2023.

Q1

The experiment setup is not clear to me whether it's sound. Is it from the matching loss function and evaluation metric, or is it from architecture innovation.

Under the rank of MAE and MAPE, the ranking of methods can differ.

A1

Thanks for pointing us this ambiguity. We actually adopt the MAE as our training loss, and we have revised the Equation-2 to make it clear. Regarding the ranking difference of MAE and MAPE, they are indeed the same metric up to a normalization (denominator), however, the denominators are not a fixed constant and change against different points (y_t). As a consequence, the ranking of the two metrics can differ even for the same method and dataset, and this is also observed in prior work [1, 2].

Q2

It was mentioned that 'the results are averaged over 5 runs'. What are these 5 runs referring to? Do they refer to different random seeds for training only?

A2

Yes, the five-runs mean different random seeds for training only, i.e., we run a method for 5 rounds with different random seeds and calculate the averaged values from the 5 rounds. We have revised the paper to make it clear now.

Q3

Introducing missing masks to feature map, motivated from vision tasks, makes sense; yet it'd be good to compare against, introducing the missing masks as input feature directly so the model can be trained with the knowledge what inputs are missing.

A3

Thanks for your suggestion. We have changed the experimental setting for all baseline methods such that they all take the missing masks as direct input features on all datasets. This can indeed improve the performance of most methods, especially for those Transformer-based ones. We have revised the manuscript to include the corresponding analysis and discussion.

Q4

It was implied that the proposed approach could capture missing pattern; yet the experiments seem to only tested the scenario where values are missing at random. In actual application, it's rare that values are missing at random. It'd be good to also test different scenarios to show the efficacy of the proposed methods.

A4

Thanks for your suggestion. We now included an additional experiment to verify the effectiveness of our proposed method under the block missing scenarios. In particular, we follow the setting of prior work [3] by adopting two block-missing rates of 0.15% and 0.2% to test different models on the Metr-LA and ETTh1 datasets. The corresponding results are presented in Table 4 of the Appendix. Notably, our proposed BiaTCGNet demonstrates its superiority consistently, achieving a remarkable 5% improvement over the best baseline method.

References:

[1] Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. Connecting the dots: Multivariate time series forecasting with graph neural networks. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (SIGKDD), 2020.

[2] Jingwei Zuo, Karine Zeitouni, Yehia Taher, and Sandra Garcia-Rodriguez. Graph convolutional networks for traffic forecasting with missing values. Data Mining and Knowledge Discovery, 2023.

[3] Andrea Cini, Ivan Marisca, and Cesare Alippi. Filling the gaps: Multivariate time series imputation by graph neural networks. In International Conference on Learning Representations (ICLR), 2022.