TSGGuide: Recommendation Guide for Multivariate Time Series Generation

Thank you very much for your comments. Our peer-to-peer responses can be found below. Also, we have uploaded a corrected PDF version of the article.

answer to w1: We have fixed these in the updated versions.

answer to w2: Thank you for your suggestion. We have added the description of hyperparameters in Parameters in Section 3.1.

answer to w3: We conducted experiments based on DLG and Air, which are from the traffic flow dataset and air dataset respectively. The experimental results are as follows:

Simulated data. At the same time, in order to demonstrate the effectiveness of our method, we used the following scheme to generate a simulated time series dataset: we adopt a framework based on the principle Y=f(X). Specifically, the process begins with sampling sequences X from a predefined random mixture distribution.

Next, we employ a binary tree structure to randomly generate mathematical expressions that define the function f(X). This mechanism ensures variability and diversity in the functional relationships represented by f.

Finally, by forward-propagating the sampled sequences X through the generated mathematical expressions f(X), we obtain the corresponding output sequences Y=f(X). This methodology provides a flexible and systematic way to simulate complex time-series data with customizable properties, which is critical for evaluating the robustness and generalization capabilities of our proposed models. The details of this dataset can be found in Appendix L.

We generated 100 simulated data sets and conducted experiments according to the experimental design in Figure 3 in the main text. The average results of the experiments are shown below:

The results demonstrate that both the diffusion-based channel-mixing framework Diffusion-TS and the GAN-based channel-independent framework CCDGAN excel across multiple evaluation metrics, with each achieving top rankings in three categories. CCDGAN outperforms in two distance-based metrics, likely due to GANs' strength in capturing temporal distance characteristics. Meanwhile, Diffusion-TS exhibits balanced performance across all metrics, highlighting the overall effectiveness of diffusion-based generation models.

answer to w4: This paper introduces a novel Central Discriminator (CCD) designed from the perspective of small-sample time series classification, which significantly outperforms COSCI-GAN (a channel-independent framework) and enhances the recommended priority of channel-independent frameworks. However, this finding is only one aspect of the recommendations provided in this paper and is not the core contribution. We have thoroughly discussed the strengths and limitations of CCD-GAN and diffusion models across different domains and scenarios and provided corresponding recommendation guidelines.

Thank you very much for your comments. Our peer-to-peer responses can be found below. Also, we have uploaded a corrected PDF version of the article.

answer to w1: Essentially, ours is a research article that provides users with a sound recommendation guide. The proposed CCD is a combination of current techniques to overcome the defect that the central discriminator in the channel-independent generation method cannot handle the small sample time series classification problem. We then conclude that the recommendation priority of the channel-independent method should be improved. We believe that the recommendation guide is valuable to the field and users.

answer to w2: We conducted experiments based on DLG and Air, which are from the traffic flow dataset and air dataset respectively. The experimental results are as follows:

Simulated data. At the same time, in order to demonstrate the effectiveness of our method, we used the following scheme to generate a simulated time series dataset: we adopt a framework based on the principle Y=f(X). Specifically, the process begins with sampling sequences X from a predefined random mixture distribution.

Next, we employ a binary tree structure to randomly generate mathematical expressions that define the function f(X). This mechanism ensures variability and diversity in the functional relationships represented by f.

Finally, by forward-propagating the sampled sequences X through the generated mathematical expressions f(X), we obtain the corresponding output sequences Y=f(X). This methodology provides a flexible and systematic way to simulate complex time-series data with customizable properties, which is critical for evaluating the robustness and generalization capabilities of our proposed models. The details of this dataset can be found in Appendix L.

We generated 100 simulated data sets and conducted experiments according to the experimental design in Figure 3 in the main text. The average results of the experiments are shown below:

The results demonstrate that both the diffusion-based channel-mixing framework Diffusion-TS and the GAN-based channel-independent framework CCDGAN excel across multiple evaluation metrics, with each achieving top rankings in three categories. CCDGAN outperforms in two distance-based metrics, likely due to GANs' strength in capturing temporal distance characteristics. Meanwhile, Diffusion-TS exhibits balanced performance across all metrics, highlighting the overall effectiveness of diffusion-based generation models.

answer to w3: We will cite TimeGAE, LDT and SDFormer in the article. At the same time, we also found a new related article SDFormer published in NIPS2024, which was published after our submission, but no official code was found for TimeGAE and SDFormer. From the results in Table 2 and Table 3 of SDFormer, we can see that the performance of SDFormer is better than DiffusionTS. Combined with our Table 8, we can increase the recommendation priority of SDFormer. At the same time, we choose Gamma using grid search in the range of 0.5 to 10. We also performed other hyperparameter analyses (such as kernel size). The specific results are shown in J.1.

answer to w4: We followed the settings in TSGBench, where the training and test sets were divided using a 9 to 1 ratio.

Thank you very much for your comments. Our peer-to-peer responses can be found below. Also, we have uploaded a corrected PDF version of the article.

answer to w1: We conducted experiments based on DLG and Air, which are from the traffic flow dataset and air dataset respectively. The experimental results are as follows:

Simulated data. At the same time, in order to demonstrate the effectiveness of our method, we used the following scheme to generate a simulated time series dataset: we adopt a framework based on the principle Y=f(X). Specifically, the process begins with sampling sequences X from a predefined random mixture distribution.

Next, we employ a binary tree structure to randomly generate mathematical expressions that define the function f(X). This mechanism ensures variability and diversity in the functional relationships represented by f.

Finally, by forward-propagating the sampled sequences X through the generated mathematical expressions f(X), we obtain the corresponding output sequences Y=f(X). This methodology provides a flexible and systematic way to simulate complex time-series data with customizable properties, which is critical for evaluating the robustness and generalization capabilities of our proposed models. The details of this dataset can be found in Appendix L.

We generated 100 simulated data sets and conducted experiments according to the experimental design in Figure 3 in the main text. The average results of the experiments are shown below:

The results demonstrate that both the diffusion-based channel-mixing framework Diffusion-TS and the GAN-based channel-independent framework CCDGAN excel across multiple evaluation metrics, with each achieving top rankings in three categories. CCDGAN outperforms in two distance-based metrics, likely due to GANs' strength in capturing temporal distance characteristics. Meanwhile, Diffusion-TS exhibits balanced performance across all metrics, highlighting the overall effectiveness of diffusion-based generation models.

We selected some representative data sets, which cover some commonly used fields and also cover data sets of different lengths.

answer to w2: Essentially, ours is a research article that provides users with a sound recommendation guide. The proposed CCD is a combination of current techniques to overcome the defect that the central discriminator in the channel-independent generation method cannot handle the small sample time series classification problem. We then conclude that the recommendation priority of the channel-independent method should be improved. We believe that the recommendation guide is valuable to the field and users.

answer to w3: For users who do not understand this field, these results and suggestions are very important, even if it is an obvious result.

answer to q1: see answer to w2.

answer to q2: see answer to w1.

answer to q3: For the same type of methods, we used the same number of iterations, and our parameters are described in detail in Section 3.1, but the models do have differences in the number of parameters.

Thank you very much for your comments. Our peer-to-peer responses can be found below. Also, we have uploaded a corrected PDF version of the article.

answer to w1: Our primary motivation was not to address the limitations of diffusion models on small-sample datasets but rather to achieve two main objectives: (1) to analyze existing time series generation methods and establish comprehensive recommendation guidelines, enabling users to select appropriate models for diverse scenarios; and (2) to enhance the effectiveness of channel-independent methods by designing the Central Convolution Discriminator (CCD), which significantly improves their recommended priority. Experimental results strongly support both objectives.

At the same time, we also compared the performance of CCDGAN and Diffusion-TS on a small sample data set. We used the Stock data set and randomly sampled 10% of the data. The results are as follows:

Among the six evaluation metrics, CCDGAN outperforms DiffusionTS on four, demonstrating its superiority on small datasets. This advantage arises from CCDGAN’s channel-independent design and the Central Convolution Discriminator (CCD), which are better suited for capturing sparse data distributions. In contrast, diffusion models like DiffusionTS rely on larger datasets for effective training, limiting their performance in such scenarios.

answer to w2: We conducted experiments based on DLG and Air, which are from the traffic flow dataset and air dataset respectively. The experimental results are as follows:

Simulated data. At the same time, in order to demonstrate the effectiveness of our method, we used the following scheme to generate a simulated time series dataset: we adopt a framework based on the principle Y=f(X). Specifically, the process begins with sampling sequences X from a predefined random mixture distribution.

Next, we employ a binary tree structure to randomly generate mathematical expressions that define the function f(X). This mechanism ensures variability and diversity in the functional relationships represented by f.

Finally, by forward-propagating the sampled sequences X through the generated mathematical expressions f(X), we obtain the corresponding output sequences Y=f(X). This methodology provides a flexible and systematic way to simulate complex time-series data with customizable properties, which is critical for evaluating the robustness and generalization capabilities of our proposed models. The details of this dataset can be found in Appendix L.

We generated 100 simulated data sets and conducted experiments according to the experimental design in Figure 3 in the main text. The average results of the experiments are shown below: