W1. Details on the evaluation of experiments

We agree that some details on the evaluation of experiments were omitted due to space limitations. Here, we provide the additional details and will incorporate them into Appendix.

 

1. Metric for TS forecasting task

Following prior works on TS diffusion models [6,18,19,26,34,38] including TSDiff [15], we utilize the Continuous Ranked Probability Score (CRPS) [9] to assess the quality of probabilistic forecasts, where we describe the details in Appendix B. Note that TS diffusion models predict the probabilistic distribution rather than the deterministic value, so CRPS accounts for the uncertainty of the predicted results, unlike MSE or MAE.

 

2. Fraction of (test) data used for inference

All datasets used in our experiments are sourced from GluonTS [2], where training and test splits are provided. We follow the setting of TSDiff [15], where the validation set is created by splitting a portion of the training dataset, with the split ratio determined by the size of the train and test datasets. Further details about the training and test sets are provided in Table A.1.

 

3. Evaluation method

For evaluation, we calculate the CRPS using only the test dataset. Specifically, we follow the approach used in TSDiff [15] by utilizing the gluonts.transform.sampler.TestSplitSampler from GluonTS [2], which leverages the final time steps of the test TS instances to predict future values.

 

Q1. Clarifying sentences and adding references

Thank you for addressing this. We are planning to improve overall writing in the revision for better clarity including your suggestions. For sure, what we meant in L27 is that noise schedules used in [24, 21] are not tailored to the TS domain (but images). Regarding M4 in L31, we have already mentioned the M4 dataset with a reference in the previous sentence (L28), but we agree that the term M4 might be confusing when used standalone.

 

Q2. Main quantitative results missing in the conclusion.

Thank you for pointing this out. As we have emphasized our results quantitatively in the introduction, we will also summarize and reiterate the main findings quantitatively in the conclusion section.

 

L1-2. Comparison with more advanced models (DLinear/FEDFormer etc).

1) Application to CSDI

We appreciate the reviewer xQJK's concern about the limited range of models used to demonstrate the effectiveness of ANT. We completely agree and, as discussed in Appendix M, we have already applied our method to another well-known TS diffusion model, CSDI [34]. We believe this further validates the applicability and effectiveness of our method across different TS diffusion models (in both univariate and multivariate settings).

 

2) Comparing with other TS forecasting models [A,B]

In short, DLinear [A] and Fedformer [B] are not compared because they are for deterministic forecasting, while our task is probabilistic forecasting; TS forecasting models for these two different tasks have been developed independently. We provide more detailed reasons as below:

• Model) TS diffusion models, including TSDiff, are probabilistic forecasting models, while models like DLinear and Fedformer are deterministic forecasting models, which have developed independently in prior works.
• Metric) For the above reason, previous works on TS diffusions [6,18,19,26,34,38] do not use them as baselines and instead employ a different metric, CRPS, to account for the uncertainty of the predicted results, rather than the MSE/MAE metrics commonly used in deterministic models.
• Dataset) TS diffusion models are mostly designed for short-term TS forecasting (STSF) [C], whereas models like DLinear and Fedformer are intended for long-term TS forecasting (LTSF). In LTSF tasks, datasets generally consist of long TS with fewer instances, and training and testing splits are usually based on different time periods of the same instances. In contrast, STSF tasks involve shorter time series with more instances, and the split is based on different instances of the TS.

Nonetheless, to address the reviewer xQJK's concern, we conducted an additional experiment with DLinear [A] using the Solar dataset [17]. Since DLinear is a deterministic model and cannot be directly compared with our task which uses CRPS as a metric, we employ a common technique for probabilistic forecasting [28] by predicting the mean and standard deviation of a normal distribution rather than exact values. The results show that DLinear yields a CRPS of 0.660, while TSDiff and {TSDiff with ANT} achieve CRPS scores of 0.399 and 0.326, respectively.

 

[A] Zeng, Ailing, et al. "Are transformers effective for time series forecasting?" AAAI 2023

[B] Zhou, Tian, et al. "Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting." ICML 2022

[C] Meijer, Caspar, and Lydia Y. Chen. "The Rise of Diffusion Models in Time-Series Forecasting." arXiv 2024

W1, Q1. Candidate schedules do not guarantee to include the optimal schedule.

As the reviewer eJYD mentioned, the candidate schedule may not include the optimal schedule. However, we note that we do not aim to find the optimal schedule; our proposed ANT is a criterion for choosing a better noise schedule from candidates based on the characteristics of the dataset.

For experiments, we selected three widely used schedules (linear, cosine, sigmoid) [4] as our candidates, incorporating five different diffusion steps and three different temperatures for non-linear schedules. This resulted in a total of 35 schedules, which we believe are sufficient, as they encompass most of the schedules used in previous TS diffusion studies [1,26,29,30].

As shown in Figure 11 and Table 12, ANT is applicable to non-trivial schedules, i.e., it gives a higher score to sensibly designed noise schedules if they perform better than trivial ones. However, as mentioned in the conclusion, finding the optimal schedule for a given dataset using our ANT score (via optimization) remains a future work.

 

W2. Relationship between ANT score and the quality of schedule

The reviewer raised concerns about the relationship between the ANT score and the quality of the schedule. However, the proposed ANT score aligns with the three desiderata of noise schedules, which can be considered indicators of schedule quality. These desiderata have been discussed in prior works and our paper:

• 1) $\lambda_{\text{linear}}$: Reducing non-stationarity on a linear scale

Previous work of DDPM [24] proposed a cosine schedule (instead of linear) and argued that maintaining a consistent noise level at each step leads to better quality, as mentioned in L38--39.

• 2) $\lambda_{\text{noise}}$: Corruption to random noise

Previous work [21] suggested that the schedule must be capable of corrupting the TS into random noise at the final step to ensure the generation of high-quality samples, as the reverse process (sampling) of the diffusion model begins with random noise, as mentioned in L40--41 and L133--135.

• 3) $\lambda_{\text{step}}$: Sufficient steps

Previous work of consistency models [31] emphasized the need for a sufficient number of steps to generate high-quality samples, as mentioned in L41--42 and L135--136. Furthermore, as shown in Table 7, using all the three components of ANT mostly results in the oracle, backing up our argument. For example, with the M4 dataset [23], using all three components results in a value of 0.026 with a Cosine(100,1.0) schedule, whereas the ANT score without considering $\lambda_\text{noise}$ yields a CRPS of 0.094 with a Cosine(10,0.5) schedule. Notably, the base schedule of TSDiff (Linear(100)) yields a CRPS of 0.036.

 

Q2. Case for higher ANT score but with better result.

A lower ANT score generally indicates a better schedule, as shown in Figure 8(a), and this tendency becomes stronger when using all three components of the ANT score. However, as the reviewer has noted, there can be exceptions where better performance is achieved with a higher ANT score, maybe because of the stochasticity of the forward diffusion process when computing ANT. Nonetheless, the schedule with the lowest ANT score should return a reasonably good performance. In our extensive analysis, we found only one such case with the Traffic dataset [3], as shown in Table 5. Among the 35 candidate schedules, Linear(50) had the lowest ANT score, yet the best performance was obtained with Cosine(75, 2.0), with a marginal difference in CRPS(0.101 vs. 0.099). It is worth noting that the base schedule Linear(100) yielded a CRPS of 0.105, which is noticeably higher than both Linear(50) and Cosine(75, 2.0).

 

Q3. Optimal noise schedule depending on diffusion models

Note that the ANT score is computed without access to the diffusion model, such that the schedule is selected based on the characteristics of the dataset, rather than the specific design choice on the model architecture being used.

Nonetheless, as the reviewer eJYD concerned, some specific design choices for diffusion models could affect the optimality of the schedule. As we had a similar concern, in Appendix M, we have applied our method to another well-known TS diffusion model, CSDI [34], where we found that ANT is effective for this model as well. We believe this further validates the applicability and effectiveness of our method across different TS diffusion models.

W1, Q1. Theoretical analysis on the ANT score

Previous works on noise schedules have demonstrated their effectiveness primarily through empirical justification rather than theoretical analysis [4,21,24]. In line with this approach, we have conducted extensive experiments to support our proposal, as highlighted by reviewers xQJK and eJYD.

Furthermore, following the previous works regarding the schedules, we focus on the intuitive aspects that schedules should meet, where the proposed ANT score aligns with the three desiderata of noise schedules:

• 1) $\lambda_{\text{linear}}$: Reducing non-stationarity on a linear scale

Previous work of DDPM [24] proposed a cosine schedule (instead of linear) and argued that maintaining a consistent noise level at each step leads to better quality, as mentioned in L38--39.

• 2) $\lambda_{\text{noise}}$: Corruption to random noise

Previous work [21] suggested that the schedule must be capable of corrupting the TS into random noise at the final step to ensure the generation of high-quality samples, as the reverse process (sampling) of the diffusion model begins with random noise, as mentioned in L40--41 and L133--135.

• 3) $\lambda_{\text{step}}$: Sufficient steps

Previous work of consistency models [31] emphasized the need for a sufficient number of steps to generate high-quality samples, as mentioned in L41--42 and L135--136.

While the advances in noise scheduling for diffusion models has mostly been made empirically, we hope our work motivates future works on the theoretical understanding of a relationship between optimal noise scheduling and the characteristics of the datasets.

 

W2, Q2. Application to other diffusion models.

We appreciate the reviewer YUJG's concern about the limited range of models used to demonstrate the effectiveness of ANT. We completely agree and, as discussed in Appendix M, we have already applied our method to another well-known TS diffusion model, CSDI [34]. We believe this further validates the applicability and effectiveness of our method across different TS diffusion models (in both univariate and multivariate settings).

 

L1. Functions used for candidate schedules.

We acknowledge the reviewer YUJG's point on expanding the range of noise functions. However, our focus is not on identifying the optimal schedule across various noise functions, but rather on proposing the criterion of choosing effective noise schedules tailored to the dataset's characteristics.

However, we emphasize that the linear, cosine, and sigmoid functions are well-established and widely used in literature [4], and recent TS diffusion models [1, 26, 29, 30] primarily employ linear or cosine schedules. Although developing the best noise function is not our primary goal, we have explored extending our approach with a more flexible noise (and non-trivial) function, such as an ensemble of cosine functions, as demonstrated in Figure 11 and Table 12, to illustrate the potential applicability of ANT for future advancements in noise functions.