Series-to-Series Diffusion Bridge Model

About the introduction of $y_\theta$ : Thank you for your constructive feedback. We acknowledge that certain references to prior work, including the introduction of $y_\theta$ in Section 3.1, have been cited without adequate contextual explanation, potentially making it challenging for readers to understand their significance directly from our manuscript. To address this, we have revised the manuscript to include a more detailed explanation of $y_{\theta}$ in Lines 173-184.

About the probabilistic forecasting performance Thank you for your valuable feedback. We have addressed this concern by including additional results for horizons of 192 and 336 in Table 8 and Table 9 of the revised manuscript. The extended results demonstrate that our Series-to-Series Diffusion Bridge Model ($\mathrm{S^2DBM}$) competes effectively with CSDI and TMDM, showcasing competitive performance in terms of $\mathrm{CRPS}$ and $\mathrm{CRPS}_{\mathrm{sum}}$ for longer horizon settings.

About the performance of linear models : Thank you for your query. Recent studies highlight that linear models, despite their simplicity, often excel due to their explainability and efficiency, sometimes outperforming more complex architectures in specific scenarios. The superior performance of linear models in certain cases remains an area for further exploration. While our Series-to-Series Diffusion Bridge Model (S2DBM) does not always outperform across all scenarios, it achieves commendable results in the majority of them. The effectiveness of linear models in specific instances also provides valuable insights for future enhancements to our model.

About the impact of F: Thank you for your interest in the functional role of $F(x)$ in our algorithm. In time series forecasting, the lookback and forecast windows often differ, making it challenging for historical sequences to provide structurally informative priors for future targets, unlike the case in image restoration. Therefore, we cannot construct a direct diffusion bridge between historical time series $x$ and future time series $y$. Instead, we employ the prior predictor $F(\cdot)$ to transform historical time series into a deterministic conditional representation $h$. This representation serves as the endpoint for the diffusion process and offers initial guidance for the reverse process, as detailed in Line 347.

To validate the impact of different implementations of $F(\cdot)$, we conduct an ablation study on the ETTh1 dataset. We vary $F(\cdot)$ among a Linear model, a NLinear model, a DLinear model and a Transformer model for point forecasting with a prediction horizon of 96. The results demonstrate robust performance across these variations, supporting our decision to employ the simple yet effective Linear model due to its efficiency and effectiveness.

About the configuration Settings of S$^2$DBM: Thank you for your suggestions on exploring various hyperparameter settings to enhance the robustness and understanding of our S$^2$DBM model. We have undertaken additional experiments to evaluate the impact of different configurations, particularly focusing on the number of diffusion steps (T) and the choice of the prior predictor $F(\cdot)$. Please see section A.4.2 and section A.4.3 in the revised manuscript.

(1) Diffusion Steps Impact: We experiment with varying the number of diffusion steps in the S$^2$DBM model. Our results indicate strong robustness across different diffusion steps, confirming the model's adaptability to changes in this parameter.

(2) About the ablation Study on $F(\cdot)$: Additionally, to validate the impact of different implementations of $F(\cdot)$, we conduct an ablation study on the ETTh1 dataset. We varied $F(\cdot)$ among a Linear model adopted in S$^2$DBM, a NLinear model , a DLinear model, and a non-linear Transformer model for point forecasting with a prediction horizon of 96. The results demonstrate robust performance across these variations, supporting our decision to employ the simple yet effective Linear model due to its efficiency and effectiveness.

About the computational efficiency of S$^2$DBM: Thank you for your question regarding the computational efficiency of our S$^2$DBM model. To provide a clear perspective on its performance, especially concerning larger datasets and real-time forecasting applications, we have conducted specific tests on the ETTh1 and Weather datasets, varying the prediction horizon $L$ to assess the inference efficiency of the proposed S$^2$DBM.

About the advantages for Using a Brownian Bridge : Thank you for your insightful query. As illustrated in Figure 1, while diffusion-based methods are adequate for probabilistic forecasting, they often fall short in point-to-point prediction accuracy, primarily due to inherent randomness. A significant source of this randomness stems from the prior distribution. When CSDI employs a standard diffusion process, it generates targets using standard Gaussian noise as the prior distribution. Although TMDM enhances the prior to be more informative of the target, the prior distribution in TMDM remains confined to a noisy representation, thus providing limited information about the generation target. A Brownian bridge is a continuous-time stochastic process where the probability distribution during the diffusion process is conditioned on both the starting and ending states. This conditioning effectively 'pins down' the process at both ends, enhancing stability by reducing the influence of noisy inputs. By leveraging the framework proposed in Example 1, our model shifts from a typical data-to-noise process to a data-to-data process, significantly reducing instability from noisy inputs and enabling the accurate generation of future features based on historical time series.

We also modified the TMDM model according to the Brownian bridge we used, and conducted prediction experiments with a prediction lengths of 96 and 192 on the ETTh1 data set. The experimental results show that our modifications can significantly reduce uncertainty.

About the robustness testing of S$^2$DBM: Thank you for highlighting the importance of robustness testing. We have conducted preliminary tests to evaluate the resilience of our S$^2$DBM model under adverse conditions involving noisy inputs. Please see Section A.4.5 in the revised manuscript. Specifically, we introduce noise to the known time series $y$, i.e., $y_{noisy} = y + a \cdot \epsilon, \quad \epsilon \sim \mathcal{N}(\mathbf{0}, \mathbf{I})$. We then use the noisy data $y_{noisy}$ as input for the S$^2$DBM model and monitor the predictive performance across various noise levels by altering the coefficient $a$. The experimental results indicate that our S$^2$DBM model demonstrates commendable robustness to noise in inputs.

We acknowledge the need for further comprehensive testing, including scenarios with missing data and varying data quality levels, to more fully ascertain the model's performance under real-world conditions. Future revisions of this work will incorporate these additional robustness tests to provide a clearer demonstration of how S$^2$DBM handles such challenges.

We sincerely thank you once again for your valuable comments. In our rebuttal, we have revised our paper and provided additional results accord to your comments. As the deadline for the public discussion phase approaches, we would greatly appreciate your response to our rebuttal. We are ready and willing to address any additional concerns you may have.

About why $h$ is chosen as the endpoint for the Brownian bridge: In time series forecasting, it is common for the lengths of the lookback and forecast windows to differ, while diffusion bridges require the lengths at both ends to be the same. Therefore, we cannot directly construct a diffusion bridge between historical time series $x$ and future time series $y$. Instead, we use the prior predictor $F(\cdot)$ to transform historical time series into a deterministic conditional representation $h$, which serves as the endpoint of the diffusion process and provides guidance at the beginning of the reverse process. We have provided explanation in the original manuscriput and it is in Line 347 of the revised version.

About the Theoretical Justification for Integrating Brownian Bridge into Diffusion Models for Time Series Forecasting : Thank you for your insightful query. As illustrated in Figure 1, while diffusion-based methods are adequate for probabilistic forecasting, they often fall short in point-to-point prediction accuracy, primarily due to inherent randomness. A significant source of this randomness stems from the prior distribution. When CSDI employs a standard diffusion process, it generates targets using standard Gaussian noise as the prior distribution. Although TMDM enhances the prior to be more informative of the target, the prior distribution in TMDM remains confined to a noisy representation, thus providing limited information about the generation target. A Brownian bridge is a continuous-time stochastic process where the probability distribution during the diffusion process is conditioned on both the starting and ending states. This conditioning effectively 'pins down' the process at both ends, enhancing stability by reducing the influence of noisy inputs [1-4]. By leveraging a Brownian bridge, our model shifts from a typical data-to-noise process to a data-to-data process, significantly reducing instability from noisy inputs and enabling the accurate generation of future features based on historical time series.

[1] Yue C, Peng Z, Ma J, et al. Image restoration through generalized ornstein-uhlenbeck bridge.

[2] Liu G H, Vahdat A, Huang D A, et al. I $^ 2$ SB: Image-to-Image Schr" odinger Bridge.

[3] Wang Y, Yoon S, Jin P, et al. Implicit Image-to-Image Schrödinger Bridge for Image Restoration.

[4] Chen Z, He G, Zheng K, et al. Schrodinger bridges beat diffusion models on text-to-speech synthesis.

About the determination of Non-Autoregressive Diffusion Processes: As far as we are aware, CSDI and SSSD are not autoregressive models. Autoregressive time-series diffusion models generate future predictions sequentially over time. In contrast, CSDI and SSSD avoid autoregressive inference by simultaneously diffusing and denoising the entire time series, making them non-autoregressive diffusion models. A more detailed discussion of the categorization of time-series diffusion models can be found in [5], which explicitly demonstrates that CSDI and SSSD are not autoregressive models.

[5] Shen, Lifeng, and James Kwok. "Non-autoregressive conditional diffusion models for time series prediction." International Conference on Machine Learning. PMLR, 2023.

About the parameters of forward process: We appreciate your attention to the typo regarding the inconsistencies, which has been corrected in line 206. As outlined in Theorem 1, Eq. (1) provides a general framework for the forward process. The proposed framework only requires that the parameters be designed to ensure that $x_t$ remains pristine at $t=0$ and undergoes maximal degradation at $t=T$. We have clarified this point in the revised version; please see line 206 in the revised manuscript. Moreover, as demonstrated in Table 1, $\hat{\alpha}_t$, $\hat{\beta}_t$, and $\hat{\gamma}_t$ offer significant flexibility, enabling various parameterization techniques that confer distinct statistical characteristics upon the diffusion models. The specific values for $\hat{\alpha}_t$, $\hat{\beta}_t$, and $\hat\gamma_t$ in our proposed S$^2$DBM are detailed in Eq. (4) (line 267), Example 1 (line 310), and Example 2 (line 319).

About the comparison with TimeDiff : We appreciate the reviewer’s feedback and would like to clarify the rationale behind the organization of our results. We kindly note that the comparison with TimeDiff is presented in Table 3 of the original manuscript. The comparison with TimeDiff is not included in Table 2 because TimeDiff uses unique settings for prediction length, which differs from those of most methods in Table 2. While TimeDiff's official code has not been publicly released, we have diligently retrained our model following the settings described in the TimeDiff paper. Subsequently, we included a performance comparison with the results reported in the TimeDiff paper in Table 3 to ensure a fair and accurate evaluation. We hope this explanation addresses the reviewer’s concerns and provides clarity regarding the alignment of our experimental setup with the existing literature.

About the novelty of our work: We would like to clarify that our work goes beyond a simple combination of TMDM and TimeDiff. The contribution of our approach lies in reducing the instability caused by noisy input and enabling the accurate generation of future time-step features from historical time series. The prior distributions in both TMDM and TimeDiff are constrained to noisy representations, providing limited guidance for generating the target outputs. In contrast, our approach introduces a Brownian Bridge process, which operates as a data-to-data diffusion mechanism rather than the conventional data-to-noise process used in existing diffusion models. This paradigm shift mitigates the instability caused by noisy input and facilitates the accurate generation of future time-step features based on historical time series data.

Moreover, we contribute a novel and unified framework that integrates and generalizes existing non-autoregressive diffusion-based time series forecasting models, including TMDM and TimeDiff. This theoretical insight offers a new perspective on the design of diffusion models for time series prediction, making our contribution conceptually significant. To substantiate the distinctiveness and effectiveness of our proposed framework, we conducted additional experiments by incorporating TMDM (whose code is off the shelf) into our framework. Notably, we preserve TMDM’s original network architecture and modify only the forward and reverse processes as outlined in Example 1 of our paper. We evaluate this model on the ETTh2 dataset with a prediction length of 192. The experimental results, summarized below, clearly demonstrate that our framework significantly enhances TMDM’s performance: