Fast and Slow Streams for Online Time Series Forecasting Without Information Leakage

I think the problem about data leakage is very important in practice. I further suggest the authors provide some concrete examples (maybe in the appendix) to better illustrate this problem, and emphasize its importance. For example:

In practice, say I have a model to predict the temperature of the next 24 days, and I want to update my model everyday. Yesterday my model has a prediction of today and the next 23 days, according to which I would like to update my model today. Of-course only today's ground truth is available: meanwhile in previous work they update model parameter today according to gt value of today and the next 23 days, and that's data leakage described in this work. Unfortunately, this problem of data leakage has been found in previous work for OTSF.

It is an important problem not only in evaluating models, but also in practice. Take quantitative trading as an example, evaluating a prediction model with such data leakage (as in previous work for OTSF) in backtesting would lead to over estimation. In trading firms they would avoid such data leakage to avoid fake profits in backtesting, but many of previous OTSF researches are not avoiding this issue, making previous research results not that practical in real scenarios.

By pointing out this problem and provide corresponding methods for OTSF, this work can potentially close the gap between OTSF researches and real scenarios for OTSF.

The description of the data leakage problem in this work is indeed complicated, and the importance of this problem might not be completely emphasized in this work. Therefore, I suggest the authors provide some concrete examples and emphasize the importance of this problem in practice.

Q1: Different look-back window lengths

Could you conduct a grid search for different look-back horizons on selected datasets, models, and forecasting horizons to demonstrate robustness with respect to look-back horizon? I'm not suggesting that you repeat all experiments. Conducting grid searches for a subset, particularly with smaller models and datasets (e.g., DLinear, iTransformers on ETTh1, Exchange; H=24), would be computationally feasible and enhance the robustness of your results.

As included in [Section A.8, line 1173],

We conduct a grid search over the look-back window length $L \in {24, 48, 96, 192}$. Contrary to the common belief that longer lookback windows provide more information and thus enhance forecasting performance, our findings suggest otherwise. The results reveal that in most cases, model performance declines as the look-back window length increases. Interestingly, a look-back window length of 24 often results in better performance, regardless of the horizon window.

Impact of lookback window length $L \in \{24, 48, 96, 192\}$:

Q2: Relationship of buffer size and look-back window lengths

I noted the experiments on different buffer-size in appendix. Would it be possible that these experiments could partially represent or replace experiments about different look-back horizon?

Intuitively, buffer sizes and look-back window size should exhibit some correlation. If a larger buffer size improves MSE performance, it implies that historical data streams are related to the current distribution being predicted. Similarly, if a longer look-back window enhances performance, it indicates that incorporating more past information is beneficial for prediction.

However, while larger buffer sizes consistently yield better results [Section A.5.1, line 943], a longer look-back window does not always lead to improved outcomes, as shown in the experiments in [Section A.8, line 1173]. This discrepancy might be due to the ER strategy used. Recall that the slow stream must wait for $L+H$ steps to ensure that only data pairs with all ground truth values are included. A larger look-back window $L$ means the data sequences have to wait longer before being added to the buffer, potentially resulting in a less informative slow stream and reducing the effectiveness of the ER procedure. Consequently, in this scenario, experiments on buffer size cannot directly relate with the results of look-back window sizes.

Q3: Adapting DSOF to foundation models

Recent work has proposed Foundation Models for Time Series Forecasting, and there are papers discussing Test Time Adaptation for Time Series Forecasting. Would it be possible that the predictions of base models can be improved by the DSOF framework?

Foundation models for time series forecasting typically involve two phases: pre-training on a large dataset and adaptation to specific tasks or datasets. DSOF could potentially enhance the adaptation phase of these models. By enabling online adaptation, the DSOF framework can be particularly beneficial for datasets where the temporal domain of the time series changes rapidly over time.

Combining the advantages of foundation models and online learning has the potential to create more accurate forecasts that can be applied effectively in real-life situations. During the pre-training phase, foundation models develop strong generalization capabilities. DSOF can further ensure that the model remains accurate and relevant by adapting in real-time to new data. However, this is a broad area that warrants further exploration and research to fully understand its potential impact.

Q4: Interaction of ER and TD Loss

In Figure 3, the combined use of ER and TD loss leads to a significant performance boost, but the individual contributions of each are less apparent. This pattern is confusing because it suggests that the benefits of each component alone are minimal, raising questions about their necessity if they do not independently improve performance. Without a clearer understanding of how these components interact, it is difficult to assess their true value within the framework. The authors should provide a more detailed analysis explaining why ER and TD loss, when used together, show such a marked improvement over their separate contributions. This might include examining how these methods complement each other and identifying specific scenarios where one is more effective than the other. Additional ablation studies or visualizations showing how each component contributes to different aspects of the model’s learning could offer deeper insights into their interactions.

In [Section A.2.3, line 772], we explore how the methods complement each other and identify specific scenarios where one is more effective than the other. We provide visualizations to illustrate these interactions.

For sequences with regular cycles, ER helps reduce variance by providing stable and confident predictions through training with batches of historical sequences. On the other hand, the TD loss component relies on pseudo labels for updates, where only the initial data point is based on ground truth, and the remaining points are less certain predictions from the teacher model. Relying solely on TD loss can lead to significant variance in predictions, even for predictable sequences with regular cycles.

For immediate changes in data distribution, TD loss can make rapid adaptations. However, the ER component, which depends on historical data, may make outdated assumptions about the current distribution, potentially leading to confident but inaccurate predictions in rapidly changing environments.

The interaction between ER and TD loss resembles a bias-variance trade-off. By utilizing both components, the model can harness their strengths to balance this trade-off, resulting in more accurate and reliable predictions across different data distributions.

Q3: Computational costs of DSOF

The paper does not provide a thorough analysis of the computational cost associated with running DSOF in real-time environments. Given the dual-stream approach and reliance on replay buffers, the method could potentially introduce higher computational demands, especially when dealing with large datasets or long lookback windows. Include a detailed assessment of the computational overhead introduced by the dual-stream approach. Specifically, measure the latency, memory usage, and training time, comparing them with baseline batch learning and other online learning frameworks. This would provide a clearer understanding of the trade-offs involved and help identify scenarios where DSOF is more efficient or where further optimizations are needed.

In addition to the online training framework of FSNet and OneNet (which we refer to as DGrad), we also investigated other online learning frameworks, specifically TFCL and DER++. Unlike DSOF, these methods — DGrad, TFCL, and DER++ — do not incorporate a student model. To facilitate a fair comparison, we also present an alternative version of DSOF that excludes the student model, referred to as DSOF w/o $\theta^{(S)}$. More details can be found in [Section 4.2, line 411], and a comparison of MSE performance is presented in [Section 4.2, line 486].

Then it is followed by an evaluation of runtime in milliseconds per iteration (ms / itr) in [Section A.1.1, line 686], GPU memory usage in mebibyte (MiB) in [Section A.1.2, line 701].

Wall time comparison (ms/itr) across various frameworks:

Processor time (ms/itr) comparison across various frameworks:

Comparison of GPU memory usage (MiB) during the online phase across different online learning frameworks:

We recognize that our method has certain drawbacks in terms of runtime. To address this, we conducted an analysis of how the main components of our method contribute to runtime, as presented in [Section A.2.4, line 794].

Wall time comparison (ms/itr) for the two main components of the framework:

Processor time comparison (ms/itr) for the two main components of the framework:

The findings clearly indicate that ER has a greater impact on runtime than TD loss. One potential strategy to mitigate this is to use a smaller replay batch size. Our experiments, outlined in [Section A.5.2, line 969], demonstrate that larger batch sizes do not yield significant performance improvements. Therefore, employing a smaller batch size of 8 or 16 in ER can help balance efficiency and accuracy.

Q2: The Neccessity of Redefining the OTSF setting

While the paper argues that existing OTSF settings introduce information leakage, it overlooks that many practical use cases update the model immediately with each new data point rather than accumulating a sequence. Although the authors highlight the risks of information leakage, real-world scenarios often prioritize fast adaptation to incoming data, even if it means some overlap in training and evaluation periods. This practice, while not perfect, remains a common approach because it ensures the model quickly adapts to recent changes. The authors should address why the proposed method of avoiding information leakage offers a practical advantage, despite differing from common real-world practices. It would also help to show more use cases where avoiding this overlap leads to clear benefits, thus ...

Before addressing the core questions regarding the necessity of our proposed setting, we would like to ensure clarity by providing a more detailed description of our approach. Let us assume the input and output windows have lengths $L = 5$ and $H = 4$, respectively.

• Previous Setting: At time $t=10$, the model uses data from $t=3$ to $t=7$ to predict values from $t=8$ to $t=11$. Once the actual data for $t=11$ is available, predictions made at $t=10$ are evaluated against the ground truth, and the model is updated using the data from $t=8$ to $t=11$. This process repeats at $t=12$, advancing predictions by one time unit.
• Our Setting: At $t=10$, the model uses data from $t=6$ to $t=10$ to forecast values from $t=11$ to $t=14$. At $t=11$, the model is updated with ground truths only up to $t=11$. The predictions made at $t=10$ are evaluated on or after $t=14$, when all actual data from $t=11$ to $t=14$ is available. This process repeats at $t=12$, advancing predictions by one time unit.

You can refer to the updated problem statement of OTSF setting in [Section 1.1, line 041] and the revised figure at [Section 1.1, line 062] for further clarification.

Clarifications:

1. "While the paper argues that existing OTSF settings introduce information leakage, it overlooks that many practical use cases update the model immediately with each new data point rather than accumulating a sequence."

   Similar to FSNet and OneNet, DSOF also makes predictions in a moving window of 1 and updates the model immediately when each new data point arrives. For instance, at $t=10$, we predict $t=11, 12, 13, 14$. When the ground truth at $t=11$ becomes available, DSOF uses this data to update the model parameters, aiding in the prediction of the next sequence $t=12, 13, 14, 15$. At $t=11$, even if we do not have actual data of $t = 12, 13,14$, the model can still be updated immediately by the fast data stream, which involves creating psuedo labels that concatenates the ground truth at $t = 11$ and the teacher model's predictions for $t= 12, 13, 14$

2. "why the proposed method of avoiding information leakage offers a practical advantage"

   Avoiding information leakage in model evaluation is crucial for ensuring accurate and reliable performance assessments in real-world applications.

   In the previous setting, information leakage occurs because the evaluation at $t=12$ includes data points from $t=9$ to $t=11$, which have already been used to optimize the model parameters at $t=11$. This overlap in time steps results in biased evaluation outcomes, leading to an overestimation of the model's effectiveness in real-world applications.

   For instance, as highlighted by reviewer oqzm, in quantitative trading, evaluating a prediction model with data leakage can lead to overestimation during backtesting. Trading firms avoid such leakage to prevent false profits in backtesting, underscoring the impracticality of previous OTSF research results in real scenarios.

3. "show more use cases where avoiding this overlap leads to clear benefits"

   Our redefined setting provides additional advantages by allowing for $H$ future predictions. In contrast, previous settings, even with $H > 1$, only forecast one future step at any given timestamp. For instance, at $t=10$, the model predicts values from $t=8$ to $t=11$, even though the ground truth for $t=8$ to $t=10$ is already known. Consequently, setting $H > 1$ in the old setting is effectively the same as setting $H = 1$. Setting $H > 1$ introduces unnecessary computational costs.

   It is important to note that our setting also involves overlap in predictions, as input data sequences are generated using a moving window with a step size of 1. However, the key distinction is that the previous setting overlaps in timestamps with known ground truth, whereas our setting overlaps in timestamps without ground truth. This means that when $H > 1$ is set, the framework predicts the data point at $t=10$ multiple times before the time actually reaches $t=10$.

Q1: Concerns Regarding OneNet's Performance

The results of the baselines, especially the comparison between OneNet and FSNet, are unexpected and raise concerns about the experimental setup. Specifically, OneNet, which is theoretically more robust and effective, shows worse performance compared to FSNet, especially on datasets like ECL, which have many variables. This discrepancy is puzzling because OneNet should typically outperform FSNet due to its more comprehensive handling of cross-variable and cross-time dependencies.

After carefully reviewing the code and comparing it with the version in the official repository, we discovered that the OCP block of OneNet was not trained as intended. After making the necessary corrections, we observed an improvement in the results, which are now more reasonable. (This also confirms the effectiveness of the OCP block in OneNet.) In most cases, there was a significant reduction in MSE. We appreciate for bringing this to our attention.

The results have been updated in the main text, which includes the average of 5 repeated runs of the experiment [Section 4.1, line 378], as well as in the appendix, which displays the standard deviation [Section E.1, line 1289].

We believe that the previous table in our earlier response was unclear due to the excessive number of columns. We have rewritten the table by removing the sub-columns for (1) FSNet and (2) OneNet trained under the batch learning setting, as well as the (3) standard deviation (SD) columns. The revised table clearly demonstrates that OneNet performs better with OCP training. Under the DSOF setting, the performance of OneNet shows significant improvement after retraining. Only one or two instances of prediction length 1 are underperforming, which may be attributed to hyperparameter issues.

OneNet's performance with DSOF:

One possible reason it might have been observed that "before retraining" performs better than "after retraining" could be that the "Batch Learn" column was looked into instead of the "DSOF" subcolumn in the previous table. Given that the OneNet's analysis is based on regret minimization in online learning, the OCP block may not be designed for batch learning. This is similar to FSNet, which tends to perform significantly better in online scenarios compared to the batch learning setting.

OneNet's performance in Batch Learning:

Novelty 1: redefined the OTSF setting to close gap of research and real-life appliations

how the proposed model addresses the information leakage?

Previous OTSF settings used in existing OTSF works ( such as FSNet and OneNet):

• At time $t=10$, the model uses data from $t=3$ to $t=7$ to predict values from $t=8$ to $t=11$. Once the actual data for $t=11$ is available, predictions made at $t=10$ are evaluated against the ground truth, and the model is updated using the data from $t=8$ to $t=11$. This process repeats at $t=12$, advancing predictions by one time unit.

Two problems can be identified from using the previous setting:

1. Information leakage occurs because the evaluation at $t=12$ includes data points from $t=9$ to $t=11$, which have already been used for parameter optimization at $t=11$. This overlap leads to biased evaluation results and overestimates the model's effectiveness in real-world scenarios.

2. The model forecasts only one future step at any given timestamp. For instance, at $t=10$, it predicts values from $t=8$ to $t=11$, even though the ground truth from $t=8$ to $t=10$ is already known. Consequently, setting $H > 1$ in the old setting is effectively the same as setting $H = 1$. Setting $H > 1$ introduces redundant computational costs.

To solve the above problems, we redefined the setting: At $t=10$, the model uses data from $t=6$ to $t=10$ to forecast values from $t=11$ to $t=14$. At $t=11$, the model can only be updated with ground truths up to $t=11$. The predictions made at $t=10$ are evaluated on or after $t=14$, when all actual data from $t=11$ to $t=14$ is available. This process repeats at $t=12$, advancing predictions by one time unit.

For further details, please refer to the updated description of the OTSF leakage problem in [Section 1.1, line 041] and the illustration in [Section 1.1, line 061]. The description of the new setting can be found in [Section 1.1, line 086].

Novelty 2: applying DSOF to the modified OTSF setting

The way to utilize teacher-student style models and experience replay seems quite standard. I suggest authors include some toy examples or a theoretical analysis to motivate the necessity proposed method from a more principled way.

Our primary contribution is redefining the OTSF setting to eliminate information leakage. Then we realized that this setting poses challenges because the predicted sequence cannot be fully updated in a supervised manner due to the lack of ground truth for subsequent time steps, as discussed in [Section 1.2, line 095]. For example, at $t=10$, we predict $t=11, 12, 13, 14$. At $t = 11$, when the ground truth at $t = 11$ becomes available, updating the prediction made at $t = 10$ poses a challenge, as we only have the actual data for $t = 11$ while the data for $t = 12, 13, 14$ is still missing. This limitation hinders the model's ability to learn the most up-to-date information effectively, as seen in conventional time series forecasting.

This challenge is further discussed in [Section 4.2, line 409], where DGrad, a simple method that eliminates information leakage by delaying ground truth feedback, does not yield optimal results and can lead to unreasonable large MSEs. Therefore, we proposed DSOF. We emphasize that DSOF is not a new model but a model-agnostic framework that enables deep time series models designed for batch learning to train online. Although teacher-student models and ER are standard approaches, from the results shown in [Section 4.2, line 486], it effectively reduces MSEs and maintain training stability.

Here we provide an introduction of the mechanisms in DSOF for tackling the above challenge within the newly defined context:

• DSOF addresses the challenges through a dual-stream approach. The fast stream processes the latest data immediately, without waiting for complete ground truth, and employs temporal difference (TD) learning to improve near-future forecasts. Meanwhile, the slow stream utilizes experience replay (ER) to stabilize the training process, reduce overfitting to noise, and ensure accurate generalization of the data distribution. This approach allows our method to effectively leverage the most recent data stream knowledge while maintaining robust generalization capabilities.

Overview of the two novelties

According to the current draft, the novelty appears limited.

We appreciate your feedback and would like to clarify the novelties in our work. Our research presents two key contributions that address the shortcomings of existing methods.

First, we redefined the OTSF setting to prevent information leakage. In the redefined setting, data included for evaluation cannot be used for model parameter updates initially. This ensures that the evaluation process remains unbiased.

Second, based on this new OTSF setting, we introduce DSOF, a model-agnostic framework that operates within this redefined setting. It allows for learning the most up-to-date information without requiring all ground truth data points of the sequence to be available beforehand.

Q5: Introducing a parameter $k$ for constructing pseudo labels

Alternatively, student updates could leverage k next real data for some k>1, at the cost of a delay of k time-steps. The student of the current work uses k=1. Alternatively, one could consider a whole spectrum (or hierarchy) of models with different values of k (for example powers of 2).

Thank you for your suggestion. The corresponding changes can be found in [Section A.6.2, line 1100] of the revised paper.

Specifically, a new parameter $k$ is introduced in constructing a pseudo label. This parameter allows for adjusting the balance between ground truth data and the teacher model's predictions. The value $k$ indicates the use of the most recent $k$ ground truth data points and the first $H-k$ steps of teacher model's prediction to construct the pseudo label.

In this experiment, instead of relying solely on the most recent data, we utilize a range of $k$ values, specifically powers of 2 that are less than the prediction sequence length as suggested.

In the following table, the setting of using powers of 2 is denoted as $k = 2^p$, while the setting $k=1$ corresponds to the original method in the pseudo label construction. The results indicate that this method is particularly beneficial for larger values of $H$. For instance, applying to the ETTh2 dataset using PatchTST with prediction length $H = 96$ provides significant improvements.

Q1: Appendices

The Appendices are missing from the pdf.

In the revised paper, the appendix has been incorporated at the end of the main text. Previously it was part of the supplementary materials.

Q2: Connection to RL and TD learning

The connection to RL and TD learning in section 3.3 seems very loose. It would be better not to use TD terminology.

We established a connection of TD learning in RL to DSOF in [Section B, line 1205]. We would be grateful if you could review it to see if it addresses your concerns.

Q3: Achieving DSOF performance with batch learning computational costs

Computational complexity: the DSOF algorithm involves two networks. Moreover, the teacher model requires to generate a long sequence of samples at each time-step, which can be very computationally costly in certain architectures like decoder transformers. I wonder if the reported outperformance could be achieved under the same computational budget as the baselines?

There are batch learning methods in time series forecasting, such as RevIN [r1], focus on enhancing data generalization in testing domains without requiring online updates. While it may be possible to achieve good results on benchmark datasets without online updates, it is important to note that the objectives of batch learning methods and online learning methods differ.

Batch learning methods aim to improve test-time generalization, making them suitable when we expect unseen data to come from a similar distribution as the training data. These methods excel at generalizing beyond the specific examples they were trained on. Conversely, online learning methods are more appropriate when test data may differ from the training distribution, as they can adapt to emerging correlations and patterns in the temporal domain that batch learning might miss.

Hybrid approaches that combine elements of both paradigms are also feasible. For instance, models can be initially trained using generalization techniques and later fine-tuned or updated with online learning techniques as new data becomes available. This approach strikes a balance between strong initial generalization and adaptability to new information.

Q4: About fast data stream

The fast learning part (Section 3.3.2) motivates by the necessity of fast updates to stay up-to-date with the recent data. For this purpose they update the student model using labels generated by the teacher. The teacher model is however out-dated. This seems like a contradiction, and it is not clear how it may resolve the problem of being up-to-date, except for the fact that the immediate next label, x_{t+1}, is used in updating the student. This seems very small updates.

In our paper, we hypothesize that model accuracy improves when more recent data points are used. For instance, when the model is at $t = 10$, given data at $t = 6,7,8,9,10$, it predicts values for $t = 11, 12, 13, 14$. Once the ground truth for $t = 11$ becomes available, the teacher model can use data from $t = 7, 8, 9, 10, 11$ to predict $t = 12, 13, 14, 15$. The inclusion of the ground truth at $t = 11$ is expected to enhance the accuracy of predictions for $t = 12, 13, 14$ made at $t = 11$, compared to those made at $t = 10$. We combine the ground truth of $t = 11$ with the teacher's predictions for $t = 12, 13, 14$ to create pseudo labels for backpropagation.

This approach is analogous to TD learning, where the value of the next state is used to update the current state, or to Deep Q-Networks (DQN) [r2], which update the policy network using the Q-value prediction of the next state from the target network. More information can be found in [Section B, line 1205]

Additionally, we assumed that in the context of $t = 11$, predictions for $t = 13$ are likely to be less accurate than those for $t = 12$. To manage the increased uncertainty of predictions at later time steps, we introduce a regularization coefficient $\gamma$. This coefficient adjusts the contribution of the error to the training loss, helping to balance the impact of inaccuracies in predictions further into the future.

[r1] T. Kim, J. Kim, Y. Tae, C. Park, J.-H. Choi, and J. Choo, “Reversible instance normalization for accurate time-series forecasting against distribution shift,” in International Conference on Learning Representations, 2022

[r2] V. Mnih, “Playing atari with deep reinforcement learning,” arXiv preprint arXiv:1312.5602, 2013

Q3: Exploring optimal values of $k$

it would be interesting to explore the optimal value of k. More specifically, instead of summing over exponential values of k, use a single k and test for what value of k=1,…,H the best performance would be achieved. This will help to understand the optimal balance between ground-truth data and teacher's predictions.

In [Section A.6.2, line 1101], we can insert a subsection to demonstrate the effects of altering the values of $k$:

• In this experiment, we set $H = 48$, with DLinear acting as the teacher model for DSOF. Our aim is to find the best ratio of ground truth values to the teacher model's predictions for constructing the pseudo label. We conduct the experiment by selecting a single value of $k$ instead of multiple values for the fast stream, and testing with evenly distributed $k$ values. Specifically, we chose multiples of 8, i.e. $k \in \{1, 8, 16, 24, 32, 40\}$. As shown in the table, smaller values of $k$ are preferred. The results align with our expectations, indicating that the model should be updated more promptly with the most recent data points.

Q4: Description of the setting in Section 1.1

I wonder why Section 1.1 begins by a numeric example before discussing the general prediction setting, as in the previous version.

We were concerned that the notation was overly complex, impacting the clarity of the paper. However, we also agree that using such notations helps maintain the formality of the prediction setting. As a result, we are revising [Section 1.1, line 041] to integrate both the notations and the example for better understanding:

• Existing works that endeavor to improve OTSF model accuracies (Pham et al., 2023; Zhang et al., 2023; 2024a) follow a uniform task setting: at time $t = i$, the model receives input data with timestamps from $t = i-L-H+2$ to $t = i-H+1$ and generates predictions from $t = i-H+2$ to $t = i+1$; at the next timestamp $t = i+1$, the actual data for $t = i+1$ becomes available, allowing the model to evaluate its previous prediction and adjust its parameters based on this new ground truth. For instance, as illustrated in Figure 1, if the input and output windows are of lengths $L = 5$ and $H = 4$ respectively, at time $t=10$, the model first takes in data from $t = 3$ to $t= 7$ and ... (The rest of the content remains identical to the main text)

Q1: Connection of DSOF to TD Learning

We acknowledge the fundamental differences between the settings of RL and OTSF. RL primarily focuses on the sequence of actions to maximize long-term rewards, whereas OTSF emphasizes short-term accuracy and continuous adaptation. Rather than completely transforming the OTSF problem into an RL framework, the term "TD" is used in this paper to indicate the inspiration from TD learning in RL, that it allows for intermediate parameter updates instead of waiting for the entire episode to finish. In the context of DSOF, "TD" highlights the process of making intermediate updates without waiting for all actual data to arrive, thus addressing the issue of missing ground truth in OTSF.

Besides, Section B of the paper is not intended to offer a rigorous mathematical formulation. Instead, its aim is to highlight the similarities in the need for intermediate updates in both RL and OTSF. However, this intention is not clearly specified in the paper. Therefore, we propose to make the following adjustments to the paper:

1. In the introduction section of the paper [Section 1.2, line 109], we can mention that TD learning serves as an inspiration (rather than being directly applied).

   • This dual-stream mechanism benefits the online training process in adaptability and stability. The fast stream, inspired by TD learning's intermediate value updates, leverages the latest information from data streams to improve near-future forecasts without waiting for the complete ground truth sequence. Meanwhile, the slow stream conducts experience replay (ER) to stabilize training, penalizes overfitting the noise in the incoming data, and ultimately learn accurate generalization of the data distribution.

2. In the methodology section [Section 3.3, line 230], we clarify that the use of the "TD" terminology is intended to highlight the process of intermediate updates.

   • This section presents the two data streams for updating the teacher-student model in Section 3.2: a slow stream that updates the model using complete ground truth data through ER and a fast stream that incorporates the latest information via a TD learning-like method in RL. The term "TD" indicates that our approach is inspired by TD learning in reinforcement learning, particularly in estimating the ground truth values at the next time step, addressing the challenge of missing ground truth in the redefined OTSF setting.

3. We can also update the introduction of [Section B, line 1208] to highlight the common challenges encountered by both TD learning and OTSF, and to explain how the principles of TD learning can be connected to OTSF.

   • As outlined in Section 3.3.2, the fast stream leverages ideas from TD learning to quickly process latest data and improve near-future forecasts. While the learning goals of RL and OTSF differ, both fields share the challenge of updating models without complete access of ground truth information. In this section, drawing inspiration from TD learning in RL, we demonstrate how the idea of estimating the value of the next state can be applied to facilitate intermediate updates in OTSF.

4. At the conclusion of [Section B, line 1236], we can add a final remark to emphasize that this section does not provide an exact formulation of the OTSF problem as an RL problem, due to the inherently different objectives of the two fields, but rather seeks to enhance OTSF performance with techniques in TD learning.

   • While insights from TD learning in RL can potentially enhance OTSF performance, OTSF should not be framed as an RL problem because their primary objectives are fundamentally different. RL focuses on optimizing sequence of actions for maximizing long-term rewards, whereas OTSF seeks accurate near-future predictions for data streams which may have changing distributions. This distinction leads to unique algorithmic settings; for instance, in RL, the current action influences the next state, whereas in OTSF, previous forecasts do not impact subsequent ones.

Q2: Mentioning the introduction of $k$ in the main text

We agree that this should be referenced in the main text. The introduction of parameter $k$ can be mentioned in [Section 3.3.2, line 264]:

• In practice, the pseudo labels can be generated with different proportions of ground truth data and teacher model predictions. Consequently, we extend Equation 5 by introducing a parameter $k$, which signifies the use of the latest $k$ ground truth data points along with the initial $H-k$ steps of predictions from the teacher model to form the pseudo label. The impact of using different $k$ values, as well as utilizing multiple $k$ values, are discussed in Section A.6.2.

Thanks to the authors for their response. My concerns about the optimal k and the first paragraphs of the introduction are addressed. The claimed connection to TD is still a pushback, because the justification in the appendix is loose and erroneous. I am inclined to raise my score to 7. However since the scores are quantized at 6 and 8, I raise the score to 8 and keep my confidence at 2.

As the discussion period nears its end, we would like to express our gratitude to the reviewers for their dedication and effort in evaluating our work. We are truly thankful for the insightful feedback and valuable comments provided. Your constructive and instructive suggestions have been instrumental in guiding our revisions, as detailed in the above post. These improvements have enhanced the clarity of our paper, particularly in articulating the problem of information leakage for a broader audience. The reviews have also encouraged us to ensure comprehensiveness in our methodology, comparisons and analysis. Your contributions have significantly strengthened the quality of our paper.