Retrieval Augmented Time Series Forecasting

Thank you for your thoughtful comments. Please see our point-by-point response.

Claims and Evidence & W1. Rationale and ablation study of multiple periods in retrieval

Thank you for raising this important point. First, we would like to clarify that our model performs retrieval across multiple periods (scales) and does not retrieve segments from identical positions across different periods. Instead, it retrieves segments at varying positions for each period based on their similarity to the input segment. Perhaps this misunderstanding arose because Figure 3 depicted keys and values from identical positions across multiple periods. In the revised manuscript, we will update Figure 3 to show different positions of retrieval in each period and revise the corresponding explanation on lines 193–197 of the right column.

In addition to our response above, we conducted new ablation tests to empirically validate the effectiveness of multi-period retrieval by comparing performance with and without multiple periods. This new result is shown in the table below, which shows multi-period retrieval shows positive impacts for most datasets. We hope this answers your question and we will be happy to add these findings in the revised manuscript in Appendix C.2.

W2, W3. Baseline comparison in Section 5.2 and 5.3

Thank you for the suggestion. As recommended, we have conducted additional experiments by comparing RAFT against strong baselines (represented in Table 1) on carefully synthesized datasets, extending our analyses presented in Sections 5.2 and 5.3. The comparison with the four strongest baselines selected based on winning ratio is summarized in the table below. Our results demonstrate that, in the experiments from Section 5.2, the proposed method showed comparable overall performance to the baselines but outperformed them in capturing rare patterns (e.g., occurrences = 1). In the Section 5.3 experiments, our method demonstrated superior performance in modeling less correlated patterns compared to all other baselines. We will include these new experimental results in the revised manuscript.

W4. Details of implementing RAFT in AutoFormer

Rather than modifying the internal Transformer architecture to integrate our retrieval module, we directly appended retrieval results to AutoFormer’s predictions at the final stage. Here are some details: AutoFormer divides the input patches into trend and seasonal components, processes them separately, and combines them by addition at the final stage. Our retrieval results, $\sum_{p \in \mathcal{P}}{g^{(p)}(\tilde{\mathbf{v}}^{(p)})}$, are added at the final stage of AutoFormer. We will clarify these details in our revised manuscript.

Suggestion 1. Impact of stride on computational complexity and performance

We agree that searching patches in extremely long time-series can be computationally intensive. To address this, we had introduced increasing a stride into the sliding window approach used during retrieval in Appendix D. Our experiments demonstrated that increasing the stride substantially decreases computational costs with minimal impact on performance. Results show that until stride 8, the performance loss is less than 5% while we could decrease the computational cost by 89%. As mentioned in our response to Reviewer 1 (W1), our method can be easily scalable to extremely long time series.

Suggestion 2. During training, if the input sequence x is located in the middle of the training dataset S, does the retrieval module search for sequences both before and after x?

To prevent information leakage during training, any patches overlapping with the input sequence x were excluded from the retrieval candidates. After removing these overlapping patches, the retrieval module utilized patches from both before and after the sequence x. At inference time, we ensured a chronological split between the training and test sets, guaranteeing that the input sequence at test time always occurs after the entire training dataset. Therefore, there is no overlap, and the retrieval module leverages the full training set.

Thank you very much for acknowledging the novelty and wide applicability of our proposed retrieval method. We plan to further revise our manuscript by including additional experiments and clarifications in the revised manuscript.

Thank you for your thoughtful comments. Please see our point-by-point response.

W1, Q1. Scalability to extremely long series.

Thank you for highlighting this important point. We fully agree that searching patches in extremely long time-series can be computationally intensive. In light of this, we had introduced a stride into the sliding window approach used during retrieval to reduce the number of searched segments in the training data (see Appendix D). Our experiments demonstrated that increasing the stride substantially decreases computational costs with minimal impact on performance. For instance, using a stride of 8 reduces inference time by a factor of 8 (which could be even parallelized) with less than 5% degradation in performance.

Another strategy could be to train a simple encoder to project the time segments of training data into low-dimensional dense embeddings and perform the search using cosine similarity (refer to Appendix C.1 - Cosine Similarity with Projection). According to our results presented in the Appendix C.1, this method maintains comparable performance with Pearson’s Correlation (default RAFT setting). This approach can significantly reduce inference-time complexity through precomputing dense embeddings of the training data segments. The entire process can be even parallelized, enabling it to handle datasets of a larger-scale such as billions of data points at inference. We can further accelerate the search efficiency via vector search solution through the approximated nearest neighbor (ANN) technique. We will incorporate this discussion into the revised manuscript.

W2, Q3. Extension to handling confounding variables (e.g., external or exogenous features)

Thank you for this comment. Our model can be extended to incorporate external features or correlated time series by training encoders to project external information into low-dimensional dense embeddings within the same embedding space as the input features. One natural extension is to consider additional external time series. In this scenario, one encoder can project external segments into dense embeddings and another encoder can project input features into the same embedding space. When embeddings are aligned within a shared space, retrieval can use cosine similarity to identify relevant external segments for each input query patch. The retrieved external segments can then be used alongside the original input for enhanced prediction and encoders would be optimized jointly during training. This idea is consistent with our ablation study described in Appendix C.1 ("Cosine Similarity with Projection"). We will include this extension and its potential benefits in the revised manuscript.

Q2. Does RAFT degrade gracefully when the time series is short (and thus historical patches are limited)? Are there diminishing returns?

It is correct that when the available time series is short, the number of historical patches for retrieval naturally decreases, potentially limiting performance gains. To investigate this phenomenon, we conducted additional experiments comparing our model with and without retrieval, varying the length of the training data from our benchmark datasets and evaluated performance on a fixed test set while adjusting the training data proportion to 20%, 60%, and 100%. Our results indicate that the impact of the training dataset's length on performance gains varies significantly across different datasets. However, our approach consistently outperformed the baseline model, even with limited historical data. We will include these findings and further discussion in our revised manuscript. We will include these findings and further discussion in our revised manuscript.

Suggestion. It would be interesting to see more real-world interpretability examples instead of the standard datasets.

Thank you for the suggestion to showcase practical use cases with real-world data, which will enhance the interpretability of our model. We will include new interpretability examples using real-world scenarios, such as disease spread data from the COVID-19 pandemic (e.g., https://data.who.int/dashboards/covid19/) in the revised manuscript.