Thank you for your detailed and thoughtful review! We will address your concerns point by point.

Q1: The statement may not hold when time series data shows no periodicity.

We appreciate your careful examination of this matter. Indeed, as you pointed out, the formulation becomes less rigorous for some irregular time series, such as in the case of a time series generated by random walks. Nevertheless, it is notable that periodicity or stationary trend constitutes a fundamental indicator of time series predictability and forecasting potential. Long-term forecasting can be very challenging if time series exhibit no periodicity or stationary trend. For example, DLinear demonstrates that a naive baseline that simply copies the most recent observation can outperform almost all SOTA deep learning models that on the aperiod financial dataset [1]. We also reach a similar conclusion by analyzing the 1st channel of Weather dataset described in Sec. 5. So, our statement holds true for time series that are predictable and have strong forecasting potential. And through extensive experiments, we demonstrate both the soundness and superiority of our approach, which is designed based on the principles of the statement, across the majority of real-world scenarios.

Also, motivated by your concerns, we will include an additional discussion section in the revised paper to elaborate on this issue, as well as other potential limitations.

Q2: Some equations exists errors.

Quite thanks for your reminder! We will carefully review all the formulas and correct the errors in the revised paper.

Q3: The details of Periodicity Injection for each dataset should be described in the experimental part.

Thanks for your valuable suggestions. We will add the below detailed description of Periodicity Injection in our revised paper according to your suggestion:

For each dataset, firstly, we use the AutoCorrelation Function (ACF) to calculate the dominant period $p$ of this dataset (the period of all datasets calculated by ACF are listed in the 5th column in Table 7 in Appendix D). Next, during the grid search process for the hyperparameter chunk size, for each experiment setting, we input the the calculated $p$ and the hyperparameter chunk size $S$ into Algorithm 1 provided in Appendix E, to obtain a modified matrix initialized via Periodicity Injection. This matrix is then used to replace the first basic correlation matrix of the initialized model, thereby completing the Periodicity Injection operation.

Q4: Please specify the reason and the advantages against SparseTSF

SparseTSF significantly reduces the number of parameters by downsampling the original time series. Additionally, it adopts a Channel-Independent strategy (Described as One Bus in Appendix B)—modeling a single shared temporal structure for all time series—to minimize the overall number of model parameters. However, the oversimplified modeling of the time series forecasting task results in severely limited representational capacity. Since it can only model a simple and singular temporal structure for a given time series system, this method fails to accurately capture the diverse temporal patterns that may exist across various time series within the system. As a result, its forecasting performance tends to be suboptimal.

In contrast, with the help of Correlation Mixing strategy, CMoS focuses on building several fundamental basic correlations that represent the diverse and inherent patterns of the whole system, and apply a specific mixing strategy for each channel to capture the channel-specific and more accurate temporal structure, thereby greatly improving the prediction performance. According to the experimental results, CMoS consistently outperforms SparseTSF with a significant margin on datasets containing more than 20 channels (more channels often contain a greater diversity of temporal structures). Therefore, when comprehensively considering the trade-off between prediction performance and parameter efficiency, our method stands out as the optimal choice among existing approaches including SparseTSF.

[1] Zeng, Ailing, et al. "Are transformers effective for time series forecasting?." Proceedings of the AAAI conference on artificial intelligence. Vol. 37. No. 9. 2023.

Thank you again for your kind review, and we hope our response can address your concerns!

Thank you for your detailed and thoughtful review! We will address your concerns point by point.

Q1: Details about Fig. 1

As an illustrative example, we use the MSE of each pair of chunks as the correlation value (labeled in the figure), and a lower MSE meaning stronger correlation. Only when the shapes of two chunks are relatively similar can the MSE between them be close to zero, indicating a strong correlation between the two chunks. In other cases, there is no significant relationship between them since their MSE is usually much greater than zero. We would be pleased if the above description can address your conerns!

Q2: The differences between your method and other chunk-based techniques.

Thanks for your valuable question. Despite all three approaches employing chunk operations on raw time series,

both Chunkformer and REWTS emphasize modeling intra-chunk relationships within individual chunks. However, this design suffers from poor interpretability, as the internal mechanisms (such as attention weights of intra-chunk correlation) are often opaque. It's hard for us to understand how these intra-chunk relationships contribute to the future predictions. Moreover, since intra-chunk modeling is highly sensitive to each value within a chunk, the model's performance is often greatly affected when the data contains high levels of noise or outliers.

In contrast, our CMoS focuses on direclty modeling the relationships between historical chunks and prediction chunks (inter-chunk relationships). This approach offers several key advantages:

• Enhanced Interpretability: Our method provides clear insights into how historical chunks influence future predictions (as shown in Sec.5).
• Improved Robustness: The focus on broader temporal relationships rather than fine-grained intra-chunk patterns makes our method more resilient to noise in the data, which is theoretically proved in Sec. 3.1 and experimentally proved in Sec. 4.3.

We will add this point to our related works to help readers better understand the differences between CMoS and other methods.

Q3: Can you compare your aggregated spatial correlations with attention-based methods?

Sure! We visualized the attention-based representation (att1&2.png, based on the attention score of PatchTST's 2 layers) and the correlation of CMoS (cmos.png) on weather dataset in anonymous repo https://anonymous.4open.science/r/kjUH.

From the picture, we find that there is a significant difference between the two. The attention-based correlation represents the attention relationships among historical data points, but it does not reveal how these relationships contribute to the prediction of future time series. As a result, it is difficult to derive an intuitive explanation of the forecasting process from the attention-based correlation. In contrast, our correlation directly captures the mapping from historical data to the future time series. This allows us to clearly see which historical chunks contribute more to the prediction of future chunks, making the model’s decision process more interpretable.

Q4: Can you further include time cost and flops?

That's a quite good suggestion! We provide the inference FLOPs, GPU memory footprint, and inference time of CMoS and other baselines on Electricity dataset using a 3090 GPU as follows. The batch size of all methods is set to 64.

Although the memory allocation strategy and powerful computational performance of the 3090 may narrow the gap in computational overhead among models, it can still be seen that, CMoS consistently maintains an advantage in computational overhead except for SparseTSF. It is also notable that the prediction performance of CMoS greatly outperforms SparseTSF, especially on those datasets with more channels. This means CMoS can achieve the best effectiveness-efficiency balance among all methods.

Q5: Some failure cases.

When the underlying data distribution shifts significantly over time (i.e. concept drift), such as sudden changes in market behavior or consumer patterns, the basic correlations may not include the new time strutures, so these rapid distribution shifts can affect the prediction accuracy of CMoS.

However, it's also important to note that this is a fundamental difficulty in time series forecasting that the entire field is actively working to address. A possible solution is to quickly update the model when facing concept drifts, and owing to the efficiency advantage, CMoS can be updated more rapidly and at a higher frequency. This rapid adaptation capability allows CMoS to mitigate the impact of concept drift more effectively than other methods.

Thank you for your detailed and thoughtful review! We will address your concerns point by point.

Q1: How could Theorem 3.2 be extended to nonlinear dependent or non-Gaussian noise (e.g., burst noise), and what empirical evidence supports its robustness under such conditions?

• Extended to Non-Gaussian noise like burst noise

Take the burst noise as an example, it can be viewed as extreme deviations that occur in the tail of a noise distribution. So we can define burst noise $B(t)$ mathematically as follows based on extreme value theory. Let $X(t)$ be a noise process. According to the Pickands–Balkema–de Haan theorem, the conditional distribution of the exceedances events exceeding a sufficiently high threshold $u$ follow a Generalized Pareto Distribution (GPD): $GPD(y ; \sigma, \xi)= P(Y(t)\le y|X(t)>u) \approx 1-\left(1+\frac{\xi y}{\sigma}\right)^{-1 / \xi}$,where $y=x-u>0$, $\sigma>0$ is the scale parameter, and $\xi > 0$ is the shape parameter. Next, since the burst noise can be regarded as discrete events, we define their occurence times ${T_i}$ follow a Poisson process with intensity $\lambda(u)$. So we can finally define burst noise $B(t)$ as $B(t) = y+u,\text{if}\ t\in\{T_i\},\ \text{otherwise}\ 0$, where $y$ is i.i.d. $GPD(\sigma, \xi)$.

However, when we attempt to replace $\delta$ with $B$ in Definition 3.1, we find it difficult to theoratically analyze $Var(\theta^TB)$ since B is sparse and thus $Var(\theta^TB)$ relies heavily on specific samples of GPD.

As an alternative, we choose to experimentally compare the performance of chunk/point-level modeling on the time series with random burst noise. Specifically, we construct several time series using sine functions, and following the above formulation, we additionally inject some burst noises into all time series. Some segments of these time series are visualized in burst.png in the anonymous repo https://anonymous.4open.science/r/kjUH. The prediction results are listed in the following table, and we can conclude that chunk-level modeling is more robust than point-level modeling when facing burst noise, which can be seen as an empirical extension of Theorem 3.2 on other non-Gaussian noise like burst noise.

• Extended to nonlinear dependency

With regard to modeling nonlinear dependency, the presence of nonlinear interactions makes it difficult to derive closed-form solutions or establish rigorous mathematical proofs. Specifically, in Definition 3.1, if $f(\theta)$ is a nonlinear function, it's hard to obtain a parametric mathematical expression for $Var(f(x';\theta)-f(x;\theta))$, which hinders the following theoretical derivations. So we design a 3-layer MLP with nonlinear activation to validate the effectiveness of chunk-level modeling. From the results, we can find that chunk-level modeling outperforms point-level modeling in most cases, indicating its robustness on multiple datasets.

Q2: Inference speed and memory footprint

That's a quite good suggestion! We provide the inference FLOPs, GPU memory footprint, and inference time of CMoS and other baselines on Electricity dataset using a 3090 GPU as follows. The batch size of all methods is set to 64.

Although the memory allocation strategy and powerful computational performance of the 3090 may narrow the gap in computational overhead among models, it can still be seen that, CMoS consistently maintains an advantage in computational overhead except for SparseTSF. It is also notable that the prediction performance of CMoS greatly outperforms SparseTSF, especially on those datasets with more channels. This means CMoS can achieve the best effectiveness-efficiency balance among all methods.

Q3: Automatically select the chunk size of the dataset.

That's quite a valuable question. As there is no enough information about period, using adhoc chunk size might not be a good choice. The good news is that since CMoS is a super-lightweight model, it is entirely affordable performing hyperparameter search within a certain range in most cases. Therefore, in practice, we can take advantage of advanced hyperparameter optimization algorithms (such as Bayesian optimization) or frameworks (such as Optuna) to automatically find the optimal chunk size for best performance on these aperiodic or irregularly sampled series.

Thank you again for your valuable review, and we hope our response can address your concerns.

Thank you for your detailed and thoughtful review! We will address your concerns point by point.

Q1: Novelty of some components

Your example is quite helpful! So we explain the two aspects you mentioned following the format you provided:

• The use of ConvNet.

Existing methods like DeepTCN or TimesNet employs one ConvNet backbone for all channels. Since different channels may exhibit different noise levels, only one backbone may struggle to resist interference from various levels of noise. In contrast, CMoS allocate a specific lightweight ConvNet for each channel to eliminate the channel-specific noise variations, thus enhancing the model's robustness.

It is also notable that compared to this specific component, the whole Correlation-Mixing framework (including ConvNet) is a more important innovation. As shown in Appendix B, compared with existing channel strategies, our Correlation-Mixing can effectively model diverse temporal and even cross-channel dependencies with great efficiency. ConvNet plays an important role in reducing the effect of noise in this framework.

• Chunk v.s. Patching.

Technically, patching only splits the historical series into segments, and patch-based models like PatchTST focus on generating aggregated representation of the correlations between these historical segments (similar to the high-level semantic information in LLMs) and then decode the representation to future time points. The black-box nature of such representations make it hard to figure out how specific segment influence the final prediction, limiting the interpretability of these methods.

In contrast, chunk splits both historical and future series, and instead of learning the high-level representations, chunk-based CMoS focus on directly modeling of the spatial correlation between historical and future segments. which is quite interpretable. Also, we proved that chunks bring benefits in robustness and efficiency.

To better understand their differences in interpretability, we visualized the patch-based representation (att1&2.png, based on the attention score of PatchTST's 2 layers) and chunk-based correlation (cmos.png) on weather dataset in anonymous repo https://anonymous.4open.science/r/kjUH. We can easily figure out how historical segments contribute to future segments for each time series in cmos.png ,while it's hard to obtain similar or other interpretable information from att1&2.png.

We will provide more details and recent works in the next version according to your valuable suggestions! Also, reviews about works before 2023 will be added.

Q2: Inject incorrect periodicity

That's an interesting question. To simulate this case, we randomly select an integer x!=period as interval in injection phase. Comparing the below results (MSE) with Table 3 in the paper, it can be seen that using incorrect period performs barely better than random initialization. So we suggest injecting the correct period only when most time series in the system passed the ACF test—a scenario that commonly occurs in real-world applications.

Q3: More Baselines

We follow your suggestion to include more baselines. The prediciton results (MSE) of naive, seasonal naive, Time-LLM and Crossformer are listed in the below. Since Time-LLM leverage very large LLM as part of the model, the trainging time is quite long (over 2 days for single setting). Therefore, we currently have only obtained partial results. The full results are on in its way.

From the results, CMoS outperforms these methods, indicating that CMoS is a quite effective model.

Q4: Other concerns

Very sorry that the response here cannot cover every point you have mentioned due to the text length limitation, and we can further discuss any uncovered points during the discussion phase. Here are the brief replies for some points:

• Parameter concerns: We determined the grid sets through extensive experiments.
• More visualizations and ablations: These are valuable suggestions. We will perform more experiments and analysis.
• K matrices are shared: As mentioned in Sec. Introduction, the K matrices are designed to learn some basic temporal structures in the system, and each channel finds a specific way to combines these basic correlations. This design brings both efficiency and robustness benefits.
• Nonlinear injection: If certain modules strongly correlate with periodicity, we can generalize Periodicity Injection to these modules. We believe this would be an interesting direction to explore.