Thank you for your valuable feedback. We have addressed each of your concerns as follows:

Q4.1: Novelty clarification.

A4.1: We point out our contributions below:

1. Our work is inspired by PatchTST's patch processing technique, which is proposed for use in the Transformer architecture. We hypothesize that the strong predictive performance observed in time series forecasting is attributed more to the patch embedding methods rather than the inherent predictive capabilities of the Transformers. Therefore, we design PatchMixer based on CNN architectures, which have shown to be faster at a similar scale to Transformers.
2. Our proposed approach, PatchMixer, has achieved state-of-the-art results, particularly enhancing the performance of convolutional networks in the field of time series forecasting. Quantitatively, PatchMixer outperforms the best-performing Transformer (PatchTST) by 3.9% on MSE and surpasses the leading CNN (TimesNet) by 21.2% on MSE.
3. PatchMixer significantly improves the efficiency of long-term time series forecasting with linear computational complexity. It is three times faster for inference and two times faster during training compared to the current state-of-the-art model.

Q4.2: Comparison to DLinear, PatchTST and TiDE:

A4.2: The method of channel independence was introduced by DLinear, while PatchTST proposed a patch processing technique to demonstrate the effectiveness of the original Transformer architecture. Based on these previous works, our paper hypothesizes that the efficacy in forecasting is largely due to the patch embedding approach rather than the Transformer's innate predictive capabilities. Thus, the replacement of self-attention with convolutions is not merely a simplification but a strategic choice to enhance forecasting performance. We also clarify the distinction from TiDE in our forecasting heads' implementation. TiDE employs global residual connections at the start, before the embedding. The whole process operates at a point-wise level. In contrast, our use of the linear connection is not global but rather post-embedding, with dimensional transformations involving both time steps and patches.

Q4.3: Model performance clarification and significant test.

A4.3: To ensure a fair comparison of the performance improvements, we have conducted extensive T-tests on both PatchTST and PatchMixer across three datasets: weather, ETTm2, and ETTh2. These tests are performed over all forecast lengths using 10 sets of random seeds to maintain statistical robustness. The results yield a T-statistic of -2.33 and a P-value of 0.02. This statistically significant outcome suggests that the improvements observed with PatchMixer are not due to random chance. The P-value, being less than the conventional threshold of 0.05, indicates that there is only a 2% probability that the observed improvements could occur due to variance in the data alone. Hence, we can assert that PatchMixer provides a consistent and significant enhancement in forecasting accuracy across datasets.

Q4.4: The effectiveness of the proposed components and different loss functions.

A4.4: Thanks for pointing these out. We argue that the performance improvements in PatchMixer are primarily attributed to the architecture itself. Our model's design and the combination of these components contribute to its superior performance. We show the additional experiments below to support our claim, which are included in the revised draft.

We have updated Table 2 and provided supplementary information to clarify this in the revised draft. We bold the best results, and supplement the prediction performance using only Dual Heads. The comparison of convolution is placed in a separate section in the appendix.

Our model PatchMixer can be represented by PatchMixer Layer + Dual Heads. As shown in the revised Table 2, it shows significant improvements compared to only Dual Heads and replacing PatchMixer Layer with Attention Mechanism. Note that all experiments adhere to the same hypermeter settings and the best results are bolded. For the small dataset (ETTh1), both our model and the utilization of only the Dual Heads component outperform the use of Attention Mechanism + Dual Heads. For medium (ETTm1) and large-scale (Traffic) datasets, PatchMixer Layer with Dual Heads demonstrates noticeable improvements over the use of Dual Heads alone. This underscores the indispensable predictive capacity of our convolutional module.

Besides, we have also conducted additional experiments to evaluate the impact of four different loss functions. The revised Table 4 now presents a comprehensive comparison between PatchMixer and PatchTST using these loss functions, with the superior results highlighted in bold. Here we provide the following key observations, which are included in the revised paper. First, PatchMixer consistently outperforms PatchTST across the board, affirming our model's superior performance. Secondly, SmoothL1loss is clearly inferior to the other two losses. Thirdly, the model trained with combined MAE+MSE loss shows a better performance on the MAE/MSE metrics than the model trained with MSE loss only.

Q4.5: Typos and redundant paragraphs.

A4.5: Thank you for pointing out the redundancy and typographical error. We have carefully modified the content of our paper, by moving the patch operation of TimesNet to the Related Work section and simplifying the descriptions of the Method section. Besides, the reference to "equation 8" has been corrected to "equations 10" in the Pointwise Convolution section.

Q4.6: Different patch representations.

A4.6: Thank you for your valuable feedback. Our primary focus is on investigating the impact of patch embedding methods on time series forecasting performance. We hypothesize that the strong predictive performance observed in PatchTST is attributed more to the patch embedding methods rather than the inherent predictive capabilities of the Transformers. Therefore in this paper, we use the same patch representation as PatchTST. As you imply, the discussion of TimesNet's patch method strays away from our motivation and focus. So we move this description from the Methods section to the Related Work section and briefly mention it.

Q3.3: Experiment setup clarification.

A3.3: The errors at different prediction lengths and comparison with transformer-based and conv-based methods are shown in Table 1 in Experiment chapter. The baselines and implementation details can be checked in Section A.1.2 in Baselines and Section A.1.3 Implementation Details of Appendix section. In short, the input length of ours follows 336 time steps, as DLinear and PatchTST. We directly use the numbers reported in their papers in our tables. We repeat our PatchMixer for 5 random seeds to verify its robustness. The patch hypermeters follow PatchTST for a fair comparison.

Q3.4: Model performance clarification and significant test.

A3.4: To ensure a fair comparison of the performance improvements, we have conducted extensive T-tests on both PatchTST and PatchMixer across three datasets: weather, ETTm2, and ETTh2. These tests are performed over all forecast lengths using 10 sets of random seeds to maintain statistical robustness. The results yield a T-statistic of -2.33 and a P-value of 0.02, indicating a significant improvement by PatchMixer is unlikely due to data variance alone. This affirms PatchMixer's accuracy enhancement.

Q3.4: Model Efficiency Analysis (where the speed-up comes from).

A3.4: Thanks for your reminder, we analyze the source of efficiency improvement from two aspects: model structure and computational cost. The details can also be found in Appendix 5 Computational Efficiency Analysis of our revised script.

1. PatchMixer leverages the strength of convolutions for local modeling while overcoming the limitation of their receptive fields. Through patch embedding, which transforms the 1D sequence into a 2D matrix, we address this limitation effectively. A moderately sized kernel (k=8) is sufficient to process stacked patches (p=16), as the number of patches is treated as the channel dimension. This novel approach allows for feature extraction within and across patches, enabling simple 1D convolutions to capture temporal patterns both locally and globally.
2. Although both PatchMixer and PatchTST utilize parallelization, when it comes to computational cost, under default configurations, PatchTST requires more than double the flops of PatchMixer (e.g., for forecasting 720 future time points in the ETTm1 dataset, the flops are 154.29M for PatchTST and 66.32M for PatchMixer). This substantial reduction in computational load without sacrificing performance underscores PatchMixer's efficiency.

Thank you for your valuable feedback. We have addressed each of your concerns as follows:

Q3.1: Clarification of contribution.

A3.1: We point out the key contributions of our paper below:

1. Our work is inspired by PatchTST's patch processing technique, which is proposed for use in the Transformer architecture. We hypothesize that the strong predictive performance observed in time series forecasting is attributed more to the patch embedding methods rather than the inherent predictive capabilities of the Transformers. Therefore, we design PatchMixer based on CNN architectures, which have shown to be faster at a similar scale to Transformers. The experimental results have verified our hypothesis.
2. Our proposed approach, PatchMixer, has achieved state-of-the-art results, particularly enhancing the performance of convolutional networks in the field of time series forecasting. Quantitatively, PatchMixer outperforms the best-performing Transformer (PatchTST) by 3.9% on MSE and surpasses the leading CNN (TimesNet) by 21.2% on MSE.
3. PatchMixer significantly improves the efficiency of long-term time series forecasting. It is three times faster for inference and two times faster during training compared to the current state-of-the-art model.

Q3.2: Table 2 clarification.

A3.2: Thanks for pointing this out. Table 2 has been modified following your suggestions in the updated draft. We have provided supplementary information on the prediction performance using only dual forecasting heads and relocated the convolution comparison to a separate appendix section.

Our model is represented by PatchMixer Layer + Dual Heads. As shown in the revised Table 2, it shows significant improvements compared to only Dual Heads and replacing PatchMixer Layer with Attention Mechanism. Note that all experiments adhere to the same hypermeter settings and the best results are bolded. For the small dataset (ETTh1), both our model and the utilization of only the Dual Heads component outperform the use of Attention Mechanism + Dual Heads. For medium (ETTm1) and large-scale (Traffic) datasets, PatchMixer Layer with Dual Heads demonstrates noticeable improvements over the use of Dual Heads alone. This underscores the indispensable predictive capacity of the convolutional module.

Dear Reviewer TNRF,

Thank you very much again for the time and effort put into reviewing our paper. We believe that we have addressed all your concerns in our response. Following your suggestions, we have enhanced the paper with additional experimental analysis. We kindly remind you that we are approaching the end of the discussion period. We would love to know if there is any further concern, additional experiments, suggestions, or feedback, as we hope to have a chance to reply before the discussion phase ends.

Thank you for the clarification and the additional experiments. The detail of model efficiency analysis helps to understand the benefit of the proposed approach.

My main concern is lack of the contributions. The authors pointed out three key contributions:

1. Patch-based Mixer with CNN architectures
2. Performance gain against Transformer-based (PatchTST) and the CNN-based model (TimesNet)
3. efficiency improvement compared to these two models

Point 1 is not the main strength since the proposed architecture mainly combines existing approaches. However, since it's the first time these techniques are applied to long-term time-series forecasting, I can count on it. Point 2 is still weak. Table 1 shows that the PatchMixer overall performs better than other models including PatchTST on most of the datasets. However, the ablation study (Table 2) indicates that their CNN-based model performs similarly to the attention-based one. It is better or similar to some metric (MSE) but not all of them. I suspect the high performance in Table 1 could be due to other factors e.g., architecture, training strategy, hyper-parameters, etc. Regarding point 3, I'm convinced that the major benefit of the proposed approach is efficiency.

Since their contributions partially convinced me, I increased my score from 3 to 6.

Thank you for the updates. I am in favor of accepting the paper. I would encourage the authors to respond to the remaining reviewers.

Best,

The reviewer.

Thank you for your detailed and constructive feedback. Please find below the actions taken in response to your comments:

Q2.1: Correction of citation and style:

A2.1: Thank you for pointing out the citation error in the related work section. This has been corrected in the revised manuscript. We have also realigned all tables to the top of the pages, placing their captions accordingly.

Q2.2: Clarification on 2D patches and methodology:

A2.2: We recognize the potential confusion between 2D patches in CV methods and our 1D patch approach in PatchMixer. The 2D patch is an image patch with C channels, in the shape of (H, W), obtained by the conv2d function. A 1D patch is a sequence patch of length P with N channels, where N is the number of patches, obtained using the unfold function. The revised manuscript now clearly differentiates between the two, with both the related work and method sections updated to enhance clarity and prevent any misunderstanding.

Q2.3: Table 2 clarification

A2.3: We acknowledge that the presentation in Table 2 may not be clear and this has been modified following your suggestions. We bold the best results and supplement the prediction performance using only dual heads. We also put the analysis between ordinary convolution and depthwise separable convolution in a separate section in the appendix. The comparison of convolution is placed in a separate section in the appendix.

As we see below, for small datasets like ETTh1, methods with only dual heads can achieve SOTA performance. For medium and large-scale datasets like Traffic and ETTm1, our PatchMixer layer is shown to be better.

Q2.4: Suggested additional Ablation Studies.

A2.4: We appreciate the reviewer's valuable suggestions on adding more ablations. We have conducted the experiments accordingly and show the results below:

●Effectiveness of Channel Independence vs. Channel Mixing: The comparative analysis of the experiments has been detailed in Appendix 7.1 of PatchTST, which highlights the superiority of channel-independent models in adaptability, training data efficiency, and overfitting prevention. To avoid duplication, we will not replicate these experiments but will reference them in the appendix of our revised manuscript.

●Impact of Different Patch Sizes: We are grateful for your comment. The selection of patch length has been analyzed in Appendix 4.1 of PatchTST, demonstrating that MSE scores remain consistent across various P values. We also select one large dataset, Electricity, to verify in our model. The result table with a prediction length of 720 time steps is as follows, affirming our model's robustness to the patch length hyperparameter:

●The Impact of Different Overlaps: The patch overlap is related to the patch step size, and the relationship is $O=P-S$. We conduct this experiment on two large datasets, Weather and Electricity. One observation is that the MSE score oscillated in a small range (between 0.003 and 0.001) with different choices of S, indicating the robustness of our model to patch overlaps. Please refer to Figure 6 in the Appendix 2 of the revised script.

●The Influence of Varying Convolutional Kernel Sizes. We have conducted the experiments and put the results in Figure 7 in Appendix 2. There was no significant pattern in the MSE score with different choices of K, the ideal patch overlap and convolution kernel size may depend on the datasets.

Q2.5: Thoughts regarding long-term dependencies without patching

A2.5: Thank you for pointing this out. In the Related Work section of our revised manuscript, we have included citations to these works to enrich the discussion around long convolutional models. Specifically, we acknowledge the following:

●S4 (Structured State Spaces): This method effectively processes long sequences through structured state spaces and has been applied in the field of large language models. Its theoretical explanation for long-term modeling provides valuable insights, particularly in understanding the underlying mechanisms of long-term dependency modeling.

●CKConv (Continuous Kernel Convolution): The approach constructs continuous convolutional kernels using a small neural network, aiming to develop a versatile CNN architecture. This innovative approach inspires our future research towards handling data of varying resolution, length, and dimension, broadening the scope of convolutional neural network applications.

●Hyena (Hierarchical Convolutional Language Models): It adeptly models long-term contexts by interweaving implicit parameterized long convolutions with data-controlled gating. This methodology presents a compelling approach that we believe could be adapted for temporal tasks, potentially enhancing the modeling of complex time series data.

Thank you for your valuable feedback. I have addressed each of your concerns as follows:

Q1.1: Motivation of model design.

A1.1: Our work is inspired by PatchTST's patch processing technique, which is proposed for use in the Transformer architecture. We hypothesize that the strong predictive performance observed in time series forecasting is attributed more to the patch embedding methods rather than the inherent predictive capabilities of the Transformers. Therefore, we design PatchMixer based on the CNN architecture, which has been shown to be faster at a similar scale to Transformers. Through theoretical analysis and practical experiments, we have demonstrated that our approach not only surpasses Transformers in predictive performance but also achieves 2-3 times faster training and inference speeds, thereby substantiating the efficacy of patch embedding in temporal prediction.

Q1.2: The results of PatchTST with MSE+MAE and SmoothL1Loss

A1.2: Thanks for the suggestions. We have conducted additional experiments to evaluate the impact of four different loss functions on our model's performance including the mentioned losses. The revised Table 4 now presents a comprehensive comparison between PatchMixer and PatchTST using these loss functions, with the superior results highlighted in bold.

Here we provide the following key observations, which are included in the revised paper.

First, PatchMixer consistently outperforms PatchTST across the board, affirming our model's superior performance.

Secondly, SmoothL1loss is inferior to the other two losses.

Thirdly, the model trained with combined MAE+MSE loss shows a better performance on the MAE/MSE metrics than the model trained with MSE loss only.

Dear Reviewer hpaR,

Thank you very much again for the time and effort put into reviewing our paper. We believe that we have addressed all your concerns in our response. Following your suggestions, we have enhanced the paper with additional experimental analysis. We kindly remind you that we are approaching the end of the discussion period. We would love to know if there is any further concern, additional experiments, suggestions, or feedback, as we hope to have a chance to reply before the discussion phase ends.