Efficient Time Series Forecasting via Hyper-Complex Models and Frequency Aggregation

Dear toAw, Thank you for your thoughtful and detailed feedback on our manuscript. We appreciate your insights, which have highlighted areas where we can further clarify and strengthen our contributions. We are committed to addressing each point raised to improve our work's clarity, rigor, and impact.

1. Justification and Complexity of Hyper-Complex (HC) Numbers: We recognize the importance of providing a clearer explanation for the introduction of hyper-complex numbers and their advantages over simpler methods. To address this, we have included an empirical evaluation in Appendix D.4, where a basic MLP structure without the hyper-complex (HC) framework was tested. The results demonstrate a notable decrease in performance, highlighting the contribution of HC algebra. While a rigorous explanation of the HC framework’s benefits remains a subject of ongoing investigation, one plausible and intuitive justification lies in its alignment with the geometry of the data. In the frequency domain, time series data often exhibit intricate relationships between components, such as amplitude and phase. Hyper-complex numbers naturally align with this structure, facilitating the model’s ability to capture cross-dimensional dependencies more effectively than traditional real-valued or complex-valued models. We are committed to exploring this phenomenon further in the final version of the paper, aiming to provide a more substantiated and logical explanation for this remarkable behavior.
2. Literature Comparison and Missing Benchmarks: We acknowledge that our literature review did not include the mentioned works [https://arxiv.org/pdf/2403.11047v1 , https://www.sciencedirect.com/science/article/pii/S0167404824002669]. Our comparison does not incorporate the study by Shen et al. (https://arxiv.org/html/2407.21275v1) due to differences in result normalization schemes. This makes a direct comparison challenging, particularly in the absence of publicly available code for the mentioned work. Additionally, the other papers cited, which utilize the STFT for time series prediction, do not report results on the datasets used in our study.Moving forward, we plan to address this limitation by providing a qualitative discussion of these methods and their potential impact in future versions of the paper.
3. Statistical Validation: We strongly agree with the reviewer’s comment on the inclusion of error bars. We will include error bars and statistical significance tests to enhance the empirical robustness of our results.
4. Role of Real vs. Imaginary Components: We appreciate your observation regarding the sufficiency of the real component in predictions. Our model, drawing inspiration from FRETS, leverages both the real and imaginary components to retain detailed information, particularly in smaller model configurations. However, our architecture's inherent flexibility enables the elimination of one component without a significant loss in performance, which suggests the potential for further model simplification.That said, when reviewing the results from FRETS, it becomes evident that ignoring either the real or imaginary part does negatively impact performance. This raises an intriguing question: is the hyper-complex structure, along with its parameter multiplicity, the underlying reason for this phenomenon? We recognize the importance of addressing this and will strive to investigate this matter further in the final version of the paper.
5. Language and Presentation Quality: We recognize the importance of readability and will engage a professional proofreader to ensure the final manuscript is clear and free from grammatical inconsistencies. Thank you again for your helpful comments. We believe these improvements will make our work clearer and more valuable.

The authors' response does not adequately address the fundamental issue regarding the necessity and benefits of hypercomplex networks.

• The new BasicMLP baseline results in Appendix D.4 (Table 9) reveal that the performance difference between hypercomplex approaches and a simple linear aggregation is minimal. This undermines the paper's central claim about the advantages of hypercomplex representations.

• The difference in the performance reported by the different methods are very small even if the authors claim that there is "a notable decrease in performance" of the basic BasicMLP. Most seriously, there are still no standard deviations in the tables, making the whole experimental evaluation pointless because it is not possible to determine if there are significant differences or not. This is a slap in the face to every statistician and, in general, researcher.

• The response superficially addresses the theoretical justification by suggesting that hypercomplex numbers "naturally align with the geometry of the data." However, this explanation lacks mathematical rigor and fails to establish why hypercomplex operations are specifically advantageous for time series data in the frequency domain.

• Finally, the extended neighborhood experiments in Appendix D.5 show that increasing the number of neighbors in WM-MLP does not improve performance, which contradicts the motivation for using HC-MLP to capture longer-range dependencies.

• The core contributions of the paper - the benefits of hypercomplex representations and the superiority of the proposed architectures - are not convincingly demonstrated. The similar performance of simpler baselines suggests that the added complexity of hypercomplex operations may not be justified for this application.

• The paper is still lacking clarity and most of the imprecisions are still present.

In conclusion, none of the issues has been addressed in a satisfactory manner. For these reasons, I maintain my original recommendation for rejection and increase my confidence score to 5.

After reading other reviewer comments, I agree to reduce my score.

Thank you for your valuable feedback and for highlighting areas for improvement. We appreciate the opportunity to clarify and enhance our work in response to your suggestions.

1.Comparison with Recent Works: Regarding the comparison with recent works, we acknowledge that most existing models do not report results on our specific scales (Mini max normalization). To conduct a meaningful comparison, it would require running all the experiments ourselves to generate these results. While this could enhance the completeness of our evaluation, we seek feedback on whether this step is essential, given the added complexity and resource requirements it entails.

2. Complexity Analysis: We acknowledge the significance of analyzing model complexity and its relevance in offering a thorough understanding of our approach. We have included a preliminary complexity analysis in Appendix D.6; however, we intend to expand and enhance this analysis in the final version to better contextualize our model's efficiency and performance.

Rest assured, the final version will include a detailed complexity analysis to further highlight the efficiency and practicality of our model.

Dear mJBU, Thank you for your comprehensive and insightful feedback. We appreciate the opportunity to address these points and clarify our contributions.

1. Terminology and Consistency: We recognize that certain terms were used interchangeably, leading to possible ambiguity. Specifically, our focus is on proposing an innovative complex-valued MLP model, termed FIA-Net, which includes both WM-MLP and HC-MLP architectures. We will refine the terminology throughout the paper to consistently reference FIA-Net and its components, ensuring clearer communication of our model's unique aspects.

2. Use of STFT: You are correct that frequency-domain techniques, including FT and wavelet transformations, have been extensively applied in time series analysis. However, while wavelet transformations are widely used, STFT remains less common. We selected STFT to capture both phase and frequency information through complex coefficients, which allows for richer representations of the signal. Unlike wavelets, which produce real coefficients, STFT provides complex coefficients that are advantageous for our model, as they offer distinct phase and frequency details that better support our multi-window approach.

3. STFT Trade-Offs: We acknowledge that STFT introduces a trade-off between time and frequency resolution. In our experiments, we examined the impact of varying the N_{FFT} parameter, which affects frequency resolution, and the number of windows, which determines time resolution. We found no single ideal configuration across all datasets; rather, optimal settings were dataset-specific. This insight will be detailed in the revised paper, and we will elaborate on how our model accommodates these variations in resolution.

4. Clarification on Complex and Hyper-Complex Numbers: The precise mathematical justification for the improved performance achieved by utilizing hyper-complex numbers remains an area of ongoing investigation. However, we can provide an intuitive explanation at this stage. In our framework, standard complex numbers encode the contributions of sine and cosine components within a signal. By processing these complex coefficients, the model inherently assumes that the real and imaginary parts—representing cosine and sine waves, respectively—are interdependent, reflecting their natural phase relationships. Extending this concept to hyper-complex numbers, we allow for interactions not only within a single dimension but also across multiple dimensions, enabling richer interdependencies between sine and cosine components across several STFT windows. This design enhances the model’s ability to capture complex dependencies over time, which is particularly beneficial for accurate long-term predictions. To validate the importance of the hyper-complex structure, we conducted an experiment described in Appendix D.4. In this test, the STFT windows were aggregated using a standard MLP architecture instead of the hyper-complex algebra-based structure. The results showed higher mean absolute error (MAE) and root mean squared error (RMSE), leading us to conclude that the hyper-complex framework provides a tangible improvement in performance.

5. Window Mixing Mechanism (WMM) and Model Complexity: The Window-Mixing MLP (WMM) operates by incorporating information from adjacent windows, with each window influenced by its immediate neighbors—both preceding and succeeding. To explore the impact of varying the number of neighboring windows, we conducted experiments where each window was influenced by two or three adjacent windows. The details of this experiment are provided in Appendix D.5.Additionally, we plan to expand the discussion on model complexity in the final version of the paper. This will include both experimental results and a detailed analysis of the asymptotic behavior, as outlined in Appendix D.6.

6. Evaluation Metrics: We primarily utilized MAE and RMSE metrics to ensure consistency with other research in the field, such as FreTS, PatchTST, and Autoformer. While we acknowledge that incorporating additional metrics could offer deeper insights, our aim is to maintain alignment with the evaluation standards commonly employed in this area of study.

7. General Formatting and Grammatical Improvements: We acknowledge the grammatical issues and formatting inconsistencies present in the current draft. All cited references, equations, and figure captions have been carefully revised to ensure accuracy and clarity. The final manuscript will include appendices as referenced and all citations will be corrected for consistency and correctness.

Thank you for your detailed and helpful feedback. We believe these revisions will improve the manuscript and make our contributions clearer and more precise.

Dear VFMH, Thank you for your thoughtful and constructive review. We appreciate your insights and the opportunity to clarify aspects of our work.

1. Similarity of Structure with FreTS and Effectiveness of Windows-Integrating Subnetwork: It is correct that the FIA-Net is a generalization of the FreTS model (as we explicitly show in the appendix), our focus is on addressing FreTS' limitation in handling long prediction horizons and minimizing error across these intervals. In shorter prediction tasks, the models perform similarly. However, for longer horizons, FIA-Net achieves up to a 20% improvement in accuracy on rmse and 14% on mae (achieved on traffic 192). Please remember that the model is trained and performs on normalized data, specifically using min-max normalization, which reduces the scale while still maintaining a significant impact in percentage terms. Additionally, it is worth noting that the Frets paper shows only marginal improvement over the PatchTST and LTSF-linear models (sometimes as little as 0.01) compared to other studies. 2.HC-MLP and Prediction Horizons: The HC-MLP architecture does aggregate all window information at each prediction step, which theoretically should improve performance for longer-term sequences. However, our experimental results indicate that information from more distant windows can sometimes introduce noise, affecting the accuracy for extended prediction horizons. Consequently, we concluded that HC-MLP is particularly beneficial for short-term predictions, whereas WM-MLP, with its focus on approximate windows, is more suited for longer-term predictions. We have revised the paper and improved the following paragraph to clarify this point, the changes are marked in red.

2. Interpretability of Figure 4: We understand that Figure 4 could be more visually distinct. In our revised version, we will enhance symbol differentiation to ensure that each operation in the HC-MLP structure is clearer, addressing any ambiguity.

3. Clarification on "8% Improvement" in Section 5.1: We have reviewed the calculations again, and you are correct. The average improvement is 5.45% for MAE and 3.75% for RMSE, as demonstrated in the Excel file here: https://file.io/s64oG9H4ADPD. We have updated the paper accordingly. Thank you very much for bringing our calculation error to our attention.

We greatly appreciate your feedback, which has helped us improve the clarity and presentation of our results. Thank you once again for your valuable comments.