Thanks for your insightful questions.

• There are quite a few bits and pieces to the model pipeline, and at times the extent to which each component contributes towards overall performance is unclear. Perhaps a few ablation studies in the supplementary material can address this?

  A: We understand that ablation study is very important to understand the impact of each part. However, FITS is a very compact model. Removing any of its part will make it not functional. And FITS only have one hyperparameter which is the interpolation rate. We have done comprehensive ablation studies on the cutoff frequency, please see the Tab.5 of the manuscript and Section D of the appendix.

• The concluding remarks of the paper focus on a few settings where the method may not be most appropriate, but I would have liked to see broader thoughts on future work, such as further reductions to model size, and possibly interpretability.

  A: Thank you for your feedback. You've raised valuable points. In response to your query about future work, we have plans to delve deeper into the exploration of complex-valued neural networks in the frequency domain, including the investigation of complex-valued attention mechanisms. Regarding interpretability, we acknowledge its importance and are actively working on improving our visualization techniques to make the results more interpretable.

  I'd like to highlight that we've included an interactive notebook in the anonymous code repository, which can assist in understanding how FITS learns various frequency combinations in the frequency domain. As for further model size reduction, we understand that it can be a challenging task, and we are currently in the process of evaluating its feasibility.

  We have add the discussion on the future work to the manuscript according to your advice.

• While I appreciated the experiment on anomaly detection, this is a field of study where (as referenced by the authors themselves), a few metrics may not be sufficient for fully capturing the success of different techniques. There is consequently a risk that the experiment featured here might not be sufficiently conclusive (although the results obtained on synthetic data are a step in the right direction).

  A: We acknowledge that relying solely on simple metrics may not adequately capture the performance of FITS in real-world tasks. However, to ensure a fair comparison, we have followed the approach of previous works, specifically the Anomaly Transformer, to demonstrate the superior performance of FITS. Nevertheless, we recognize the importance of interpretability and plan to delve deeper into this aspect by investigating the interpretability of FITS using synthetic datasets. For a more comprehensive visualization and interpretation, we invite you to explore the jupyter notebook available in our anonymous repository.

Thanks for your insightful questions. We will address your concerns with three replies due to the length limit.

• The presentation of FITS could be more comprehensive (more details in the questions). Furthermore, I would like to see more "basic" baselines (e.g., ARIMA, DeepAR for forecasting, and Random Cut Forest for Anomaly detection). To further substantiate the claims about performance (and to put them on a statistically sound foundation), a critical difference plot[1] may be a useful analysis tool. Lastly, the method employs several heuristics that need to be tuned in the real world, a drawback it shares with other methods.

  A: In line with previous studies like PatchTST, the effectiveness of statistical methods like ARIMA and RNN-based methods such as DeepAR and LSTM has been found to be inferior compared to Transformer-based methods. Hence, we did not include them as baselines in our study. Additionally, DeepAR is specially designed as a probabilistic forecasting model which may not be appropriate to compare with it. It's worth mentioning that RCF, a proprietary anomaly detection algorithm by Amazon, is not included in our analysis. In our manuscript, we present results for Isolated-Forecast, an ensemble and decision tree-based algorithm.

  If we understand the ‘heuristic’ correctly, FITS only have one hyper-parameter i.e., the cutoff frequency. It is easy to find the optimal value comparing to the large hyper-parameter space of other models.

  We use the ARIMA implementation provided in the LTSF-Linear. However, as described in the repo, ARIMA runs extremely slow. As a result, we only report the MSE performance on ETTh1 and ETTm1.

  Datasets	96	192	336	720
  ETTh1	1.0642	1.1053	1.125	1.158
  ETTm1	1.4870	1.7228	2.0984	3.3069

  We have added the critical difference plot in the Appendix I. FITS outperforms all the baselines with statistical significance. The critical difference plot also showcases the robustness of FITS' performance across various datasets, reinforcing its reliability.

• Can you explain why x(t-τ) is a shift forward in time, rather than backward?

  A: When we talk about shifting forward in time, it means we are essentially repositioning the starting point of our time axis in the direction of increasing time values. Alternatively, you can think of it as sliding the entire function in the opposite direction, which is the negative part of the time axis. Consider a time series represented as F(t). Shifting forward by a duration of τ means that we are relocating the original reference point τ units ahead in time. This action results in a new representation of the function, denoted as F(t-τ). In simpler terms, it's like moving our viewpoint along the time axis to where τ becomes the new starting point.

• Can you add a more thorough caption to Fig. 2? As it stands, it is not comprehensible enough to fully understand your method. Is it just a single linear layer/what is the output dimension? What are the typical dimensions of ? Are (i)RIN and (i)rFFT performed in parallel or sequentially? Please elaborate more on how RIN works, either in the Figure or its caption.

  A: We have updated the caption for Figure 2 according to your advice.

  Yes, as described in Section 3.2, the complex linear layer is just a single linear layer with complex-valued weights.

  Suppose the input X dimension is (B, L, C). The $\tilde{X}$ should be the dimension of its rFFT which is (B, L//2+1, C). After a LPF, the $\tilde{X}$ is cut to the length of l according to the cutoff frequency. The input dimension of complex linear layer is (B, l, C) (Batch size, Length, Channel) and the output is (B, ηl, C) which extend the length according to the interpolation rate.

  As mentioned in the paper, RIN stands for Reversable Instance Normalization. It is a simple normalization method that removes the bias and variance for each data instance before feeding into the model. After model processing, the variance and bias are added back to the result to reverse it back to the original data distribution. In most cases, RIN is used to eliminate the distribution shift so that models can reach a better result. However, FITS use RIN mainly to remove the bias of time series which can introduce a large 0 frequency component. Please check the paragraph under Figure 2 for detailed information.

  RIN and FFT runs in sequential. As described, RIN is performed on the time domain. And we employ RIN to eliminate the bias in time series and use iRIN to add it back. Thus, RIN is performed ahead of the rFFT and iRIN in performed after the irFFT.

We apologize for any confusion caused. As you correctly mentioned, we utilized a pre-existing repository for generating the critical difference (CD) plot. It is worth noting that the CD plot is commonly employed in classification tasks, where the evaluation metric is accuracy (higher is better). However, in our case, we evaluated a forecasting task based on mean squared error (MSE), where lower values indicate better performance. Consequently, being ranked sixth actually means that our model performs the best in terms of MSE.

To create the CD plot, we originally only considered the MSE performance over a 336-forecasting horizon, and we have now updated the CD plot in accordance with all the evaluation settings. Kindly refer to the appendix for the updated version. We also attempted to reverse the order of the models in the plot. However, we encountered some issues with the plotting repository when activating the reversal. Consequently, we have retained the original order for now. We are actively working to rectify this issue and will update the diagram in the final version.

The updated CD plot now demonstrates that our model, FITS, is no longer connected to the other methods, indicating statistical significance in its performance superiority.

We apologize once again for any confusion caused and hope that our responses have adequately addressed your concerns.

Thanks for your insightful questions.

1. The result on anomaly detection is not remarkable. Especially on the SMAP and MSL dataset. Some more in depth analysis is needed to find out the reason.

   A: As analyzed in the conclusion. These two datasets are majorly composed with binary data where time-domain modeling is preferable as the raw data format is sufficiently compact. The binary-valued data also have messy frequency presentation which is difficult for FITS to learn. We provide a visualization in the interpretability notebook in the anonymous repository to show such finding.

2. Even though the FITS have far fewer parameters comparing to the DLinear, it still need more time to inference on the GPU (0.6 and 0.4 ms respectively). Does this mean the DLinear is still the best choice for the real-time application?

   A: The inference time of FITS is sub-ms level which is comparable to the communication overhead, the 0.2ms overhead does not make a dealbreaker to fail FITS to work as a real-time time series analysis model. Furthermore, the inference time on GPU does not match the common case on the edge-device real-time application. The edge-device often only have single core computation unit which should resemble the inference time result on the single-CPU setting which FITs achieves better result. The inference time is also influenced by the cuda optimization. As far as we know, the complex-valued neural network in cuda and pytorch is still an experimental feature, which indicate that the optimization is still not as perfect as the normal linear layer. This may cause some performance downgrade. Furthermore, the edge devices often have very constrained memory resources such as the arm development board we use (STM32F303K8T6, which is with ARM Cortex M4 core at 72 MHz, 64 Kbytes of flash memory, 16 Kbytes of SRAM). The parameter of DLinear can not fit into a board with 64 KBytes of flash memory. As a result, FITS is still more suitable for the edge-device deployment.

3. How do author deploy the model on devices that do not support the complex computation?

   A: We are aware of that many edge devices or even RTX4090 do not support complex computation. Thus, we provide a FITS-real model to simulate the complex-valued linear projection with normal real-valued linear layer as following:

   $y_{r}= W_r * x_r - W_i * x_i$

   and,

   $y_{i}=W_r * x_i + W_i * x_r$

   The complex-valued weight matrix is divided as two real-valued linear projection $W_i, W_r$.

   The FITS-real model achieves the same result as the original FITS, and they have the same memory footprints.

Thanks for your insightful questions.

• The method's limitation lies in its inability to generate probabilistic forecasts, a crucial requirement for numerous industrial applications.

  A: We agree on the limitation that mentioned by the reviewer. As a compact model FITS is not designed to generate probabilistic forecasts. But it can be an interesting future work to generate probabilistic forecasts over frequency domain.

• Moreover, the evaluations conducted on benchmark datasets may not accurately reflect the real-world scenarios of edge devices. These benchmark datasets, such as those related to traffic and weather, typically do not require processing on edge devices and can be handled in offline settings. It remains uncertain how well the model will perform when applied to data from edge devices, like healthcare devices or industrial sensors, which present different challenges and requirements.

  A: We conduct evaluations on benchmark datasets primarily to ensure a fair and standardized comparison with other baseline models, demonstrating the performance of FITS in controlled conditions. We acknowledge that some of these benchmark datasets may not perfectly align with real-world edge device scenarios. However, it's important to note that our ETT dataset, sourced from sensors on an electrical transformer, closely matches our intended use case. Notably, FITS has demonstrated outstanding results on this dataset, reinforcing its suitability for relevant applications.

  Furthermore, we are actively extending the applicability of FITS to various real-world tasks, including those involving healthcare devices. As part of this effort, we offer a script for generating an ONNX runtime, facilitating the deployment of FITS on embedded systems. We will further evaluate FITS on new datasets in our future works.

• Did you try fitting a larger neural network for frequency interpolation task? It would be interesting to see if the performance of this architecture scales with the size.

  A: Thank you for inquiring about this aspect. We share your curiosity regarding the performance of deep complex-valued neural networks in the context of frequency interpolation tasks. It's worth noting that the frequency domain inherently involves complex numbers, making complex-valued neural networks a natural choice for processing such data.

  However, it's important to acknowledge that the development of complex-valued neural networks is still a work in progress. Critical components, including activation functions and loss functions tailored for complex numbers, are not widely available or well-established in the current landscape. This limitation poses challenges when attempting to construct large-scale complex-valued neural networks.

  As part of our future research efforts, we are actively exploring strategies to overcome these obstacles and scale up complex-valued neural networks for applications in the frequency domain. This includes investigating complex-valued frequency domain Transformers as a potential avenue for advancement. Your question underscores the importance of this exploration, and we look forward to sharing our findings in this area as we make progress.