FM-TS: Flow Matching for Time Series Generation

1. The paper does not do enough efforts to distinguish itself from similar works in the literature, such as CFM-TS in ICML 2024.

2. This paper's experiment does not compare with ODE for time series works, which could also be used for time series generation. What is unique in flow matching that is beneficial for time series modeling?

3. It seems to me that the predictive score is most important -- unless the authors could suggest other usage of generated fake time series, if not for privacy-protected learning. However, the proposed model does not seem significantly better than baselines in the predictive score.

4. Figure 1 does not seem to be a comprehensive comparison in efficiency. It only compares FID against one baseline.

5. Very limited interpretation/reasoning about the experimental results. Mostly it is only about listing all the numerics, but readers can hardly understand why the result looks like the ones presented in the paper and the implication.

W1 The paper lacks clarity in presenting the core model illustrated in Figure 2. Although rectified flow is explained thoroughly as a preliminary concept, the main components of the model are only briefly introduced in the final paragraph of Section 3, without adequate explanation. The authors should explain the model components, clarify the rationale behind the chosen architecture and its relevance to time series.

W2 In its current form, the paper appears to be an application of rectified flow to time series without addressing the specific challenges in adapting rectified flow from image data to time series such as causality of time series, seasonality, trends; limited novelty.

W3 The experimental evaluation is insufficient.

• W3.1 What is the motivation behind choosing squared error as the evaluation metric? Squared error is a metric of evaluation for point prediction. For a generative time series model, evaluating solely on squared error for forecasting and imputation is inadequate. A more suitable evaluation would be based on metrics like Continuous Ranked Probability Score (CRPS) for univariate or Negative Log-Likelihood for both univariate and multivariate distributions.
• W3.2 What is the reason for choosing only the current set of baselines for forecasting and imputation? It would be beneficial to compare FM-TS against point-estimation models since the chosen evaluation metric is squared error; ex. PatchTST and TS-Mixer for forecasting tasks.
• W3.3 For the imputation task, the choice of only 70% and 80% missing data rates should be justified. For direct comparison with existing work, consider the settings from Toshiro et al. (10%, 50%, and 90%) and Alcaraz et al. (70%, 80%, and 90%). -W3.4 Since the main motivation for the FM-TS is the computational inefficiency of diffusion models, authors should show the runtime of training FM-TS and other baseline models (runtime per epoch, number of epochs until convergence, and/or total training time). Figure 3 attempts to do this job in terms of inference speed only.

Minor:

• M1 In line 242, should the function $v: {R}^{l \times d} \times [0,1] \to {R}^{l \times d}$ include $t \in [0,1]$ as an argument?
• M2 Please increase the font size of legends and labels in Figures 4 and 5 for readability.
• M3 Enhance the captions for Tables 3 and 4 to clarify the evaluation metrics used.

1. The overall writing quality is good, but some statements are confusing or misleading. For example, the descriptions about the capability of diffusion models in handling long-term dependency are contradictory in line 058 and line 061. The former states that diffusion models can capture long-range dependencies and generate diverse, high-quality samples, while the latter asserts that diffusion models struggle to preserve long-range dependencies and intricate patterns in time series data.
2. There is insufficient discussion on the conditional generation, particularly on why the unconditional models are adapted for conditional tasks by Algorithm 1. The introduction of concepts like t-power sampling lacks sufficient context and explanation, which makes it challenging for readers unfamiliar with them to understand their implication. Can the authors provide a brief example of how Algorithm 1 adapts unconditional models for conditional tasks? Can you also expand the intuition behind t-power sampling and its role in this adaptation?
3. The ablation study on the logit-normal distribution does not convincingly demonstrate its superiority; its performance is comparable or inferior to uniform sampling. Could the authors provide more analysis on why the logit-normal distribution is beneficial despite the results shown in Table 4? Are there any qualitative differences or theoretical advantages not captured by the metrics used?

• The paper’s flow and organization present significant challenges in readability, largely due to unclear transitions between key concepts such as computational efficiency and generalization. It seems that the authors do not consistently differentiate these concepts in their approach. The dense and complex sections lack clear explanations, detracting from the paper’s overall coherence.

• There is an unusual conflation of generalization and computational requirements, resulting in ambiguity. For instance, the authors assert that diffusion contributes to generalization on lines 056–057, yet they appear to refute this on lines 057–058, leading to further confusion.

• The paper also lacks reproducibility experimentation (no available code or scripts), as no implementation details or code are provided. Including code would facilitate verification of the results and support a broader understanding. Moreover, the appendix consists of only a few lines of explanation in general. It seems that the paper is not ready for publication at this stage.

• The claim that FM-TS can generalize to conditional generation tasks without retraining is intriguing but underdeveloped. The paper lacks comparisons with models specifically designed for conditional tasks, and no compelling evidence is presented to validate FM-TS’s performance in such scenarios. A deeper exploration of FM-TS’s generalization capability would strengthen this claim.

• The authors assert that their model outperforms the current state-of-the-art (SOTA); however, the results in Table 2 do not support this claim, as high standard deviation values suggest inconsistent performance. It would be valuable for the authors to discuss these variations and integrate them into their analysis.

• The paper suggests that imputation and forecasting tasks are nearly identical, differing only in the choice of point masking $M$. This assumption oversimplifies the nature of imputation, which often requires bidirectional information to accurately infer missing points. In contrast, forecasting typically operates with unidirectional data. Recognizing these differences is essential for model design and performance.

• The implementation details of "t power sampling" are missing. Without an explanation of how this method improves results, it is difficult to assess its functional role. Providing a detailed description of the sampling process would enhance transparency and reproducibility, offering insight into whether this is an optimization layer or a refinement in sampling for conditionality.

• The paper concludes with a vague claim that the unconditional model can be “directly used for conditional generation.” However, no details or references are given to substantiate how the model adapts to conditional tasks without retraining. A brief explanation or citation would clarify this point.

At this stage, it is challenging to recommend acceptance of this paper, primarily due to concerns regarding reproducibility. Without access to code, it remains unclear how to replicate the authors' results. Furthermore, the improvements in the paper's tables do not align well with the contributions claimed in the introduction.

References

[1] Qi, M., Qin, J., Wu, Y., & Yang, Y. (2020). Imitative non-autoregressive modeling for trajectory forecasting and imputation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 12736-12745).