Sundial: A Family of Highly Capable Time Series Foundation Models

YES

There are no theoretical claims.

The experimental designs and the conducted analysis seem to be acceptable.

I find the idea interesting and the results pertinents.

The following paper is not referenced in the submitted paper, even though it achieves high-quality image generation without the need for discrete tokenization, thus having some connections to the proposed method: Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vector quantization. arXiv preprint arXiv:2406.11838, 2024.

The methods and evaluation criteria proposed in the paper are fundamentally well-aligned with the challenges of time series foundation modelling. The authors recognize a critical limitation in existing approaches—specifically, the tension between discrete tokenization (which limits the representation of continuous values) and parametric densities (which restrict distribution modelling capacity). Their proposed TimeFlow Loss offers a theoretically sound solution by enabling Transformers to learn flexible predictive distributions without requiring tokenization or prior distribution specification, which is particularly appropriate for the heterogeneous nature of large-scale time series corpora.

The patch-based continuous tokenization approach sensibly balances computational efficiency with representation quality, addressing the unique characteristics of time series data while maintaining compatibility with Transformer architectures. Technical adaptations like Pre-LN for stability and RoPE for temporal causality demonstrate thoughtful consideration of the domain-specific challenges in time series modelling.

Regarding evaluation criteria, the authors employ a comprehensive benchmarking strategy that convincingly validates their claims. The use of both point forecasting metrics (MSE, MAE) and probabilistic metrics (CRPS, MASE, WQL) across three established benchmarks—Time-Series-Library, GIFT-Eval, and FEV—ensures thorough performance assessment. The GIFT-Eval benchmark is particularly appropriate as it encompasses 23 datasets with diverse characteristics, providing a robust measure of generalization capability. The evaluation against both statistical methods and competing foundation models offers the necessary context for interpreting performance gains.

The TimeBench dataset, with its unprecedented scale of 1 trillion time points and diverse sources spanning different frequencies and domains, constitutes an appropriate foundation for pre-training models intended for broad applicability. The careful exclusion of test datasets from pre-training data demonstrates methodological rigour in preventing data leakage that could invalidate zero-shot performance claims.

The ablation studies comparing TimeFlow Loss with alternative approaches (Table 3) and the exploration of inference trade-offs (Figure 8) provide crucial scientific validation of design choices. However, the paper would benefit from more domain-specific evaluations to complement the general-purpose benchmarks and from deeper analysis of prediction diversity beyond visual showcases. Nevertheless, the overall approach and evaluation strategy are well-matched to the foundational goals of developing generalizable, probabilistic time series models capable of effective zero-shot performance.

The paper's main theoretical foundation rests on the flow-matching framework originally proposed by Lipman et al. (2022), which the authors adapt to time series forecasting. The primary theoretical contribution—TimeFlow Loss—is formulated in Section 4.1.3, where the authors extend the conditional flow-matching objective to handle sequential time series data. While the mathematical formulation is clearly presented (Equations 4-7), the authors don't provide formal proofs for the new theoretical properties of this adaptation. Instead, they demonstrate its effectiveness through empirical validation across multiple benchmarks.

The theoretical formulation builds directly upon the preliminaries established in Section 3.1, where the standard flow-matching framework is introduced. The authors correctly present the existing theoretical elements including the velocity field ODE (Equation 1), the Flow-Matching objective (Equation 2), and the Conditional Flow-Matching objective with Gaussian formulation (Equation 3). These equations are properly cited to the original work. The adaptation to time series involves conditioning the flow-matching process on a learned representation h_i, which appears mathematically sound, but the paper doesn't provide a rigorous proof of its properties beyond empirical results. The inference procedure (Algorithm 1) is a straightforward application of numerical ODE solving, consistent with standard flow-matching literature.

The claim that TimeFlow Loss mitigates mode collapse during pre-training is primarily substantiated through qualitative visual showcases (Figures 13-14) and comparative experiments (Table 3) rather than through theoretical guarantees. This empirical validation approach is reasonable for a systems paper but limits the theoretical depth of the contribution.

From my understanding, while the paper presents a coherent mathematical framework for applying flow-matching to time series forecasting, it does not contain novel theoretical proofs requiring verification. The mathematical soundness of the approach derives from its faithful extension of established flow-matching theory, with innovations focused on architecture and application rather than fundamental theoretical advances.

The authors present a comprehensive evaluation framework for their proposed time series foundation models, with experiments that are generally well-designed but I think it contains several methodological limitations that merit consideration. I am providing some of them here.

(1) The zero-shot forecasting evaluation methodology demonstrates strong validity through the use of established benchmarks (Time-Series-Library, GIFT-Eval, and FEV) and clear separation between pre-training and evaluation datasets. The authors implement appropriate metrics (MSE/MAE for point forecasting and MASE/CRPS/WQL for probabilistic forecasting) that align with community standards. The careful exclusion of evaluation datasets from the pre-training corpus (explicitly denoted by dashes in Table 1) strengthens the experimental integrity of zero-shot claims. However, the comparison methodology raises concerns regarding controlled evaluation conditions. While Table 6 documents architectural differences between models, the paper doesn't explicitly confirm whether all baseline models had access to identical context lengths during inference, which could significantly impact forecasting performance. Additionally, the reliance on officially reported results from other papers (noted in Table 7) introduces potential inconsistencies in evaluation protocols that could affect fair comparison.

(2) The ablation study for TimeFlow Loss (Table 3) is methodologically sound in maintaining architectural consistency while varying only the training objective. However, this analysis is limited to reporting point forecasting metrics (MSE) without evaluating probabilistic metrics, which undermines a complete assessment of the central claim regarding improved distribution modelling. The qualitative comparisons in Figures 13-14 partially address this gap but lack objective quantification of prediction diversity.

(3) The scaling behaviour analysis in Figure 7 presents valid training curves but would be strengthened by more intermediate model sizes to establish clearer scaling laws. Similarly, the inference speed versus performance trade-off analysis (Figure 8) systematically varies generation parameters but doesn't properly quantify the relationship between these parameters and actual inference time.

(4) Most concerning is the absence of statistical significance testing or confidence intervals across all experimental results, particularly important given the high variance typically observed in time series forecasting performance. This omission limits the robustness of performance comparisons, especially for closely-matched results.

I think this work makes several notable contributions that both build upon and diverge from established research trajectories in time series modelling. The authors' central contribution—TimeFlow Loss—represents a significant advancement in how foundation models handle continuous-valued time series data. This approach extends the flow-matching framework of Lipman et al. (2022) to the autoregressive time series domain, creating a bridge between continuous generative modelling and sequential forecasting that has been largely unexplored in prior literature. The authors' decision to embrace generative modeling rather than discrete tokenization strategically positions their work as an alternative to the language-modeling inspired approach taken by Chronos (Ansari et al., 2024). While Chronos adapted techniques from NLP by discretizing continuous values, Sundial's native approach avoids the information loss inherent in quantization—addressing a fundamental limitation recognized but unresolved in previous work. Similarly, the TimeFlow Loss offers greater flexibility than the parametric mixture distributions employed by Moirai (Woo et al., 2024), which the authors correctly identify as potentially constraining when modelling heterogeneous time series distributions at scale.

The architectural adaptations, while individually derived from existing techniques like RoPE (Su et al., 2024) and Pre-LN (Xiong et al., 2020), represent a thoughtful integration of components that specifically address the challenges of time series forecasting. This integration draws from disparate research streams that have not previously been united for time series foundation models. Particularly, the deliberate incorporation of FlashAttention and KV Cache reflects an understanding of the efficiency challenges faced by practitioners—a consideration often neglected in academic time series research but well-established in large language model literature.

TimeBench's trillion-point scale represents an order-of-magnitude increase over previous datasets like those used in Time-MoE (300B, Shi et al., 2024b) and Timer (231B, Liu et al., 2024a,b). This scaling aligns with the broader foundation model literature's emphasis on dataset size as a critical factor in model capability, while specifically addressing the unique challenges of time series heterogeneity. The authors' careful compilation of diverse frequency data connects to emerging work on scaling laws for time series (Shi et al., 2024a), extending these insights into previously unexplored data volumes.

I think the empirical results position Sundial at the intersection of deterministic and probabilistic forecasting research streams. The comparative analysis against both MSE-trained models and diffusion-based alternatives provides a valuable empirical bridge between these previously separate approaches. This integration of multiple modelling paradigms within a unified foundation model framework represents a meaningful synthesis of previously disparate research directions in time series forecasting.

N/A

The proposed method and evaluation criteria make sense for the problem.

There is no proof or theoretical claims.

Experimental designs and analyses are sound and valid.

This paper contributes to the literature of time series foundation models. Existing models mainly use parametric densities or language modeling, while this paper introduces a new perspective of generative modeling. It also proposes a large TimeBench for model training and a family of foundation models with SOTA performance on point and probabilistic forecasting.

There are no essential references not discussed.