A Geometry-Aware Metric for Mode Collapse in Time Series Generative Models

We sincerely thank Reviewer VZKP for their detailed and constructive review. Your feedback has highlighted a few key areas where we can significantly improve the paper’s clarity, and we appreciate the opportunity to clarify them. We have taken the feedback seriously and believe the proposed revisions will substantially strengthen the paper.

[Question 1] Clarifying the Theoretical Motivation and Formulation of DMD-GEN. We have revised Section 3.4 to better explain the metric’s design. Our approach addresses two fundamental questions: first, how to compare the dynamics of any two individual time series, and second, how to scale this comparison to entire sets (batches) of time series to detect mode collapse.

A. The Core Component: A Geometric Distance Between Two Time Series

The first challenge is to define a meaningful distance between the dynamics of a single original time series ($X_i$) and a single generated one ($X̃_j$).

• The Problem: The key dynamics of each series are captured by its DMD modes, which form a mathematical subspace. Because these subspaces can be oriented differently, they do not share a common basis, making a simple Euclidean distance invalid.

• Our Solution ($d_{\text{proj}}$): We use the principled, geometry-aware geodesic distance on the Grassmann manifold (defined in Equation 3 and Theorem 3.6). This measures the shortest path between the two dynamic subspaces, providing a mathematically sound way to compare their core dynamics.

B. The Main Goal: Comparing Distributions to Detect Mode Collapse

Mode collapse is a distributional problem: a model fails if the distribution of its generated dynamics does not match the distribution of the original data's dynamics.

• The Problem: How do we compare the entire set of original dynamics to the entire set of generated dynamics?

• Why Simple Averaging is Insufficient: Just averaging the pairwise $d_{\text{proj}}$ distances can be misleading. A generator might create a single ''average'' sample that is reasonably close to all original samples but fails to reproduce any of their specific, distinct dynamic modes.

• Our Solution (Optimal Transport): Optimal Transport (OT) is the established mathematical framework for comparing two distributions. It finds the "minimal effort" required to transform one distribution into the other by finding the best possible matching between their respective samples.

Putting It All Together: The DMD-GEN Metric ($d_{\text{DMD}}$)

Our final metric integrates these two solutions into a single, robust framework:

1. We use our geometric distance, $d_{\text{proj}}$, as the cost to match any two time series.

2. We build a cost matrix $C$, where each entry $C_{ij}$ is the pairwise geodesic distance $d_{\text{proj}}$($X_i$,$X̃_j$).

3. Optimal Transport uses this cost matrix to find the minimum total effort to align the generated distribution with the original one. This minimum effort is our final metric, $d_{\text{DMD}}$.

A low $d_{\text{DMD}}$ signifies that the distributions of dynamics are similar (low mode collapse), while a high cost indicates a poor match (severe mode collapse). This principled, two-level approach ensures that DMD-GEN is both geometrically sound at the sample level and distributionally aware at the set level. We will incorporate this clearer explanation into the revised manuscript.

[Question 2] Experimental Evidence for DMD-GEN’s Advantages and Efficiency. We acknowledge that Table 1 only shows consistency and does not, by itself, demonstrate the primary advantages of DMD-GEN. The key advantages are sensitivity, efficiency, and interpretability, which we will highlight more clearly with the following new additions to the paper.

• Superior Sensitivity to Mode Collapse: Our primary claim is that DMD-GEN is more sensitive to mode collapse than existing metrics. This is experimentally proven in Table 2. In our controlled synthetic experiment, a minor 10% mode imbalance causes DMD-GEN to increase by +681%, signaling a strong detection. In contrast, the Predictive Score barely changes (<1%) in absolute value, and the Discriminative Score gives a noisy signal that is not stable across severities (flips sign). We will explicitly state this in the main text as a key takeaway.

• Quantitative Comparison of Computational Complexity: DMD-GEN’s ‘‘training-free’’ nature provides a significant practical advantage. Additionally, we report below the average time needed to compute DMD-GEN w.r.t different number of samples.

Computational time (in seconds) on the stock dataset.

[Question 3] Proving the Interpretability of DMD-GEN. We agree that the claim of interpretability requires direct experimental proof. We thank the reviewer for raising this point. To validate this claim, we added a new section to the appendix with visualizations that demonstrate how DMD-GEN identifies which modes are poorly captured.

• Our Visualization Strategy: As you suggested, we will visualize the captured patterns. Our process is as follows:

1. For a batch of original and generated data, we compute the DMD-GEN score and the optimal transport plan $\gamma^\star$.

2. The transport plan $\gamma^\star$ tells us exactly how the original modes are ‘‘mapped’’ to the generated modes. We can identify a cluster of original time series that are poorly matched (i.e., have a high transport cost to all generated samples). These are the ‘‘poorly captured modes.’’

3. We will then create a plot showing a representative time series from this ‘‘poorly-captured’’ cluster alongside a visualization of its dominant DMD mode (e.g., a specific high-frequency oscillation). This provides direct, visual proof that DMD-GEN has identified a specific dynamic pattern in the original data that the generative model failed to reproduce.

This new analysis will provide the experimental evidence for our interpretability claim.

[Question 4] Typos. Thank you for your careful reading and for catching these errors. You are correct about both typos. We have corrected the reference in line 248 to point to the correct equation, and we have numbered the previously unnumbered equation and updated the reference in line 253. We appreciate your attention to detail.

We are confident that these clarifications and new experimental results directly address your concerns and have significantly improved the quality and clarity of our paper. We hope these changes will convince the reviewer of our work’s merit. We thank you again for your valuable guidance.

We are very grateful to Reviewer zvbY for their thorough review and positive assessment. We are especially encouraged that they recognized the novelty of our approach and the importance of the problem. We appreciate the insightful weaknesses raised, which will help us improve the paper significantly.

[Weakness 1] Discrepancy in Benchmark Results (Table 1)

Thank you for this observation. The reviewer is correct that the absolute scores for the baseline metrics (Predictive/Discriminative Score) on TimeGAN may differ from those in other papers. This is not an inconsistency in our metric but a result of ensuring a fair and controlled comparison within our study.

To eliminate confounding variables, we implemented and trained all baseline models (TimeGAN, TimeVAE, DiffusionTS) and all evaluation metrics under a single, unified experimental setting (i.e., identical data splits, preprocessing, and training hyperparameters). While this ensures our internal comparisons are rigorous and fair, it can lead to different absolute scores compared to papers that may use slightly different experimental setups.

We will add a footnote to Table 1 to clarify this point, stating that all models and metrics were evaluated in our controlled environment to ensure a fair head-to-head comparison.

[Weakness 2] Ambiguity and Apparent Contradiction in Figure 1

The reviewer makes an excellent point about the potential confusion caused by Figure 1. Our core argument is that a robust quantitative comparison requires using the DMD subspaces (eigenvectors), as raw eigenvalues are not invariant. We recognize that Figure 1 was unclear and will revise it.

The only purpose of Figure 1 is to provide a high-level, qualitative intuition for how a model’s dynamics evolve during training. The visual clustering of the generated eigenvalues (blue dots) around the original ones (red dots) suggests that the model is learning the correct range of frequencies and decay rates. It is an illustration, not a part of our proposed metric.

To resolve this ambiguity, we will revise the caption of Figure 1 to explicitly state: ‘‘This figure provides a qualitative illustration of spectral properties. It is intended for intuition only. Our proposed metric, DMD-GEN, is based on the principled and robust comparison of DMD subspaces, not raw eigenvalues.’’

[Weakness 3] Demonstrating Novel Insights Provided by DMD-GEN

This is a crucial point. A new metric should ideally offer insights that existing ones do not. DMD-GEN provides unique advantages in both sensitivity and interpretability, which we will emphasize more clearly in the revised manuscript.

1. Superior Sensitivity to Emerging Mode Collapse: DMD-GEN can detect subtle mode collapse that other metrics miss. This is demonstrated in our synthetic experiment in Table 2, which shows the metric’s high sensitivity to mode collapse, and is further supported by our bootstrapping experiment in Figure 2, which shows its responsiveness to the preservation of temporal structure. When we introduce a minor 10% mode imbalance:

• DMD-GEN's score changes dramatically (+681%), clearly flagging the problem.

• Predictive Score barely moves (-0.54%), completely failing to detect the issue. This shows that DMD-GEN provides a much clearer and more sensitive signal, which is a significant practical advantage over existing aggregate metrics.

2. Diagnostic Interpretability (Pinpointing a Failure Mode): Beyond a single score, DMD-GEN can help diagnose what kind of dynamics are being lost. As we will add in a new appendix figure (in response to reviewer feedback), the optimal transport plan from our metric can identify specific original (real) samples that are poorly matched by the generator. By visualizing the DMD modes of these ''unmatched'' samples, a user can see exactly which dynamic patterns (e.g., a high-frequency oscillation, a slow decay) were missed. This diagnostic capability is a novel insight that aggregate scores from metrics like Predictive Score or Context-FID cannot provide.

We will add a paragraph and an illustrative figure to our experiment section to explicitly highlight these two unique insights demonstrated by our results.

We thank the reviewer again for their supportive and constructive feedback. We are confident that these clarifications and additions will strengthen our paper and more clearly articulate the novel contributions of DMD-GEN.

We sincerely thank Reviewer jdT8 for their detailed and highly constructive review. The feedback has helped us identify a few areas where our experimental validation and explanations can be significantly strengthened. We have run new experiments to directly address your concerns, and we are confident these additions improve the paper.

1. On Hyperparameter Choices and Sensitivity

• [Question 1] Choice of τ=0.95: We chose τ=0.95 as it is a standard and well-established practice in spectral analysis literature for capturing a system’s dominant energy/variance without overfitting to noise. Our experiments (e.g., Table 1) confirm that this value provides stable and meaningful results across diverse datasets. A dynamic, data-dependent selection of τ is non-trivial, as it depends on the singular value spectrum from SVD, a non-differentiable operation that cannot be easily optimized. We will clarify this rationale in Section 3.1.

• [Limitations] Sensitivity to Sequence Length: We agree that the metric’s sensitivity to sequence length is an important limitation to discuss. Shorter sequences provide fewer snapshots for DMD to reliably estimate modes. To quantify this, we ran a new experiment on the Stock dataset (with DiffusionTS) varying sequence length. The results confirm that as sequence length increases, the DMD-GEN score becomes more stable (lower variance), reflecting the metric’s improved ability to capture richer dynamics from more structured data. We will add this analysis to the limitations section.

2. Strengthening Experimental Validation and Presentation

• [Question 2] Extreme Mode Collapse Scenarios: This is an excellent suggestion. We have run the synthetic experiment for more extreme mode collapse cases (λ=80% and 90%) and will add them to Table 2. The results further validate our claims: DMD-GEN and Context-FID remain highly sensitive, while the Predictive score is far less responsive. (Note: These settings are symmetric to the 20% and 10% cases already in the paper).

• [Question 4 & Concern 1] Clarifying Baselines and Failures:

o [Question 4] MTopDiv: We agree that the rationale for not including MTopDiv in the main comparison should be clearer. MTopDiv is not designed for sequential data; it flattens time series, ignoring temporal dependencies. Our experiments in the supplementary material show it produces scores with excessively high variance, making it unreliable for this task. We will clarify this in the main text.

o [Concern 1] Missing Values in Table 1: The dashes ("—") denote computational failures. For certain datasets such as Stock Dataset, training the auxiliary models required by metrics like Context-FID was numerically unstable. We will add a note to Table 1 explaining this. This highlights a key advantage of DMD-GEN: its training-free nature makes it more robust and broadly applicable.

3. On the Interpretation of Visual Results (Figures 1 & 2)

• [Question 3] Eigenvalue Overlap in Figure 1: The reviewer correctly observes the significant overlap post-training. This is not a sign of insensitivity, but rather a visual confirmation of successful learning. At Epoch 0, the eigenvalues are distinct because the model’s dynamics are random. The post-training overlap signifies that the generator has learned to reproduce the characteristic frequencies and growth/decay rates of the real data. We will revise the text to make it clear that this overlap is the desired outcome.

• [Concern 2] High Variance in Figure 2: The reviewer correctly notes the high variance for small block sizes. This is not a flaw in our metric but an expected outcome that DMD-GEN correctly captures. The high variance stems directly from the Moving Block Bootstrap (MBB) process itself, not our metric: when the block size is small, each bootstrapped time series becomes a highly randomized shuffle of the original patterns. Because this resampling introduces significant variability, the resulting samples have very different dynamic structures from one another. Our metric’s high variance in this regime accurately reflects this induced randomness. Conversely, once the block size is large enough to preserve the underlying temporal dynamics, the DMD-GEN estimates stabilize. This confirms that the metric is reliable and robust, provided the resampling window is appropriate for the data’s structure. We will ensure this full explanation is included in our discussion of Figure 2.

Once again, we thank the reviewer for their valuable insights, which we believe have helped us improve the quality of our work.

We sincerely thank Reviewer CvCr for their positive assessment and insightful feedback. We are encouraged that the reviewer found our work to be a technically solid contribution that addresses an important and underexplored area. We address the reviewer's valuable suggestions below.

[Weakness 1] Relation to Image Generation Evaluation Metrics. We thank the reviewer for recommending these references. While our main focus is on evaluating temporal generative models, we agree that incorporating a discussion of image-based diversity metrics such as RKE and FKEA-Vendi would enhance our work and helps situate our contribution within the broader context of diversity evaluation. In the revised manuscript, we have added a discussion of these methods in our related work section (Section 2).

Conceptual Distinction (FKEA-Vendi): A key distinction is that FKEA-Vendi is a reference-free metric, designed to measure the intrinsic diversity of a generated set. While valuable, our work specifically targets mode collapse, a phenomenon that, by definition, requires a comparison between the generated samples and the ground-truth data distribution. Therefore, a reference-based metric like DMD-GEN is not only appropriate but essential for detecting and identifying mode collapse, as it directly quantifies how well the diversity of the original data is preserved.

Experimental Comparison (RRKE): To directly address the reviewer’s request, we conducted new experiments comparing DMD-GEN with RRKE. To apply RRKE to time series, we flattened each multivariate sample into a single vector. While this process unfortunately discards the temporal structure that DMD-GEN is designed to analyze, it provides a valuable point of comparison. As shown in the table below, the model rankings produced by RRKE are consistent with DMD-GEN on the ETTh and Sines datasets. This consistency with an established eigendecomposition-based metric further validates our approach. We will include the complete RRKE results for all datasets in the final manuscript.

[Weakness 2] Illustrative Toy Example. Thank you for the valuable suggestion. In our revised version, we added a new figure featuring an intuitive toy example. This example shows a ground-truth dataset with two distinct, easily visible dynamic modes (e.g., a decaying sinusoid and a growing linear trend). We then compare this to a ‘‘mode-collapsed’’ generated set that only captures one of the modes. This figure will visually demonstrate how the DMD eigenvalues of the generated set fail to match the full spectrum of the original, and how DMD-GEN assigns a high (poor) score, directly aligning with visual intuition. This new visual example will be a simple introduction to the more detailed tests in Section 4 (Table 2 and Figure 2), where we rigorously test DMD-GEN’s sensitivity across a spectrum of synthetically-controlled mode collapse severities.

[Question 1] Required Number of Samples for Convergence. This is an excellent question regarding the practical application and efficiency of our metric. To demonstrate the sample efficiency of DMD-GEN, we performed an ablation study on the Stock dataset using TimeGAN, varying the number of samples per batch.

The results demonstrate the stability of DMD-GEN. The mean value remains highly consistent even when the batch size is reduced to just 16 samples, with only a minor, acceptable increase in variance. This highlights that our metric is efficient and can provide reliable estimates without requiring a large number of evaluation samples. We added this analysis to the appendix to provide clear guidance for future users.

[Question 2] Visualization of Samples for Interpretability. Thank you for this constructive suggestion, as it directly ties into a core contribution of our work: the interpretability of DMD-GEN. Visualizing the modes is a perfect way to make this concrete. In the revised manuscript, we will add a new figure to illustrate this. Specifically, our analysis will: 1) Compute the DMD modes for a batch of original time series. 2) Use the optimal transport plan \\gamma^\\star (from the DMD-GEN calculation) to cluster these original time series based on the similarity of their dynamic mappings to the generated data. 3) For each resulting cluster, we will visualize a representative time series sample. 4) Alongside the sample, we will plot its dominant underlying DMD modes (those with the largest eigenvalues). This visualization will provide a clear, interpretable link between the raw time series and the specific dynamic patterns that are (or are not) being successfully captured by the generative model, showcasing the diagnostic power of our metric.

We thank the reviewer once again for their time and valuable feedback, which has helped us to significantly improve the clarity and completeness of our paper.