ASCENSION: Autoencoder-Based Latent Space Class Expansion for Time Series Data Augmentation

Thank you for the review. Below, we clarify the novelty and motivation of our work and provide new results supporting our key hypotheses

Summary

"C4.1 Claimed novelty lies in latent space and use of VAE for DA in time-series, but these are not fundamentally new C4.2 Motivation of the paper is unclear C4.3 Paper doesn’t clearly explain th limitations in time-series and how ASCENSION addresses class expansion."

This work is motivated by the fact that most state-of-the-art DA methods for time series focus on intra-class generation. We hypothesize that controlled, progressive class boundary expansion in latent space can boost classification. While prior works (e.g., Modals, [Wa24, Wa24b]) explore class expansion, they do not include gradual control. Our key contribution, the α-scaling mechanism, enables this, going beyond clustering loss combination. Experiments show ASCENSION consistently outperforms baselines: with ResNet, gains of 2.8%–8.5%, and up to 30.2% vs. KoVAE; with FCN, 1.3%–13.2%, and up to 31.7%. A new ablation study (added to rebuttal) attributes 7%–61% of the gains to α-scaling. Due to space limits, we refer the reviewer to our response to Reviewer h2Y9 (C3.5) for more details about this ablation study. These new results will be included in the revised paper.

[Wa24] Wang, T. et al. (2024) Fine-grained Control of Generative Data Augmentation in IoT Sensing. Advances in Neural Information Processing Systems, 37, 32787-32812.

[Wa24b] Wang, T. et al. (2024) Data augmentation for human activity recognition via condition space interpolation within a generative model. ICCCN (2024)

"C4.4 Paper lacks clarity and coherence; Key details are scattered across sections, making it hard to follow..."

Thank you for pointing out the language issues - we’ve corrected them. We’re also open to any specific suggestions the reviewer may have regarding content organization."

Methods And Evaluation Criteria

"C4.5 Evaluation method focuses only on overall classification accuracy and does not provide enough detail to demonstrate how ASCENSION effectively expands class probability densities/boundaries. Baseline models are not the most recent"

Section 4.2.6 and Fig. 5 qualitatively assess class expansion risks, with quantitative details in Appendix D. Based on reviewer suggestions, we added ImagenTime, Diffusion-TS, and KoVAE to our baselines. ASCENSION outperforms all: for FCN, they rank 3rd, 5th, and 9th in total accuracy; for ResNet, 4th, 6th, and 9th. Summary results appear below; Full results available at: https://github.com/ASCENSION-PAPER/ASCENSION/tree/main/comparison_results

• ResNet

• FCN

Theoretical Claims

"C4.6 The method lacks detailed proofs or a deeper theoretical explanation to support its claims."

Despite its simplicity, extensive empirical comparisons substantiate our method’s effectiveness against SoTa alternatives. The new ablation study quantifies the α-scaling mechanism and contrastive loss impact (cf response to Reviewer h2Y9 - C3.5). Although contrastive loss benefits are established (>2% accuracy increase), notably, the α-scaling mechanism contributes significantly (7 to 44%) of accuracy improvement over ResNet and 10 to 61% over FCN.

Existing sections will be refined to enhance clarity of the justification."

Supplementary Material

"C4.7 No. I didn't find the materials."

Supplementary materials are available on our anonymous GitHub (see Sec. 6): https://github.com/ASCENSION-PAPER

Relation To Broader Scientific Literature

"C4.9 Paper fails to explain limitations of SoTa methods and how ASCENSION addresses them."

Appendix A outlines key limitations (e.g., temporal distortion, GAN instability) and explains how ASCENSION advances the state of the art. Figure 6 gives a timeline of all baselines.

We appreciate the reviewer’s detailed and constructive comments. We address each of the raised concerns below. We clarify the novelty and rationale of ASCENSION compared to the referenced works below. We also provide ablation study results.

Claims And Evidence & Relation To Broader Scientific Literature

"C3.1 the assertion that 'no SoTa DA method for time-series classification enables progressive (iterative) and meaningful class boundary expansion...' may require reconsideration. [1] also appears to propose a progressive class boundary expansion... prior research has explored the potential of generating inter class synthetic samples [1,2]. It would be beneficial to do a wider survey and incorporate the findings into the related work."

References [1, 2] explore class boundary expansion via inter-class interpolation, but our approach differs in two key ways: (i) we extrapolate within a single class for controlled expansion, and (ii) the process is iterative and progressive—starting small and expanding gradually (cf. Fig. 1) —while assessing risk during sample generation, eliminating the need to assign class labels. We thank the reviewer for pointing out these works and will include them in the revised Related Work (App. A). Unfortunately, as public implementations are not yet available, we could not include them in our comparison. Nonetheless, we added three new DA baseline methods—ImagenTime, DiffusionTS, and KoVAE—as suggested by Reviewer SfpM. Full version of results available at: https://github.com/ASCENSION-PAPER/ASCENSION/tree/main/comparison_results

Methods And Evaluation Criteria

"C3.2 how is the clustering loss computed during training? Is the distance loss calculated for each data point within a batch? how does the weighting of the loss terms impact the final performance?"

$\mathcal{L}_{cluster}$ is a contrastive loss computed per batch and back-propagated with the total loss via SGD. We initially used uniform weights, but following the reviewer’s suggestion, we are running a grid search over 256 combinations. So far, 2 have been tested, showing only a 0.02% accuracy variation—too early for conclusions, but more results will be included in the revised version and Supplementary Material.

"C3.3 How does this approach compare to or differ from contrastive learning?"

We thank the reviewer for noting the imprecise description, $\mathcal{L}_{cluster}$ is in fact a contrastive loss and this will be clarified in the revised version.

"C3.4 The rationale behind this (iterative training) choice is not clearly explained. A more straightforward alternative could be to control class expansion by simply adjusting the clustering loss. The authors' own analysis at Line 651 suggests that the iterative training method is prone to instability ... they need to explain its fundamental advantages over other potential methods"

The rationale for iterative training stems from our hypothesis that progressively and controllably expanding class boundaries in latent space improves classification. While prior works (e.g., Modals and [1]) have explored class expansion, they lack mechanisms for gradual control. Our α-scaling mechanism preserves clear class boundaries before extrapolating outside a class space—unlike approaches that simply reduce clustering loss early on. Despite some instability, ASCENSION shows consistent improvements across all UCR 102 datasets: with ResNet, gains range from 2.8%–8.5% (Table 1) and up to 30.2% vs. KoVAE; with FCN, 1.3%–13.2%, and up to 31.7% vs. KoVAE. Due to space limits, we refer the reviewer to the new results table in our response to Reviewer SfpM (see C2.2). An ablation study (detailed below) attributes 7%–61% of these improvements to progressive class expansion. These findings will be included in the revised version.

Experimental Designs Or Analyses

"C3.5 A significant issue is the absence of an ablation study; it would be beneficial to understand how each of these elements independently contributes to the overall performance."

An ablation study was conducted for the rebuttal to assess the individual roles of the clustering loss and α-scaling mechanism - full results at: https://github.com/ASCENSION-PAPER/ASCENSION/tree/main/ablation_study/

• the α-scaling mechanism significantly improves performance, with median gains of 0.3–0.5% and top-quartile gains over 1.9%, accounting for 7–44% (ResNet) and 10–61% (FCN) of the accuracy gap with baselines. On 102 datasets, the performance delta (mean accuracy step>1 – step=1) is:

• removing the clustering loss yields only a 1.3% averafe gain (vs. 4% with the full method), with benefits limited to the first augmentation step. Subsequent degradation highlights its key role in supporting progressive expansion

We thank the reviewer for the detailed and constructive critique. We provide the requested clarifications below and provide comparison results with the suggested baselines. We will include all this in our revised manuscript.

Claims And Evidence & Methods And Evaluation Criteria

"C2.1 Several notations and definitions ($\mathcal{L}_{class}$, $K_y$) are poorly explained, reducing clarity around the contribution"

To enhance clarity, these notations will be further explained in the revision. Specifically, $L_{class}$ denotes the contrastive loss, while $K_y$ represents the number of points for class $y$ at the current augmentation step. The paper will be reorganized so that $\mathcal{L}_{class}$ and $K_y$ are defined as soon as they become relevant.

"C2.2 The absence of fair comparisons with strong generative baselines such as ImagenTime [1], DiffusionTS [2], KoVAE[3], GT-GAN [4]"

We have conducted additional experimental comparison studies for the rebuttal, incorporating the suggested methods. These new results will be incorporated into Section 4 (Results). Unfortunately, we were unable to reproduce GT-GAN, as the available code contains hard-coded parameters and would require substantial modifications. As it stands, the implementation appears difficult to reproduce without further clarification or updates.

A summary of the new results is provided below, demonstrating that ASCENSION significantly outperforms the evaluated baselines. ImagenTime, Diffusion-TS, and KoVAE are respectively ranked -- in terms of "total" accuracy performance -- 3rd, 5th and 9th for FCN (among all the evaluated baselines), and 4th, 6th and 9th for ResNet.

Full version of the new experiments (for all datasets) is available at: https://github.com/ASCENSION-PAPER/ASCENSION/tree/main/comparison_results

• ResNet

• FCN

Experimental Designs Or Analyses

"C2.3 Fairness of experimental baselines is questionable due to conditional generation mismatch."

We are not sure to understand what the reviewer means by "conditional generation mismatch". We would be happy to follow up on this topic if the reviewer can clarify.

"C2.4 No ablations are presented to show the necessity of the clustering loss or the α-scaling expansion mechanism independently"

"Two ablation studies were conducted to assess the individual roles of the clustering loss and α-scaling mechanism. These new studies will be added to the revised version. Full results: https://github.com/ASCENSION-PAPER/ASCENSION/tree/main/ablation_study

The results and findings of these two new studies are presented below:

• the α-scaling mechanism significantly improves performance, with median gains of 0.3–0.5% and top-quartile gains over 1.9%, accounting for 7–44% (ResNet) and 10–61% (FCN) of the accuracy gap with baselines. On 102 datasets, the performance delta (mean accuracy step>1 – step=1) is:

• Removing the clustering loss yields only a modest average accuracy gain of 1.3%, compared to 4% with the full method. Notably, improvements are mostly observed in the initial augmentation step, while later progressive steps lead to rapid accuracy degradation without the clustering loss, thus highlighting its essential role in sustaining performance throughout the augmentation process.

"C2.5 Questions For Authors - Why $\mathcal{L}_{cluster}$ is used and not simply contrastive loss?"

The term $\mathcal{L}_{cluster}$ in our paper refers to a contrastive loss. We thank the reviewer for highlighting the lack of clarity and will revise the manuscript accordingly.

We thank the reviewer for taking the time to read our work and offer such constructive feedback. We appreciate the recognition of the systematic nature of our experiments, as well as the suggestions to discuss the application of our method to multivariate data and expanding our ablation analyses.

Methods And Evaluation Criteria

"C1.1 It would make more sense if the authors can also apply their method on UEA datasets for multivariate time series classification tasks."

We agree with the reviewer that our method can theoretically apply to multivariate time series classification tasks. However, this would require changing the encoder to account for the multiple dimensions, as well as a specific parameter tuning. This is why we focus our extensive study on univariate time series. We will include a sentence discussing the potential extension to multivariate data, as the rebuttal period is (unfortunately) too short to conduct such adaptations and experiments.

Experimental Designs Or Analyses

"C1.2 One issue is that the ablation experiments are conducted on very specific datasets. Can the authors provide additional rationale on using certain datasets (instead of using other ones) for ablation analysis?"

Regarding the study associated with Figures 7 to 16 (impact of the number of iterations on classification performance), we chose -- for conciseness -- to present results for one representative dataset per UCR category, e.g., one from the 6 ECG Signal datasets, one from the 8 Device datasets, etc. Full results are available in the supplementary material at: https://github.com/ASCENSION-PAPER/ASCENSION/tree/main/alpha_study

We believe this subset sufficiently captures the overall trends, which align with the discussion in Appendix C. If the remaining graphs are deemed essential, we are happy to generate and include them in the Supplementary Material (given this requires significant computation time). We thank the reviewer for highlighting this and will add a clarifying note in the revised version for transparency.

Essential References Not Discussed

"C1.3 VAE-based augmentation method is classical and has been widely investigated, and the authors can provide slightly more comprehensive literature search to include more related work."

In the initial version of the paper, we did include a comparison with VaDE, a seminal VAE-based data augmentation technique. In addition of VaDE, we incorporated a recent method, KoVAE [Na23], in our new experiments (conducted for the rebuttal), as well as two additional diffusion model-based DA methods, Diffusion-TS [Yu24] and ImagenTime [Na24], in order to strengthen the benchmark study. A summary of the new results is provided below, demonstrating that ASCENSION significantly outperforms all the evaluated baselines (incl., KoVAE). Full version of the new experiments (for all datasets) is available at: https://github.com/ASCENSION-PAPER/ASCENSION/tree/main/comparison_results

These new results will be added to the result section (Sec. 4), and methods discussed in the Related Work (Appendix A), which covers the history of VAE-, diffusion model- and GAN-based data augmentation methods. We are happy to discuss additional references that the reviewer would provide.

• ResNet

• FCN

[Na23] Naiman, I. et al. (2023). Generative modeling of regular and irregular time series data via Koopman VAEs. arXiv preprint arXiv:2310.02619.

[Na24] Naiman, I., Berman, N. et al. (2024). Utilizing image transforms and diffusion models for generative modeling of short and long time series. Advances in Neural Information Processing Systems, 37, 121699-121730.

[Yu24] Yuan, X., & Qiao, Y. (2024). Diffusion-ts: Interpretable diffusion for general time series generation. arXiv preprint arXiv:2403.01742.