Breakthrough Sensor-Limited Single View: Towards Implicit Temporal Dynamics for Time Series Domain Adaptation

Thanks for the helpful feedback!

Q1: Modules are composed of well-known ideas with limited novelty

We appreciate reviewer's insightful feedback regarding novelty assessment. Many high-level ideas (curriculum learning and fusion mechanism) are broad research paradigms, as evidenced by surveys [1-3]. However, novelties of methods mentioned in surveys [1-3] are recognized in top venues (NeurIPS, ICML, ICLR). Hence, solely using high-level ideas cannot limit novelty; the novelty lies in how to design special curricula and fusion mechanisms for special scenarios.

Similarly, novelty of EDEN lies in special designs that address unique challenges of TSDA.

1. Theoretical Insights

In UDA theory (Eq. 2 in paper), existing TSDA methods fail to consider transferability ($d_{\mathcal{H}\Delta\mathcal{H}}$) and discriminability ($\lambda^*$) to design special curricula, fusion mechanisms and consistency regularization. EDEN bridge this gap.

2. Method-Specific Innovations

MSCA: Theory-Inspired Curriculum Design

• Contrary to heuristic curricula, we propose the first coarse-to-fine scale curriculum to consider transferability and discriminability guided by TSDA theory.

Quality-Aware Feature Fusion: Theory-Inspired Fusion Mechanism

• Replaces heuristic fusion with a learnable feature scoring module based on transferability-discriminability trade-offs.

Temporal Coherence Learning: Consistency at Score-Level

• The first to leverage TSDA theory-inspired scores for consistency learning.

[1] A survey on curriculum learning. TPAMI, 2021.

[2] Curriculum learning: A survey. IJCV, 2022.

[3] Deep multimodal data fusion. ACM computing surveys, 2024.

Q2: Difference between explicit domains and multi-scale or multi-view learning

"Explicit domains" emerged naturally from this context: temporal dynamics are inherently implicit, while time series data are constrained by human-controlled explicit conditions.

Difference from multi-scale or multi-view learning: EDEN's core contribution lies not in obtaining explicit domains but rather in how to use generated multiple explicit domains to tackle the unique challenge in TSDA: the entanglement of domain shift and intricate temporal patterns. This is beyond existing multi-scale or multi-view learning methods.

• From terminological perspective: different connotations

  • multi-scale vs. explicit domains Multi-scale is one of three types of explicit domains, emphasizing different sampling rates, but it cannot cover multi-subspace and multi-segment included in our explicit domains.
  • multi-view vs. explicit domains Multi-view learning refers to datasets that contain different modalities or types of data, such as an object described by text, video, and audio [1], where multiple views already exist within the dataset. In contrast, explicit domains emphasize being artificially derived from original time series through changes in sampling rate, record duration, and time-frequency transform.

• From the perspective of representation learning: different purposes

  • multi-scale vs. explicit domains The existing multi-scale learning approaches in time series leverage inherent multiperiodicity of time series, averaging forecasting series across multiple scales to enhance overall performance [2]. In contrast, multi-scale in EDEN emphasizes the transferability differences of different-scale time series, revealing that coarse-scale features exhibit better cross-domain transferability. This is a unique challenge in UDA that general multi-scale learning cannot address.

  • multi-view vs. explicit domains Existing multi-view learning in time series is to achieve self-supervised learning by utilizing the consistency between different augmented data [3]. However, EDEN is designed based on UDA theory insight.

    $\epsilon_{Q}(h) \leq \epsilon_{P}(h)+ \frac{1}{2}d_{\mathcal{H}\Delta\mathcal{H}(P, Q)} + \lambda^*$

    In UDA theory, $d_{\mathcal{H}\Delta\mathcal{H}(P, Q)}$ means transferability, while $\lambda^*$ means discriminability. Both jointly determine UDA performance. From perspectives of transferability and discriminability, EDEN explores the relationships between different scales, subspaces, and segments.

[1] Deep multi-view learning methods: A review. Neurocomputing, 2021.

[2] Timemixer: Decomposable multiscale mixing for time series forecasting. ICLR, 2024.

[3] Multi-view Self-Supervised Contrastive Learning for Multivariate Time Series. ACMMM. 2024.

Q3: Performance gains are modest (4.8%).

For comparison, we calculate improvements of methods relative to their respective SOTA at that time. Compared to baselines, EDEN achieves comparable performance gains. However, as field develops, performance will gradually approach its limits, and the rate of absolute improvement will also slow down. To further illustrate, we demonstrate performance gains of five general UDA methods relative to SOTA at the time of their publication. It is evident that the proportion of gains becomes smaller. However, this does not detract from the fact that these works are representative.

[1] Domain adaptation with conditional transferable components. ICML, 2016.

[2] Conditional adversarial domain adaptation. NeurIPS, 2018.

[3] Bridging Theory and Algorithm for Domain Adaptation. ICML, 2019.

[4] Gradually Vanishing Bridge for Adversarial Domain Adaptation. CVPR, 2020.

[5] Reusing the Task-specific Classifier as a Discriminator: Discriminator-free Adversarial Domain Adaptation. CVPR, 2022.

Q4: Lack strong ablations: clarify each module's contribution and why all three modules' combination is optimal

We respectfully note that this analysis is already provided in our submission: In Table 2, comprehensive ablation studies have been included, containing Module Ablation, MSCA Ablation and QFusion Ablation.

In Module Ablation, we not only investigate the contribution of each module, but also analyze their interaction.

• Each module's contribution

  As detailed in Line 292~295 of our paper and Row 1~4 of Table 2, EDEN’s three modules show performance gains when added individually: MSCA yields an average absolute accuracy gain of 1.57 across three datasets; QFusion improves accuracy by 2.56; and TCL enhances performance by 1.30.

• Optimal combination

  As detailed in Line 292~295 of our paper and Row 1~8 of Table 2, using one or two of these modules achieves suboptimal results, and full integration of the three modules (EDEN) achieves best performance. Against MSCA+TCL, EDEN improves +1.37 accuracy gain; against MSCA+QFusion, EDEN improves 1.24; against QFusion+TCL, EDEN improves 1.15. Thus, integrating all three modules emerges as the best choice.

Q5: Limited computational cost analysis

In Appendix D.1 of our paper, we present training time of EDEN on FD dataset.

We additionally report GPU memory consumption, FLOPs (Floating Point Operations), and Parameters, where FLOPs quantifies overall computational complexity. Our method achieves comparable computational cost to SOTA baselines, while delivering significant performance enhancement.

Q6: Evaluation is limited to classification

Forecasting constitutes a regression task, as is the mainstream approach (reconstruction) for anomaly detection. UDA problem for regression tasks remains unresolved.

First, regarding the development of UDA, the target error bound of UDA theory holds exclusively for classification. Subsequently, many works have continued to improve classification UDA based on this bound [1, 2]. Occasionally, other fields, such as CV, have explored regression problems, but they have lost theoretical guarantees [3]. Currently, there are few UDA methods that simultaneously address both DA classification and regression problems. Researching on classification is the mainstream approach in DA field.

Second, TSDA methods (e.g. baselines in our paper) universally assume classification tasks unless explicitly stated otherwise, and there are no methods that can simultaneously address both classification and regression issues.

Our paper focuses on theoretically-grounded design. MSCA and QFusion are designed inspired by the target error bound of UDA theory, which applies to classification tasks and is not suitable for forecasting or anomaly detection.

[1] A theory of learning from different domains. Machine learning, 2010.

[2] Bridging theory and algorithm for domain adaptation. ICML, 2019.

[3] Representation Subspace Distance for Domain Adaptation Regression. ICML, 2021.

Q7: Lack clear comparisons with more recent TSDA baselines

In experimental section, we select five representative TSDA baselines published in the last five years: ACON(24), RAINCOAT(23), CLUDA(22), AdvSKM(21), and CODATS(20). Among these, ACON is SOTA. In Related Work, we provide detailed introductions and comparisons of these works. In addition to these works, if the reviewer could provide specific baselines that would facilitate further comparison, we would greatly appreciate it.

Please do let us know if you have any further questions.

Dear Reviewer aFd7,

We sincerely thank you for your invaluable insights and constructive feedback. We deeply appreciate this opportunity to address your concerns and refine our work.

In our response, we have addressed the following queries:

• Clarification on EDEN's novelty from the perspectives of theoretical insights and method-specific innovations (W1&L1 in the original review -> Q1 in our rebuttal).
• Explanation of the fundamental differences between explicit domains and multi-scale or multi-view learning from the perspectives of the terminological perspective and representation learning (W2&Q3&L2 in the original review -> Q2 in our rebuttal).
• Justification of compelling performance gain (W3&L3 in the original review -> Q3 in our rebuttal).
• Clarification on inclusion of strong ablations, which not only directly demonstrates the contribution of each module but also proves that the full integration of the three modules is the optimal choice (W6&W7&Q2 in the original review -> Q4 in our rebuttal).
• Supplementary analysis of the computational cost, which includes not only the training time analysis already presented in our paper but also additional analyses of memory, FLOPs, and parameters (W4&L4 in the original review -> Q5 in our rebuttal).
• Justification of classification-focused evaluation scope from the perspectives of the development of UDA and the existing TSDA methods' evaluation protocol (W5&Q1&L5 in the original review -> Q6 in our rebuttal).
• Clarification on comprehensive baseline comparison (W6 in the original review -> Q7 in our rebuttal).

We hope our responses have adequately addressed your concerns. If you have any further concerns or questions, please do not hesitate to let us know, and we will be more than happy to address them promptly. Thanks for your recognition of our work and the thoughtful and constructive feedback again!

Best Regards,

All Authors

Thank you for your dedicated time and effort in reviewing our paper. We sincerely appreciate your thoughtful follow-up and positive assessment.

We understand that for senior machine learning researchers, the standard of novelty is high and is influenced by personal research taste. Therefore, it is understandable if there are remaining concerns regarding the novelty of our method. We are delighted that there is still an opportunity before the discussion to delve deeper into what constitutes the novelty of our paper.

Original Review: "The proposed modules are largely based on established ideas (curriculum learning, adaptive fusion, segment-level consistency), with limited methodological novelty."

As you noted, our method is indeed inspired by well-known ideas (curriculum learning, adaptive fusion, and segment-level consistency). However, contrary to your view, we respectfully maintain that utilizing such high-level ideas does not preclude novelty in our method. We would like to present objective arguments demonstrating that our contributions about methodological novelty meet and exceed the standards of the top conferences.

1. The novelty of Multi-Scale Curriculum Adaptation

As a general idea in Machine Learning, Curriculum Learning [1] was ranked first in the Most Influential ICML Papers of 2009 (https://resources.paperdigest.org/2024/05/most-influential-icml-papers-2024-05). We acknowledge that our paper may not match the groundbreaking impact of the first work to propose the general idea of curriculum learning, nor can it rival the most influential ICML 2009 papers in terms of historical significance.

However, based on the idea of curriculum learning, what specific problem to address and how to design a novel and effective curriculum can constitute sufficient novelty. For this reason, over the past 16 years, hundreds of curriculum learning-inspired works have been published at top conferences like ICML, NeurIPS, and ICLR. These papers did not propose fundamental ideas comparable to curriculum learning. Instead, they applied the ideas of curriculum learning to specific tasks, designing tailored curricula, which were also recognized by the most prestigious venues. Here, we list three influential works:

• Reference [2] published in CVPR 2020 with more than 700 citations: CurricularFace: Adaptive Curriculum Learning Loss for Deep Face Recognition.
  • Propose an adaptive curriculum learning strategy from easy samples to hard samples for deep face recognition.
• Reference [3] published in NeurIPS 2021 with more than 1000 citations: FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling.
  • Propose a curriculum learning approach from a lower threshold to a higher threshold to leverage unlabeled data.
• Reference [4] published in NeurIPS 2020 with more than 100 citations: Safe Reinforcement Learning via Curriculum Induction.
  • Propose a curriculum induction from soft reset interventions to hard reset interventions to gradually relax safety constraints to approach the original environment.

To this day, the concept of curriculum learning continues to thrive across diverse domains, inspiring emerging fields such as AI4Science [5], diffusion model training [6], and vision foundation model fine-tuning [7]. While these works are largely based on the established idea of curriculum learning, they nevertheless design distinct curricula to address specific challenges, constituting their primary novelty contribution.

Similarly, the novelty of Multi-Scale Curriculum Adaptation stems from the proposed first coarse-to-fine scale curriculum to consider transferability and discriminability guided by TSDA theory, to the best of our knowledge, which remains unexplored in existing curriculum learning methods and TSDA methods.

[1] Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In ICML, 2009.

[2] CurricularFace: Adaptive Curriculum Learning Loss for Deep Face Recognition. In CVPR, 2020.

[3] FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling. In NeurIPS, 2021.

[4] Safe Reinforcement Learning via Curriculum Induction. In NeurIPS, 2020.

[5] Curriculum Learning for Biological Sequence Prediction: The Case of De Novo Peptide Sequencing. In ICML, 2025.

[6] Denoising Task Difficulty-based Curriculum for Training Diffusion Models. In ICLR, 2025.

[7] Curriculum Fine-tuning of Vision Foundation Model for Medical Image Classification Under Label Noise. In NeurIPS, 2024.

Thank you for your helpful feedback! We are grateful that the reviewer highlights our motivations, technical solution, and theoretical insights. To clarify, we summarize the original weaknesses or questions and adjust the order of the issues.

Q1: Validation in low-resource scenarios (few-shot)

Thank you for your suggestion regarding additional experiments under the setting of few-shot source data to further validate EDEN's robustness.

• Modifications to the UDA setting

  We have modified the original UDA setting to a few-shot UDA setting. Specifically, we retain only K samples per class from the original labeled source data, while keeping the unlabeled target data unchanged. On the UCIHAR dataset, we conducted experiments with 5-shot, 10-shot, and 15-shot settings.

• Performance

  As shown in the table, EDEN not only achieves excellent performance under the original UDA setting, but also exhibits strong robustness and generalization capabilities in the few-shot UDA scenarios.

Q2: Comparisons with TS-TCC and TimeMAE

TS-TCC and TimeMAE are time series self-supervised representation learning methods. In the UDA setting, we first perform self-supervised learning on the feature extractor using labeled source domain data and unlabeled target domain data. Subsequently, we fine-tune the feature extractor and classifier using labeled source data to perform classification on the target domain. We conduct experiments on the UCIHAR dataset.

Compared to the Source-only method, these representation learning methods extract generalized features through self-supervised learning. However, since the fine-tuning of the classifier still relies on labeled source data, the model tends to overfit the source domain, unable to mitigate domain shift, resulting in relatively limited performance gain.

In summary, these representation learning methods primarily focus on general representation learning capabilities, while EDEN, with strong general representation learning abilities, places more emphasis on transferable representation learning. We will include these discussions in the related work section and add these comparisons in the experiment section in the next version.

Q3: Single-dataset time series classification

Following the setups and data splits of TS-TCC and TimeMAE, we conduct single-dataset classification experiments on the UCIHAR dataset.

• Modifications to EDEN

  Since only a single domain exists without domain shifts, we make minor modifications to EDEN. Specifically, we remove the domain discriminator in MSCA while retaining the curriculum mixing of coarse and fine-scale features. Additionally, we eliminate the transferability criterion in QFusion, performing feature fusion solely based on the discriminability criterion.

• Performance

  EEDEN achieves comparable experimental results in single-dataset time series classification compared to the advanced representation learning method TimeMAE and the general time series analysis method TimesNet [1], verifying its generalized feature extraction capability.

  Method	Accuracy
  TS-TCC	89.22±0.19
  TimeMAE	95.11±0.18
  EDEN	94.25±0.28
  TimesNet	96.93±0.24

  [1] Timesnet: Temporal 2d-variation modeling for general time series analysis. In ICLR, 2023.

Q4: Visualizations of domain shift

We adopt t-SNE to visualize the feature distributions on the UCIHAR dataset. However, due to the limitations of the rebuttal format, we're unable to upload images or external links. We can only provide the Euclidean distances $d$​ between the source and target features reduced to 2D space via t-SNE to show the domain gap.

The second column of the table represents the gap between the centers of the two domains, reflecting the discrepancy between the feature marginal distributions. The third column indicates the average gap between the centers of each class in the two domains, reflecting the discrepancy between the feature conditional distributions.

After applying MSCA to align the distributions, we observe a significant reduction in the discrepancy of marginal distributions, demonstrating that MSCA effectively aligns the marginal distributions of the two domains. However, a considerable gap in conditional distributions still remains.

When we add QFusion, we observe a substantial reduction in the discrepancy in conditional distributions. This validates the motivation behind QFusion and confirms the fusion mechanism's effectiveness. Specifically, the feature quality of the same class varies significantly across different subspaces. By using transferability and discriminability criteria, we can adaptively select higher-quality features, thus achieving better alignment at the class level.

Q5: Duplicate references in Lines 378 and 380

Thank you for pointing out this error. We will correct it in the next version.

We hope our responses were able to address any remaining concerns. Please do let us know if you have any further questions as well as what would be expected for score improvement. Even a slight increase in score would greatly help in recognizing the potential of our work. Thank you again for providing insightful comments.

Thank you for your helpful feedback! We are grateful that the reviewer highlights our motivations and experimental results.

W1: The abstract and introduction lack clarity in conveying the core ideas behind "multi-scale" and "multi-subspace" modeling.

Thank you for your valuable feedback.

We will revise the abstract and introduction to improve clarity and add explanations:

• "Multi-Scale Curriculum Adaptation progressively integrates coarse and fine-scale features to obtain the final extracted features, facilitating curriculum adaptation from coarse-to-fine by aligning the mixed feature between source and target domains. "
• "Quality-Aware Feature Fusion measures feature quality through transferability and discriminative scores for adaptive fusion. The transferability criterion is determined by A-distance, which reflects the prediction confidence of the domain classifier, while the discriminative criterion is based on the entropy and confidence of the classifier. To enhance score diversity at the class level, we also introduce diversity perturbation by encouraging the coefficient of variation."

Q1: The assumption that coarse-scale features are easier and more domain-invariant needs clearer theoretical or empirical support.

Empirically, in Figure 2 (a)(b) of our paper, we use A-distance [1] to quantify domain discrepancy of features at different scales. We further report the A-distance on three additional datasets in the following table. Combined with Figure 2(a)(b), the results validate our assumption on all six datasets, demonstrating that aligned features of coarse-scale time series exhibit smaller domain discrepancies. This quantitative analysis adequately supports the proposed coarse-to-fine curriculum.

Theoretically, as noted in Proposition 4.1 of our paper, A-distance $d_A = 2(1-2\epsilon)$is determined by the generalization error $\epsilon$ of discriminating between source and target examples, and serves as a proxy of $\mathcal{H}$-divergence [2] between the source domain $P$ and target domain $Q$ , $d_\mathcal{H}(P, Q)=2\sup_{h\in\mathcal{H}}|Pr_P[h]-Pr_Q[h]|$​. This metric guides the family of domain adversarial learning methods (e.g., DANN, CDAN), providing valid estimates of domain discrepancy.

[1] Domain-Adversarial Training of Neural Networks. JMLR, 2016.

[2] A theory of learning from different domains. Machine learning, 2010.

Q2: Handle different time scales of source and target data

In existing time series analysis work, it is generally assumed that all training and testing data for the same feature extractor should have the same resolution. When there is a significant difference in resolution among data, it is typically addressed by ensuring consistency through downsampling. This is because time series data has a high information density; when the sampling rates differ, a segment of length 5 may capture variations within 5 hours in one case and variations over 5 days in another, leading to completely different distributions. Therefore, it is not feasible to use the same feature extractor for both.

Q3: Only three types of explicit domains are explored. other forms—e.g., modality, condition, or context—be useful?

Thank you for your valuable insights. Leveraging other forms for domain adaptation is a promising research direction. However, it is important to note that the other forms, e.g., modality, condition, or context, typically depend on external knowledge, which is not universally available across datasets.

In contrast, our proposed three explicit domains are derived from the original time series without relying on any external information, and are universal across multiple datasets. These explicit domains are the only three explicit domains, which is fundamentally determined by the intrinsic limitations of time series sampling (sampling rate, record duration) and the inherent property that discrete signals can be transformed into the frequency domain.

We believe that external modalities, conditions, or contexts can enhance UDA performance from two aspects.

• Meta-information guided transfer For example, in action recognition tasks, we know the context of the subjects' dominant hands—one being left-handed and the other right-handed. Using this information, we can adjust the distribution of channels in the original series to achieve spatial alignment at the channel level.
• External conditions assisted classification For example, in action recognition tasks, the data not only includes time series but also contains determining conditions. For instance, if the main frequency is below a specified value, it indicates a stationary state. We can use this information to assist in classification.

Q4: Module Naming Consistency

Thank you for your suggestion. We will revise the module names in the next version to ensure naming consistency.

We hope our responses were able to address any remaining concerns. Please do let us know if you have any further questions as well as what would be expected for score improvement. Even a slight increase in score would greatly help in recognizing the potential of our work. Thank you again for providing insightful comments.