Phase-Driven Domain Generalizable Learning For Nonstationary Time Series Classification

L225 : What is the distribution of $\mathbf{k}$? How can it be uniformly distributed on the non-negative integers rather than a finite subset? Why should the window length $w$ be random at all?

$p_i$ is a random non-negative integer, and the only constraint is $2^{p_i}<\Xi$.

The choice of $w$ results in varied resolution for the STFT spectra which is a hyperparameter in this case.

What is the overall intuition behind this approach? Why is it expected to work better than using the signal in the time domain, or even the original signal concatenated with its phase-shifted version?

We cannot disentangle the task-irrelevant nonstationarity from the task-relevant components if we maintain only the temporal view. Translating the time series to the time-frequency spectrum allows us to leverage the magnitude response that is maintained during the augmentation and diversify the phase component and also allow it to be used as a residual deeper in the network.

Def. 2.3 : What does it mean for a domain to be "built on an input variable"?

We are referring to that a domain is defined as the data with input and label variables, $\mathbf{x}$ and $y$, respectively.

Theorem 2.5 : How is risk defined?

The risk is the Gibbs risk, i.e., $R_{\mathcal{D}}[\rho]=\mathbb{E}_{(\mathbf{x}, y) \sim \mathcal{D}} \mathbb{E}_h \mathrm{I}[h(\mathbf{x})\neq y]$ where $h \sim \rho$.

L343 : Why is it beneficial that the distribution is altered? Is the distribution of the Hilbert-transformed output comparable to the original data from some domain? If not, it seems counterintuitive that adding "different data" would simplify classification tasks.

The Hilbert-transformed output alters the temporal statistics while retaining the spectral information which allows the model to learn generalizable time-series representation for the task semantics.

Baseline Methods : Why are no strong baseline methods for time series classification included?

We want to clarify that we are focused on domain generalization for non-stationary time-series classification and do not claim to address general time-series classification. Accordingly, our choice of datasets includes those with distinct domain discrepancies [1], and in our setting, we do not access domain labels or target samples from unseen domains. To that end, we have carefully selected datasets to reflect different design choices and to justify our performance in addressing this problem.

[1] "AdaTime: A Benchmarking Suite for Domain Adaptation on Time Series Data", TKDD, 2023.

Only accuracy is reported. Are the datasets balanced, making accuracy a reasonable metric?

We present the class distributions of all the datasets used in the manuscript in Figure 8 of the Appendix. Only WISDM and Sleep Stage Classification (SSC) datasets demonstrate certain imbalances among a few classes. We have presented the AUC score and the accuracy metric for the WISDM dataset in Figure 9 of the Appendix, and our conclusions remain consistent. Generally, the past works utilizing these datasets had adopted accuracy as the performance metric, and hence we adhered to the same in the submission. From our additional analyses, we can see that the choice of metric does not change our inference.

Why is the "Related Work" section positioned at the end of the paper? This structure seems non-standard.

We decided to place the "Related Work" section toward the end because we wanted readers to dive into our methods and design principles right from the start. This way, the technical ideas come across more directly, with related work added afterward for broader context. Many papers have taken this approach, and it felt like the best way to tell the story here. We hope this stylistic choice doesn’t detract from the main ideas and appreciate your understanding.

L508 : "Our study is the first to rigorously address the impact of non-stationarity on time-series out-of-distribution classification." This is the first mention of OOD classification in the paper. How is this addressed?

Our work focuses on domain generalization, where the model is trained on source domains using design elements such as diversification and the reintroduction of a nonstationarity dictionary, which aid in learning generalizable representations. The model is then evaluated on unseen target domains, which consist of out-of-distribution data. This nomenclature is not introduced in this paper but is instead a standard practice in works on domain generalization.

We appreciate your thorough feedback and recognition of the novelty of our design elements and the importance of the problem of domain generalization for nonstationary time series classification.

Before addressing the more specific questions, we want to discuss our choice of baselines to address one of your key concern. Below is a breakdown of the various baselines we have carefully selected to reflect different design choices and to justify our performance in addressing this problem.

• Domain-Generalization for time-series classification: Diversify (ICLR'23)
• Domain-Adaptation for time-series classification: MAPU (KDD'23)
• Non-Stationary representation learning for time-series: NSTrans (Neurips'22), Koopa (Neurips'24)
• Time-series Foundation Model: Chronos (TMLR'24)
• Domain-Generalization for audio applications: BCResNet (DCASE'21)
• Domain-Generalization for vision applications: ERM, ANDMask, RSC, DANN, GroupDRO (standard benchmark from Gulrajani et.al in ICLR'21)
• PhASER + model-agnostic add-ins: RevIN (ICLR'21)

While we understand that many works can be considered relevant, it is not feasible to implement all baselines in our testbed and compare performance against every possible model. Therefore, we selected baselines that provide insights into our performance relative to specific areas of interest, such as nonstationarity modeling, domain generalization with domain labels (as in vision applications), and domain adaptation using target samples (as in MAPU). If you have a particular baseline that you believe would enhance our discussion, we would be more than willing to consider it and aim to provide timely results for your consideration during the rebuttal period.

To support reproducibility and future benchmarking, we have followed the data splits established by previous works and ensured comprehensive coverage of baselines by referencing previous domain generalization work on time-series classification using these dataset splits.

Questions

Definition 2.1 excludes stationary time series. How well is the proposed method expected to perform if the time series is indeed stationary?

Most real-world signals with domain discrepancies are nonstationary in nature. However, if the signals were purely stationary, the task of representation learning becomes much simpler, and our method would not offer any significant advantage in such cases. For instance, consider the GR dataset, which has the smallest ADF statistics, indicating it is the least nonstationary signal among all the datasets we tested. On this dataset, PhASER and DIVERSIFY perform nearly equivalently (refer Table 4 in the main manuscript). Therefore, while our framework can be applied to all time-series domain generalization tasks, it truly shines when the signals are nonstationary.

Def. 2.2 :

• What do $\mathcal{S}$ and $\mathcal{T}$ represent?

  • The former is the marginal input distribution while the latter is the marginal label distribution.

• Are the samples assumed to be independent?

  • Yes, they are independent.

• How is it determined which source domain(s) $\mathcal{S}_i$ a sample $\mathbf{x}_i$ is drawn from?

  • It is drawn from all sources

• What is being defined in "Definition 2.2"?

  • We define the basic setups and objective of Time-Series Domain Generalization problem.

• Equation (2) suggests that each observation $\mathbf{x}_i$ comes from a single source. How is this related to $\mathcal{T}$?

  • $\mathbf{S}$ is the joint domain that unifies all $\mathcal{S}_i$.

• $\mathcal{T}$ might denote "any potential unseen target domain." Is it fixed for optimization purposes, or could it vary?

  • $\mathcal{D}_\mathrm{U}$ is not fixed, we just use it to denote a particular unseen target domain.

• The composition $f \circ g$ is interpreted as $g \circ f$, which is non-standard.

  • We put $g$ behind F to illustrate their positional relationship.

L108: "Note that the joint distributions of different source domains are similar but distinct." In what sense are they similar?

They are similar in their spectral content but their temporal statistics are different.

L180 : What is the explicit effect of the phase shift on the distribution? While the data may appear different, how does it affect the distribution? (Partially addressed in Thm. 2.6.)

The mean and variance of the distribution changes while the energy contained in time-varying frequency components is maintained.

L186 : Why is it assumed that the time series $\mathbf{x}_i$ can be characterized by a deterministic function? This seems counterintuitive, as time series are typically modeled as random processes.

We claim this assumption on a time series signal observation to better define Hilbert Transform with mathematical symbols.

L200 : Does each time series $\mathbf{x}_i$ essentially produce a second, phase-shifted time series $\mathbf{x}_i'$?

Yes, it does.

We have run a new baseline, InceptionTime [2]—a top-ranked deep-learning time-series classification model from a recent bake-off challenge [1]—on all our datasets and included the results. It performs particularly well on one of the datasets, HHAR, and is our best baseline there. We have updated our results to reflect this appropriately. We hope this helps aid your assessment of our empirical evidence.

[1] Bake-Off Redux: A Review and Experimental Evaluation of Recent Time Series Classification Algorithms, Data Mining and Knowledge Discovery, 2024.

[2] InceptionTime: Finding AlexNet for Time Series Classification, Data Mining and Knowledge Discovery, 2020.

To assist in your assessment, we have also added statistics to our ablation study.

Additionally, we have made significant improvements to the mathematical notations and explanations of our theoretical justification. We hope you can refer to these updates and provide us with specific feedback on which formulations still seem imprecise to you.

Your feedback is important in improving our paper for the research community, and we greatly appreciate it. Thank you once again for your time.

Questions:

The authors present their ideas with remarkable clarity; however, I have one question: Could the authors provide a more detailed explanation of the benefits of the Phase-driven Residual Broadcasting method employed in this study, particularly in comparison to other multimodal fusion approaches? What specific advantages does it offer in this context?

A residual broadcasting-style network is used in many audio-event detection applications and is known to support generalizable learning while remaining conservative in model size [4, 5]. It may not be accurate to compare it with general multimodal fusion architectures (early fusion, late fusion, etc.), as we do not have distinct modalities but rather distinct representations (magnitude and phase) of the same modalities in the residual-broadcasting connection. It is more comparable to residual connections since this broadcasting unit can be repeated several times if the network is deeper.

[3] Broadcasted Residual Learning for Efficient Keyword Spotting, Interspeech, 2021

[4] Domain Generalization on Efficient Acoustic Scene Classification Using Residual Normalization, DCASE, 2021

We thank you for your time, and feedback and for commending our work on originality and contribution. We would like to respond to your questions and concerns below.

Weaknesses:

The paper employs the Hilbert Transform primarily because it effectively extracts phase information from signals and possesses non-parametric properties, making it particularly useful for handling nonstationary time series. However, wavelet transform offers significant advantages in processing nonstationary signals, especially in time-frequency analysis and multi-resolution feature extraction. Therefore, why is wavelet transform not utilized in this context?

We want to walk you through our design intuitions and justify our choice of using the Hilbert Transform's analytic expression to conduct a phase shift in nonstationary signals:

• Wavelet transform can help in applying random phase shifts at localized temporal scales, but the choice of the mother wavelet which is application-specific heavily governs the performance of this transform. In contrast, the Hilbert transform is very simple and ensures consistent magnitude response too, making it an attractive choice across various applications for conducting phase augmentation.

• As for the randomness of the phase augmentation within a signal, our preliminary analysis in Section 3.2, Figure 5, and the design explained in Section D.7.2 in the Appendix (Figure 10) using a modified version with Hilbert transform was not particularly beneficial. So, we don't find the motivation to go to extra lengths to discover a generally good wavelet to facilitate random phase augmentations through Wavelet transform.

• Although, we did explore the empirical model decomposition technique where we could extract instantaneous frequency, and by combining it with the complex representation of the signal, we can achieve instantaneous amplitude and phase thus facilitating any arbitrary phase shifts too. However, our initial results were not promising as they heavily depended on the choice of the number of intrinsic mode functions based on the application. We briefly discuss this in Section B of the Appendix (lines 1073-1078).

But, we agree that a wavelet transform is an interesting tool that can be harnessed to explore other time-frequency representations instead of the STFT if we want to employ special signal processing elements for a given application, such as multirate signal analysis or localization of a particular frequency component. However, for purposes of a generic way to conduct phase-diversification through in-sample augmentation, we advocate the use of Hilbert Transform.

The paper highlights the importance of phase information in the classification of nonstationary time series. Could you elaborate on how phase information influences the model's learning process? What specific roles does phase information play in different application scenarios?

Our intuition is that phase can serve as an approximate proxy for non-task-specific nonstationarities. We refer to Table 1, row 2, where using only phase features results in the lowest accuracy for activity recognition in the WISDM dataset. Task-specific nonstationarities are characterized through the Short-Time Fourier Transform (STFT), which ensures that information is not lost and allows the neural network to learn it if it aids in optimizing the objective. We demonstrate this intuition through our design process and show empirical evidence on 5 datasets and 11 baselines.

Image augmentation techniques, such as rotation, may significantly impact the input frequency and phase modalities, particularly when processing time series data. How can I determine the extent of these transformations to enhance performance without compromising the characteristics of the original data?

Yes, the direct application of augmentation techniques from the vision domain may not be suitable for all the time-series applications. Particularly for rotation, some past works on Human-activity recognition (HAR) [1] including a very recent work [2], suggest that it can help emulate different positions of placement of the Inertial-Measurement-Unit (IMU) which is usually the key sensor for wearable HAR. Therefore, we also explore it in the context of HHAR dataset briefly in Figure 5.

We do not attempt to propose a new type of time-series augmentation for general time-series representation learning; rather, we present our findings that phase anchoring through multiple stages supports domain generalization in time-series classification tasks, with augmentation being one of those stages.

[1] Generalizable low-resource activity recognition with diverse and discriminative representation learning, KDD, 2023

[2] UniMTS: Unified Pre-training for Motion Time Series, Neurips, 2024

Thank you for the question.

We do not mean to suggest that phase is the only way to assess the non-stationarity of a signal (as noted in our response to Reviewer VbRq). Through this work, we aim to demonstrate that phase can be diversified in a way that does not alter the spectral properties of a signal and by incorporating this diversified phase-derived embedding deeper into the network to emulate more "difficult" non-stationarities possible from unseen target domains, we can ensure that the model learns only task-relevant semantics overcoming the irrelevant nonstationarities for time-series classification. Our overall design intuition is supported by the controlled pilot study in Figure 3, where, despite increasing the non-stationarity of a signal, PhASER can perform consistently well.

We appreciate your recognition of the novelty in our approach and thank you for your time and feedback on improving our work. We address your concerns below.

The proposed approach, which adds phase information using the HT that in turn is processed in parallel with the signal's magnitude, is relatively simple. On its own, this may not be sufficient to justify publication in ICLR.

While we propose a simple, in-sample, non-parametric augmentation using the Hilbert Transform, our work also introduces two key model-related designs: separate magnitude-phase feature encoding and the use of phase residuals in a broadcasting operation. We evaluate our approach comprehensively across five datasets and against 11 diverse baselines, some of which are more complex, assume access to target domain samples, or leverage domain labels in their learning. Despite these advantages for the baselines, our method outperforms them in generalization performance.

As noted by you, "incorporating phase information in this manner for the purpose of time-series classification is new." Through this work, we aim to showcase the value of integrating signal-processing concepts into time-series deep learning analyses and present our findings on the advantage of a phase-anchored design scheme for generalizable representation learning in time-series classification tasks without accessing target samples or domain labels. Additionally, we provide benchmarks (leveraging data splits proposed by past works [1] and re-running older baselines on them) that the community can use for future research in this direction.

We believe that simplicity is a key strength of our approach, and we systematically demonstrate the value of each (carefully designed) component through ablation studies, supported by theoretical grounding. We hope that the community finds these ideas beneficial.

[1] "AdaTime: A Benchmarking Suite for Domain Adaptation on Time Series Data", TKDD, 2023.

The theoretical justification in Section 2.5 is unclear, and Theorem 2.6, in particular, is unconvincing. Numerous features of a signal, other than the phase, could exhibit changes in "nonstationary statistics”. Additionally, the term "nonstationary statistics" is used frequently throughout the paper but is not clearly defined. Question : I would like to ask the authors to clarify the meaning of their theoretical results, specifically a clearer explanation of Theorem 2.6 and a formal definition of "nonstationary statistics”.

We would like to draw your attention to Definition 2.1, especially Equation 1, where we define nonstationary time series. Regarding Theorem 2.6, we agree that attributes other than phase can contribute to a signal's nonstationarity. Therefore, we do not claim that phase is the only way to diversify the source domains' nonstationarity, but rather one of the ways.

We would also like to reiterate that our design takes into account the spectral nonstationarity inherent in real-world signals. While formally proving this is challenging without conducting signal decomposition under system-specific assumptions—unlike statistical nonstationarity, which can be characterized for representative samples using Augmented Dickey-Fuller (ADF) tests, as shown in Table 8 of the Appendix—our approach leverages transforms such as STFT and HT. These are standard tools for analyzing nonstationary or multirate signals while preserving the energy content of time-varying frequency components that are likely relevant to task semantics. By modifying only the phase, our method introduces diversity in temporal nonstationarity, thereby enriching the representation space without compromising the spectral characteristics critical for nonstationary time series classification.

We would greatly appreciate your further specific feedback on this justification to help improve our paper.

We thank you for your time and feedback. We address your main concerns below.

Many definitions and statements are not mathematically sound.

In Eq.(1), the left side represents the probability density function of the variable on the right side.

We have assumed the real-valued signal is characterized by a function $x(t)=\mathbf{x}_t$, thus use $x(t)$ in the definition of HT.

When doing the ADF test, the authors should provide the p-values instead of the raw statistics, which are not interpretable.

We have updated Table 8 in the Appendix to include p-values. Please refer to the updated pdf. Initially, we only reported the ADF statistics, following the reporting standards used in prior works [1, 2] on nonstationarity.

[1] "Non-stationary transformers: Exploring the stationarity in time series forecasting", Neurips, 2022.

[2] "Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictors", Neurips, 2024.

In Definition 2.2, is there a connection between $\mathcal{S}_i$ and $\mathcal{X}_\mathbf{S}$?

Yes, there is, as $\mathcal{X}_\mathbf{S}$ is the marginal input distribution of the joint domain that unifies all $\mathcal{S}_i$.

The authors introduce a beta divergence, which is not the usual one. Furthermore, it does not satisfy the properties of a divergence. Also, Equation 9, on which the theoretical analysis relies, must be clearly stated (not all terms are defined).

The widely used divergence in cross-domain learning is defined on hypothesis disagreement (based on task module)[3], which cannot characterize the nonstationarity in the raw features of time series. Beta-divergence is defined on the raw data space, thus suitable for our problem. All terms in Eq. (9) have been defined, $q$ is a non-negative constant while $\mathrm{RD}(\cdot)$ is the Renyi divergence.

[3] Mansour, Y., Mohri, M., and Rostamizadeh, A. Domain adaptation: Learning bounds and algorithms. In COLT, 2009.

There needs to be more clarity between the claims and what the proposed methodology can achieve.

We want to clarify that we are doing domain generalization (without accessing the domain labels) and not domain adaption which assumes access to target domain labels.

Our simplified intuition is as follows: after phase diversification while retaining the magnitude spectrum, we perform separate magnitude-phase encoding. Subsequently, we reintroduce the nonstationarity dictionary (diversified phase-derived embeddings) as a residual connection deeper in the network. This step mimics the aggressive task-irrelevant nonstationarity of unseen domains by corrupting the learned embeddings with phase embeddings—representations that are only weakly relevant to the task. If our model can still learn meaningful representations, it will generalize well.

Regarding more justification and literature for residual broadcasting-style network, it is often used for on-device audio-event detection applications and is known to support generalizable learning while maintaining a conservative model size [4, 5]. Design elements like sub-spectral normalization [6] help disambiguate high- and low-frequency information, enabling device-related generalization to account for factors such as sensor drifts or other slowly varying components (like Direct Current, DC) that typically manifest in the lower frequency spectra. Inspired by this intuition, applied in audio domains with Mel-Frequency Cepstral Coefficients (MFCCs), we extend this approach to multivariate time-series data using an STFT-based time-frequency representation, incorporating a phase-based residual with a more phase-anchored design.

[4] Broadcasted Residual Learning for Efficient Keyword Spotting, Interspeech, 2021 [5] Domain Generalization on Efficient Acoustic Scene Classification Using Residual Normalization, DCASE, 2021 [6] SubSpectral Normalization for Neural Audio Data Processing, ICASSP, 2021

We thank you for your response and patience while we address your concerns.

1. $\mu_{i, t}$ and $\sigma_{i,t}$ are defined in Definition 2.1 and represent the mean and variance of time-series signals from domain $\mathcal{S}_i$ at timestamp , $t$, respectively.

We have also highlighted other definitions, such as $\mathcal{D}_\mathrm{U}$, which were pointed out as undefined but are already described in Definition 2.2. We have also added an explanation to lines 108-109, and 160-161 upon your advice for clarity. Please let us know if you have any other notation-specific concerns, and we will address them promptly.

2. Regarding further theoretical justification about Theorem 2.5 : As mentioned in the paper, we regard $\beta$-divergence as a proxy of non-stationarity, but it is infeasible to approximate it in the raw feature space. Therefore we approximate the $\beta$-divergence in the representation space, and specifically, we approximate it in the low-level representation space extracted by the phase feature encoder $F_{\mathrm{Pha}}$ and in the high-level representation space extracted by the temporal feature encoder $F_\mathrm{Tem}$. To better minimize these two levels of approximations, we connect them via a residual connection. We have clarified this in the updated manuscript.

3. Regarding further explanation of Theorem 2.6: By using phase augmentation to modify the input data statistics, we encourage the model to learn task-relevant semantics despite diverse conditions of non-stationarity, which helps in learning more generalizable representations, ultimately leading to higher performance even for unseen target domains.

4. We have updated all the references to reflect their final venue of publication (except Chronos which got accepted to TMLR on 11 November and does not have a Digital Object Identifier yet).

5. Additionally, we have run a new baseline, InceptionTime [2] -- top-ranked deep-learning time-series classification model from a recent bake-off challenge [1] -- for all our datasets and included the results. We hope this helps in aiding your assessment of our empirical evidence.

[1] Bake off redux: a review and experimental evaluation of recent time series classification algorithms, Data Mining and Knowledge Discovery, 2024

[2] InceptionTime: Finding AlexNet for Time Series Classification, Data Mining and Knowledge Discovery, 2020

We thank you for taking the time to read and provide feedback on our paper. However, we would like to clarify a few points that may have been misunderstood:

1. Our paper does not focus on introducing a new type of time-series augmentation for general time-series representation learning; rather, it presents our findings that phase anchoring through multiple stages supports domain generalization in time series classification tasks, with augmentation being one of those stages.

2. While we propose a simple in-sample, non-parametric augmentation using the Hilbert Transform, we also introduce two other model-related designs: separate magnitude-phase feature encoding and the use of phase residuals for a broadcasting operation.

3. Beyond a comprehensive evaluation across 5 datasets and against 11 baselines, we systematically demonstrate the value of each component through ablation studies. We reiterate the respective average improvements achieved by separate phase-magnitude encoding and phase residuals, as shown in Table 5 below.