CaPulse: Detecting Anomalies by Tuning in to the Causal Rhythms of Time Series

Q3. How to determine an anomaly judgment? A threshold?

A4. Yes, a threshold can indeed be applied to the anomaly score to classify instances as anomalous or normal. For evaluation, we follow previous density-based anomaly detection methods [9,10], using the ROC curve calculated from anomaly scores to evaluate our method.

Q4. What is the meaning of Figure 6c?

A4. Figure 6c provides a visualization of the flattened representations to help understand the structure of the cause pyramid representation. We have clarified this in the revised manuscript (Line 452). Thank you for your helpful question.

Q5. In Figure 6a, I don't understand why CaPulse is the only one that can accurately predict the anomalies because I don't know why they are anomalies through the time-series plot.

A5. Thank you for the question. Figure 6a demonstrates that CaPulse is uniquely capable of detecting anomalies because it identifies the true causal factors underlying these anomalies, rather than relying solely on patterns in the time-series data, which is exactly our motivation of the design of our framework. However, we admit that the anomaly labels do not specify the exact type of anomaly (e.g., hardware failure), and we also lack labels for potential user misoperations. Therefore, our analysis in this part is based on inferences drawn from observation and the results.

We really appreciate your invaluable comments, which provided us with useful suggestions to make our work more solid and clearer. Your feedback means a great deal to us. We have revised our paper accordingly and hope that these changes address your questions and concerns. Thank you once again for your thoughtful input!

Reference

[1] Matrix profile I: all pairs similarity joins for time series: a unifying view that includes motifs, discords and shapelets. ICDM. 2016.

[2] Breaking the Time-Frequency Granularity Discrepancy in Time-Series Anomaly Detection. WWW 2024.

[3] TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis. ICLR 2023.

[4] A dataset to support research in the design of secure water treatment systems. In International conference on critical information infrastructures security. 2016.

[5] Causality Inspired Representation Learning for Domain Generalization. 2022.

[6] Causal-Debias: Unifying Debiasing in Pretrained Language Models and Fine-tuning via Causal Invariant Learning. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics. 2023.

[7] ​​Robusttad: Robust time series anomaly detection via decomposition and convolutional neural networks. 2021.

[8] Through the dual-prism: A spectral perspective on graph data augmentation for graph classification. 2024.

[9] Graph-augmented normalizing flows for anomaly detection of multiple time series. ICLR. 2022.

[10] Detecting multivariate time series anomalies with zero known label. AAAI. 2023.

[Questions]

Q1. If some non-causal factors can also be considered as causal factors depending on the domain?

A1. Thank you for the insightful question. In the proposed SCM, the non-causal factors $\boldsymbol{U}$ and the causal factors $\boldsymbol{C}$ are defined as unobserved latent variables that represent a range of potential influences. Based on whether a factor directly causes $\boldsymbol{y}$ or only affects $\boldsymbol{X}$ without impacting $\boldsymbol{y}$, we can categorize it as either a causal factor $\boldsymbol{C}$ or a non-causal factor $\boldsymbol{U}$. This distinction is, therefore, flexible and may vary depending on the specific domain or scenario, exemplified by the additional healthcare example discussed in our answer for W1. This adaptability allows our model to accommodate different domains. We have added this discussion in Appendix G.1, and we really appreciate this valuable comment.

Q2. What is the rationale for augmenting the raw input? Is it enough to simulate real-world disturbances? The performance of it? Would it have multiple ways of augmenting?

A2. Thank you for the questions about the augmentation operation. We address your questions point by point:

• Rationale for augmentation: As described in the first part of Section 4.1, the purpose of augmentation is to implement a causal intervention. Since $\boldsymbol{C}$ should be separated $\boldsymbol{U}$ according to the Common Cause Principle (Section 3.2), we perform an intervention on $\boldsymbol{U}$ does not make changes to $\boldsymbol{C}$. Then, by encouraging similarity between representations with and without $\boldsymbol{U}$, we aim to learn only the causal component, $\boldsymbol{C}$, as established in previous works [5,6].
• Simulating real-world disturbances: We acknowledge that real-world scenarios can be more complex, and our approach aims to simulate the essential parts of this complexity. As described in Lines 234-239, based on insights from prior literature [7,8], non-causal elements frequently appear randomly, similar to noise typically found in the high-frequency components of time series data. Therefore, we perform causal intervention by adding noise to the high-frequency (less significant) components of the input data to simulate these non-causal disturbances while preserving the core causal structure.
• Performance impact: As one of the core components of our framework, it indeed enhances the framework’s performance. This can be observed in our ablation study (Table 2), where the model without causal intervention (w/o CI) shows reduced performance, demonstrating the positive impact of causal intervention on our framework. We have also conducted additional experiments on more datasets, as shown in the following table, and hope to address your concern.

• Multiple augment methods: We agree with the suggestion to explore multiple augmentation methods. To address this, we have conducted additional experiments of different augmentation methods and combinations on two datasets (PSM and SMD), as shown in the following table. | Aug. Method | PSM| SMD | |--------|---------|--------| | HighFreq| 0.753 ± 0.042 | 0.906 ± 0.009 | | LowFreq| 0.743 ± 0.015 | 0.902 ± 0.007 | | Shift| 0.728 ± 0.011 | 0.885 ± 0.022 | | HighFreq + LowFreq| 0.725 ± 0.009 | 0.905 ± 0.005 | | HighFreq + Shift| 0.727 ± 0.011 | 0.884 ± 0.021 | | LowFreq + Shift| 0.725 ± 0.008 | 0.881 ± 0.018 | | HighFreq + LowFreq + Shift | 0.729 ± 0.014 | 0.874 ± 0.010 |

HighFreq refers to adding noise to the high-frequency components, which was our initial approach. LowFreq denotes adding noise to the low-frequency components, and Shift indicates shifting the input time series by 20 time steps. The "+" symbol represents a combination of different methods. According to the results, adding noise to high-frequency components yields the best performance, with LowFreq also performing well but slightly below HighFreq. Shifting the time series has a lesser impact, and combining multiple augmentation methods does not improve performance beyond HighFreq alone, suggesting that excessive variability may obscure meaningful causal patterns. We have included this analysis in Appendix G.2, and we appreciate the valuable feedback that encouraged us to explore these augmentation strategies.

We sincerely thank you for your valuable comments and for recognizing the novelty of our approach, the effectiveness of Periodic Normalizing Flows for periodicity handling, and our focus on interpretability. Below, we address the concerns you have raised point by point.

[Weaknesses]

W1. The causal view of TSAD might not hold in various real-world settings.

A1. Thank you for your insightful comments. We acknowledge that real-world scenarios can be more complex and dynamic. However, we believe that the fundamental patterns in the real world can be abstracted and represented within the proposed SCM for TSAD. To further support this point, in addition to the cloud computing platform example in Section 3.1, we provide another real-world scenario in healthcare: here, $\boldsymbol{X}$ could represent biometric data (e.g., heart rate, sleep patterns) from wearable devices, with anomalies $\boldsymbol{y}$ indicating potential health issues. In this context, $\boldsymbol{U}$ might represent environmental factors or background noise that can influence the reading $\boldsymbol{X}$ without indicating true bodily anomalies, while $\boldsymbol{C}$ can be factors like medication side effects, which directly impact both $\boldsymbol{X}$ and $\boldsymbol{y}$. We have added a discussion in Appendix G.1 to clarify these points. Thank you for this valuable comment which has significantly improved our paper.

W2. The limited number of baselines and benchmark datasets.

A1. Thank you for your valuable feedback. To address your concern, we added five additional baselines, including three classical methods (MatrixProfile [1], KNN, and KMeans) and two recent baselines (DualTF [2] and TimesNet [3]), as well as an additional dataset (SWaT [4]). The preliminary five-run results are shown in the table below (Italicized entries indicate newly added experimental results). Due to the time constraint, we commit that all experiments will be conducted in full, and be included in our final revision.

W3. Lack of friendly explanation for Figure 7.

A2. Thank you for your comments, and we apologize for the unclear explanation of Figure 7. We have provided a more reader-friendly explanation below: In Figure 7, red (positive SHAP values) indicates a push towards anomaly detection, while blue (negative SHAP values) indicates a shift towards normal behavior. The magnitude of each value reflects the strength of each cause’s contribution to each sample. According to the results, causes like $c_1$ consistently demonstrate a strong positive influence on anomaly detection, suggesting that their representations are closely linked to anomaly-indicating patterns (e.g., "hardware failure" in a cloud service context). Conversely, causes like $c_{7}$ ​ tend to shift predictions toward normal behavior, indicating that these causes are more reflective of regular instances (e.g., "user misperception"). Figures 7a and 7c illustrate the contributions of a specific sample, while Figure 7b provides an overview across all samples. We have included this friendly version's explanation in our revised paper (Lines 460-500). Thank you again for your valuable suggestion, which has greatly helped to improve the clarity of our paper!

Thank you for addressing the concerns I had with this paper. I recommend you revise the paper by adding all the experiments you said during the rebuttal period. My concerns have been addressed through rebuttal and I raise my score.

Q3. Lack of comparison with the latest methods.

A3. Thank you for your valuable feedback. To address your concern, we plan to add three additional baselines, including two recent baselines (DualTF [7] and TimesNet [8]) and one classical method (MatrixProfile [9]). Due to time constraints, we commit that all experiments will be conducted in full, and be included in our final revision.

We greatly value your feedback and have included the necessary revisions in our updated version. We hope our responses clarify your concerns. As these do not pertain to fundamental technical issues, we kindly request you to reconsider our score. Thank you for your time and thoughtful review!

Reference

[1] Graph-Augmented Normalizing Flows for Anomaly Detection of Multiple Time Series. ICLR. 2022.

[2] Detecting Multivariate Time Series Anomalies with Zero Known Label. AAAI. 2023.

[3] Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy. ICLR. 2022.

[4] Breaking the Time-Frequency Granularity Discrepancy in Time-Series Anomaly Detection. WWW. 2024.

[5] CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecasting. ICLR. 2022.

[6] Towards Out-of-Distribution Sequential Event Prediction: A Causal Treatment. NeurIPS. 2022.

[7] Breaking the Time-Frequency Granularity Discrepancy in Time-Series Anomaly Detection. WWW 2024.

[8] TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis. ICLR 2023.

[9] Matrix profile I: all pairs similarity joins for time series: a unifying view that includes motifs, discords and shapelets. ICDM. 2016.

We appreciate your thoughtful feedback and your recognition of the wide range of experiments and the novelty of our approach. We address your concern point by point as follows.

[Weaknesses]

Please see the answers to the Questions section.

[Questions]

Q1. The ablation experiments on the other datasets?

A1. Thank you for your suggestion. Following your suggestion, we have conducted ablation studies on two additional datasets to further evaluate the generalizability of our approach. The results, presented in the tables below, show that removing any single component leads to noticeable performance drops, ranging from 3.46% to 4.1% on Cloud-B and 3.59% to 3.98% on PSM. In contrast, the full CaPulse model consistently achieves the highest performance. These findings underscore the critical contribution of each core component to the overall effectiveness of the model. Additionally, this analysis has been included in Appendix F.1 in our revision for further reference.

Q2. Causal inference and frequency domain-based methods have already been proposed before. Concern about novelty.

A2. Causal inference and frequency domain-based methods are broad fields encompassing a wide range of tasks and approaches. As discussed in Section 2.2, we have already reviewed relevant works and highlighted the gaps our approach aims to address. To further clarify, we have provided a comprehensive comparison with related work in the table below.

Based on this, we would like to restate our contribution as follows: We propose CaPulse, a causal framework for time series anomaly detection that identifies anomalies by capturing disruptions in underlying generative processes and multi-period dynamics, providing both high accuracy and interpretability.

We have also added an extended discussion of the comparison in Appendix G.3 of our revised submission, which we hope will address your concerns. If there are specific related works you believe we may have overlooked, please let us know, and we will be happy to provide further clarification.

We sincerely appreciate your thoughtful comments. Below, we address the concerns you have raised in detail.

[Weakness]

W1. The objective function’s detail is unclear.

A1. Regarding the objective function, we believe there may be a misunderstanding, and we would like to clarify the following points:

1. The objective function is conditioned solely on $C_{ind}$ and not on $C_{o}$ (an intermediate result), as outlined in Eq. 3 and 5.
2. We do not explicitly select $C_{ind}$; rather, it is learned automatically through the PaCM and MpCF modules, as illustrated in Figure 3 of our framework. This learning process is also detailed in Section 4.2 and Appendix B.1.
3. Our approach is not a two-stage optimization strategy. As stated in Section 4.4, the total loss is defined as: $\mathcal{L} = \mathcal{L}{\text{nf}}+\alpha \mathcal{L}{\text{sim}} + \beta\mathcal{L}{\text{ind}}$, indicating the model is trained in an end-to-end manner.

Thank you for your comments, we have revised the manuscript to ensure that these details are more easily accessible and explicitly highlight the parameters to be learned, as suggested (Line 328-329).

W2. Some algorithms’ operations and rationales behind the design choice are unclear.

A2. We appreciate your question and make the following clarification:

1. Use of FFT: Thank you for raising this point. As described in Lines 233–235, we use FFT to perform causal intervention by adding noise to the less significant parts of the data. FFT is a widely used method for separating high- and low-frequency components, enabling targeted intervention on high-frequency noise. We have clarified this in our revision (Lines 242–243).
2. Orthogonality and causality: We employ orthogonality to satisfy the Independent Causal Mechanisms assumption, which requires none of the factorizations of C to entail information from others, thus it enforces the mutual independence of the causal factors. This is described in Lines 186 - 189.
3. Pyramid structure: The pyramid structure is specifically designed to capture multi-period information, as highlighted in the introduction (Lines 90–91).
4. Noise injection approach: As described in Lines 234-239, based on insights from prior literature [1,2], non-causal elements frequently appear randomly, similar to noise typically found in the high-frequency components of time series data. Therefore, we perform causal intervention by adding noise to the high-frequency (less significant) components of the input data to simulate these non-causal disturbances while preserving the core causal structure. We acknowledge that real-world scenarios can be more complex, thus we have conducted additional experiments of different augmentation methods and combinations on two datasets (PSM and SMD), as shown in the following table.

HighFreq refers to adding noise to the high-frequency components, which was our initial approach. LowFreq denotes adding noise to the low-frequency components, and Shift indicates shifting the input time series by 20 time steps. The "+" symbol represents a combination of different methods. According to the results, adding noise to high-frequency components yields the best performance, with LowFreq also performing well but slightly below HighFreq. Shifting the time series has a lesser impact, and combining multiple augmentation methods does not improve performance beyond HighFreq alone, suggesting that excessive variability may obscure meaningful causal patterns. We have included this discussion in Appendix G.2.

We sincerely appreciate your thoughtful comments and hope our responses have adequately addressed your concerns. If there is anything we haven’t fully explained, please don’t hesitate to let us know—we are more than happy to provide further clarification. As these points are not fundamentally technical issues, we kindly ask you to reconsider our score. Thank you very much for your time and consideration!

Reference

[1] ​​Robusttad: Robust time series anomaly detection via decomposition and convolutional neural networks. 2021.

[2] Through the dual-prism: A spectral perspective on graph data augmentation for graph classification. 2024.

[Questions]

Q1. How does the computational complexity scale with time series length and dimensionality?

A1. Thanks for your question. The information you requested has already been included in Appendix B.3 of our original paper. Specifically, the theoretical complexity of our approach is $\mathcal{O}(T \log T) + \mathcal{O}(N^2 D_h)$. The first term, $\mathcal{O}(T \log T)$, results from the FFT used to extract local periods. The second term, $\mathcal{O}(N^2 D_h)$, corresponds to the attention mechanism in the MpCF module. Notably, the PeNFs are linear, contributing minimal additional complexity.

Q2&Q4. What are the computational requirements and the comparison with parameter-efficient baselines?

A2. Please see the answer for weakness.

Q3. What is the optimal hyperparameter configuration?

A3. Thank you for your questions, and we apologize for missing the details of the implementation. For each method, we employed publicly available implementations and configured hyperparameters for fair comparisons. The hidden dimension was uniformly set to 64 for most methods, with two layers in their network architecture, unless otherwise specified, e.g., adversarial methods like DROCC and ALOCC used additional parameters such as gamma (set to 2) and lambda (set to 0.0001) in DROCC, while USAD included specific configurations for $\alpha$ and $\beta$, both set to 0.5. Transformer-based approaches like AnomalyTransformer had a window size of 60, 8 attention heads, and a feedforward network dimension of 512. We have accordingly included these details in our revision of Appendix D. We hope this addresses your concerns, and thank you again for pointing this out.

Thank you for your valuable comments. We have addressed them in our revised submission and hope our responses resolve your concerns. Since these are not critical technical flaws, we sincerely ask for a reassessment of our score. We truly appreciate your review and consideration!

Thank you for your insightful comments and for recognizing the clarity and experimental design of our approach. We address each of your concerns below.

[Weaknesses] The proposed method appears computationally intensive with many hyperparameters, and the empirical comparison focuses only on similarly complex baselines, neglecting simpler algorithmic methods.

Thank you for raising your concern regarding the computational complexity of our approach. We address your points in detail as follows:

A1. Theoretical computational complex: As discussed in Appendix B.3, the theoretical complexity of our approach is $\mathcal{O}(T \log T) + \mathcal{O}(N^2 D_h)$. The first term, $\mathcal{O}(T \log T)$, results from the FFT used to extract global and local periods, with the global period calculated as a preprocessing step. The second term, $\mathcal{O}(N^2 D_h)$, corresponds to the attention mechanism in the MpCF module. Notably, the PeNFs are linear, contributing minimal additional complexity.

A2. Efficiency comparison: Theoretically, the complexity of Matrix Profile (MP) is $\mathcal{O}(T_l^2 \log T_l)$ [1], where $T_l$ represents the total length of the time series (typically, $T_l \gg T$). Thus MP’s theoretical complexity is higher than that of our approach. To further address your concern, we conducted experiments on four datasets and measured the time costs. However, we think that a direct efficiency comparison may be unfair for several reasons:

1. Methods like MP can only be run on the CPU, while DL methods such as CaPulse can leverage GPU acceleration.
2. MP operates directly on the test data, which is smaller (about one-third of the training set size), whereas CaPulse is trained on the full training set.
3. Training epochs vary across datasets and can be adjusted, making the total training time flexible.

Thus, to provide additional context, we also included a comparison with a recent DL-based method, DualTF [2].

Based on the table, we observe that CaPulse achieves significantly lower time costs per epoch and consistently outperforms Matrix Profile and DualTF in ROC, demonstrating both efficiency and effectiveness.

A3. Performance comparison with classical methods: Thank you for your suggestion to include more classical baselines. Following your advice, we have added three additional baselines — MP, KNN, and K-means — to our evaluation, as shown below. The results indicate that CaPulse consistently achieves superior ROC scores compared to classical methods, reinforcing its robustness and accuracy in detecting anomalies across diverse datasets.

Thank you for your valuable comments. We have included this discussion and experimental results in Appendix F.2 of our revised paper.

[1] Matrix profile I: all pairs similarity joins for time series: a unifying view that includes motifs, discords and shapelets. ICDM. 2016.

[2] Breaking the Time-Frequency Granularity Discrepancy in Time-Series Anomaly Detection. WWW. 2024.

Dear Reviewers,

We would like to inform you that we have just completed all experiments and have further updated our revised manuscript. Specifically, we have included results from five additional baselines in Table 1, as per your suggestion: three traditional methods (MP, KNN, and KMeans) and two recent approaches (DualTF and TimesNet), evaluated across all datasets.

Additionally, we would like to kindly remind you that the discussion phase is ending soon on Nov. 26th. We sincerely hope reviewers who have not yet had the opportunity to review our initial response can take a look at our response and the revised manuscript and ensure that all concerns have been addressed.

We greatly appreciate your constructive feedback and remain available to provide further clarifications or make additional refinements if needed. Thank you for your time and effort in reviewing our work!

Best regards,

Authors

Dear Prof. Eamonn Keogh,

Thank you for your valuable comments. We really appreciate the contributions of Matrix Profile (MP) in time series data mining, and would like to provide the following clarification:

• Computational Complexity: As suggested by Reviewer q2Qb, we have included the comparison with the MP method [1] in our revision. The theoretical computational complexity of MP we stated in our response is referred from [1], as stated in the first line of Section III.C in the original paper: “The overall complexity of the proposed algorithm is $\mathcal{O}(n^{2} log n)$, where n is the length of the time series”. In contrast, the complexity of our method is $\mathcal{O}(T \log T) + \mathcal{O}(N^2 D_h)$. Based on this distinction, we made the claim regarding complexity in our work. Thanks to your comments, we will carefully review the new literature [2] you provided to explore potential improvements in the implementation of MP.

• Our Scope and Focus: Recently, deep learning methods have achieved superior performance in TSAD, compared to conventional approaches (e.g., KNN) [3-4]. Following this trend, our work is centered on Deep Learning (DL) for TSAD with an emphasis on accuracy, which guided our selection of baselines [5–12]. As such, MP-based methods fall outside the primary scope of our paper, as well as the papers in the DL-based TSAD literature. To ensure fairness, we have considered comparisons of both efficiency and effectiveness with DL-based approaches. In our response to Rq2Qb’s W2, we explained why we believe a direct comparison between DL-based methods and MP-like approaches is not ideal. However, to address the concern raised, we still have included this comparison in our work.

Thank you once again for your comments.

Best regards,

Authors

Reference:

[1] Matrix profile I: all pairs similarity joins for time series: a unifying view that includes motifs, discords and shapelets. ICDM. 2016.

[2] Matrix Profile XXIV:Scaling Time Series Anomaly Detection to Trillions of Datapoints and Ultra-fast Arriving Data Streams. Yue Lu, Renjie Wu, Abdullah Mueen, Maria A. Zuluaga and Eamonn Keogh. ACM SIGKDD 2022

[3] Deep Learning for Time Series Anomaly Detection: A Survey. 2024.

[4] A Comparative Analysis of Traditional and Deep Learning-based Anomaly Detection Methods for Streaming Data. 2019.

[5] Breaking the Time-Frequency Granularity Discrepancy in Time-Series Anomaly Detection. WWW 2024.

[6] TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis. ICLR 2023.

[7] Graph-augmented normalizing flows for anomaly detection of multiple time series. ICLR. 2022.

[8] Detecting multivariate time series anomalies with zero known label. AAAI. 2023.

[7] Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy. ICLR. 2022.

[9] USAD : UnSupervised Anomaly Detection on Multivariate Time. KDD. 2020

[10] Deep autoencoding gaussian mixture model for unsupervised anomaly detection. ICLR. 2018.

[11] Deep semi-supervised anomaly detection. ICLR. 2020.

[12] Towards a Rigorous Evaluation of Time-series Anomaly Detection. AAAI. 2022