ScatterAD: Temporal-Topological Scattering Mechanism for Time Series Anomaly Detection

We sincerely thank Reviewer jjez for the detailed comments and insightful questions.

Q1: Analysis of the universality of the scattering phenomenon.

To address your concerns about the universality of the scattering phenomenon, we present the quantitative results on the scattering phenomena for all six benchmark datasets used in our study. Furthermore, to evaluate the model's robustness, we inject additive Gaussian noise of three different intensities (σ =0, 0.5, 1.0, 2.0) into the normalized test data (with σ=0.0 as the no-noise baseline). We quantify the impact of noise on the model's internal discriminative power by calculating the average scattering score for normal and anomalous samples, which we define as the mean Euclidean distance of samples to their respective class center, and analyzing their separation ratio.

The results in the table above clearly reveal a consistent anomaly scattering pattern across six different real-world datasets. In addition to this quantitative evidence, we would like to draw the reviewer's attention to Figure 3 in our paper, which qualitatively supports the consistency of the scattering behavior from another perspective. Furthermore, we wish to highlight the case of the WADI dataset, where under medium-to-high noise, the scattering score of normal samples surpasses that of anomalies. We attribute this phenomenon primarily to WADI's high dimensionality (123 dimensions), where strong noise may have a compounding effect and lead to complex changes in inter-variable correlations. This also indicates that while ScatterAD is robust in various noisy environments, its performance can be sensitive to strong noise when dealing with extremely high-dimensional datasets with subtle anomaly patterns. Thank you again for this insightful question, which helps us to more comprehensively characterize the performance boundaries of our method.

Q2: Clarification on the reproducibility of experimental results.

Thank you for this question, which is crucial for ensuring the rigor and reproducibility of our work. The experimental results reported in our paper, including the baseline evaluations, are based on the average of three independent runs. We will add standard deviations for all core experimental results in the revised version to more comprehensively demonstrate the stability and statistical reliability of our findings.

Q3: Regarding the random initialization strategy for the scattering center $c$.

We sincerely thank you for your question regarding the initialization strategy for the global scattering center $c$. This insightful question is also raised by Reviewer WWNK, and we kindly invite you to refer to our detailed response in Section W1. In brief, our experiments show that the random initialization of $c$ is a simple yet effective strategy. For a detailed analysis, including stability tests, we kindly direct you to the Section W1.

We greatly appreciate Reviewer WWNK for the comprehensive and insightful comments.

W1: Regarding the random initialization strategy for the scattering center $c$.

Your concerns about training stability and suboptimal solutions are entirely valid, and these are key considerations in our model design.

First, we wish to clarify the core role of $c$. In our framework, $c$ is initialized only once at the beginning of each training run and remains fixed throughout the training process. It is not a parameter learned via backpropagation. Its purpose is twofold:

• (1) Provide a fixed anchor point: $c$ offers a convergent target direction for the representations of normal samples, enabling the model to learn to map these representations into a compact region on the hypersphere.
• (2) Break symmetry and regularize: If we fix $c$ to a constant (e.g., [1, 0, ..., 0]), it might introduce an undesirable bias, inducing the model to learn a trivial solution where all normal samples map to a specific axis. Randomly initializing $c$ avoids this risk, compelling the model to learn more generalizable representations. This can be viewed as a form of implicit stochastic regularization, preventing the model from overfitting to a pre-defined direction in the latent space.

To empirically address your concerns about instability and suboptimal solutions, we conduct a new experiment: we repeat our full training and evaluation process with 5 different random seeds on the MSL and PSM datasets. A different random seed results in a different scattering center $c$. We report the mean and standard deviation of the Aff-F1 and A-ROC metrics.

Furthermore, we design several variants to compare the performance of different center initialization strategies, including:

• (1) Zero: Initialize the center at the origin (zero vector) to test the simplest symmetric starting point.
• (2) Fixed-Radius: Initialize the center on a hypersphere with a fixed radius to test the model's sensitivity to the initial magnitude.
• (3) Multi-center: Use multiple independent scattering centers to test the model's ability to capture potentially multi-modal normal data.

In the table below, Num Centers denotes the number of scattering centers used, and Radius specifies the initial magnitude for the fixed_radius strategy. We compare these strategies with the Random-in-Ball (randomly selecting a point within the unit hypersphere) strategy proposed in our paper.

The results confirm that for most datasets, ScatterAD is not sensitive to the specific initialisation strategy of $c$, and the random_in_ball strategy proposed in our paper is a simple and effective choice.

W2: Discussion on the effectiveness of cosine similarity in high-dimensional representation space.

Your point that distances between random vectors in high-dimensional space tend to converge is entirely valid for unprocessed or randomly distributed high-dimensional data. However, the core of our method is to avoid processing such data directly. Instead, the basic task of ScatterAD is to learn a nonlinear mapping from the original high-dimensional space to another well-structured high-dimensional representation space. In this learned space, the distribution of vectors is not random, specifically:

• (1) $L_{scatter}$: Actively pulls the representations of normal samples towards the same fixed center $c$, forming a compact cluster locally.
• (2) $L_{time}$: Enforces temporal continuity in the representation space by minimising the distance between representations of adjacent timesteps.
• (3) $L_{contrast}$: Ensures that the representations from different views ($online$ and $target$) remain structurally consistent, further reinforcing the geometry of the representation space.

Therefore, we are not relying on the properties of cosine similarity in a random high-dimensional space; rather, we leverage our model to create a representation space that possesses both spatial structure and temporal continuity. Furthermore, we provide a comparison of our anomaly scoring strategy with two variants based on classic distance metrics:

The results show that our simple inductive bias based on Euclidean distance is more effective than methods that attempt to fit complex correlations (Mahalanobis distance) or probability distributions (KL Divergence). This also serves as evidence that our model successfully learns a well-structured representation space.

We sincerely thank Reviewer 9v8x for the insightful comments and valuable suggestions. We will address your concerns one by one below and will integrate these clarifications into the final version of our paper.

W1: Ambiguity of the notation $h_i$.

Thank you for pointing out this ambiguity. We will use $z_i = h_t^t + h_i^{(l)}$ to denote the final representation and will update this notation consistently throughout the rest of the revised manuscript.

W2: Lack of specific model architecture details.

To clarify, the online encoder $f_\theta(·)$ and the target encoder $f_\phi(·)$ share the exact same backbone architecture, which consists of a causal convolution module (Eq. 1 in our paper) followed sequentially by a Graph Attention Network (GAT) layer (Eqs. 2 and 3 in our paper). However, the outputs of these two encoders undergo asymmetric processing steps before being used in the final loss computation, as detailed below:

This asymmetric design is a well-established and highly effective strategy in momentum contrastive learning frameworks (Kaiming et al., 2020; Byeongchan et al., 2023). It prevents the model from collapsing by breaking the symmetry through an additional learning task (predicting the target representation). We will update the methods section in the final version of our paper with a detailed process description and diagram to ensure the clarity and full reproducibility of our work.

W3: Unexplained parameters.

As per your suggestion, we will define all parameters upon their first appearance in the revised manuscript and include their default values. We provide the clarifications here:

• (1) $\tau$ in Section 3.2: This is the look-back window size of the temporal graph, which defines neighbourhood relations as $(u_{t-k}, u_t)$, where $k \in [1, \tau]$. In our experiments, we use $\tau=2$.
• (2) $l$, $k$ in Equation (1): $l$ represents the layer index of the causal convolution, and $k$ is the kernel size. We use two two-layer ($l=2$) with a kernel size of $k=3$.
• (3) $\xi$ in Equation (8): This represents the parameters of the target encoder $f_\phi(·)$. We use $\xi$ to maintain consistency with the momentum update literature, where $\theta$ is often used for the online encoder. For clarity, we will unify the notation to use $\phi$.

W4: Ambiguity of feature aggregation.

It is important to note that the aggregation operation $h_i = h_t^t + h_i^{(l)}$ is a simple element-wise addition. To be precise:

• (1) $h_t^t$ ("temporal features") is the output of the initial causal convolution encoder (from Eq. 1).
• (2) $h_i^{(l)}$ ("topological features") are the attention-weighted features learned from neighbouring nodes via the GAT layer (from Eq. 3).
• (3) $h_i$ ("node features") is the final representation obtained by the element-wise addition of these two components.

We will revise the relevant text in Section 3.2 in the revised manuscript, following your suggestion, to explicitly state that this is an element-wise addition and to clarify the origin of each term. Furthermore, as per your suggestion in W1, we will resolve the ambiguity caused by the notation $h_i$.

---

W5: Relationship between the formulas for $c$.

Thank you for requesting clarification on this. The two parts of this formula describe a two-step sequential process for initializing the scattering center $c$.

• (1) Step 1: $c' = \text{Randn}(\cdot)$: First, a vector $c'$ is sampled from a standard normal distribution.
• (2) Step 2: $c = \frac{c'}{\||c'\||} \cdot \epsilon$: The sampled vector $c'$ is then L2-normalized, projecting it onto the unit hypersphere, and subsequently multiplied by a random scalar $\epsilon \in [0, 1)$. This ensures that $c$ is strictly located inside the unit ball.

W6: Comparison with classic baselines.

Following your suggestion, we provide extensive comparisons with classic baselines such as ModernTCN, iTransformer, and DCDetector. The results for all baseline models are reproduced by strictly following the configurations and guidelines from their official open-source codebases. Due to space constraints, we present four key scores here; the complete results will be updated in the revised manuscript.

---

Q1: Encoder architecture specifications and the origin of $h_{online}$, $h_{target}$.

We provide detailed specifications in our response to W2. We will also explicitly state these details in Section 3 of the revised manuscript to improve clarity and reproducibility.

Q2: Definitions and values of key parameters.

Please see our response to W3. We will ensure that all parameters are clearly defined upon their first appearance and will add a table of key hyperparameters in the appendix of the revised manuscript.

Q3: Connection between the theoretical result $I(Z_T; Z_G | G)$ and the practical implementation.

Thank you for this crucial question regarding the connection between our mutual information theory and its practical implementation. The link is as follows:

(1) Theoretical Goal: Our theory (Appendix A) shows that maximizing the conditional mutual information $I(Z_T; Z_G | G)$ is a principled way to learn complementary representations from the temporal ($Z_T$) and topological ($Z_G$) views, given the graph $G$.

(2) Practical Implementation: The contrastive fusion loss $L_{contrast}$ (Eq. 7) serves as a tractable, differentiable lower bound for this mutual information. By minimising $L_{contrast}$, we effectively maximise this lower bound, thus driving the model to achieve the theoretical objective. In our implementation, the graph structure $G$ is the temporal graph described in Section 3.2, where nodes are time points at timestep $t$, and edges connect temporally adjacent nodes (e.g., $(t-1, t)$). This graph $G$ constrains the entire process, defining the neighbourhood for the GAT's aggregation operation and providing positive pairs for the contrastive loss.

References

He, Kaiming, et al. "Momentum contrast for unsupervised visual representation learning." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020.

Lee, Byeongchan, and Sehyun Lee. "Implicit contrastive representation learning with guided stop-gradient." Advances in Neural Information Processing Systems 36 (2023): 30885-30897.

We sincerely thank Reviewer xfeK for the detailed and constructive comments, which are instrumental in further improving our paper.

W: Regarding hyperparameter sensitivity, theoretical assumptions, and their impact on generalization.

To systematically evaluate the robustness of our model, we conduct a detailed sensitivity analysis in Appendix E of our paper on key hyperparameters, including the time window size, number of encoder layers, hidden dimension, EMA decay rate, and the anomaly threshold $δ$. The results show that ScatterAD maintains stable performance across a wide range of parameter settings. And, we detail the impact of changes in graph structure in Q1.

Q1: Regarding the performance of a static graph when the topology changes.

To simulate dynamically changing graph topologies, we extend our data loading process to generate different graph structures for each input time window during training and inference. We design two dynamic topology generation strategies:

(1) Random Topology: For each time window, we randomly establish connections between nodes with a certain probability (edge_prob=0.3). To ensure basic graph connectivity, we preserve the connections between adjacent nodes. This strategy simulates scenarios where connections are irregular and appear randomly.

(2) K-Nearest Neighbours (KNN) Topology: For the topology constructed within each time window, we compute the Euclidean distance between each node and all other nodes, connecting it to its k-nearest neighbours ($k=3$).

The results indicate that while dynamic topology strategies lead to marginal performance improvements in some cases (e.g., Random Topology on MSL and KNN Topology on PSM), they introduce additional computational overhead. Therefore, ScatterAD exhibits strong robustness to graph topology, and employing a static graph strikes a good balance between efficiency and effectiveness.

Q2: Clarification on the choice of Euclidean distance and its sensitivity to feature scaling.

Our goal is to find a simple yet effective inductive bias. Compared to complex methods that require covariance estimation and rely on distributional assumptions, Euclidean distance, being a computationally simple and assumption-free geometric metric, is a more direct and robust choice to achieve our objective. To address the potential issue of feature scaling, we first apply standard Z-score normalisation to all input data. Second, we perform L2 normalisation on the target representation $h_{target}$ before calculating the scatter loss ($L_{scatter}$) (Equation 5), ensuring robustness to feature scales.

To address the concerns regarding different distance metrics, we conduct a new experiment comparing our method with two other classic distance metric variants: Mahalanobis distance and Kullback-Leibler (KL) Divergence. In the Mahalanobis distance variant, we estimate the global covariance matrix from the representations of the training set, following standard practice. We report the performance on three representative datasets below.

Q3: In-depth analysis of performance discrepancy on the NIPS-TS-SWAN dataset.

First, we would like to clarify that ScatterAD successfully generates separable anomaly scores for the NIPS-TS-SWAN dataset. The high PA-F1 score (0.736) demonstrates that anomalous events are assigned higher scores than normal ones. Furthermore, to objectively analyse the characteristics of the Aff-F1 and Point-wise F1 metrics, we design a new experiment to simulate the metric sensitivity, which is independent of our model and aims to test the robustness of the Aff-F1 metric itself on the NIPS-TS-SWAN dataset. We assume a model whose predictions can perfectly match the ground truth. We then simulate a minor, realistic localisation error by shifting the entire prediction sequence by $n$ timesteps and observe how the Aff-F1 and Point-wise F1 scores change.

The results show that for the NIPS-TS-SWAN dataset, a localisation shift of just one timestep can cause the Aff-F1 score of the model to catastrophically collapse from 1.0 to 0.065. This explains why many other SOTA models (as shown in Table 1 of our paper) also perform poorly on this metric. In contrast, the more traditional Point-wise F1 score shows only a modest decline. This is why we report both scores.

To further validate our model's detection capabilities at a macro level, we compiled overall prediction statistics for ScatterAD on the NIPS-TS-SWAN dataset.

The statistics show that the total proportion of anomalies predicted by ScatterAD (32.1%) closely aligns with the ground truth (32.6%), demonstrating its accurate detection capability. However, the higher number of predicted segments suggests that the model may tend to respond to a single continuous anomaly as several smaller segments. Consequently, its performance is underestimated by the Aff-F1 metric under these specific data conditions due to such minor, unavoidable localisation offsets.