Towards a Unified Framework of Clustering-based Anomaly Detection

It is unclear why the student’s t-distribution should work fundamentally better than a Gaussian mixture model.

The choice of the Student’s t-distribution over a Gaussian mixture model (GMM) is motivated by its heavy-tailed nature, which enhances robustness to outliers—a critical factor in anomaly detection. Unlike GMMs, where Gaussian tails decay rapidly and outliers can skew parameter estimates, the Student’s t-distribution mitigates this by assigning lower influence to extreme values. Our ablation study’s empirical evidence (Table 3) shows that performance significantly decreases when switching to a GMM.

The explanation of why vector information fundamentally improves the anomaly score is vague and should be elaborated better. ...

Thank you for your feedback. Due to space limitations, we provided a detailed explanation of the benefits of our scoring design in Appendix C and illustrated its differences from traditional clustering scores in Figure 4. We will emphasize these advantages more clearly in the next version to better highlight the rationale behind our approach.

Since the authors test their methods on tabular data only, they should compare to more recent works on this problem. ...

Thank you for introducing these excellent research works. Following your suggestion, we have properly cited these articles in our revised manuscript. Additionally, we have incorporated NeuTraLAD and DROCC as comparison baselines and reported the results in the following anonymous link: https://anonymous.4open.science/r/ICML2025_9292-4E31.

The method seems very overengineered, and many parts of the pipeline lack a clear motivation or intuition other than working better on the test set in the experiments.

Thank you for your input. We hope to clarify your misunderstanding regarding the design of our method. Far from lacking clear motivation, every module in our pipeline is purposefully designed, rooted in a unified optimization objective and theoretical framework. Our approach’s strength is validated through rigorous, fair comparisons across 30 diverse datasets spanning multiple domains, where it consistently achieves superior average performance, highlighting its robustness and broad applicability.

Other Comments Or Suggestions

Thank you for your detailed feedback. We will improve the clarity of Figure 2’s caption, define "MM" in Section 3.1.1, fix the broken reference in Section 4.3, and add an impact statement in the revision.

Could the authors provide a better intuition or motivation for adding directional information to the anomaly score? ...

Directional information enhances anomaly scoring by modeling samples as influenced by multiple cluster “forces,” identifying anomalies with weak or conflicting resultant forces. It may hurt in datasets with irregular or overlapping clusters, where directionality could misclassify points. For normal samples within clusters, since the "force" exerted by the center of their own cluster will be much greater than that of other clusters, the direction information of other cluster centers will be avoided from influencing the assessment of the abnormality degree of the samples.

Is there a particular reason why the authors learn representations using an autoencoder? ...

Thank you for your comment. Autoencoders are widely used as a foundational framework for anomaly detection due to their ability to learn low-dimensional representations via reconstruction loss. We appreciate your suggestion and agree that exploring alternative representation learning objectives holds potential for future work. However, the focus of our approach is on anomaly-aware maximum likelihood optimization, which better guides representation learning. Our ablation study (Table 3) demonstrates that performance significantly drops when only the autoencoder objective is used.

How did authors tune hyperparameters?

We used fixed hyperparameters ($K=10$, $l=1\%$) across all datasets for a fair comparison with baselines, which used default settings. These were effective on average, but tuning could improve results, as shown in the sensitivity analysis in Section 4.4.

Is there a clear intuition as to why student’s t-distribution mixture models work better than Gaussian mixture models?

Yes, the t-distribution’s heavy tails (Equation 3) make it robust to outliers, unlike GMMs’ exponential tails. This enhances clustering and scoring in anomaly detection, as shown in Table 3 (AUC-ROC drops with GMM).

Under which conditions does it make sense to view anomaly detection from a clustering perspective? ...

Thank you for your question. A clustering perspective is advantageous for anomaly detection when the data has natural groupings or multiple modes, allowing anomalies to be identified as deviations from these structures. However, in cases where clusters are absent or irrelevant to anomalies, density-based or distance-based methods may be more effective.

However, the paper does not elaborate on the specific scenarios in real datasets that this method can address. Although this scoring method theoretically holds advantages, further clarification is needed regarding its applicable scope.

We thank the reviewer for highlighting the need for more detail on specific application scenarios. We acknowledge that the original manuscript lacked sufficient elaboration on real-world use cases. To address this, we will revise the paper by adding a new subsection that discusses the method’s applications in diverse anomaly detection tasks. Specifically, we will include examples such as financial fraud detection (e.g., identifying irregular transaction patterns), cybersecurity intrusion detection (e.g., detecting anomalous network traffic), and medical diagnostics (e.g., flagging unusual patient records). These examples will be supported by experimental results to demonstrate the method’s performance advantages in these domains.

On the experimental specifics of the gravity-inspired scoring function: The paper states this function can more comprehensively measure sample-to-cluster relationships but does not clearly define its scope of application. What types of anomalies does this method excel at detecting? Can the authors provide concrete examples of its improved performance on different anomalies?

We appreciate the reviewer’s request for clarification on the gravity-inspired scoring function’s scope. This method excels at detecting anomalies in datasets with complex, multi-cluster structures, particularly where anomalies lie at cluster boundaries or exhibit ambiguous cluster affiliations. We will add synthetic and real-data experiments showcasing improved detection of these anomaly types compared to traditional methods.

On verifying the independent contribution of the gravity-inspired scoring function: The paper claims this function significantly boosts overall performance, yet lacks in-depth validation of its individual contribution. Could ablation studies be conducted to analyze its impact? For instance, comparing model performance with and without this function under identical experimental conditions would clarify its specific effects.

Thank you for your valuable feedback. We would like to clarify that in Table 1, we have already compared the two anomaly scoring methods and treated them as two separate variants of the model. We apologize for not clearly indicating this distinction in the ablation section. We will revise the paper to highlight this and improve the explanation, ensuring that the independent contribution of the gravity-inspired scoring function is more clearly addressed.

The paper lacks comparisons with the latest baseline methods. Among the 17 baseline methods included, only 3 are from after 2020, and there are no methods from 2024.

Thank you for noting the need for updated baselines. Given the rapid advancements in unsupervised anomaly detection, we have added recent methods, including NeuTraLAD (2024) and DROCC (2020), to our revised experiments. These methods are evaluated on the same datasets and metrics as UniCAD to ensure a fair and comprehensive comparison. The results are available at the following anonymous link: https://anonymous.4open.science/r/ICML2025_9292-4E31

Although the paper proposes an efficient iterative optimization strategy, the computational complexity of UniCAD remains high, especially for large-scale datasets. The iterative nature of the EM algorithm and the processing of high-dimensional data may lead to long training times, limiting its applicability in real-time scenarios.

We appreciate this observation. The computational complexity of UniCAD, detailed in Section 3.4 as $\mathcal{O}(tN(\log N + Td(D + K)))$, is manageable with reasonable settings for iteration count $t$ and cluster number $K$. Table 2 demonstrates that UniCAD’s training and inference times are competitive with deep learning methods like DAGMM and DCOD. To address concerns about large-scale and real-time scenarios, we will provide additional runtime results on larger datasets and explore optimization techniques (e.g., parallelization or approximation) to further reduce training time, enhancing its practical utility.

Two minor issues. In 4.3, 'with additional experiments on other data domains presented in Appendix ??' In 3. Methodology, 'Finally, we present an efficient iterative optimization strategy to optimize this model and provide a complexity analysis for the proposed model.' However, I fail to find complexity analysis in the text.

Thank you for pointing out these issues. For the appendix reference in Section 4.3, we will ensure the additional experiments are fully detailed in the appendix and correct the placeholder “??”. Regarding the complexity analysis, it is provided in Appendix D.4; we will revise Section 3 to explicitly reference this section for clarity.

Please explain the difference between the proposed method and the typical learning of Gaussian Mixture Models (GMMs) with EM?

UniCAD distinguishes itself from traditional GMM-EM through several key innovations. Unlike GMM-EM which operates directly on raw features, UniCAD incorporates deep representation learning via neural networks to project data into a lower-dimensional space. It introduces an anomaly indicator function $δ(x_i)$ to dynamically filter out outliers during optimization, overcoming GMM-EM's assumption of normal-only samples. For scoring, UniCAD replaces conventional likelihood probabilities with a novel gravity-inspired vector-sum approach that better captures complex sample-cluster relationships. Additionally, it adopts Student's t-distribution mixture models instead of Gaussian distributions for improved outlier robustness.

Why is Anomaly Scoring with Vector Sum necessarily beneficial? Although the authors provide an explanation in Section 3.2.3, they do not discuss scenarios such as the following: If a sample point belongs to a normal cluster that happens to be centrally located in the feature distribution, could it inadvertently receive a high anomaly score?

The vector-sum anomaly scoring is beneficial because it comprehensively assesses a sample’s relationship with all clusters via directional “gravitational” forces, unlike scalar methods that consider only magnitude. For a sample in a normal cluster’s center, its strong “force” from that cluster dominates, yielding a large resultant force and thus a low anomaly score. In contrast, anomalies, weakly influenced by all clusters or pulled between them, have a smaller resultant force, increasing their score.

It is recommended to carefully reconsider the use of gravitation as an introductory concept in Section 3.2. In Section 3.2.1, the analogy between the equation of universal gravitation and the anomaly scoring formula appears somewhat forced, as their similarity is essentially limited to both being expressed as fractions. In Section 3.2.2, the connection is further reduced to merely involving vector summation. While the gravitational analogy provides an intuitive perspective, its relevance to the proposed method is not sufficiently substantiated.

We appreciate this critique and will revise Section 3.2 to de-emphasize the gravitational analogy as a direct equivalence. Instead, we will frame it as an intuitive inspiration, focusing on how vector-sum scoring models sample-cluster interactions as “forces” to capture complex relationships. We will provide detailed explanations and visualizations to substantiate its relevance, ensuring the method’s merit stands independent of the analogy.

Experimental settings are not fair Default parameters for baseline methods were used, which raises concerns. All methods have to re-run and be re-optimized for the datasets, similar to what you do for your solution

Thank you for highlighting the concern regarding the fairness of our experimental settings. In the paper, we also used the default parameters for our method, in line with the experimental setup used for the baseline methods. Specifically, we followed the ADBench benchmark [Han et al., 2022] to ensure a fair and consistent comparison. We acknowledge that tuning parameters could potentially improve performance, but as this is an unsupervised task, it is difficult to rely on techniques such as cross-validation for hyperparameter optimization. Therefore, we opted to maintain the same default settings across all methods to ensure fairness in the comparison.

The solution seems novel but several of its components are based on existing ideas. Clarifying novelty vs. only citing earlier work can help strengthen the contribution

We appreciate your feedback on the novelty of UniCAD. While components like mixture models and the Student’s t-distribution draw from existing ideas, the core contribution of UniCAD lies in its unified theoretical framework that integrates representation learning, clustering, and anomaly detection via anomaly-aware maximum likelihood estimation, alongside a novel gravity-inspired anomaly scoring mechanism. Specifically: (1) the joint optimization of these tasks with theoretical grounding, (2) the gravity-inspired vector-sum-based anomaly scoring capturing complex sample-cluster relationships, and (3) an efficient iterative optimization strategy are unique to our approach. To clarify this, we will revise the paper to explicitly distinguish between foundational concepts and our original contributions, supported by comparative experiments and theoretical analysis to underscore UniCAD’s distinctiveness.

Substantial runtime overhead

We acknowledge the concern about runtime overhead. In the paper (Table 2), we provided a runtime comparison showing that UniCAD is competitive with deep learning baselines like DAGMM and DCOD, despite the iterative EM algorithm. The efficient iterative strategy and parameter updates reduce computational overhead, with a complexity of $\mathcal{O}(tN(\log N + Td(D + K)))$, making it feasible for large-scale datasets.