Weakly Supervised Anomaly Detection via Dual-Tailed Kernel

We thank the reviewer for the valuable feedback.

Usage of Lemma 4.1 in the main paper.

Lemma 4.1 is the formal stepping-stone for showing that a single kernel cannot serve as both a strictly light-tailed and a heavy-tailed function. By demonstrating $\lim_{d\to\infty} \kappa_{\mathrm{light}}(d)\,/\,\kappa_{\mathrm{heavy}}(d) = 0$, it clarifies that any kernel “light” enough to enforce tight in-class clustering cannot simultaneously retain the broader margin necessary for out-of-class separation. This result underpins the transition to a dual-kernel scheme in Theorem5.3, where one kernel is dedicated to pulling in-class points close, and a second kernel enforces out-of-class separation. In our revised version, we plan to reference Lemma 4.1 more explicitly in the main text (e.g., just before Theorem5.3), making its critical role in motivating the dual-kernel approach more transparent.

Validation of Lemma 5.1, 5.2 and clarification of moderate distances

In Lemmas 5.1 and 5.2, “moderate distances” refers to the typical radius range where in-class samples are neither so close that their similarity is almost one nor so far that the kernel decays to zero. This region is precisely where the difference between a rapidly decaying (light-tailed) and a more gradual (heavy-tailed) function is most pronounced. Although “moderate” is inherently qualitative, Table 4 in the appendix shows that in real datasets, most points do indeed occupy this intermediate distance regime in the learned representation. By measuring average distances between each sample and its assigned (or opposing) center, one sees that in-class samples form tighter clusters under light-tailed similarity, and out-of-class samples remain substantially farther under heavy-tailed similarity—exactly matching the theoretical claims of Lemmas 5.1 and 5.2.

Proof $L_{total} < 2$

To prove this property, we further need to prove the minimum $L_{\text{total}}(\theta)$ is below 2 and $L_{\text{separation}}$ is not zero.

In the main paper, we define the total loss as

$$L_{\mathrm{total}}(\theta) = L_{\mathrm{separation}}(\theta) + L_{\mathrm{diversity}}(\theta).$$

Corollary 5.6 shows that collapsing each class onto a single point yields a diversity loss of exactly 2 while driving the separation term arbitrarily close to zero, so the total loss under this degenerate arrangement is approximately 2. To demonstrate that degeneracy does not globally minimize the total loss, one can construct a non-degenerate arrangement in which each class’s points lie near (but not identically at) its center. Placing the points close to their respective centers makes $L_{\mathrm{separation}}(\theta)$ arbitrarily small, say $\varepsilon$, while ensuring that the points are not all coincident reduces $L_{\mathrm{diversity}}(\theta)$ below 2 by some margin $\delta > 0$. Because these adjustments can be made largely independent of each other, one can tune $\delta$ to exceed $\varepsilon$, forcing

$$L_{\text{total}}(\theta) = L_{\mathrm{separation}}(\theta) + L_{\mathrm{diversity}}(\theta) \le \varepsilon + \bigl(2 - \delta\bigr) < 2.$$

Since the degenerate arrangement’s total loss is 2, whereas this non-degenerate arrangement achieves a strictly smaller value, degeneracy cannot be optimal. Consequently, $L_{{separation}}$ cannot vanish at the true minimum, and the trivial collapse solution is excluded as a global minimizer.

Explanation regarding 0 label and ensemble size.

In Section3.1, “0” denotes unlabeled samples whose true status is unknown—mostly normal but potentially anomalous. Our method uses the labeled anomalies (y=1) to drive separation, exposing hidden anomalies among the unlabeled. We will clarify in the revision that “0” does not guarantee normality. For ensemble size (1, 3, or 5 splits), more splits can marginally improve accuracy but fragment the data and increase runtime. We find five splits to be the best balance of performance and cost, and will make this rationale explicit in the revised manuscript.

Justification for single normal center.

In classical one-class anomaly detection, using a single center for normal data aligns well with margin-based and complexity-theoretic reasoning: keeping the normal region compact and described by a minimal boundary (e.g., a single enclosing ball) not only simplifies optimization but also reduces the risk of overfitting, particularly under limited anomaly labels. This approach is seen in canonical methods like DeepSVDD or DeepSAD, which assume that normal data occupy a single contiguous cluster in feature space. While multiple centers can capture diverse normal modes, each adds hyperparameters. Empirically, a single center often suffices, as normal instances instances typically share enough common structure to cluster tightly around one latent coordinate.

W1

We will improve the structure of our manuscript in the revision, addressing section formatting and organization.

We thank the reviewer for the helpful feedback.

Q1

From a theoretical standpoint, WSAD-DT applies margin-based reasoning to handle both partial contamination in the unlabeled set and extremely limited anomaly labels. Assigning each class its own center and using a dual-tailed kernel—light-tailed for in-class compactness, heavy-tailed for out-of-class margins—maintains separation even when anomalies are scarce or intermixed with unlabeled data. The heavy-tailed component preserves a “push” at moderate distances, which is crucial under weak supervision where a handful of labeled anomalies must guide the entire boundary. Meanwhile, the diversity term prevents degenerate “collapse” solutions, ensuring normal and anomalous samples maintain sufficient variability. The ensemble mechanism gives each ensemble model a distinct view of normality while sharing the same anomaly labels ensures consistent guidance. Aggregating these diverse detectors improves robustness and generaliza- tion under limited anomaly labels. Furthermore, our experiments under different weak-supervision setups—1%, 5%, and 10% labeled anomalies, sometimes as few as five labeled anomalies total—demonstrate that WSAD-DT remains effective even in extremely label-scarce conditions. As shown in our contamination experiments (Appendix M), the method’s performance degrades only slightly when unlabeled data are partially contaminated, underscoring both its theoretical and practical robustness in real-world weakly supervised scenarios.

Q2

We compared WSAD-DT against SOEL (using the authors’ code from https://github.com/aodongli/Active-SOEL), which explicitly handles anomalies in the unlabeled set, using each approach’s default configurations. Table https://anonymous.4open.science/r/weakly_anomaly_detection/Table4.png shows that WSAD-DT generally achieves higher AUC-ROC on most datasets, occasionally by a wide margin. These results indicate that while SOEL is effective at identifying unlabeled anomalies, the dual‐tailed separation and diversity regularization in WSAD-DT provide a stronger results in weakly supervised scenarios.

Q3

To isolate the effect of the kernel design and the diversity term, we fix the ensemble size to 1 in both the full WSAD-DT and a simplified variant that uses only a single Gaussian kernel with no diversity regularization. As shown in Table https://anonymous.4open.science/r/weakly_anomaly_detection/Table5.png, we compare the full WSAD-DT (dual-tailed + diversity) to a simplified version using only a single Gaussian kernel without the diversity term. Across most datasets, WSAD-DT achieves higher AUC-ROC. Empirically, this suggests that relying on a single kernel alone and omitting the diversity regularization is insufficient for robust anomaly detection under weak supervision. In contrast, combining light- and heavy-tailed kernels with a diversity penalty improves stability and overall performance.

Q4

We appreciate the reviewer’s suggestion regarding the use of more realistic datasets, such as MedicalMNIST. In response, we have extended our experiments to include this dataset, applying the commonly used "one-vs.-rest" evaluation protocol. This protocol treats each class in turn as normal while randomly selecting samples from other classes as anomalies, following the standard practice for anomaly detection tasks on multi-class datasets. Our experiments on the MedicalMNIST dataset have yielded strong results, demonstrating the robustness and scalability of our method in more complex, real-world scenarios. As shown in the Table https://anonymous.4open.science/r/weakly_anomaly_detection/Table6.jpg, we compare the performance of our method (WSAD-DT) against several competitive methods across various datasets in the MedicalMNIST suite. For example, on the Derma dataset, WSAD-DT achieved the highest AUC-ROC score of 0.9002, outperforming methods such as DeepSAD, DevNet, and XGBOD. Across several datasets, our method consistently ranks among the top performers, reinforcing its effectiveness in handling medical data with weak supervision. We believe these results demonstrate that our method is not only effective on tabular datasets but also performs well in more complex, real-world applications like medical anomaly detection.

We sincerely thank the reviewer for their perceptive review.

Q1

We adopt the 70/30 train/test split protocol following the approach used by AdBench (Han et al., 2022) [1], and this does not contradict the principle of weak supervision. What determines the “weakness” here is not how much of the dataset is allocated to training, but rather how few anomalies are actually labeled among that training portion. Even though 70% of the data is used for training, only a small fraction of those anomalies—sometimes just five labeled anomalies—is available for explicit supervision. The rest of the training set remains unlabeled. This imbalance in labeled anomalies, rather than the overall training size, is what makes the setting truly weakly supervised and differentiates it from the kind of fully supervised approach that would require extensive or comprehensive anomaly labeling.

Q2

We set the bandwidth parameters to reflect the distinct behaviors of normal and anomalous data, using a smaller value for the normal center to enforce tighter in-class clustering and a larger value for the anomaly center to accommodate its more varied distribution. These parameters essentially control how rapidly similarity decays with distance for the light-tailed (Gaussian) and heavy-tailed (Student-$t$) kernels, respectively. A Gaussian kernel with a smaller bandwidth enforces compactness for normal samples, whereas a slightly larger bandwidth for anomalies prevents overly constraining their more diverse representations. Likewise, the Student-$t$ kernel parameter determines how “heavy” the tail is, maintaining a broader margin for out-of-class points. We provide an ablation study showing that the method remains robust across reasonable ranges [0.1-1] of these bandwidth and tail parameters (Appendix O).

Q3

We performed an additional set of experiments comparing WSAD-DT with IDK (using the authors’ code from https://github.com/IsolationKernel/Codes) and Isolation Forest (IForest) [2] (code from https://github.com/yzhao062/pyod), adopting each method’s default parameter settings. In Table https://anonymous.4open.science/r/weakly_anomaly_detection/Table2.png, WSAD-DT consistently outperforms IDK and IForest in AUC-ROC across nearly all datasets, often by a substantial margin. Although isolation-based methods are well-known for their speed and simplicity, these results suggest that incorporating a small set of labeled anomalies, along with a dual-tailed kernel and diversity regularization, yields more accurate detection. Even on large or high-dimensional datasets, WSAD-DT achieves better separation between normal and outlying samples than both IDK and IForest. These findings are consistent with [1], where even minimal supervision—such as having only five labeled anomalies—can substantially surpass purely unsupervised methods.

Q4

We conducted an additional experiment where we replaced the dual-tailed setup (Gaussian for in-class + Student-$t$ for out-of-class) with a single Gaussian kernel using two different bandwidths—one smaller for in-class (e.g., $\sigma=0.1$) and one larger for out-of-class (e.g., $\sigma=2$). To isolate the effect of the kernel design and the diversity term, we fix the ensemble size to 1 in both setting. Table https://anonymous.4open.science/r/weakly_anomaly_detection/Table3.png shows that while using two distinct Gaussian bandwidths can perform reasonably well, the full dual-tailed configuration (Gaussian + Student-$t$) generally achieves stronger AUC-PR score. Larger $\sigma$ in a Gaussian slows its decay somewhat, but that decay remains fundamentally exponential, whereas a Student-$t$ kernel falls off more gradually and does not saturate as quickly at moderate distances. This long-tail behavior preserves a stronger “push” for out-of-class points, leading to a broader margin and, ultimately, more robust anomaly separation. This suggests that having a genuinely heavy tail is important for pushing out-of-class points farther away and not just a matter of scaling the same kernel. In practice, a true heavy-tailed kernel—like Student-$t$—better preserves moderate similarities at larger distances, thereby preventing early saturation and improving separation of anomalous instances.

[1] Han, S., Hu, X., Huang, H., Jiang, M., & Zhao, Y. (2022). Adbench: Anomaly detection benchmark. Advances in neural information processing systems, 35, 32142-32159.

[2] Liu, F. T., Ting, K. M., & Zhou, Z. H. (2008, December). Isolation forest. In 2008 eighth ieee international conference on data mining (pp. 413-422). IEEE.

We thank the reviewer for the valuable feedback.

**Ablation stay extreme case(e.g., 0.1%) **

We have conducted additional experiments with a 0.1% fraction of labeled anomalies. Table https://anonymous.4open.science/r/weakly_anomaly_detection/Table1.png summarizes AUC-ROC results under this extreme scenario. Despite having only 0.1% labeled anomalies, WSAD-DT achieves state-of-the-art or near-best performance on most datasets. Notably, even with such sparse labels, dual-tailed kernels and our ensemble design still effectively leverage the limited anomaly examples to separate normal vs.\ abnormal data.

Q1

The “well-separated” assumption in Theorem 5.3 is a common theoretical device in margin-based analyses, akin to the strict separability often assumed in classical SVM proofs. Although real data typically feature some overlap between anomalies and normal samples, this idealized scenario highlights a core theoretical insight: having distinct margins reveals how dual-tailed kernels outperform single-kernel designs by simultaneously promoting in-class compactness (via the light-tailed kernel) and sustaining a long-range “push” for out-of-class points (via the heavy-tailed kernel). In practice, WSAD-DT remains effective when anomalies are only partially separable because the heavy-tailed kernel preserves nontrivial gradients even at moderate distances, preventing anomalies close to the normal center from collapsing into negligible similarity. Meanwhile, the light-tailed kernel keeps each class tightly clustered, and the diversity term further prevents degenerate collapsing by penalizing over-concentration. Moreover, our experiments across different weak-supervision scenarios—1%, 5%, or 10% labeled anomalies, sometimes only five anomalies total—show that WSAD-DT remains effective under label scarcity. As demonstrated in our contamination experiments (Appendix M), performance degrades only mildly when unlabeled data contain anomalies, confirming that while Theorem 5.3 holds under clean-margin assumptions, WSAD-DT’s dual-tailed design still excels under real-world overlap.

Q2

We focus on light‐ vs. heavy‐tails, not specific forms. Any light‐ (e.g., Laplacian) or heavy‐tail (e.g., Cauchy) kernel is valid, as long as it preserves the fast‐ vs. slow‐decay principle. In Appendix O, we vary kernel parameters to test different tail decays and observe stable performance.

Q3

WSAD-DT is primarily designed for static, batch-oriented anomaly detection and does not explicitly account for streaming data or concept drift, similar to methods like DeepSAD, DevNet, or XGBOD. Nonetheless, it can be adapted by updating the normal and anomalous centers as new data arrive and retraining periodically to accommodate changing patterns. The ensemble structure also naturally extends to dynamic contexts by allowing older models to be replaced with ones trained on more recent data. This preserves the dual-tailed kernel advantage while helping WSAD-DT remain effective in environments, such as fraud detection, where data distributions evolve over time.

Q4

Classical kernel-based anomaly detection methods, such as One-Class SVM, rely on a single kernel applied uniformly across the data [3]. In contrast, multi-kernel and multi-view approaches blend multiple kernels—often through linear combinations—to capture richer similarity structures [2]. For example, Zhao and Fu introduce a dual-regularized framework that factors multi-view data into cluster indicators and sample-specific errors [1]. However, these methods still learn a single, blended kernel function that is applied globally to all points. By contrast, our approach conditionally assigns two distinct kernels to each data point: a light-tailed kernel for in-class distances and a heavy-tailed kernel for out-of-class distances. This explicitly changes tail behavior based on whether a point is close to the normal or anomaly center—rather than learning a single global kernel mixture. As we demonstrate in Section 5, this dual-tailed design achieves tighter in-class clustering and broader out-of-class separation—two conflicting goals that a single kernel or linear blend cannot simultaneously satisfy.

Typos

We thank the reviewer for pointing out the typos, and we will correct them in the revised manuscript.

[1] Zhao, Yue, and Yun Fu. "Dual-Regularized Multi-View Outlier Detection."

[2] Gönen, Mehmet, and Ethem Alpaydın. "Multiple Kernel Learning Algorithms."

[3] Schölkopf, Bernhard, et al. "Estimating the Support of a High-Dimensional Distribution."