GOLD: Graph Out-of-Distribution Detection via Implicit Adversarial Latent Generation

We would like to express our sincere gratitude for your recognition of our work and your insightful suggestions. We have carefully addressed your comments as follows.

Re W1: Performance analysis

For our model evaluation, we use several widely adopted metrics to assess OOD detection performance: AUROC, AUPR, and FPR95. AUROC and AUPR measure the trade-off between true positive rates (TPR) and false positive rates at various thresholds, while FPR95 focuses on performance under high-sensitivity conditions—specifically, the rate of misclassifying ID samples as OOD when the TPR is at 95%. Thus, FPR95 provides a more noticeable score gain under stricter criteria, reflecting improvements in detection performance.

Moreover, with respect to the dataset characteristics, considering the multi-graph Twitch dataset, which consists of 3 real OOD graphs. As shown in Table 1 of the manuscript, the Non(Real)-OOD Exposure baselines GNNSafe(++) achieves an average FPR95 of 76.24% (33.57%), while our GOLD improves this to 1.78%. This improvement is likely because the real OOD graphs exhibit a distribution similar to the training ID data, making it difficult for Non-OOD exposed GNNSafe to distinguish between ID and OOD. In contrast, GNNSafe++ benefits from additional OOD training data, helping it better differentiate between ID and test OOD samples.

A closer look at individual OOD test graphs performance (see Table 11 in Appendix A.9 and below) reveals that the baselines perform significantly better on one particular OOD test graph, Twitch-RU. This suggests that the higher-performing OOD graph has a greater distributional divergence from the ID data, while the lower-performing OOD graphs are more similar to the ID data. Notably, GNNSafe++ appears to strongly benefit from the exposure of an OOD graph that may be distributionally similar to Twitch-RU, boosting detection performance.

However, this may introduce bias, as the model’s performance could vary depending on the similarity between the OOD test data and the OOD training data. Our proposed model addresses this bias by simulating pseudo-OOD instances directly from the ID data and incrementally diversifying them to improve detection across a broader range of OOD distributions. Thereby, alleviating potential bias from the exposed OOD, greatly improving performance across diverse OOD test sets.

Dear Reviewer yx8w,

Thank you once again for your recognition and thoughtful feedback on our submission. We will ensure the clarified information goes to the future version of the paper.

We greatly appreciate your comments, as they are helpful in improving our work. We have thoroughly reviewed your questions and have addressed as below.

Re W1: Computational cost

We would like to thank the reviewer for the careful attention to detail. As noted, we acknowledge the inherent training cost associated with using latent diffusion models. Demonstrated in Tables 1 & 5 of the manuscript, GOLD is able to achieve SOTA OOD detection performance and also delivers inference times that match current baselines. Given these improvements, we believe the trade-off between increased cost and enhanced performance is justified. We also appreciate the reviewer’s recognition that this cost increase is acceptable.

Additionally, to further improve efficiency, we have experimented with an additional lightweight variant by using a latent VAE to replace the latent diffusion. This approach also delivers competitive performance, which demonstrates the efficacy of our method. Importantly, we would like to emphasise that our model maintains (almost) the same inference speed as the baseline methods, ensuring that the performance gains do not come at the expense of test-time efficiency.

Re Q1: Notation clarification in Eq. 10)

Apologies for the unclear notation, the subscript represents the label of the corresponding logit value from the binary classifier (MLP) model after applying the Softmax function (i.e., [0] represents the ID class 0 and [1] represents the OOD class 1).

We have highlighted the revision in Section 3.2 of the manuscript.

Re Q2: Different backbones

We provide the following **experiments with two additional backbones: GAT $1$ and MixHop $2$ **, following GNNSafe and NodeSafe, where GAT and MixHop achieve second and third places, while GCN achieves the highest performance similar to their ablation studies. We compare these architectures against GNNSafe and NodeSafe, as well as their OOD-exposed variants. To ensure a fair comparison, we maintain the same configuration as the original GCN implementation, with a hidden dimension of 64, two layers, 8 attention heads for GAT, and two hops for MixHop. The results, shown below, demonstrate that GOLD outperforms other methods across the evaluated backbones.

We have also updated this as highlighted in Appendix A.13 of the manuscript.

[1] Veličković et.al., Graph Attention Networks. ICLR18.

[2] Abu-El-Haija et.al., MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing. ICML19.

We sincerely appreciate your constructive opinions for our paper. We have thoroughly reviewed the concerns and provided the following clarification.

Re W1: Blank Figure

Thank you for pointing out the issue with the possibly missing Figure 1(d), we have attempted to download the manuscript PDF from OpenReview, and the figure appeared to be available. This scatter plot with many dots has used a PNG figure to reduce the file size instead of using a huge PDF file. The rendering issue may potentially be a system error with OpenReview. We have revised the figure (highlighted caption) in the updated manuscript by replacing it with an even smaller-sized PNG image.

Re Q1: Difficulty of pseudo-OOD samples overly close to ID distribution

We thank the reviewer for raising an insightful point about the potential confusion in the OOD detector when pseudo-OOD samples are too close to ID data. In GOLD, pseudo-OOD data is generated via an LGM, which intentionally creates embeddings that initially lie close to ID samples - this proximity is by design, as it helps create challenging and representative instances.

To mitigate any confusion caused by this closeness, GOLD employs an implicit adversarial optimisation framework along with divergence regularisation (Eq.13), which iteratively maximises the energy gap between ID and pseudo-OOD embeddings. This training strategy ensures sufficient separation between the two distributions while maintaining the challenging nature of the synthetic samples. An illustration of this embedding separation process is provided in the motivation for GOLD in Figure 1 of the manuscript, where the initially close pseudo-OOD samples are effectively separated after training. To further control the proximity and diversity of pseudo-OOD data, GOLD employs different mechanisms based on the generative model used: with the default LDM, diversity is managed through the initial noisy vectors, while in the VAE variant, it is controlled by sampling the mean and variance in the latent space.

Moreover, the benefit of using difficult pseudo-OOD that are close to ID data is to prevent the detector from being biased by or overfitting to the OOD data. If the OOD used for training is too easy or too far away from in-distribution data, (e.g., using molecule graph as OOD for social network), the detector will easily overfit towards this direction, which cannot handle the difficult real-world OOD situation that could be much closer to the in-distribution data (e.g., detecting Place A’s data as OOD from Place B’s data in social network).

Re Q2: Large variance in the FPR95 Metric for Amazon and Coauthor dataset

Thank you for your insightful question. The result in Table 1 is obtained by averaging the different OOD test sets, where the variance reflects the difficulty of each of these test sets instead of the stability of the GOLD model on a single test set. For example, the $3.16 \pm 5.46$ result of GOLD on the Coauthor dataset is obtained from the average performance on three separate test sets with different types of OOD, Coauthor-S (structure manipulation OOD), Coauthor-F (feature interpolation OOD), and Coauthor-L (label leave-out OOD), respectively.

The stability of GOLD is demonstrated by the variance in the detailed evaluation tables for each testing set, which is shown in Table 11~15 in the Appendix. It can be seen that the variance within the individual subsets remains low.

We have revised the clarity in the updated version of the manuscript in the description of Table 1 and Section 4.1 (highlighted).

Dear Reviewer SRXq,

We highly appreciate the insightful feedback and the score increase we have received. We will ensure that all the updates are included in the future version of the paper.

We greatly appreciate the time and effort dedicated to providing us with thought-provoking comments and questions. We have thoroughly addressed your concerns.

Re W1 & Q1: Novelty and Contribution of GOLD to existing work

We thank the reviewer for recognising that our proposed divergence regularisation (Eq.12) is a key novelty. Building on this insight, GOLD also introduces a second key novelty: an implicit adversarial framework for synthesis-based OOD detection, which differentiates its purpose and realisation from existing works.

Traditional generative approaches primarily focus on preserving in-distribution (ID) characteristics, generating samples that retain or mimic information from prior or ID data $1$ . As the reviewer rightly points out, such methods are widely applied across various tasks, including continual learning for experience replay. When applied specifically to out-of-distribution detection, existing approaches typically use generative models to create samples from low-density boundary regions of ID data, relying on boundary-synthesis approaches based on non-parametric nearest neighbour distances $2, 3,4$ . These methods create synthetic OOD data by selecting points near the ID boundary, which may result in limited representation of the real OOD space.

GOLD takes a fundamentally different approach via our implicit adversarial learning framework. It employs a novel two-phase process: (1) generating ID-like samples using an LGM, and (2) transforming these samples into OOD instances using a novel energy-guided approach within the adversarial framework, guided by OOD-specific regularisation. This iterative process is distinctive both in its objective—producing semantically meaningful pseudo-OOD samples closer to real-world scenarios rather than synthesising boundary samples—and its realisation, as it leverages a unique adversarial optimisation rather than traditional boundary-synthesis techniques.

Thus, the key novelties of GOLD lie in two interconnected components: (1) the divergence regularisation for effective OOD detection, and (2) the implicit adversarial framework that iteratively balances ID-like sample generation with purposeful OOD divergence. While generative models and energy-based detection methods are common, GOLD's comprehensive training framework integrates these components in a novel manner, producing pseudo-OOD samples that are more representative and effective compared to prior methods. This framework also offers a practical solution to address the limitation of graph data, where pre-trained generative models like StableDiffusion are unavailable. The effectiveness of GOLD is further validated through empirical studies.

Re W2 & Q3: Ablation studies on different divergence regularisations

Thank you for your suggestion of analysing the regulariser (Reg) ($\mathcal L_{\text{Unc}}$, $\mathcal L_{\text{EReg}}$, and $\mathcal L_{\text{DReg}}$) individually to better illustrate the effectiveness of our proposed method. We present a detailed evaluation of the various combinations of these regularisers below (using the Twitch dataset), where a \checkmark indicates the inclusion of a given regulariser (removing or applying each individually):

The results demonstrate that the GOLD configuration, which incorporates all three regularisers, achieves the highest performance, highlighting the critical role each regulariser plays in the model’s design. Additional extended analysis on more datasets and subsets is available in Table 4 of the main text and Appendix A.9 (Tables 18-20).

[1] Goodfellow et.al., Generative Adversarial Networks. NIPS14.

[2] Tao et.al., Non-Parametric Outlier Synthesis. ICLR23.

[3] Du et.al., Dream the Impossible: Outlier Imagination with Diffusion Models. NIPS23.

[4] Lee et.al., Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples. ICLR18.

[5] Liu et.al., Towards Graph Foundation Models: A Survey and Beyond. 2024.