Explaining the Out-of-Distribution Detection Paradox through Likelihood Peaks

Thank you for the insightful review and praise for our ideas and presentation. We address your concerns point-by-point below:

Soundness

1. You expressed some concerns about the validity of our Taylor approximation of $\log \rho_r(\mathbf{x})$. Part of this was based on how $\hat{\rho}_r$ ranks in-distribution and OOD data in Figure 3 in the manuscript. In fact, Figure 3 shows them behaving precisely as would be expected. For very negative $r$, the OOD data has higher mass because it is assigned higher likelihood under the model. For values closer to zero (here above $r \approx -7$), the Taylor approximation loses its accuracy since the mass is estimated over too large of a region. The slopes of the curves are also appropriate, as the more negative slope for OOD data indicates it has lower LID. To further alleviate these concerns, we are conducting experiments to verify the suitability of our LID estimator on benchmark datasets, and we will include them before the end of the discussion period.
2. You shared a concern about how we set $r$ with reviewer urvk; please find our response to this in the general rebuttal.

Presentation

1. You asked us to be more precise in the work about manifolds, submanifolds, and probability mass. Thank you for the suggestion; we will update the language around manifolds in the manuscript accordingly. To summarize,

   • The in-distribution data sits on a submanifold (of zero Lebesgue measure).
   • The likelihood-based DGM, which represents a full-dimensional density, concentrates mass around the (complex) in-distribution submanifold in such a way as to encode the submanifold’s dimensionality [A].
   • Due to its inductive biases, the DGM density also inadvertently concentrates a negligible amount of mass around out-of-distribution submanifolds when these are “simpler” (i.e. of lower intrinsic dimension).

   Since there is a distinction between the probability mass of the DGM and the underlying (measure zero) submanifolds being modelled, there is no underlying error in the way we use these concepts, though we certainly take your point that they can be made clearer.

2. We thank you for pointing this out. You are correct that the notion that the OOD paradox must be due to lower probability mass was not originally proposed by us, although we do highlight that the paper you point out does not propose a way to empirically verify this. We will reword the manuscript to make this clear. Our main contributions are linking this explanation to intrinsic dimension, and exploiting this newfound connection to build a highly-performant OOD detection algorithm.

3. You suggested a rewording to make the jump from a uniform distribution on a high-dimensional ball to a Gaussian a bit clearer. Thank you for this; we will clarify this paragraph in the manuscript.

You also had two additional questions:

1. Thank you for bringing this point up. We are currently conducting a set of experiments where the ground truth values of $\rho_r$ and $\frac{\partial}{\partial r} \rho_r$ are either known or can be accurately computed. We will include these results in the manuscript before the end of the discussion period.
2. You also asked us about whether choosing hyperparameters for $r$ based on SVHN-CIFAR-10 does better than choosing another arbitrary statistic from a generative model, and whether our method is robust to this choice. In the general rebuttal we propose a new intuitive way to select $r$ without hyperparameter tuning; we hope this resolves your concerns on this front. Otherwise, could you please clarify what kind of arbitrary statistic you have in mind? We do point out that the “complexity correction” and “likelihood ratios” baselines we compare against could be loosely interpreted as leveraging arbitrary statistics, namely the compression length of images and the likelihood of another model, respectively.

[A] Tempczyk et al., “LIDL: Local Intrinsic Dimension Estimation Using Approximate Likelihood”, ICML 2022.

• (W3): Please see our discussion above as to why OOD samples being assigned negligible probability mass is indeed well established. We also point out that we do not claim OOD data is always assigned higher density, nor that it always has lower intrinsic dimension. In our work we often discus non-pathological cases like training on SVHN and evaluating on CIFAR10, where CIFAR10 data is not assigned higher likelihoods (e.g. Figure 14 for this pair). This is completely consistent with previous literature, which has found OOD data is only assigned higher likelihoods when OOD data is “simpler” than in-distribution data, which we quantify through intrinsic dimension. You also ask what would happen if we used our method in non-pathological settings. First, note that Table 1 contains several such examples, e.g. models trained on MNIST and tested on FMNIST, or trained on SVHN and tested on CIFAR10, are not subject to the standard paradox (and the intrinsic dimension of OOD data is not lower in this case), yet our method nonetheless succeeds at OOD detection. If random noise was used as OOD data instead, despite not having a low-intrinsic dimension, it would achieve a lower likelihood and still be detected by the other threshold in our dual threshold method (Algorithm 1). We will include such a result by the end of the discussion period.
• (W4): We respectfully disagree with this characterization of our work. We hope that all the points we made above elucidate why, and that you will consider reevaluating our work at the end of the discussion period once we have made the promised manuscript updates. Otherwise, please let us know aspects you believe are “weak” or “empirical” and we will engage in discussion around them.

[A] Nalisnick et al., “Do Deep Generative Models Know What They Don’t Know?”, ICLR 2019.

[B] Choi et al., “WAIC, but Why? Generative ensembles for robust anomaly detection”, 2018.

[C] Zhang et al., “Understanding Failures of Out-of-Distribution Detection with Deep Generative Models”, ICML 2021.

We thank you for your time, effort, and feedback, and we appreciate the detailed review. Here we discuss the concerns you listed. You also asked about the applicability of this method to VAEs and diffusion models; for this, please see the general rebuttal.

Before addressing your concerns, we think it relevant to clarify at a high level some of the overarching themes and methodology in our work as follows.

The core paradox of the study is that, while a trained DGM generates samples that look much more like its training data (e.g., CIFAR 10), it can assign higher likelihoods to data that is very clearly outside its training data distribution (eg. SVHN). Our work shares its motivation with the rich body of literature initiated by [A] and [B] which we contour in our Background and Related Work sections. Though there may exist subtle differences between generated samples and training data, the much starker dissimilitude between in-distribution/OOD pairs such as CIFAR-10/SVHN make the aforementioned paradox remarkably surprising.

The notion that in-distribution points must belong to regions of higher probability mass than OOD points is broadly accepted in the literature (e.g. [A] section 2.1 and [C]) and follows naturally from first principles: if a CIFAR10 model never generates SVHN data, it must assign negligible probability to OOD regions (where OOD here is defined as SVHN). As a result, we respectfully disagree that it is a “strong assumption” without “sufficient support”.

But we do not take this notion as an assumption; instead, we take it as a hypothesis (we will clarify this in the manuscript). Recall the following pair of facts outlined in our Methods section:

• If a sample has low likelihood, it is in a region of low probability mass.
• If a sample has high likelihood and low intrinsic dimension (i.e. the density is highly peaked), then it is also in a region of low probability mass.

In our experiments (e.g. Figure 4), we see that OOD data tends to have low probability mass according to one of the above criteria, which verifies the hypothesis. We use this information to develop an OOD detection method, which as reviewer urvk noted, produced significant improvements over competing approaches.

Having clarified our perspective, here we respond to your individual concerns:

• (W1): As mentioned above, we emphasize that the stark dissimilarity between OOD and in-distribution data does make the behaviour extremely puzzling, even when models are imperfect, as evidenced by the large existing literature on the topic. We also point out that all models are imperfect (some more so than others of course), and so OOD detection with imperfect likelihood models remains a relevant problem.

  We agree with you that there is “no guarantee that a deep generative model would never generate OOD samples”, and indeed since the model assigns high likelihoods to OOD data there must be some (vanishingly small) probability of sampling them. However, for all practical purposes, the well-trained models used in our examples do not generate OOD data.

  Finally, we also point out that if we redefine in-distribution samples as generated samples from the (imperfect) model, OOD samples still achieve higher likelihoods. See for example Figure 2 (we note that this is a novel observation of ours). Because in this hypothetical situation our model is equal to $p(x_\text{IID})$, these results show that “puzzling behaviour” persists regardless of model fit.

• (W2): Indeed several explanations have been proposed in the literature, as we summarized in the Background and Related Work sections, including [A]. In particular, the one you bring up [A] has strong assumptions, namely that the in- and out-of-distribution densities have overlapping support and that they are both Gaussian. When, as in our case, in- and out-of-distributions are semantically different, the support overlap assumption is unrealistic [C]. Our explanation and resulting method do not require either of these assumptions (note our local Taylor approximation results in a local Gaussian approximation, not a global one).

We have added an extra set of experiments in response to your question in (W3) about whether our algorithm is capable of detecting random noise as OOD data because it would have high LID. In these experiments, we consider a model trained on one of the datasets in the paper and then feed it random images; each pixel intensity is chosen uniformly at random from the values 0 to 255. As you pointed out, the densities for these random images would not be sharply peaked, so they would have no low-dimensional structure and hence would have high LID.

We validated our method for all datasets, and in every case the OOD datapoints are completely separated from in-distribution points by the likelihood threshold. Hence, in every case the AUC-ROC is a perfect 1.00:

For a visualization of why our method is perfectly effective here, please see Figure 1 of the newly-uploaded supplementary material, which depicts estimated LID values and likelihoods for four of our models. As in the paper, each point represents a datapoint with its colour indicating whether it is in-distribution, generated from the model, or OOD. In red, we have visualized both the LID and likelihood thresholds chosen by our dual-threshold OOD detector. As illustrated, the likelihood threshold easily divides in-distribution from out-of-distribution. This is a “non-pathological” case, where the OOD datapoints lie on a flat region of the density obtaining high LID estimates while simultaneously obtaining extremely low likelihoods. The density does not peak around random noise.

We are grateful for your insightful comments and your decision to adjust your score favourably. Your feedback is invaluable, and in light of your suggestions, we will take extra care to articulate our paper more clearly, ensuring that it accurately conveys our methodology and addresses potential ambiguities.

Thank you for your positive feedback on our ideas, methodology, and experimental results.

1. You asked whether our analysis is generalizable to models outside of normalizing flows; please see the general rebuttal for more discussion on this point.
2. You also asked a couple of questions about setting $r$.
   • You asked about a more principled way of setting it; please see the general rebuttal, where we propose an improvement over our original approach.
   • You also asked why we cannot simply set $r$ to a very small value; we agree this is a relevant question and we will further elaborate on it when updating the paper. An intuitive way to think about this is that the hyperparameter $r$ effectively chooses a scale at which we probe the density model, and that it determines how sensitive the estimate is to slight perturbations in the density. This is completely analogous to how LIDL [A] sets the various amounts of noise it uses to estimate LID, and to how commonly used estimators of global intrinsic dimension set the number of nearest neighbours as a hyperparameter [B]. Please refer to section 2.4 of [A] for a discussion on how they heuristically choose the scale variable according to how sensitive they require their estimator to be.

[A] Tempczyk et al., “LIDL: Local Intrinsic Dimension Estimation Using Approximate Likelihood”, ICML 2022.

[B] Levina and Bickel, “Maximum Likelihood Estimation of Intrinsic Dimension”, NeurIPS 2004.

We deeply appreciate your valuable feedback. Your suggestion to illustrate the generalizability of our approach using additional model types, such as diffusions or VAEs, is well-received.

As promised above, we ran experiments on every dataset pair using the new “principled” way of choosing $r$ which we described in response to the concerns of reviewers urvk and i5V7.

A table containing ROC-AUCs for both Naïve Likelihoods and our method on each task is below. Each row contains two tasks - one where generated samples from the model are considered in-distribution and one where the test data is considered in-distribution. The results are generally similar to those presented in the paper, with the key exceptions of (1) CIFAR-10 vs CelebA, in which our method now outperforms the best flow-based baseline, and (2) MNIST vs. Omniglot and FMNIST vs. Omniglot, in which our method achieves AUCs about 10-12% lower than our original method in the paper. We also note that in general this method tends to perform better on RGB datasets than our original method.

Since setting $r$ using L-PCA performs comparably, but is done in a more principled way, we will update the manuscript to incorporate our results with this method.

As promised, we have updated our manuscript, incorporating new results in response to the valuable feedback provided by the reviewers. The changes are in blue for ease of reading. We extend our sincere gratitude to all reviewers for their insightful contributions. Their input has not only enhanced the substance of our work but also improved its presentation. Please let us know if there is anything else that we can do.