Local-Prompt: Extensible Local Prompts for Few-Shot Out-of-Distribution Detection

Q3: Performance without learnable negative local prompts.

A3: As the negative local prompts represent potential outlier knowledge, they are randomly initialized since we have no access to outlier samples during training, therefore it can not be replaced with hand-crafted prompts in the method. So we assume the reviewer hopes to remove the negative local prompts along with hard-negative samples (which are used to optimize the negative local prompts). The results are shown in Fig.4 termed as w/o LNP. We showcase the detailed results as follows:

It can be seen that with negative local prompts, the model gets consistent and substantial improvements, showcasing the effectiveness of negative local prompts for enhancing local outlier knowledge with respect to few-shot OOD detection.

As for the qualitative effectiveness of the negative local prompts, we provide additional visualization in Fig.5 to demonstrate that they actually learn outlier knowledge in different scenarios.

Q4: Question about Figure.2 .

A4: We sincerely appreciate your raising this point. First, we clarify that augmented inputs indeed refer to the image patches from the randomly cropped images (marked with yellow in the top diagram of Fig.2), which is in line with your understanding. In the bottom diagram of Fig.2, we take one cropped image as an example, and the orange patches are local features from the same augmented image. For loss calculation and evaluation, the most related regions (patches after regional selection in the diagram) are selected for enhancing OOD detection performance. We are grateful for the reviewer's rigorous reading, and hope the explanation could address your concerns.

We are truly grateful for the reviewer's interest in our methods with detailed reviewing, and hope the explanation could address your concerns. If you have any further questions, reply to us at your convenience and we will try our best to address your concern.

[1] Learning to Prompt for Vision-Language Models. IJCV 2022.

[2] Conditional Prompt Learning for Vision-Language Models. CVPR 2022.

[3] LoCoOp: Few-Shot Out-of-Distribution Detection via Prompt Learning. NeurIPS 2023.

[4] Delving into out-of-distribution detection with vision-language representations. NeurIPS 2022.

[5] ID-like Prompt Learning for Few-Shot Out-of-Distribution Detection. CVPR 2024.

[6] Energy-based Out-of-distribution Detection. NeurIPS 2020.

[7] Non-Parametric Outlier Synthesis. ICLR 2023.

We appreciate the reviewer for the valuable comments and appropriately respond to them as follows. We will add the suggested experiments and illustrations in the revised version.

Q1: Justification for using few-shot fine-tuning.

A1: Few-shot OOD detection is proposed by LoCoOp[3]. We agree with their settings and introduce them briefly in the introduction. Here we provide the explanation in detail.

"On the one hand, the zero-shot methods do not require any training data, but they may encounter a domain gap with ID downstream data, which limits the performance of zero-shot methods[4]. On the other hand, fully supervised methods utilize the entire ID training data but may destroy the rich representations of CLIP by fine-tuning, which also limits the performance despite requiring enormous training costs. To overcome the limitations of fully supervised and zero-shot methods, it is essential to develop a few-shot OOD detection method that utilizes a few ID training images for OOD detection. By finding a balance between the zero-shot and fine-tuned methods, few-shot OOD detection has the potential to provide a more efficient and effective solution for OOD detection."

The effectiveness and efficiency have been demonstrated by previous works[3,5], and few-shot OOD detection has the potential to get competitive results against full-tuning methods[6,7]

For the reviewer's concern that the performance may be biased toward the examples of the class being used, in the paper, we conduct standard processing of few-shot selection following existing methods in both representation learning[1,2] and OOD detection[3], and select few-shot samples using exactly the same procedure. Therefore, all comparisons are conducted under the same setting. As for the performance, we are not sure about the random seed used by previous works. In our work, all of the results are averaged for three runs and we do not observe obvious fluctuations.

We hope the analysis could help the reviewer have a better understanding of few-shot OOD detection. If you still have concerns or other questions, respond to us and we will make in every effort to address your concerns.

Q2: Few-shot and many-shot performance.

A2: For OOD detection performance on SUN and Places datasets, on the one hand, the phenomenon is common under the setting of few-shot OOD detection, with similar trend is observed in ID-like[5] (1-shot and 4-shot in Places, shown in table below) and LoCoOp[4] (4 shot and 16 shot in SUN and Places shown in Table.1). Possible reasons for the phonomenon are: 1) the ability of few-shot learning for effectively gaining outlier knowledge and enhancing OOD detection performance with scarce samples, certificating the utility; 2) Moreover, for few-shot OOD detection, more samples may not necessarily bring better performance, and incorporating more samples may confuse the discrimination, which should be the characteristic of the two datasets.

Dear reviewer GNZR:

We respectfully appreciate again for your insightful and thoughtful comments! As the suggestions from the reviewer, we give thorough explanation and update the manuscripts accordingly.

As it is approaching the end of the discussion period (November 26 at 11:59 pm AoE) and we do not receive feedback, we sincerely hope you could look through our response and have a further comment at your convenience if you have any questions about the paper. We will do our best to address the issues of the reviewer.

Best wishes,

Submission 6694 Authors.

We sincerely thank the reviewer for taking the time to view the rebuttal and making valuable suggestions to the paper. As for the two additional questions, we would like to make some further clarifications:

1. Adding more shots

As can be seen from the main table and the explanation in the first-round discussion, we would like to emphasize two key points: 1) the phenomenon is commonly observed in various OOD detection methods, including but not limited to ID-like and LoCoOp, this part we have adequately showcase in the first round discussion. 2) the phenomenon should be attributed to the characteristics of the two datasets.

To further support our point of view, we calculate the similarity of prompts between both OOD datasets and ID-dataset with different shots and observe the trends of change:

It can be seen from the table that 1) the similarity of the four OOD datasets are related to the OOD detection performance on them. Specifically, more similar to ID datasets (Texture) leads to poor performance, which is in line with the analysis and experimental results; 2) as the number of shot increases, the discrimination between SUN and Places (0.01 and 0.02 decrease) is vaguer than that in iNaturalist and Texture (0.09 and 0.06 decrease, respectively). It partially proves that the gain brought by more shots from the two middle datasets is smaller. It is the characteristic of the two datasets, and is irrelevant to the algorithm used for OOD detection.

From the analysis above, we would like to explain that we are not the only one that encounters the phenomenon, and we give possible explanation from our perspective of view to showcase that the phenomenon does exist other than the bias brought by few-shot setting, which indicates the rationality of the phenomenon. As it is not the focus of our paper, the effectiveness of our proposed method that our proposed method achieves consistent and substantial improvements should not be ignored. The reviewer raises an interesting point and we will take investigation of balance between different kinds of datasets as future research points.

If the reviewer still has concerns about the phenomenon, point out in detail which part of the explanation above you have questions, and we are pleased to make further explanation.

2. Improvement from negative local prompt

We would like to emphasize that negative local prompt is one core contribution of the proposed method and our contributions can be concluded as: 1) utilization of local prompts (enhance local information), 2) local negative prompts (provide outlier knowledge) and 3) local based OOD metrics. We give a detailed comparison of the proposed method with LoCoOp in a progressive manner to demonstrate the effectiveness of each component.

We summarize from the table that 1) compared with LoCoOp (with the same OOD metric), we get a substantial improvement (4.62% FPR95), which firmly validates the effectiveness of local prompts; 2) our method equipped with local and outlier enhancement further improves the performance by 2.32% on FPR95. We shall say all these improvements are made by our local enhancement framework and should not be ignored.

Last but not least, please allow us to say that improvement on few-shot OOD detection is challenging. Given that we already substantially improve the performance by about 4.62% on FPR95 compared to LoCoOp with the same OOD metric, which is approximately the promotion from 4-shot to 16-shot, such an improvement building on several methods that proposed by us, i.e., ours_{GL} to ours_{w LNP} should not be considered to be marginal.

We really appreciate the reviewer's patience and valuable time to view the rebuttal, and rates a positive points for the paper. We hope the additional response could address the remaining questions from the reviewer. We are more than pleased to discuss with the reviewer about any of the problems and discover some observations that are worth paying attention to.

I think the authors addressed most of my concerns and I would like to stick to my previous rating of marginally above acceptance. It has some interesting observations.

Couple of comments:

• I don't think I understand authors' point of adding more shots causing more confusion for OOD detection, because intuitively having more shots should make the model more confident about in distribution data.

• The improvement from negative local prompt seems to be marginal, despite being a key selling point of the paper. Is there any intuition why?

Q3: Explanation of hard negative samples.

A3: Actually, what counts for OOD detection is that the space is not ID regions, so separating ID and OOD regions is of great significance in the method, and employing hard negative samples perfectly suits the requirement as carefully selected OOD data may not contribute to separating ID and OOD regions in the scene. Theoretically, hard negative samples have the most similar distribution with ID samples than real outliers. Once our model is trained on hard-negative samples, it naturally gains the ability to discriminate outlier samples as detecting hard negative samples (similar to ID samples) is harder than real outliers. We emphasize that the goal is not easy to realize, as we carefully design both local prompts to learn local information of ID categories, negative local prompts to simulate potential outliers, and use a diversity regularization to make negative local prompts cover more unseen classes. All the analysis above strongly demonstrates the effectiveness of the proposed method.

To support our point of view, 1) Quantitatively, we conduct experiments about the number of negative local prompts in Table.7 and find that local negative prompt is well sufficient to achieve good results, indicating that the local negative prompts represent features of potential outliers well. We additionally carry out experiments to calculate the similarity of both ID-dataset, hard negative samples, and OOD datasets with local prompts and list qualitative results as follows:

It can be seen in the table that without local enhancement, the model fails to discriminate between ID-datasets and hard negative samples, and a large margin exists with OOD-datasets. After local enhancement, the difference between ID-datasets and hard negative samples obviously improves, and hard negative samples share similar similarity with OOD-datasets. The results demonstrate the effectiveness of hard negative samples and the rationality for hard negative samples serving as a form of outlier samples.

2) Qualitatively, we visualize both local negative prompts in Fig.5 and attention regions of local prompts in Fig.9. It is vividly shown in these visualizations that the negative local prompt is discriminative about potential outliers in the scene (Fig.5), and pushing away from these potential outliers (e.g., background) is helpful for the model to concentrate on ID regions (Fig.9) and therefore improve the performance.

We sincerely hope our analysis could help the reviewer have a better understanding of the strength of the proposed method and the rationality for hard negative samples serving as a form of outlier samples. If you still have questions, point out in detail and we will try our best to make it more clear.

Q4: Initialization of local prompts and negative local prompts.

A4: The global prompts are hand-crafted prompts "a photo of {class}" (line237) and set frozen during training and evaluation. The local prompts are initialized with embeddings of "{learnable prefix}+{class}" (line264) and optimized during training. Negative local prompts are initialized randomly due to lack of real outlier information and diversity regularization $\mathcal{L}_{reg}$ guarantees that negative local prompts cover more unseen spaces. Intuitive explanation can also be found in diagram Fig.2. Their dimensions are the same as the dimension of text embeddings, which is 512 in our paper.

Q5: Further clarification of prompts.

A5: We would like to make an explanation that $t_c$, $t$ and $\hat{t}$ stand for features of global, local and negative local prompts in our paper (defined in line 237, 264 and 265, respectively). As explained in Q4, global prompts are hand-crafted prompts in the form of "a photo of {class}" and set frozen. In practical application, local prompts are initialized with embeddings of "{learnable prefix}+{class}" and "learnable prefix" are learned in end-to-end optimization with the weighted sum of loss function. Local prompts in Eqn.4 share the same meaning with the symbol $t_k$ (k as the subscript from 1 to number of category). Intuitive explanation can also be found in diagram Fig.2. We thank the kind remind from the reviewer and make a clear explanation in the updated manuscript. If the reviewer has any further questions about the notation, point it out and we will make a clear and detailed explanation.

[1] LoCoOp: Few-Shot Out-of-Distribution Detection via Prompt Learning. NeurIPS 2023.

[2] Zero-Shot In-Distribution Detection in Multi-Object Settings Using Vision-Language Foundation Models.

We appreciate the reviewer for the valuable suggestions and appropriately respond to them as follows. We will add the suggested experiments and explanations in the revised version.

Q1: Definitions of crucial symbol.

A1: We thank the reviewer's kind reminder, $\mathcal{T_k}(x)$ is the sum of $*k*$ largest elements in $x$ (line184). In our paper, the shape of $x$ is the number of local tokens/features. $\hat{t_i}$ is the notation of local negative prompts (line266), with subscript i from 1 to number of negative local prompts $N_{neg}$ in Eqn.6 and Eqn.7. We give a detailed explanation of relevant notations in the updated manuscripts. If the reviewer still has doubts about any of the symbol definitions, we are pleased to address your questions.

Q2: The difference between the proposed method and GL-MCM[Miyai et al. 2023b].

A2: We would like to emphasize the difference between our method and GL-MCM[Miyai et al. 2023b]. First, from the motivation, GL-MCM focuses on ID detection, which is a totally different OOD setting (ours OOD detection as a reference). Consequently, it merely needs to detect any of the regions that are similar to ID categories, so the maximum process is reasonable. However, cases are totally different in the OOD detection setting, which has to detect any possible outlier regions. By contrast, the motivation of our method is to detect hard OOD regions. Moreover, we kindly remind that the proposed score is a small part of our method. Concretely, we propose local prompt and negative local prompts to enhance local outlier knowledge. We focus on hard OOD regions with generated hard OOD samples, and optimize the corresponding prompts with proposed loss functions to obtain outlier information.

To further enhance the explanation, we conduct OOD detection using GL-MLM score, the results are shown in the table below.

It can be concluded that 1) compared with GL-MCM, our method achieves a consistent and substantial promotion (10% and 3% on average, respectively), which is in line with the analysis above that GL-MCM fails to achieve satisfying results in OOD detection; 2) our R-MCM once more gets improvement against our GL-MCM, demonstrating that the proposed method better fits for enhancing OOD detection with fine outlier knowledge.

We compare the mentioned work in detail to show the unique strength and advantage of our method both theoretically and experimentally. If the reviewer still has questions, raise specific questions and we will try our best to address them.

Dear reviewer KX2t:

We respectfully appreciate again for your insightful and thoughtful comments! As the suggestions from the reviewer, we give thorough explanation and update the manuscripts accordingly.

As it is approaching the end of the discussion period (November 26 at 11:59 pm AoE) and we do not receive feedback, we sincerely hope you could look through our response and have a further comment at your convenience if you have any questions about the paper. We will do our best to address the issues of the reviewer.

Best wishes,

Submission 6694 Authors.

Q2: Results using Regional OOD score.

A2: Following the suggestion of the reviewer, we additionally conduct experiment of OOD score solely dependent on Regional OOD score. Results are shown in table below.

From the table, it can be concluded that 1) Regional OOD score can not achieve satisfying results in our experiment and must rely on global score; 2) the proposed R-MCM achieves the best results with respect to all metrics. This is in line with analysis that global prompt/score provides global and fundamental discrimination between ID and normal OOD samples (Sec.4.1) and local prompts/score provides fine discrimination results between ID and hard OOD samples (Sec.4.2). Consequently, it demonstrates that our proposed score integrating gliobal and local information outperforms the strategy that solely relies on the Regional OOD score. We kindly highlight that we aim to propose a novel approach to enhance OOD detection performance from the perspective of enhancing local information but not to demonstrate that local prompts are better/more important than global prompts and can replace their position.

We would appreciate it if our explanation could address the review's concerns. If you have any other questions about the method or the experiment, reply to us and we will try our best to address your confusion.

[1] ID-like Prompt Learning for Few-Shot Out-of-Distribution Detection. CVPR 2024.

[2] Negative Label Guided OOD Detection with Pretrained Vision-Language Models. ICLR 2024.

[3] Deep Anomaly Detection with Outlier Exposure. ICLR 2019.

[4] Zero-Shot In-Distribution Detection in Multi-Object Settings Using Vision-Language Foundation Models.

2) NegLabel

Neglabel[2] designs a novel scheme for the OOD score with negative labels. It leverages real outlier information with negative labels from extensive corpus databases, which we discuss in line38-41. This kind of knowledge helps to a great extent pointed out by OE[3] and is inconsistent with real-world application, where negative categories are infinite. As we have no access to any true outlier information, the comparison is unfair. Moreover, while all of the mentioned works mainly focus on negative prompts, we highlight that we concentrate on the connection between global and local, and negative prompt is a small part of our method. By contrast, the advantage of our method is that it incorporates local information with pseudo local outliers and finds a novel way to take local information into both training and OOD score calculation. As the reviewer suggests, we provide comparison and analysis shown below:

We observe that 1) although without outlier, our model achieves better average performance against NegLabel, demonstrating the effectiveness of our local information refinement strategy; 2) NegLabel has an obvious overfitting in iNaturalist and SUN, which mainly consist of natural scenery. These datasets share different data distributions with images like Texture, which are detailed texture images. Consequently, our method achieves better balance between different kinds of OOD datasets, which strongly validates the effectiveness of incorporating local information and strengthens its application in diverse and infinite read-world scenarios.

We include these two relevant works in the revised version (line 426-447). We hope the comparison and analysis would help reviewer have a better understanding of the unique contributions and advancements of the proposed method.

We would like to emphasize that the core is to better utilize local outlier information to strengthen the ability of OOD detection, which has not been explored before, and some kind of negative prompts are just the way to achieve it. Moreover, our method can well adapt to all global prompt-based methods (see the experiments in Table.1 and above) and integrating with them achieves better performance, which strongly demonstrates the potential and generalization ability. Despite the similarity on the surface, the starting point is essentially different. We hope the analysis above can help the reviewer have a better understanding of the strengths and unique contributions of our paper. We are grateful that the reviewer points out the related work and we update in the revised manuscripts.

We thank the reviewer for the constructive comments and respond to them appropriately as follows.

Q1: Analysis of related works.

A1: As the reviewer suggests, we compare with related work ID-like prompt and NegPrompt and respectively show the detailed analysis.

1) ID-like

We conduct thorough comparison with ID-like[1]. First, from the perspective of motivation, our method differs from ID-like in that ID-like still focuses on global prompts optimization, with random crop generating ID-like samples. As it treats the whole image overall, it can not solve the problem of hard OOD samples with subtle differences in certain regions as described in line52-53. By contrast, our method optimizes local prompts and can well integrate with ID-like to further enhance the performance. Empirically, we compare OOD detection performance and showcase the results in the table below.

It can be concluded that: 1) our model achieves better average performance against ID-like, demonstrating the utility of our local information enhancement strategy; 2) ID-like obviously performs poorly in SUN and Places, which mainly consist of scenery with diverse scenarios; 3) integrating the global prompt into our model further enhances the performance as shown in the last row. We attribute it to the advantage that it brings well-trained global prompts that better tailor for OOD detection rather than the hand-crafted template "a photo of {class}". It is helpful to our method in distinguishing outlier samples that have overall-dominant features, further showcasing unique advantage of local prompts and the potential of local prompt optimization as an orthogonal direction for OOD detection.

We additionally showcase the examples that are not detected by previous global based method in Fig.6. By contrast, the subtle differences can be discriminated by our method, which further demonstrates the effectiveness of our method.

Dear reviewer dbE1:

We respectfully appreciate again for your insightful and thoughtful comments! As the suggestions from the reviewer, we give thorough explanation and update the manuscripts accordingly.

As it is approaching the end of the discussion period (November 26 at 11:59 pm AoE) and we do not receive feedback, we sincerely hope you could look through our response and have a further comment at your convenience if you have any questions about the paper. We will do our best to address the issues of the reviewer.

Best wishes,

Submission 6694 Authors.

Dear reviewer dbE1:

We sincerely appreciate again for taking the time to provide valuable suggestions for our work! As the suggestions from the reviewer, we give thorough comparison with the mentioned papers to demonstrate the strength of the paper both theoretically and empirically and and update the manuscripts accordingly (Sec.5.1). We additionally carry out experiments to validate the effectiveness of the proposed OOD metric (shown in the first-round discussion).

As it is approaching the end of the discussion period and we do not receive feedback, we are not sure if our responses address the concerns from the reviewer. We sincerely hope you could look through our response. We would be grateful if you could have a further comment at your convenience. If you have any questions about the paper, We will do our best to address them.

Best wishes,

Submission 6694 Authors

Dear Reviewer dbE1:

Thank you for taking the time to review our paper and propose constructive suggestion. Could you please kindly review our updated manuscript or previous comments? We have provided detailed responses to each question raised by the reviewer and we believe we have addressed all the concerns in our updated paper and previous responses. Thus, we would appreciate it if the reviewer could view the rebuttal and reevaluate the score for our paper. If the reviewer has more questions, we will try our best to address the problems.

Best wishes,

Submission 6694 Authors

Q4: Influence of the two hyperparameters.

A4: In Table.9, we aim to explain that the proposed method is not sensitive to the hyperparameter in a wide range, at least in the entire ablation process of our practical experiment (in our experiment 1-10 for $\lambda_{neg}$ and 0.1-1 for $\lambda_{reg}$). Empirically, we find that the ratio of the two corresponding coefficients should be within a wide range (approximately 2-10), and do not carefully select the hyperparameters. We use the two coefficients just for rigorous description of the entire training procedure in Eqn.8 and we highlight that our method still gets robust results without any coefficient parameter ( $\lambda_{neg}=1$, $\lambda_{reg}=1$ in table below) and outperforms previous methods like LoCoOp. We also carry out ablation study about $\lambda_{neg}$ to demonstrate the effectiveness of different modules.

Q5: Nonlinear change in FPR95.

A5: the attribute of nonlinear characteristics in few-shot learning has been widely observed in the fields of OOD detection[1]. For example, similar trend is observed in ID-like[2] (1-shot and 4-shot in Places shown in table below) and LoCoOp[1] (4 shot and 16 shot in SUN and Places shown in Table.1). The reason for the phenomenon could be: 1) the effectiveness of few-shot learning that effectively learns knowledge of downstream tasks especially in the very few samples; 2) Moreover, for few-shot OOD detection, more samples may not necessarily bring better performance, and incorporating more samples may confuse the discrimination, which should be the characteristic of the two datasets. In Table.8, the reason for the nonlinear change can partly be attributed to that our local-based model learns characteristics of OOD detection with efficiency and gets the best discrimination performance between ID and OOD in FPR95, and more samples just confuse the process.

[1] Few-Shot Out-of-Distribution Detection via Prompt Learning. NeurIPS 2023.

[2] ID-like Prompt Learning for Few-Shot Out-of-Distribution Detection. CVPR 2024.

We appreciate the reviewer for the valuable suggestions to our work. We reply to them sequentially and will complement in the revised version.

Q1: Standard deviation of metrics.

A1: All of the results are averaged for three runs and we do not observe obvious fluctuations as the variance shown in the table below. It showcases that the improvements are stable and do not attribute to the randomness. We speculate the reason is that background information of the images is rich enough to capture local outlier knowledge and thus achieve stable results, which are validated by visualization in Fig.4. Moreover, we would like to emphasize that few-shot OOD detection is challenging and the improvement is not small, which is shown in table in Q2 that our method obtains 22.04% (FPR95 )and 2.0% (AUROC) relative gain compared with LoCoOp.

Q2: Accuracy differences between w/ LNP and w/o LNP.

A2: We showcase the detailed results of w/ LNP and w/o LNP of 16 shot in the table below. We would like to highlight that the benefit margin is gradually decreasing and the improvement is actually not small. We take the improvements from 4-shot to 16-shot as a reference. Compared to w/o LNP, ours with LNP achieves 0.82 (2.62% relatively) and 0.32 (0.4% relatively) gain with respect to FPR and AUROC, respectively. By contrast, the improvements from 4 shot to 16 shot is 1.77 and 0.45, respectively as a reference. From the results, it can be seen that the improvement is not small, therefore demonstrating the effectiveness of the proposed LNP OOD metric.

Q3: Examples of OOD images.

A3: We appreciate the reviewer for detail and thorough understanding of the proposed method. As the reviewer suggests, we showcase several examples where global based method fails to detect hard OOD samples, and add them in the updated manuscript (line 525 and Fig.6). For example, in the first image, previous global-based method fails to detect the outlier as it is similar to ID category ocean liner on the whole, with only subtle difference mechanical devices on the deck indicating that it is actually an icebreaker (also demonstrated by the iceberg next to it). The same phenomenon are also observed other examples (the shape of the sunflower center in the second image and the calyx of the apple in the third image). From the examples, it is vividly illustrated in the example that global-based method focuses on overall representations and fails in hard outliers with subtle difference in regions. By contrast, our method enhances local information and successfully solves the problem by outlier knowledge, qualitatively showcasing the superiority and effectiveness of our method.

Dear reviewer NiJR:

We respectfully appreciate again for your insightful and thoughtful comments! As the suggestions from the reviewer, we give thorough explanation and update the manuscripts accordingly.

As it is approaching the end of the discussion period (November 26 at 11:59 pm AoE) and we do not receive feedback, we sincerely hope you could look through our response and have a further comment at your convenience if you have any questions about the paper. We will do our best to address the issues of the reviewer.

Best wishes,

Submission 6694 Authors.

Dear reviewer NiJR:

We would like to express our gratitude for your kind response and make further explanation about your concerns. We will explain them sequentially.

With respect to W3, we would like to emphasize that correctly classification with regional difference is just one of the reasons that cause wrong OOD detection results, and other type of wrong OOD detection should not be ignored. We understand the meaning of the reviewer that the OOD detection fails mainly because ocean liner and icebreaker both belong to "ship" in some way (kind reminder that ocean liner is acutally a wrong result because the icebreaker cannot serve as a passenger ship). In other words, previous method fails in this kind of outliers because they share similar global meaning.

However, it is not the only way that previous global-based OOD detection method fails. It also indeed comes from the misclassification where ID and OOD are also with subtle differences in certain regions. This kind of wrong detection is also challenging and by no means easier than the first type. For example, in the third example, the ID (Rose hip) is almost the same as OOD (Apple) and is hard to discriminate even from the perspective of local information. It should not be dismissed that the difficulty in discriminating them is as hard as the first type of example and that ID and OOD samples do not necessarily belong to similar parent class. Here we just want to cover more types of sources that cause wrong OOD detection results. We thank the reviewer's suggestion and will provide more visualizations in the updated manuscripts.

With the help of visualizations, we also would like to further illustrate the uniqueness of our method. The first example that the reviewer agrees to be persuasive just comes from the visualization of the original paper (we are unable to provide an external link but we sincerely wish the reviewer to look through it at the third example of Fig.7 on page 8). We believe it firmly demonstrates that the best global-based OOD detection method can only concentrate on outlier samples with overall differences and fails in hard samples that only have subtle differences in certain regions, whichi is the core issue that we aim to address.

Therefore, we would like to express that the core motivation and contribution of our method lies in the utilization of refine local information, and the training pipeline and design of loss function to achieve this goal, which is not explored before. The motivation and starting point of the two method is totally different, and the so-called "negative samples" is just the way to generate pseudo samples. Empirically, we adequately demonstrate that our local-based method achieves competitive results and further gains substantial improvements with no doubts. We sincerely hope that our explanation could give the reviewer a better understanding that 1) outlier information from local perspective and 2) extensibility to integrating global prompts to get further improvements is the core contribution of our paper. We will add more explanation of the comparison with the method in the updated manuscript.