ReweightOOD: Loss Reweighting for Distance-based OOD Detection

We thank reviewer for their presented concerns which we address them below.

It's important to note that the primary focus is to demonstrate the effectiveness of our proposed method in OOD detection. Furthermore, we study the reweighting approach’s efficacy for the first time in OOD detection. Such work in the domain has been missing, which makes our work novel in OOD detection.

We perform the comparison of our method with various approaches well suited for OOD detection. The key takeaway lies in the superior performance of our approach compared to competitive approaches like SupCon, CSI, and CIDER. Moreover, as presented in the paper, our work demonstrates the distinctively remarkable performance in comparison to strong post-hoc methods too. As requested by reviewer XxBj, we haver updated the manuscript with additional posthoc methods.

The hyperparameters are determined from the validation set.

The sensitivity study of these hyperparameters using ResNet18 network with CIFAR100 datasets is presented below.

We thank the reviewer for the suggestions. We address the concerns below.

• We will clarify the improvement relative to the SoTA approaches by being more specific in the abstract in the upcoming versions. We believe the performance improvement compared to the current competitive approaches such as SupCon and CIDER is still significant.

• Orthogonal to the papers presented in CVPR2023, our work has the distinction in that it adopts a reweighting approach which is novel to the best of our knowledge.

• The ImageNet-1k experiments adopting the ImageNet100 setup is presented below:

  Method	iNaturalist	SUN	Textures	Places	SSB Hard	Ninco	Openimage	Average FPR $\downarrow$
  SupCon	61.93	68.85	25.43	75.28	90.79	78.82	55.64	65.25
  CIDER	53.54	61.27	23.95	68.76	91.52	79.15	54.22	61.77
  (ReweightOOD) Ours	50.70	59.46	24.27	66.27	90.18	76.76	50.97	59.80

  It can be observed that the proposed approach has noticeable gains over SupCon and CIDER demonstrating its superiority.

We thank the reviewer for the concrete suggestions. We address the presented concerns below:

• The updated manuscript now contains the suggested comparisons and we do cite all the mentioned works in the manuscript.

• The experiments with the Imagenet-1k dataset following the setup of the ImageNet-100 experiment are shown below.

  Method	iNaturalist	SUN	Textures	Places	SSB Hard	Ninco	Openimage	Average FPR $\downarrow$
  SupCon	61.93	68.85	25.43	75.28	90.79	78.82	55.64	65.25
  CIDER	53.54	61.27	23.95	68.76	91.52	79.15	54.22	61.77
  (ReweightOOD) Ours	50.70	59.46	24.27	66.27	90.18	76.76	50.97	59.80

  It can be observed that the proposed approach has noticeable gains over SupCon and CIDER demonstrating its superiority.

• We thank the reviewer for the possible further extensions. We aim to undertake such an approach in future work.

• As can be observed from the above ImageNet-1k experiments, our approach demonstrates performance gain even when scaled to an even more challenging (higher number of categories) setting.

We thank Review NpEj for the presented concerns. We address the concerns below.

• We have resolved the issue in the latest manuscript by referencing Figure 3 and Figure 4.

• Apologies for the overleaf rendering issues, the correctly rendered visualization is updated in the manuscript.

• Thanks for noticing the space consideration. We'll be mindful of it in the subsequent version.

• Thanks for pointing it out. Adopting the training setup of ImageNet100, we present the requested comparison with ImageNet-1k below:

  Method	iNaturalist	SUN	Textures	Places	SSB Hard	Ninco	Openimage	Average FPR $\downarrow$
  SupCon	61.93	68.85	25.43	75.28	90.79	78.82	55.64	65.25
  CIDER	53.54	61.27	23.95	68.76	91.52	79.15	54.22	61.77
  (ReweightOOD) Ours	50.70	59.46	24.27	66.27	90.18	76.76	50.97	59.80

  It can be observed that the proposed approach has noticeable gains over SupCon and CIDER demonstrating its superiority.

We thank Reviewer 4EKT for raising insightful concerns and queries. We address them below.

A detailed clarification of the distinction between our proposed approach against CurricularFace and AdaFace is presented below.

• CurricularFace adopts a curriculum-style learning approach, wherein early/late training dynamics are taken into account. It emphasizes easy samples at the earlier stage while prioritizing the hard samples at the later stage. ReweightOOD does not impose curriculum-style learning at all, as it does not distinguish between early and late training phases. The reweighting function of ReweightOOD, in the form of a sigmoid, strategically suppresses easy samples from the outset, directing attention toward challenging samples. AdaFace, another proxy-based approach, adopts the quality of the image -- approximated by feature norm -- into the weighting of the sample's importance. In AdaFace, hard samples from higher-quality images receive increased weighting, while easy samples from lower-quality images are weighted higher. In contrast, our approach focuses solely on the cosine similarity when reweighting pair instances. Moreover, ReweightOOD facilitates distinct treatment of positive and negative pairs due to the presence of a transformation function with separate hyperparameters. The utilization of two sets of linear transformations, specific to positive and negative pairs, enables flexible weighting tailored to achieving optimal OOD detection. Additionally, both curricularFace and adaFace adopt cross-entropy-based optimization while ReweightOOD uses similarity maximization/minimization optimization as shown in equation (3).

Regarding the experiments, we address the queries below:

• The result of ImageNet100 including CIDER is presented below:-

  Method	iNaturalist	SUN	Textures	Places	SSB Hard	Ninco	Openimage	Average FPR $\downarrow$
  Baseline	3.07	2.39	4.57	5.47	35.39	29.15	7.05	12.44
  SupCon	2.43	1.98	2.59	5.43	34.25	25.58	5.28	11.08
  CIDER	5.07	1.85	1.77	5.70	37.35	27.93	5.73	12.20
  (ReweightOOD) Ours	2.18	1.97	2.73	5.29	32.00	24.63	5.06	10.55

  As can be observed from the above Table, our approach yields the minimum average FPR of 10.55.

• The updated MES Table with the results of CIDER is given below:

  Method	Apples	Aquarium Fish	Baby	Bear	Beaver	Bed	Bee	Beetle	Bicycle	Bottles	...	Mean
  Baseline	1.09	1.11	1.07	0.93	0.97	1.07	1.03	1.07	1.10	1.07	...	1.05
  SupCon	0.97	1.01	0.97	1.00	1.00	0.98	0.99	0.97	1.04	1.03	...	1.01
  CIDER	0.96	0.94	0.89	0.91	0.86	0.88	0.87	0.88	1.04	0.99	...	0.92
  (ReweightOOD) Ours	0.95	0.90	0.90	0.89	0.84	0.90	0.89	0.91	0.99	0.96	...	0.90

  As evidenced from the above Table, the average MES of ReweightOOD is minimum among all. Moreover, the class centroid dispersion of ReweightOOD (0.63) is comparable to that of CIDER (0.64) despite not explicitly encouraging the inter-class dispersion.

• The random hyperparameter optimization is employed with respect to the validation set (gaussian noise). The hyperparameter set to (5, −4, 2, 1) consistently demonstrates strong performance in experiments conducted on both CIFAR100 and ImageNet100 datasets, across all architectures used in the paper.

• Apologies for the overleaf rendering issues, the correctly rendered visualization is updated in the manuscript.