ReDebias: Exploring Residual energy based Debias learning

1. The writing in this manuscript presents significant challenges, particularly for readers who are not well-versed in the nuances of long-tailed distribution learning. For instance, in line L079-P2, the authors state, “we observe that Negative log-likelihood (NLL) loss, which heavily relies on softmax, struggles to distinguish correct from incorrect predictions when their predicted probabilities for the target class are similar.” However, this assertion leaves me perplexed. What exactly do the authors mean by this? Additionally, why does this occur? If such claims are central to the manuscript's argument, providing empirical evidence to substantiate them would greatly enhance the reader's understanding.

2. Similarly, in line L085-P2, the authors discuss connecting the long-tailed problem to causal inference by framing long-tailed datasets as instances of data that are Missing Not At Random (MNAR). However, the acronym MNAR is not sufficiently explained. While I interpret it as indicating that the missing data follows a certain distribution, the authors' articulation of MNAR as “only a subset of data is observable” is not clear and may lead to further confusion.

3. Another point of confusion arises when comparing the Residual-Energy score to established model calibration techniques. The authors fail to clearly articulate the distinct role of this score within the context of model calibration, which typically aims to align prediction confidence with posterior error rates. In line L090-P2, the authors describe the Residual-Energy score by stating, “where lower values indicate larger errors, and higher values reflect more accurate predictions.” However, this description does not effectively clarify how the Residual-Energy score differs from traditional model calibration methods or its unique contributions.

4. The manuscript’s motivation also raises questions about its significance. The authors suggest that previous methods, such as LGLA and DODA, demonstrate suboptimal performance in head classes, as illustrated in Figure 1. However, the differences in performance appear minimal, given that the Sacrificial Accuracy axis ranges from -1% to 1%, which may be seen as trivial. If my interpretation is correct, this diminishes the overall persuasive power of the motivation for the research.

5. Furthermore, any marginal improvement over previous works—such as the roughly 1% enhancement shown in Table 2—further reduces the perceived significance of this contribution. When considered alongside the limited advancements in performance metrics, the paper fails to convincingly establish itself as a significant advancement in the field.

6. Lack of conclusion section.

In summary, I find that the manuscript does not effectively convey its novelty or importance, primarily due to unclear writing and insufficient explanations of key concepts. The performance results also do not indicate a revolutionary contribution. As such, I recommend rejection of this submission.

1. It is unclear why the author chooses to use the Residual-energy score instead of the original energy score. The motivation for removing the logits of the target class is not discussed. If the Residual-energy score is indeed superior to the original energy score, an experimental comparison or theoretical analysis is necessary to substantiate this claim.

2. The original energy score includes a temperature parameter T, which is neglected in the Residual-energy score. If setting T = 1 is the best practice, at least a theoretical or empirical comparison is needed to validate this. If not, there may be room for improvement.

3. The paper claims in Section 3.2 that the energy score can reflect more sensitive changes than the softmax score, but it only provides a illustrative figure in Fig. 2, lacking a theoretically robust proof.

4. Hyper-parameter analysis in Appendix A. 4 merely lists a wall of numbers, lacking an analysis of the data. Additionally, the result for ViT-B/32 with 200 Epochs in Appendix Table 2 is missing, and the author does not explain why the table is incomplete.

5. The manuscript appears to have not been carefully proofread, containing various typos, including 'tipycal' at line 269, 'Finnally' at line 348, and 'traning from scratch' at line 373, etc.

W1 and W2 are my primary concerns.

1. The paper lacks sufficient innovation, and its motivation is not compelling. In Figure 2, the authors illustrate their approach using only two samples, which is unconvincing.

2. The paper lacks theoretical explanations; while the experiments show that Re-Debias outperforms other methods, there is insufficient clarification on why Re-Debias works effectively.

I have not found significative weakness in the paper except for some minor typos and errors. Following some of them:

• typo around line 250 (A highlighted)
• I guess Eq 9 contains a typo in the summation indexes (j \ne 1)
• typo around line 348 (Finnally)

The main weaknesses of this paper are editorial. Unfortunately, there are many typos in the manuscript, making it at times difficult to read and to thoroughly understand the contributions of the paper. These editorial issues lead to confusing statements. For instance, it is not clear to me whether the adjusted logits $g_y(x)$ should be $g_y(x) = f_y(x) + \log (C\cdot \pi_y)$ or $g_y(x) = f_y(x) - \log (C\cdot \pi_y)$. This is because in Equation 6, the $\arg\max$ is taken over $\exp (f_y(x) - \log (C\cdot \pi_y))$. The authors also claim that $C\cdot n$ is often unknown in the real world, yet the closed form for $\pi_s = \frac{n_c}{|\mathcal{O}|}$, the label frequency, is given and used throughout the manuscript. Finally, it is not clear to me why the residual energy score excludes always the first class. At least, from the definition given in Equation 9, $E(x, \bar{y}) = -\log \sum_{y\neq 1}^C e^{f_y(x)}$, this seems to be the case. Should $E(x, \bar{y})$ accounts for all the classes except the class of $y$ or what?