Approximately Correct Label Distribution Learning

Many thanks for your precious comments! We have provided point-by-point responses to your questions below.

Comment 1: Some necessary justifications about the existing deep-rooted problems of LDL are needed. In Introduction: "For years, there are some deep-rooted problems in the field of LDL:" How to draw this conclusion? Combining with the related work (Section 6), More detailed elaboration is needed.

Response: The "deep-rooted problems" refer to the following two issues:

• Poor discriminability of traditional metrics: Early work (Geng, 2016) demonstrated that Clark and Canberra metrics suffer from oversensitivity to small values. Subsequent studies (Xu & Zhou, 2017) revealed KLD's unreliability with sparse predictions, prompting calls for alternative measures.

• Overfitting from minimizing average measurement: For example, AA-BP (Geng, 2016), a simple 3-layer network, minimizes MSE but underperforms due to overfitting. Current methods implicitly address this through ad-hoc regularization (Ren et al., 2019a; b; Jia et al., 2023a; b), indicating the field of LDL lacks principled solutions.

We will expand on these points in Section 6 to clarify the motivation for our work.

Comment 2: In Robustness testing, shown in Figure 4, why only other five methods were used to compare? In Table 2, the authors comapre the proposed δ-LDL with the existing other eleven methods.

Response: We selected representative methods from different performance tiers to maintain clarity in the visualization. Including all 11 methods would make the plot overcrowded and hard to interpret. Tables 2 & 3 provide the complete comparison for readers interested in detailed results.

Comment 3: In Table 2, the detailed experimental results on different datasets, there is no need to ranking the other eleven competitors, which reduces the readability of the paper.

Response: The ranking was intentionally included to demonstrate that while $\mu$ is derived from KLD, it produces meaningfully different evaluation outcomes than KLD alone. As discussed in Section 5.2, this highlights how $\mu$ addresses known limitations of traditional metrics. We will add a clearer explanatory note in the table caption to improve readability while maintaining this important comparative information.

Comment 4: Although the current experiments are good enough, it is more convinced if the authors can explain the reason why these seven datasets are used in the current experiments.

Response: The seven datasets were carefully selected to represent diverse real-world LDL applications across multiple domains: Aesthetic perception ($\mathtt{M}^{\mathtt{2}}\mathtt{B}$ & $\mathtt{fbp5500}$), facial emotion recognition ($\mathtt{RAF\\_ML}$ & $\mathtt{SBU\\_3DFE}$), multi-class image classification ($\mathtt{Natural\\_Scene}$), and artistic emotion perception ($\mathtt{Painting}$ & $\mathtt{Music}$). Due to the complexity of human subjective perception, these tasks particularly exhibit the characteristic of label polysemy - a crucial challenge LDL aims to address.

Comment 5: In the last paragraph of Introduction, the authors highlight the contributions of the paper. It looks more like contributions and organizational structure.

Response: Yes. We will revise the subsection title to better reflect its dual purpose.

Many thanks for your precious comments! We have provided point-by-point responses to your questions below.

Comment 1: Non-professional users may not intuitively understand how a specific distance or similarity value represents the closeness between the predicted and ground-truth label distributions.

Response: Traditional metrics reflect the closeness from different perspectives. For example, Euclidean distance measures geometric separation, while K-L divergence quantifies distributional differences. Our metric, i.e., Eq. (11), built upon these foundations, remains intuitive. It directly corresponds to an area ratio. Let us explain this better with Fig. 2 (b):

• Numerator of Eq. (11): The integral corresponds to the area under the curve, bounded by the gray line and axes.

• Denominator of Eq. (11): $\delta_0$ represents the area of the gray rectangle, reflecting the ideal model’s performance.

Comment 2: The paper also has some imperfections, such as the arbitrary naming of methods or metrics (e.g., $\mu$ and DeltaLDL), which does not reflect the characteristics of the methods and metrics. ... It is recommended to assign a name to the evaluation metric to more accurately reflect its characteristics, rather than using the Greek letter ($\mu$) directly.

Response: We appreciate this thoughtful suggestion. While we originally adopted Greek-letter notation $\mu$ for consistency with established metric conventions like Spearman's $\rho$ and Kendall's $\tau$, we agree that more descriptive names would better reflect their characteristics. We will provide the aliases as follows:

• $\mu$: improvement ratio (clearly indicating performance gain).

• DeltaLDL: AC-LDL (Approximately Correct LDL).

These new names better capture the methods' essential features while maintaining readability. Thank you for helping improve our paper's clarity.

Comment 3: In Equation (13) and Equation (19), it should be clarified whether the symbol $c$ is also included within the logarithmic operation.

Response: In both Equations (13) and (19), the parameter $c$ should indeed be included within the logarithmic operation. We will revise these equations to explicitly show this by adding parentheses.

Comment 4: Why should $\delta_0$ be defined as in Equation (12), and what advantages does this approach have compared to directly allowing users to pre-set a threshold?

Response: Our theoretical analysis illustrates that, $\delta_0$ reflects the worst-case divergence. Therefore, values larger than $\delta_0$ (for distance metrics) would imply tolerating worse-than-random errors, which doesn't make much sense. While smaller $\delta$ could be explored, they require strong assumptions about data training difficulty (e.g., label noise), i.e., we can't decide how small $\delta$ should be. Without such prior knowledge, $\delta_0$ provides a neutral starting point.

Allowing non-professional users to set thresholds does not provide any advantages. We provide results of $\mathfrak{D}(\text{KL}, \delta; f)$ w.r.t. $\delta$ on $\mathtt{SBU\\_3DFE}$ & $\mathtt{Natural\\_Scene}$, where $f$ always outputs a uniform label distribution matrix.

$\mathtt{SBU\\_3DFE}$:

$\mathtt{Natural\\_Scene}$:

Key findings: 1) $\mathfrak{D}(\text{KL},\delta;f_0)$ shows high sensitivity to $\delta$ changes (non-linear response); 2) optimal $\delta$ ranges vary significantly across datasets, which is not a mutually referable setting. This sensitivity highlights the dangers of ad-hoc parameter choices, justifying our pursuit of a parameter-free solution.

Many thanks for your precious comments! We have provided point-by-point responses to your questions below.

Comment 1: My main concern is whether the new metric proposed in the article can truly distinguish between superior and inferior models. In Eq. (11), there is a coefficient of 1/delta_0 before the integral. What does this coefficient mean? Why does the area under the curve (AUC) need to be multiplied by such a coefficient? Moreover, the smaller this coefficient is, the larger \mu becomes. This makes one wonder whether it is the subsequent integral, this coefficient, or the interaction between the two that plays a role in the measurement. I think this point needs further explanation.

Response: The metric $\mu$ is not a raw AUC but a ratio of areas. Let us explain this better with Fig. 2 (b):

• Numerator: The integral in Eq. (11) corresponds to the area under the curve, bounded by the gray line and axes.

• Denominator: $\delta_0$ represents the area of the gray rectangle, reflecting the ideal model’s performance.

Thus, $\frac{1}{\delta_0}$ normalizes the $\mu$ metric to [0, 1], ensuring fair comparison (no arbitrary scaling) between metrics. We will clarify this critical point in the manuscript.

Comment 2: Setting \delta_0 as the expected KL divergence between the label distribution and the vector v is somewhat heuristic. It is recommended to give more explanations or conduct more experiments for exploration.

Response: Our theoretical analysis illustrates that, $\delta_0$ reflects the worst-case divergence. Therefore, values larger than $\delta_0$ (for distance metrics) would imply tolerating worse-than-random errors, which doesn't make much sense. While smaller $\delta$ could be explored, they require strong assumptions about data training difficulty (e.g., label noise), i.e., we can't decide how small $\delta$ should be. Without such prior knowledge, $\delta_0$ provides a neutral starting point. We provide results of $\mathfrak{D}(\text{KL}, \delta; f)$ w.r.t. $\delta$ on $\mathtt{SBU\\_3DFE}$ & $\mathtt{Natural\\_Scene}$, where $f$ always outputs a uniform label distribution matrix.

$\mathtt{SBU\\_3DFE}$:

$\mathtt{Natural\\_Scene}$:

Key findings: 1) $\mathfrak{D}(\text{KL},\delta;f_0)$ shows high sensitivity to $\delta$ changes (non-linear response); 2) optimal $\delta$ ranges vary significantly across datasets, which is not a mutually referable setting. This sensitivity highlights the dangers of ad-hoc parameter choices, justifying our pursuit of a parameter-free solution.

Comment 3: How can we determine which samples are difficult to learn? Perhaps it would be best to have a warm-up process, or there could be a process of gradually changing the hyperparameter \delta.

Response: The identification of difficult-to-learn samples is gradually facilitated by the "ReLU + margin" mechanism during training, since initializing $\delta$ to $\delta_0$ is sufficiently inclusive, allowing the model to first learn from easier patterns. While we prioritize simplicity and avoid introducing additional hyperparameters, we acknowledge that a warm-up strategy or adaptive scheduling for $\delta$ could be explored in future work.

Comment 4: In Remark 2.3, please give some explanations that why the closed-form solution of Eq. (8) does not exist.

Response: The partial derivative of Equation (8) is given by:

$$\frac{\partial \ell_{\text{MLE}}}{\partial \alpha_k} = -m \left( \Phi \left( \sum_{j=1}^{c} \alpha_j \right) - \Phi (\alpha_k) \right) - \sum_{i=1}^{m} \ln d_{\boldsymbol{x}_i}^{d_k}\text{,}$$

where $\Phi(x) = \frac{\mathrm{d} \ln \Gamma(x)}{ \mathrm{d} \Gamma(x)}$ is the digamma function. The equation exhibits highly nonlinear behavior and implicitly forms a globally coupled nonlinear system, making a closed-form solution for Equation (8) intractable.

Many thanks for your comprehensive comments! Responses to your concerns are as follows.

Response about $\delta$: Our theoretical analysis illustrates that, $\delta_0$ reflects the worst-case divergence. Therefore, values larger than $\delta_0$ (for distance metrics) would imply tolerating worse-than-random errors, which doesn't make much sense. While smaller $\delta$ could be explored, they require strong assumptions about data training difficulty (e.g., label noise), i.e., we can't decide how small $\delta$ should be. Without such prior knowledge, $\delta_0$ provides a neutral starting point. We provide results of $\mathfrak{D}(\text{KL}, \delta; f)$ w.r.t. $\delta$ on $\mathtt{SBU\\_3DFE}$ & $\mathtt{Natural\\_Scene}$, where $f$ always outputs a uniform label distribution matrix.

$\mathtt{SBU\\_3DFE}$:

$\mathtt{Natural\\_Scene}$:

Key findings: 1) $\mathfrak{D}(\text{KL},\delta;f_0)$ shows high sensitivity to $\delta$ changes (non-linear response); 2) optimal $\delta$ ranges vary significantly across datasets, which is not a mutually referable setting. This sensitivity highlights the dangers of ad-hoc parameter choices, justifying our pursuit of a parameter-free solution.

Response about $\varepsilon$: The tolerance parameter $\varepsilon$ & maximum recursion depth $\xi$ both ultimately control iteration limits. However, $\varepsilon$ is usually set as a fixed small value (1e-7) following numerical analysis conventions. We instead focuse on more adjustable $\xi$, analyzed in Fig. 2 (a).

Response about downstream tasks: We appreciate this constructive suggestion, but directly applying LDL to downstream tasks risks objective mismatch (W. & G., '19; '21b). However, we can conduct additional vision experiments in the context of pure LDL.

Setup:

• Dataset: $\mathtt{JAFFE}$ (256 $\times$ 256 facial images).
• Arch.: ResNet-50 backbone.
• Training: 10-fold CV, Adam optimizer (lr=1e-4, bs=32, 100 epochs).
• Baseline: AA-BP (G., '16) with the same configuration.

Results (bold = better):

$\delta$-LDL achieves better ($\uparrow$0.73% $\mu$) or comparable results.

Response about the complexity: We provide runtime comparisons ($\delta$-LDL vs. baselines on $\mathtt{SBU\\_3DFE}$). Results:

Runtimes are normalized to $\delta$-LDL’s execution time (1.x) for direct comparison. $\delta$-LDL shows similar training times to SCL/LDLF methods. Note that such comparisons may lack fairness since PT-Bayes, AA-$k$NN, LDLSF, SA-BFGS & LCLR do not employ deep learning architectures in their implementations.

Response about the assumption: While we use Dir. distributions for theoretical analysis due to their mathematical tractability and natural fit for probability simplex, theoretical assumptions are relaxed in implementation. Precisely because of the consideration of deviations in real-world scenarios, our method does not strictly require this assumption in practice; our method is empirically robust to distribution-free label structures, as demonstrated in experiments.

Response about the applicability: We appreciate the suggestion to explore broader applicability. However, the continuous nature of label distributions differs fundamentally from the discrete logic of multi-label annotations, making DeltaLDL and its derivatives ($\mu$ & $\delta$-LDL) more suited to pure LDL tasks. Let‘s discuss some examples:

• Some MLL evaluation metrics, such as subset accuracy, inherently involve discretization in their computation. These metrics must be calculated on mini-batch test sets and are unsuitable as optimization objectives, making them irrelevant to the core problems we aim to address.
• However, metrics like Hamming/Jaccard could potentially be applicable. When approximating $\delta_0$ for these cases, one would need to substitute uniform vectors with a random binary matrix and modify the margin mechanism for discrete outputs.

Multi-label metrics often prioritize accuracy over distributional fidelity, conflicting with LDL’s paradigm. This presents an interesting and meaningful direction for future work.