Concentration Distribution Learning from Label Distributions

Thank you for your time and effort on reviewing our paper. In what follows, we will address your questions in detail.

Experimental Designs Or Analyses 1: The traditional LDL experimental datasets may differ from real-world scenarios because the last dimension may not serve as the background and its generation assumption may not be consistent with the assumption proposed in this paper (Eq. (1)).

The paradigm of concentration distribution learning is a new concept, so there are no available CDL datasets. To this end, we construct the first real-world CDL dataset in our paper and conduct experiments on it to verify the effectiveness of our model of exvacating background concentration. And by hiding the last dimension of LDL datasets, we transform LDL datasets into CDL datasets. If our model can recover this hidden dimension, which is regarded as the background concentration, its ability to learn concentration distribution can be proved.

Experimental Designs Or Analyses 2: Can the authors elaborate more on the success of the proposed method on the CDL dataset? Also, it might be helpful to include more experimental results on more data points.

Thanks for your advice. In the final version of this paper, we will present more results on the constructed CDL dataset and some other data points.

Weakness 1 & 2: The writing of the paper needs to be improved. There are some typos and unclear expressions in the paper, and some equations seem to be redundant.

Thanks for pointing out our shortcomings. In the final version of our paper, we will carefully correct these errors and simplify our equations.

Weakness 3: The assumption of the background vector generation process is too simple, which may limit the applications in the real-world scenarios.

The paradigm of concentration distribution learning is a new concept raised by this paper. Since we have no further prior knowledge about the background concentration distribution pattern, the assumption that it is evenly spread across each class of the real label distribution is general. Furthermore, the experimental results also validate the rationality of this assumption.

Other Comments Or Suggestions 1: The writing of the paper can be improved and revised to be clearer.

Thanks for your advise. This paper will be carefully revised in its final version.

Other Comments Or Suggestions 2: More justification of the data generation model and its practicality in real-world applications should be given.

For the data generation model, please refer to Weakness 3 of this response. Additionally, in the final version of this paper, we will present more real-world applications of concentration distribution learning.

Other Comments Or Suggestions 3: Theoretical analysis of the proposed method can be done to improve the paper.

Please refer to the response of Weakness 1 to reviewer 3FJ2.

Question: Fig. 2 is not similar because the difference, i.e. the expression, may not correspond to the background. It is the main object of the image. Therefore, I am not sure that the core idea of the paper that the background of the image is important can be applied.

The "background concentration" is named after the background, but it is an abstract concept and does not actually refer to the background of the image. In the case of facial expression, the background concentration doesn't refer to any expression, and it represents the description degree of “no emotion”. In other words, the stronger the emotion is, the lower the background concentration it should be assigned.

Thank you for your precious suggestions. After careful consideration, our responses to the questions you mentioned are listed as follows.

**Claims And Evidence: The authors believe that in the existing label distribution learning, the labels in the label space are insufficient to describe the samples, which contradicts the basic assumption in the consensus definition of label distribution learning.

The assumption that "the labels in the label space can fully describe each sample" is correct in the cases of label distribution learning, which aims to learn the relative description level of each sample. However, in concentration distribution learning, we focus on learning the absolute description level. In these cases, the labels in the label space are insufficient to describe the samples because they overlook the existence of background concentration. In other words, CDL is an extension of LDL, which are not contradicted with each other.

Methods And Evaluation Criteria: I am rather skeptical about the significance of comparing the experiments in this paper, which involve label distribution learning with concentration terms, to traditional label distribution datasets that do not include concentration terms.

The paradigm of concentration distribution learning is a new concept, so there are no available CDL datasets. To this end, we construct the first real-world CDL dataset in our paper and conduct experiments on it to verify the effectiveness of our model of exvacating background concentration. And by hiding the last dimension of LDL datasets, we transform LDL datasets into CDL datasets. If our model can recover this hidden dimension, which is regarded as the background concentration, its ability to learn concentration distribution can be proved.

Experimental Designs Or Analyses: Would it be better to add a two-tailed t-test to verify the superiority or inferiority against each comparison algorithm?

Thank you for pointing out our shortcoming. The following is the result of the two-tailed t-test. In the significance level of $\alpha=0.01$, the null hypotheses that "there is no significant difference between the ranks of our proposed CDL-LD and the baseline algorithm" are all rejected, which means that the control model CDL-LD is significantly different to all the other baseline algorithms in rank. This result further proves the effectiveness of the proposed model.

Weakness 1: The real-world examples used by the authors to illustrate the role of concentration are inappropriate; both sets of examples could resolve the mismatch in descriptive strength by adding extra labels, which is not convincing.

The concept of background concentration cannot be replaced by simply adding extra labels. In Fig. 1(a) and (b), background concentration can be interpreted as the background of the image. However, it's hard to describe the background by adding labels like “mountain” or “water” since there are lots of contents in the background except mountain and water. Even if we make out all objects in the background, the label distribution of them is still costly and time-consuming to obtain. For other cases of Fig. 1(c), (d) and Fig.2, the meaning of background concentration is too abstract that it's impossible to be labeled. So we have no choice but to introduce the concept of background concentration.

Weakness 2: The validation of the algorithm’s performance should not be considered as one of the contributions of the entire paper.

Our contribution is mainly reflected in coming up with the concentration distribution learning paradigm, constructing the first real-world CDL dataset and proposing a model to learn CDs from LDL datasets. We will rearrange the part of contribution in the final version of this paper.

Weakness 3: No theoretical analysis or claims are provided to further validate the robustness and effectiveness of the proposal.

Please refer to the response of Weakness 1 to reviewer 3FJ2.

Other Comments Or Suggestions: The authors should use appropriate examples to match the methods of solving the problem with the issues and motivations they propose, and they should avoid giving the impression that contradicts the fundamental assumption of label distribution learning.

Please refer to Claims And Evidence and Weakness 1 of this response.

Questions For Authors: If an additional labels, such as “mountain” or “water,” is added to make the label set meet the condition of “being able to fully describe each sample,” does the issue discussed by the authors no longer exist in the example they used?

Please refer to Weakness 1 of this response.

Thanks for your valuable reviews, and your suggestions will effectively help us improve our work. Below are our responses to your questions.

Claims And Evidence: The authors claim that "excavating the background concentration makes full use of the information in the datasets and benefits the downstream tasks", however, the experiments are not conducted on several downstream tasks, I suggest the authors should be cautious when making such a statement.

Thank you for your advice. We think that in the experiments on the real-world CDL dataset, our model exvacates the hidden strength of emotions and helps us distinguish the images better, which can also be regarded as a downstream task. So we drew the conclusion that our proposed model can benefit the downstream tasks, which is reasonable.

Essential References Not Discussed: There is one essential reference that should be discussed, that is [1] "Label distribution changing learning with sample space expanding." Journal of Machine Learning Research 24.36 (2023): 1-48.

Thanks for pointing out our missing reference. We will add this to the final version of this paper.

Theoretical Claims: No theoretical claim is provided in this paper.

Please refer to the response of Weakness 1 to reviewer 3FJ2.

Weakness 2: Datasets used in the experiments are small, larger datasets should be considered.

Thank you for your advice. We carry out further experiments on the human gene dataset (with 17,892 instances). The results are listed as below, with the best result of each metric shown in boldface. The extensive experiment further proves the superiority of the proposed CDL-LD.

Question 1: Why do you choose the Dirichlet distribution(line 149 ), please give the reasons behind.

Dirichlet distribution is a probability distribution on a multidimensional space and the sum of its components is 1, which makes it suitable for representing the probability distribution of multiple mutually exclusive events. Label distributions represent the probability distribution of multiple labels which are mutually exclusive, so we choose Dirichlet distribution in this paper.

Question 2: In Eq. (1), assuming that the background concentration is evenly spread on each class of the real label distribution vector b in probability." Is this assumption reasonable?

The paradigm of concentration distribution learning is a new concept raised by this paper. Since we have no further prior knowledge about the background concentration distribution pattern, the assumption that it is evenly spread across each class of the real label distribution is general. Furthermore, the experimental results also validate the rationality of this assumption.

Question 3: Why define Eq.(10) in such a form? Can you make a detailed explanation?

$\vert\vert\boldsymbol{y}-\boldsymbol{p}\vert\vert_2^2$ is the MSE loss mentioned above in the paper, and $\frac{1}{B(\boldsymbol{\alpha})}\prod^c_{i=1}p_i^{\alpha_i-1}$ is the probability density function of Dirichlet distribution. Integrating the product of these two terms gives the final loss function of the neural network, which aims to minimize the average value of MSE loss over the whole Dirichlet distribution.

Thanks for your valuable reviews, and your suggestions will effectively help us improve our work. Below are our responses to your questions.

Claims And Evidence 1 & 2: Why does the paper assume that each $alpha\_i$ is composed of the sum of the dataset $e\_i$ and the background concentration $u\_i$, instead of treating the background as a whole?

The paradigm of concentration distribution learning is a new concept raised by this paper. Since we have no further prior knowledge about the background concentration distribution pattern, the assumption that it is evenly spread across each class of the real label distribution is general. Furthermore, the experimental results also validate the rationality of this assumption. By defining $\alpha_i=e_i+u_i$, we make the problem mathematically computable and finally draw the conclusion that all $u_i$ equal to one, which indicates that we should treat background concentrations as a whole. In other words, without initially dividing the background concentrations, the value of $u$ cannot be decided.

Methods And Evaluation Criteria 1: The paper uses MSE loss in a way that seems somewhat unreasonable. Have you tried using CE loss instead?

By applying MSE loss, the final loss function can be arranged in the form of Eq. (10) in the paper, which minimizes the predictive square error ($(y_i-\frac{\alpha_i}{S})^2$) and the variance of the Dirichlet distribution ($\frac{\alpha_i(S-\alpha_i)}{S^2(S+1)}$) simultaneously. Optimizing these two terms makes our experimental results more precise and stable. The CE loss, however, does not have this property.

Relation To Broader Scientific Literature: The paper studies label distribution learning, but only ancient references are cited.

Thank you for your pointing out our shortcoming. We will incorporate some recent works in the final version of our paper.

Disadvantage 1: Some of the theoretical assumptions in the paper lack proper analysis or proof.

Thank you for pointing out our shorcoming. We apply rademacher complexity to explain the theoretical bound of our proposed model.

Let $\mathcal{H}$ be a family of functions. For any $\delta>0$, with probability at least $1-\delta$, for all $h \in \mathcal{H}$ such that

$$\mathcal{L}(h) \leq \mathcal{L}\_{S}(h)+\widehat{\mathcal{R}}\_{S}(\mathcal{H})+3\sqrt{\frac{\log 2 / \delta}{2 n}},$$

where $\mathcal{L}(h)$ and $\mathcal{L}\_{S}(h)$ are the generalization risk and empirical risk with respective to h, and $\widehat{\mathcal{R}}\_{S}$ is the empirical rademacher complexity bounded by $(\mathcal{H})\leq\mathbb{E}\_{\boldsymbol{\sigma}}\left[ \frac{1}{n} \sum\_{i=1}^{n} \sigma\_{i} \mathcal{L}\_{AMSE}(\mathbf{\alpha}\_i) \right]$ with $\sigma\_{i}\in[0,1]$. $n$ represents the number of instances.

Because $\mathcal{L}\_{AMSE}(\mathbf{\alpha})=\sum^c\_{i=1}(y\_i-\frac{\alpha\_i}{S})^2+\frac{\alpha\_i(S-\alpha\_i)}{S^2(S+1)}>0$, we get

$$\mathcal{L}(h)- \mathcal{L}\_{S}(h) \leq \mathbb{E}\_{\boldsymbol{\sigma}}\left[ \frac{1}{n} \sum\_{i=1}^{n} \sigma\_{i} \mathcal{L}\_{AMSE}(\mathbf{\alpha}\_i) \right]+3\sqrt{\frac{\log 2 / \delta}{2 n}} \leq \frac{1}{n} \sum\_{i=1}^{n} \sum^c\_{j=1} \left[(y\_{ij}-\frac{\alpha\_{ij}}{S\_i})^2+\frac{\alpha\_{ij}(S\_i-\alpha\_{ij})}{S\_i^2(S\_i+1)}\right]+3\sqrt{\frac{\log 2 / \delta}{2 n}},$$

in which $c$ is the number of classes. We assume that the neural network gives $e>0$ for every instance, then $\forall i,j, \alpha\_{ij}>1, S\_i>c$. So we have

$$\mathcal{L}(h)- \mathcal{L}\_{S}(h) \leq \frac{1}{n} \sum\_{i=1}^{n} \sum^c\_{j=1} \left[(y\_{ij}-\frac{\alpha\_{ij}}{S\_i})^2+\frac{\frac{\alpha\_{ij}}{S\_i}(1-\frac{\alpha\_{ij}}{S\_i})}{S\_i(S\_i+1)}\right]+3\sqrt{\frac{\log 2 / \delta}{2 n}} \leq \frac{1}{n} \sum\_{i=1}^{n} \left[1+\frac{1}{4c(c+1)}\right]+3\sqrt{\frac{\log 2 / \delta}{2 n}}.$$

When the number of instances $n$ tends to infinity, the bound finally becomes $1+\frac{1}{4c(c+1)}$, which indicates that this bound shrinks when the number of classes $c$ increases. This conclusion is intuitive because the background concentration tends to zero when $c$ tends to infinity, degrading the CDL problem to learnable LDL problem.

Experimental Designs Or Analyses & Disadvantage 3: The paper provides a detailed introduction to the experimental design and analysis, but the datasets used seem relatively small. Have you tried testing on larger datasets? & The datasets used in the experimental section are relatively small, and no experiments have been conducted on larger datasets.

Thank you for your advice. We carry out further experiments on the human gene dataset (with 17,892 instances). The results are listed as in the response of Weakness 2 to Reviewer 7WaC due to the character limit, which further proves the superiority of the proposed CDL-LD.