Alleviating Label Shift Through Self-trained Intermediate Distribution: Theory and Algorithms

Thank you for your comments. For the first weakness, since our source dataset has the restriction of label distribution, we try to verify the performance of ALS on a larger dataset with a larger number of categories. However, due to time constraints, we did not obtain new results. But, we add MLLS, which is an advanced method for last two years, as a comparison method to illustrate ALS more effectively. For the second weakness, we directly adopt the code provided by the authors of the compared methods. Since the comparison method can be embedded in our method, the network architecture and hyperparameters are set according to the compared methods. We illustrate it in the paper for a clear illustration. For the third weakness, we use the parameter $\alpha$ to determine the label distribution. We choose $\alpha=1$ as the upper bound of the shift, because $\alpha=0.5$ may cause some classes to have zero data, which will break the algorithm. For the forth weakness, the new class can be seen as a special case of extreme imbalance. One of the directions for future research is how to correct the model in the case of extreme imbalance. This article focuses more on how to use the data in the target domain, and thus does not consider extreme imbalanced cases specially. For the first question, we adopt micro-Fscore for multi-class classification.

Thank you for your comments. For the first weakness, we analyze the time cost through experiments in subsection A.3.5 of the supplementary material and the experimental results show that our method does not have a particularly high time cost. Although if we pick more pseudo-labels, we will spend more time training the reweighted target classifier. However, we only select samples with the highest confidence that can maintain the label ratio in this paper, so it does not result in too much time cost. For the first question, we revise this paper in the following two parts. (1) In view of the grammar and symbol usage problems of the article, we again checked and revised it to ensure that there were no symbol misuse and syntax error problems. (2) We propose a new method to extend ALS with MLLS, which is an advanced method for last two years, and call it MLLS-ALS. Then we compare MLLS-ALS with MLLS, and the experimental results in subsection A.3.4 of the supplementary material show that MLLS-ALS achieves good performance in most cases.

Thank you for your comments. For the first weakness, as the reviewers said, ALS can be viewed as an additional trick for improving performance on most label shift methods. Thus we propose a new method to extend ALS with MLLS, which is an advanced method for last two years, and call it MLLS-ALS. Then we compare MLLS-ALS with MLLS, and the experimental results in subsection A.3.4 of the supplementary material show that MLLS-ALS achieves good performance in most cases. For the second weakness, the reason why we use BBSE and RLLS as basis methods is to fancy their nice theoretical properties, which can be combined with ALS to obtain Proposition 7. If the basis method, such as MLLS, has no theoretical guarantees, it is difficult for us to obtain the desired theoretical properties. While there are no theoretical guarantees for MLLS, experiments can be conducted to illustrate the performance of ALS. The experimental results is shown in subsection A.3.4 of the supplementary material. For the third weakness, we illustrate the meaning of ALS in the part of Introduction, i.e., 'our approach, abbreviated as ALS, learns a self-trained intermediate distribution...'. We used 'ideal' earlier to better distinguish between the true and pseudo-label-guided intermediate distributions. Borrowing from the reviewer's suggestion, we drop the 'ideal' for clearer presentation. In view of the grammar problems of the article, we have checked and revised it again. For the first question, we compare the proposed MLLS-ALS with MLLS in subsection A.3.4 of the supplementary material to show the effectiveness of ALS.

Thank you for your comments. For the first weakness, since these notations are explained by the cited literature and also used in Proposition 7, for reasons of space, we only explain these notations in the proof of the supplementary material. Taking into account the suggestions of the reviewers, we give instructions on the first use of these notations for a clearer illustration. For the second weakness, we analyze the time cost through experiments in subsection A.3.5 of the supplementary material and the experimental results show that our method does not have a particularly high time cost. Although if we pick more pseudo-labels, we will spend more time training the reweighted target classifier. However, we only select samples with the highest confidence that can maintain the label ratio in this paper, so it does not result in too much time cost. For the first question, we reinterpret these notations in the main text.