Weakly-Supervised Contrastive Learning for Imprecise Class Labels

Dear reviewer LHXf:

Thanks for your valuable suggestions, we will try to address your concerns and we are eager to engage in a more detailed discussion with you.

W1: Lack of in-depth analyses of experiment results...

• We analyze the effectiveness of the proposed method in detail in the theoretical understanding of Section 3. It can be roughly understood in the following order:
  • Supervised information improves representation learning. Theorem 3.4 describes how much the quality of learned features can be improved when using accurate supervised information. It can be seen that the features learned by the graph constructed with supervised information usually have a smaller error upper bound than those learned by only self-connection.
  • Weakly supervised information can approximate supervised information. Through Theorem 3.7, we try to show that the features estimated using weakly supervised information can approximate the features obtained by supervised information to a certain extent. This theorem also indirectly illustrates how the proposed method works in weakly supervised scenario.
• Your proposal made us realize that there is a lack of some supplementary explanations between our theoretical analysis and experimental verification. We will emphasize the reasons why the model behind the theoretical results is valid in the experimental part when the revised version is uploaded.

W2: Some typos.

We are very sorry for this. We will check typos carefully and correct them in the next revision.

Q1: Question on hyperparameter settings.

The setting of the hyperparameters $\alpha$ and $\beta$ is related to the number of categories in the dataset. Denote $C$ is the number of class, setting the ratio of $\beta$ to $\alpha$ at $2C$ can achieve a promising performance in general.

Dear reviewer kufB:

Thank you for your valuable suggestions. We truly appreciate your review and are committed to addressing your concerns with care and attention. We sincerely look forward to engaging in a more in-depth discussion with you, as your insights are essential in helping us improve and refine our approach.

W1 & Q1: There's a discrepancy between the caption and the content of Table 1.

We are very sorry for this, this is a typo on our part, and we will correct the inconsistency between the caption and the table in the next revision.

W2 & Q2: In Table 7, PiCO outperforms the proposed method with a significant performance advantage under the partial ratio setting of 0.1 on the CUB-200 dataset. The 5.89% performance advantage is remarkable. It would be beneficial to provide an explanation for why the proposed method performs poorly in this scenario.

• In addition to using contrastive learning to learn similar representations for samples from the same class, PiCO also uses class prototype-based pseudo-labeling mechanism. The synergy between the contrastive learning and the above mechanism has been proven to be very effective in several fine-grained classification tasks [1].
• Our approach avoids the need for additional complex techniques. By incorporating our contrastive loss into the training objective, we achieve an 8.69% improvement over GFWS in this setting, highlighting the potential for even greater performance gains.
• Although no other complex techniques are used, our method achieves the best performance in most of the remaining experiments, which also shows the effectiveness of our method in most scenarios.

Reference:

[1] Partial label learning: Taxonomy, analysis and outlook. Neural Networks 161 (2023): 708-734.

Q3: Lack of the validation on instance-dependent data on PLL.

Following the works in [1, 2], we have expanded our experimental results to include the settings of biased tags associated with four fine-grained instance-dependent partial label datasets. The table below demonstrates that our method achieves marginally better performance than the current state-of-the-art (SOTA) approaches, and notably, without relying on any additional tricks or techniques. In the next version of our paper, we plan to incorporate more comprehensive comparative experiments to further validate our findings.

The experimental details are shown in the table below ($\mathrm{resnet34}^{*}$ denote resnet34 pre-trained on Imagenet)：

Q4: Only one state-of-the-art (SOTA) method published in 2024 is adopted for comparison, while the others were published in 2022 or earlier.

• Recent advances in partial label learning have primarily concentrated on instance-dependent partial label problems, exemplified by several notable methodologies: CEL [1], a class-wise embedding-guided disambiguation approach; DIRK [2], a self-distillation-based label disambiguation framework; NEPLL [3], which employs normalized entropy for sample selection; and IDGP [4], a generative method modeling candidate label generation processes.
• While these methods provide thorough investigations into label disambiguation mechanisms for instance-dependent scenarios, they notably neglect the critical exploration of leveraging ambiguous information for representation learning—a research gap addressed by our work. Comprehensive comparative experiments under instance-dependent partial label settings will be included in our subsequent manuscript version, accompanied by detailed discussions in related work sections.

Reference:

[1] Mixed Blessing: Class-Wise Embedding guided Instance-Dependent Partial Label Learning (KDD'2025)

[2] Distilling Reliable Knowledge for Instance-Dependent Partial Label Learning. (AAAI'2024)

[3] Candidate-aware Selective Disam-biguation Based On Normalized Entropy for Instance-dependent Partial-label Learning. (ICCV'23)

[4] Decompositional Generation Process for Instance-Dependent Partial Label Learning. (ICLR'2023)

Dear reviewer x5zp:

Thank you for your valuable suggestions. We truly appreciate your review and are committed to addressing your concerns with care and attention. We sincerely look forward to engaging in a more in-depth discussion with you, as your insights are essential in helping us improve and refine our approach.

W1 & Q1: It seems this paper relies on uniform class distribution for theoretical guarantees—how does this hold for imbalanced data?

(1) Practical Applicability to Imbalanced Data:

• Our Algorithm 2 explicitly handles imbalanced scenarios by estimating $\mathcal{L}_{wsc}$ without any class prior assumption ($\mathbb{P}(\boldsymbol{y})$). This ensures practical validity under arbitrary class distributions. Algorithm 1 and Proposition 2.3 are merely simplified special cases under class balance, not prerequisites for imbalance handling.

(2) Theoretical Scope:

• The theoretical guarantee of unbiased representation learning under weak supervision (as formalized in Theorem 3.7: "Learning from Weak Supervision Approximates Supervised Learning") is inherently class-prior-invariant, requiring no assumptions about label distribution. However, the downstream generalization analysis under supervised information Theorem 3.4 currently requires balanced assumptions for tractability. Hence, the adverse impacts of extreme long-tailed distributions on our framework remain unclear. We will conduct further research on the framework's application under extreme long-tailed conditions in future work.

W2: While t-SNE plots (Fig 2) suggest better representations, deeper qualitative insights are missing.

In addition to Figure 2, we also theoretically analyze that the proposed method helps improve the quality of representation learning features.

• Theorem 3.4 bounds linear probe error of feature derived from perturbation graph through properties of self-supervised connectivity graph and perturbation coefficients, and demonstrates that the perturbation graph increases the density of intra-class connections without adding extra-class connections.
• In Theorem 3.7, we further show that the features learned through finite samples and weakly supervised information can approximate the features learned from supervised information.
• Finally, we present the linear probe error of the features learned by our framework in Corollary 3.8, which can be regarded as a qualitative analysis of the learned features.

Q2: What’s the computational overhead of building/maintaining the similarity graph compared to standard contrastive learning?

Compared with standard contrastive learning, our additional computational overhead is mainly reflected in the single-step matrix multiplication used to construct semantic similarity, that is, $S((\tilde{x},\tilde{q}), (\tilde{x}',\tilde{q}'))$. This step of calculation does not bring too much additional computational cost and is negligible compared to standard contrastive learning. It can be considered that the computational overhead of our proposed method is basically the same as that of standard contrastive learning.

Q3: Are there scenarios where discrete labels would still outperform continuous similarity (e.g., clean labeled data)?

• The concept of continuous semantic similarity we proposed is designed for weakly supervised learning, and our method is very effective under weakly supervised information empirically.

• Theorem 3.7 explains the difference between the two features. The third and fourth terms on the right side of the inequality characterize the error induced by the usage of weakly supervised information, where the third term can be regarded as the estimated error term, and the fourth term describes the extent to which the weakly supervised information affects the learned representation. If the supervision information is completely accurate, it can be seen from the analysis that our proposed framework can be approximately equivalent to supervised contrastive learning.

Dear reviewer jhEr:

Thank you for your valuable suggestions. We truly appreciate your review and are committed to addressing your concerns with care and attention. We sincerely look forward to engaging in a more in-depth discussion with you, as your insights are essential in helping us improve and refine our approach.

W1. The paper does not provide sufficient evidence that shows its superiority in representation learning.

In addition to Figure 2, we also theoretically analyze that the proposed method helps improve the quality of representation learning features.

• Theorem 3.4 bounds linear probe error of feature derived from perturbation graph through properties of self-supervised connectivity graph and perturbation coefficients, and demonstrates that the perturbation graph increases the density of intra-class connections without adding extra-class connections.
• In Theorem 3.7, we further show that the features learned through finite samples and weakly supervised information can approximate the features learned from supervised information.
• Finally, we present the linear probe error of the features learned by our framework in Corollary 3.8, which can be regarded as a qualitative analysis of the learned features.

Q1. Can the proposed method be applied to weakly-supervised multi-label learning problems?

• The method we proposed is based on the semantic similarity between samples. The construction of semantic similarity depends on the transfer conditions approximately satisfied by each sample. It is not specially designed for multi-label classification tasks and cannot be simply applied to multi-label classification.

• We believe that by further expanding the concept of semantic similarity, it is possible to apply our method to weakly-supervised multi-label learning problems. We will explore this problem in future work.