Balancing Positive and Negative Classification Error Rates in Positive-Unlabeled Learning

Q1. Lack of sensitivity analysis of introduced parameters.

Thank you for your comment. We conduct comprehensive parameter sensitivity analysis. For the parameter $\tau$, our experiments show optimal performance at $2\times10^{-3}$ across most datasets, which aligns with our theoretical analysis that moderate penalty parameters achieve balance between constraint strength and optimization stability. For the parameter $\beta$, we find that the range $[0.4,0.5]$ performs optimally across different datasets, which is significant because $\beta$ controls the update speed of dynamic lower bound. Too small value of $\beta$ leads to overly loose constraints losing the balancing effect, while too large value of $\beta$ causes optimization instability. Our experiments demonstrate that DC-PU exhibits relative robustness to these hyperparameters, with parameter variations within reasonable ranges not significantly affecting performance, further validating the practicality of our method.

Q2. Lack sufficient analysis of the theoretical results.

Thank you for your valuable suggestion. Theorem 3.3 establishes the fundamental generalization bound that explains DC-PU's superior performance through four distinct components: two Rademacher complexity terms $\mathfrak{R}\_{n_u,p}(\mathcal{G})$ and $\mathfrak{R}\_{n_p,p_p}(\mathcal{G})$, a sampling fluctuation term $\chi_{n_p,n_u}$, and a distribution discrepancy term $\Delta$.

The asymptotic convergence properties. Under standard assumptions, as $n_p, n_u \to \infty$, two Rademacher complexity terms gradually decrease and converge to zero, and so does the sampling fluctuation term $\chi_{n_p,n_u}$. The bias term $\Delta$ is independent of sample size and can be treated as a constant. Consequently, as $n\to\infty$, $R(\widehat{g}_{\mathrm{DC-PU}}) \to R(g^*)$.

The convergence rates. The convergence rates of the first and second terms are governed by the Rademacher complexities $\mathfrak{R}\_{n_u,p}(\mathcal{G})$ and $\mathfrak{R}\_{n_p,p_p}(\mathcal{G})$, corresponding to rates of $O(1/\sqrt{n_p})$ and $O(1/\sqrt{n_u})$, respectively. The fluctuation term $\chi_{n_p,n_u}$ contributes an additional rate of $O(1/\sqrt{n_p}+1/\sqrt{n_u})$. Consequently, the overall convergence rate is characterized by $O\left(\max(1/\sqrt{n_p},1/\sqrt{n_u})\right)$. It indicates that DC-PU achieves minimax optimal convergence while maintaining balanced performance, contrasting with methods that optimize overall accuracy at the expense of class balance.

Additionally, the exponential decay rate of bias probability provides crucial insight into DC-PU's stability advantages. It shows that when classes have natural difficulty separation $\alpha-\beta > 0$, our constraint mechanism provides exponentially strong concentration guarantees. This explains why DC-PU performs particularly well on datasets with inherent class differences.

The sampling fluctuation term. The sampling fluctuation term $\chi_{n_p,n_u} = 2\pi/\sqrt{n_p} + 1/\sqrt{n_u}$ provides explicit mathematical guidance for experimental design. This term shows that when positive class prior $\pi$ is small, positive sample size $n_p$ becomes more critical for maintaining tight bounds, while unlabeled sample size $n_u$ provides consistent benefit regardless of class imbalance.

The distribution discrepancy term. The distribution discrepancy term $\Delta$ represents the controlled bias introduced by our constraint mechanism. Unlike uncontrolled bias in traditional methods, this term decreases exponentially with probability, ensuring that our method maintains consistency while providing stability benefits. It explicitly connects sample sizes, class difficulty separation, and theoretical guarantees: $\Delta \leq \pi' C_\ell \exp(-2(1-\pi)^2(\alpha-\beta)^2/C_\ell^2 \cdot 1/(\pi^2/n_p + 1/n_u))$

Q3. Is the weak constraint still necessary?

Thank you for your comments. We argue that the weak constraint cannot be considered as being incorporated within the strong constraint.

First, since the expected risk is generally intractable, empirical risk is commonly used as its approximation. Compared to the strong constraint, which enforces strict equality between the empirical risks of the positive and negative classes, and thereby introduces potential estimation error. The weak constraint imposes only a lower bound on the negative class risk, which makes it a more tolerant and robust form of estimation under approximation.

Second, from a convergence perspective, using the strong constraint alone tends to lead to rapid convergence and overfitting in the early stages of training. In contrast, the weak constraint contributes to a more stable training process and helps mitigate overfitting by introducing a lower-bound formulation.

Finally, ablation studies indicate that the weak constraint can sometimes achieve better performance than the strong constraint, indicating that the two constraints are complementary in practice.

Q4. Why the method outperforms baselines in "last" but not in "max" on certain datasets?

Thank you for the suggestion. This phenomenon actually demonstrates the strength of our method rather than a limitation, as the "max" performance represents peak instantaneous performance that may include overfitting to particular batches, while "last" performance reflects stable, converged performance under full constraint enforcement that is more representative of true generalization capability. For example, the results of FOPU* on Alzheimer show that it tends to overfit in later stages while it can find a strong model during training, resulting in lower average performance in the final epochs. This indicates that the model trained by FOPU* is less stable than ours.

Thanks for the author's reply, I tend to keep the original score.

Q1. How these theoretical results translate into the improved performance observed in experiments?

Thank you for your suggestion. Our theoretical framework provides direct mathematical explanations for the experimental phenomena.

The training stability. Our bias analysis in Lemma 3.1 reveals that DC-PU's positive bias occurs with exponentially decaying probability, which directly explains the improved training stability observed in our experiments. While uPU suffers from severe overfitting because $\widehat{R}\_u^-(g)-\pi \widehat{R}\_p^-(g)$ tends to be less than 0 during model training, and non-negativity constraint of nnPU leads to the negative-class risk estimator frequently approaching zero, DC-PU's dual-constraint mechanism prevents both problems with high probability. The asymptotic convergence properties further illuminate this advantage that the generalization error bound in Theorem 3.3 contains four major terms, including two Rademacher complexity terms $\mathfrak{R}\_{n_u,p}(\mathcal{G})$ and $\mathfrak{R}\_{n_p,p_p}(\mathcal{G})$, a sampling fluctuation term $\chi_{n_p,n_u}$, and a distribution discrepancy term $\Delta$. Under standard assumptions, as $n_p, n_u \to \infty$, the Rademacher complexity terms and sampling fluctuation term converge to zero while the bias term $\Delta$ remains constant, consequently ensuring $R(\widehat{g}_{\mathrm{DC-PU}}) \to R(g^*)$. The convergence rates analysis shows that the overall rate is characterized by $O(\max(1/\sqrt{n_p}, 1/\sqrt{n_u}))$, which explains the smooth and consistent learning curves we observe in Figures 3 and 4.

The generalization. The estimation error bound in Theorem 3.3 provides explicit guidance on how our design choices improve generalization performance. The controlled bias term ensures that DC-PU doesn't suffer from the systematic over-play of negative-class risk. This theoretical framework explains why DC-PU consistently achieves superior "last" performance metrics in Tables 2 and 3. As the exponentially small bias probability ensures robust behavior regardless of specific dataset characteristics.

The GAP. The explicit equality constraint $\hat{R}_p^+(g) = \frac{\hat{R}_u^-(g) - \pi\hat{R}_p^-(g)}{1-\pi}$ in our DC-PU theoretically guarantees that the empirical risks of positive and negative classes remain close throughout training. This mathematical constraint directly translates into the consistently low GAP values shown in Figure 2, where DC-PU maintains balanced error rates across all datasets.

Q2. Lack the choice of other hyperparameters.

Thank you for your comment. We conduct comprehensive parameter sensitivity analysis. For the parameter $\tau$, our experiments show optimal performance at $2\times10^{-3}$ across most datasets, which aligns with our theoretical analysis that moderate penalty parameters achieve balance between constraint strength and optimization stability. For the parameter $\beta$, we find that the range $[0.4,0.5]$ performs optimally across different datasets, which is significant because $\beta$ controls the update speed of dynamic lower bound. Too small value of $\beta$ leads to overly loose constraints losing the balancing effect, while too large value of $\beta$ causes optimization instability. Our experiments demonstrate that DC-PU exhibits relative robustness to these hyperparameters, with parameter variations within reasonable ranges not significantly affecting performance, further validating the practicality of our method.

Q3. How DC-PU applies to the recent dataset DIABETES?

Thank you for your comment. We add the experimental results on the DIABETES dataset, as shown below. The results on DIABETES dataset provide compelling evidence for the effectiveness of our method in real-world tabular data scenarios. It demonstrates consistent performance throughout training and indicates that DC-PU maintains balanced performance across both classes, which is crucial for real-world medical applications. The most significant finding lies in the GAP metrics: 4.78±0.009 (min) and 10.12±0.009 (last), substantially lower than competing methods like uPU and nnPU. This dramatic reduction in classification error imbalance demonstrates that our dual-constraint mechanism effectively addresses the fundamental challenge of imbalanced error rates even in complex tabular data with intricate feature interactions. The consistent low GAP values across all metrics confirm that our mathematical constraints translate effectively to practical performance improvements in scenarios.

Q1. Correction of the proof of Lemma 3.1.

Thank you for pointing out this error. It begins with a mistake in which we wrote the upper bound condition of $\widehat{R}\_p^+(g)$, i.e., $0 \le \sup_{g \in \mathcal{G}} \widehat{R}\_p^+(g) \le \beta$, as a lower bound condition in the current manuscript. When applying the correct condition $0 \le \sup_{g \in \mathcal{G}} \widehat{R}\_p^+(g) \le \beta$, the inequality $(1-\pi)R\_n^-(g)-\bigl(\widehat{R}\_u^-(g) - \pi \widehat{R}\_p^-(g)\bigr)\ge (1-\pi)(\alpha-\beta)$ is valid.

This correction actually strengthens our theoretical analysis by demonstrating that the separation between the true negative class risk and our estimator depends on the gap $\alpha-\beta$ between the lower bounds of negative and positive class risks. The lemma shows that our bias probability bound $P(D_\omega^-(g)) \leq \exp(-2(1-\pi)^2(\alpha-\beta)^2/C_\ell^2 \cdot 1/(\pi^2/n_p + 1/n_u))$ has an exponential decay rate that explicitly depends on this separation. When $\alpha > \beta$, the exponential decay becomes stronger, providing better theoretical guarantees. It explains why DC-PU performs particularly well on datasets with natural class difficulty differences, as our constraint mechanism automatically adapts to these inherent class characteristics while maintaining theoretical consistency.

Q2. The paper lacks discussion of the relationship between flooding.

Thank you for your suggestion. Flooding uses a fixed constant flood level $\lambda$ where the loss becomes $\max\\{\ell(f(x), y)-\lambda,0\\}$, creating a uniform lower bound across all samples and classes. It addresses general overfitting by preventing zero loss but lacks class-specific considerations essential for PU learning scenarios. In contrast, our DC-PU employs a dynamic lower bound, creating a class-aware constraint that automatically adapts to the actual positive class risk. The exponential moving average ensures stability while allowing adaptation, contrasting sharply with flooding's static threshold.

More critically, our explicit equality constraint $\widehat{R}_p^+(g) = \frac{\widehat{R}_u^-(g) - \pi \widehat{R}_p^-(g)}{1-\pi}$ introduces a mathematical relationship between classes that flooding cannot capture.

Overall, DC-PU ensures that class-level risks remain proportionally balanced, addressing the fundamental challenge of PU learning where class imbalance emerges from the absence of labeled negative samples rather than general training instability.

Q3. There are several typos.

Thank you for your corrections. We will revise them in the next version.

Q4. Theorem 3.3 lacks discussion.

Thank you for your comment. Theorem 3.3 establishes the fundamental generalization bound that explains DC-PU's superior performance through four distinct components: two Rademacher complexity terms $\mathfrak{R}\_{n_u,p}(\mathcal{G})$ and $\mathfrak{R}\_{n_p,p_p}(\mathcal{G})$, a sampling fluctuation term $\chi_{n_p,n_u}$, and a distribution discrepancy term $\Delta$.

The asymptotic convergence properties. Under standard assumptions, as $n_p, n_u \to \infty$, two Rademacher complexity terms gradually decrease and converge to zero, and so does the sampling fluctuation term $\chi_{n_p,n_u}$. The bias term $\Delta$ is independent of sample size and can be treated as a constant. Consequently, as $n\to\infty$, $R(\widehat{g}_{\mathrm{DC-PU}}) \to R(g^*)$.

The convergence rates. The convergence rates of the first and second terms are governed by the Rademacher complexities $\mathfrak{R}\_{n_u,p}(\mathcal{G})$ and $\mathfrak{R}\_{n_p,p_p}(\mathcal{G})$, corresponding to rates of $O(1/\sqrt{n_p})$ and $O(1/\sqrt{n_u})$, respectively. The fluctuation term $\chi_{n_p,n_u}$ contributes an additional rate of $O(1/\sqrt{n_p}+1/\sqrt{n_u})$. Consequently, the overall convergence rate is characterized by $O\left(\max(1/\sqrt{n_p},1/\sqrt{n_u})\right)$. It indicates that DC-PU achieves minimax optimal convergence while maintaining balanced performance, contrasting with methods that optimize overall accuracy at the expense of class balance.

Additionally, the exponential decay rate of bias probability provides crucial insight into DC-PU's stability advantages. It shows that when classes have natural difficulty separation $\alpha-\beta > 0$, our constraint mechanism provides exponentially strong concentration guarantees. This explains why DC-PU performs particularly well on datasets with inherent class differences.

The sampling fluctuation term. The sampling fluctuation term $\chi_{n_p,n_u} = 2\pi/\sqrt{n_p} + 1/\sqrt{n_u}$ provides explicit mathematical guidance for experimental design. This term shows that when positive class prior $\pi$ is small, positive sample size $n_p$ becomes more critical for maintaining tight bounds, while unlabeled sample size $n_u$ provides consistent benefit regardless of class imbalance.

The distribution discrepancy term. The distribution discrepancy term $\Delta$ represents the controlled bias introduced by our constraint mechanism. Unlike uncontrolled bias in traditional methods, this term decreases exponentially with probability, ensuring that our method maintains consistency while providing stability benefits. It explicitly connects sample sizes, class difficulty separation, and theoretical guarantees: $\Delta \leq \pi' C_\ell \exp(-2(1-\pi)^2(\alpha-\beta)^2/C_\ell^2 \cdot 1/(\pi^2/n_p + 1/n_u))$

Q1. The theoretical exposition is insufficient.

Thank you for your valuable suggestion. Our theoretical framework addresses a fundamental challenge in PU learning where existing methods suffer from the imbalanced error rates between positive and negative classes. The risk decomposition in Eq.3 reveals the core mathematical insight: under the SCAR assumption, we can express the negative class risk as $R_n^-(g) = \frac{R_u^-(g) - \pi R_p^-(g)}{1-\pi}$, which allows us to estimate negative class performance without labeled negative samples. However, this unbiased estimator $\widehat{R}_u^-(g) - \pi \widehat{R}_p^-(g)$ frequently becomes negative during training, leading existing methods like nnPU to truncate it at zero, resulting in the negative class being over-played.

Our dual-constraint mechanism in Eq.8 fundamentally transforms this problem through two innovations. The dynamic lower bound $\omega = (1-\pi)\widehat{R}_p^+(g)$ automatically adapts to the current positive class performance, creating a balanced constraint that evolves with the model's learning progress. The explicit equality constraint $\widehat{R}_p^+(g) = \frac{\widehat{R}_u^-(g) - \pi \widehat{R}_p^-(g)}{1-\pi}$ mathematically enforces that the empirical positive and negative class risks remain equivalent throughout training, directly addressing the class imbalance problem at its theoretical root. The mathematical elegance lies in how these constraints work together: while the dynamic bound prevents degenerate solutions, the equality constraint ensures that neither class dominates the optimization landscape. This theoretical foundation explains why DC-PU achieves consistently low GAP values across most experimental settings, as the constraints translate directly into balanced error rates between positive and negative classes.

Q2. The legends are incorrectly labeled.

We sincerely apologize for this critical labeling error in figures. We have corrected all figure legends to properly reflect "DC-PU" as our method name and will ensure this correction is implemented in the next version.

Q3. The proposed method relies on the SCAR assumption, defining a boundary condition for the method's applicability.

Thank you for your suggestion. This is indeed a fundamental limitation that we should acknowledge more prominently in our main text. As the SCAR assumption can indeed be violated in real-world scenarios, we still believe our method provides robustness beyond strict SCAR assumption.

Firstly, the dual-constraint mechanism inherently provides stability against moderate SCAR violations by preventing extreme class imbalance that could arise from distribution shift, while our dynamic lower bound adapts to the actual observed data distribution rather than relying solely on theoretical distributional assumptions.

Furthermore, our evaluation on the real-world Alzheimer dataset represents a realistic medical diagnosis scenario where SCAR assumptions are likely violated to some degree, yet DC-PU still demonstrates clear improvements in balanced error rates and overall performance. We will expand our limitations discussion in the next version to explicitly address scenarios where SCAR may not hold.

Q4. The experiments are conducted on multi-class datasets that have been converted into binary PU tasks.

Thank you for your comment. In fact, in addition to the artificially converted datasets, our evaluation also includes the real-world Alzheimer dataset, which represents a genuine medical diagnosis scenario with inherent positive-unlabeled structure rather than artificial conversion. The experimental results are presented in Tables 3 and 4 of our paper. Furthermore, we are committed to evaluating on additional real-world PU datasets and will include evaluation on the DIABETES dataset from TableShift as suggested by Reviewer k4vm.

Dear Reviewer,

Thank you for submitting the Mandatory Acknowledgement. However, I noticed there is no visible engagement in the discussion thread. We kindly encourage you to engage with the authors, and clarify any outstanding issues.

Thanks a lot.

Best regards,

AC