Towards Calibrated Deep Clustering Network

Question 3: Loss in Equation 4

Equation 4 is the sum of the two individual losses, $L_{cal}$ and $L_{en}$. There is no additional hyperparameter for weighting the two losses because they are designed to operate on similar scales. Specifically, $L_{cal}$ denotes the average entropy per sample, while $L_{en}$represents the negative entropy of the average prediction across all samples.

The use of the negative entropy loss $L_{en}$ is inspired by prior works on deep clustering methods (e.g., CC, SCAN, SPICE and SeCu). It helps avoid assigning all samples to a single cluster, promoting a more balanced clustering structure.

Following your suggestions, we conduct an ablation study that varies the scale of the coefficient $w_{en}$ for $L_{en}$, i.e., $L=L_{cal}+w_{en} L_{en}$. The results presented in Table Q3 below. The findings indicate that the proposed model is robust to $w_{en}$, so we fixed it at $w_{en} = 1$ for simplicity in the experiments. Moreover, removing the negative entropy loss $L_{en}$ will degrade the clustering performance, especially on CIFAR-20.

Table Q3: Influence of coefficient $w_{en}$ on clustering accuracy (ACC %) and expected calibration error (ECE %) across STL-10 and CIFAR-20 datasets.

Question 4: Out-of-Distribution (OOD) Behavior

Thank you for your question, which is crucial for evaluating our method's ability to withstand distribution shifts. Following your suggestion, we have conducted additional experiments. The specific settings and results are as follows:

Settings

• Datasets: We employ CIFAR-10 and CIFAR-20 as in-distribution (ID) datasets when mixing data from unrelated datasets. We use Near-OOD datasets (CIFAR-20 and Tiny ImageNet for CIFAR-10; CIFAR-10 and Tiny ImageNet for CIFAR-20) and Far-OOD datasets (SVHN, Textures, Places365).
• Procedure: For the methods of CC, SCAN, SPICE, SeCu, and CDC (Ours), we use max softmax scores (MSP) of ID and OOD samples to assess the model's performance on AUROC and FPR95 metrics.

Results: As shown in Table Q4-1 and Table Q4-2 below, our method demonstrates superior OOD performance compared to comparative methods, indicating the significant role of calibration in rejecting unknown samples. For instance, CDC trained on CIFAR-10 achieved a 2.9% increase in AUROC and a 19.5% decrease in FPR95 compared to the most effective baseline method.

We also plot the density-confidence in Figure 11 of the revised paper to illustrate that the OOD regions of our method are more concentrated in areas of low confidence, demonstrating enhanced separation ability between in-distribution and OOD samples.

Thank you for your valuable comments.

Weakness 1: Improvement for Paper Structure and Organization

Thank you for your constructive suggestions regarding the structure of the paper. We have reorganized the paper in the revised version. Specifically, in Section 3, we first give an overview on the dual-head framework , and introduce the calibration head and clustering head in sequence. Then we introduce the initialization strategy, the overall training scheme and prediction method.

Furthermore, we have moved an ablation study of "Mini-cluster Number K" from the appendix B.5 of the original paper into the main body of the revised paper to provide a comprehensive evaluation.

We also need to point out that the initialization strategy is important or our work. As evidenced in Figure 4 of the original paper, the initialization technique we proposed promotes the overall performance and stability of the dual-head network.

Weakness 2: Inconsistent and Ambiguous Notation

The flowchart in Figure 2 of the original paper can provide a more intuitive explanation. We will add formal expressions in the revised version:

• We will modify the notation section (Line 141-147) as: $f (\varTheta; x_i)$ be the feature extractor, $g(θ_{clu};\cdot)$ and $g(θ_{cal};\cdot)$ be the clustering head and the calibration head, $\sigma(\cdot)$ be the softmax function.
• New definitions of strong augmentation function $\mathcal {A} (\cdot)$and weak augmentation function $\mathcal {W} (\cdot)$.
• In Line 230 and Line 235, the clustering head and the calibration head respectively predict un-augmented samples as: $p_i^{clu} = \sigma(g(\theta_{clu}; f(\varTheta; x_i)) )$ and $p_i^{cal} = \sigma(g(\theta_{cal}; f(\varTheta; x_i)))$.
• In Line 303, the calibration head provides predictions for weakly augmented samples: $p_i^{w cal} = \sigma(g(\theta_{cal}; f(\varTheta; \mathcal{W}(x_i))))$.
• In Line 323, the clustering head provides predictions for strongly augmented samples: $p_i^{s clu} = \sigma(g(\theta_{clu}; f(\varTheta; \mathcal{A}(x_i))))$.

These changes will make the methodology clearer and more accessible to readers.

Weakness 3: Title of Section 3

We will change the title of Section 3 to "Proposed Method" as suggested.

Question 1: Clarification of 'classifier g(θ,⋅)'

The classifier $g(θ;\cdot)$ indeed refers to both the clustering head $g(θ_{clu};\cdot)$ and the calibration head $g(θ_{cal};\cdot)$. Its role is to project the feature embeddings $f (\varTheta; x_i)$ into the respective output spaces, specifically, $g(θ_{clu};f (\varTheta; x_i))$ and $g(θ_{cal};f (\varTheta; x_i))$ for the clustering head and calibration head respectively.

We agree simply using $g(θ;\cdot)$ is unclear. We have replaced $g(θ;\cdot)$ with $g(θ_{clu};\cdot)$ and $g(θ_{cal};\cdot)$ in the revised paper, including in the Notation section and Algorithm 1.

The function pipeline of our network is: $x_i$ (input) $\rightarrow$ $f (\varTheta; x_i)$ (backbone) $\rightarrow$ $g(θ_{clu};f (\varTheta; x_i))$ (clustering head) and $g(θ_{cal};f (\varTheta; x_i))$ (calibration head).

Question 2: Prediction Algorithm

As we mentioned in “Final Prediction” (subsection 3.4) in Line 333-334 of the original paper, the final prediction of i-th sample $x_i$ is given by the calibration head. We will add the formula to clarify the presentation, i.e., $p_i^{cal} = \sigma(g(θ_{cal}; f(\varTheta; x_i)))$.

Table Q4-1: The AUROC (%) and FPR95 (%) of various deep clustering methods trained on CIFAR-10 for out-of-distribution (OOD) detection.

Table Q4-2: The AUROC (%) and FPR95 (%) of various deep clustering methods trained on CIFAR-20 for out-of-distribution (OOD) detection.

Dear Reviewer 1qud,

Thank you once again for your time and efforts in reviewing our paper. We appreciate your valuable feedback. Other reviewers have indicated that their concerns have been addressed and have acknowledged our contributions. Although you mentioned that "I am satisfied with the revised manuscript and appreciate the detailed experiment results", we noticed that your score is still not positive. Thus, we would like to know if there are any additional concerns we can address. We would be more than happy to provide further responses.

We appreciate your consideration.

Best regards,

The authors

While no specific questions were raised, we remain open to any suggestions or clarifications that would help improve the presentation and impact of the work. Thank you for your constructive feedback and for taking the time to review our paper.

Dear Reviewer KQtX,

Thank you for dedicating your time and effort to reviewing our manuscript. Initially, we noted that you did not provide any comments in the Weaknesses and Questions section. However, we observed that you quietly adjusted the rating from 6 to 5 without indicating any specific issues, which left us feeling perplexed.

We have noticed that you have now included a comment in the Weaknesses section stating, "According to the discussion between the authors and other reviewers, I altered my rating." We kindly request that you please elaborate on your specific concerns. As in the current discussion period, all the other reviewers say their concerns have been addressed, and some have raised their scores accordingly. For example, both Reviewer yP9a and Review bpNw raised their score from 6 to 8.

It is through constructive feedback and clear communication that each of us can contribute effectively as professional reviewers, thus enhancing the quality of our community.

Thank you.

Table W2-1: The clustering performance ACC, NMI, ARI (mean±std %) and calibration error ECE (mean±std %) of various deep clustering methods trained on CIFAR-10, CIFAR-20 and STL-10.

Table W2-2: The clustering performance ACC, NMI, ARI (mean±std %) and calibration error ECE (mean±std %) of various deep clustering methods trained on ImageNet-10, ImageNet-Dogs and Tiny-ImageNet.

Thank you for your valuable comments.

Weakness 1 & Question 1: Hyperparameter Selection and Sensitivity
The influences of K are shown in Appendix B.5 of the original paper. On the whole, we selected hyperparameters B, K, and H based on empirical evaluation. Specifically:

Batch size and Number of mini-clusters (B, K): As demonstrated in Table W1-1 below, our model exhibits considerable robustness to variations within a reasonable range for B and K, especially on challenging datasets such as CIFAR-20. Variations of K by ±20% and B by ±50% only result in the changes in overall accuracy of ±1.4%.

Table W1-1: Influence of (B, K) on clustering accuracy (ACC %) and expected calibration error (ECE %) across STL-10 and CIFAR-20 datasets.

• Insight for B: We constrain GPU memory usage to within 9GB. For datasets with smaller image size, larger batch sizes are utilized (e.g., B=1000 for CIFAR-10 and CIFAR-20, which have image sizes of 32x32). Conversely, smaller batch sizes are employed for datasets with larger image sizes (e.g., B=500 for ImageNet-10 and ImageNet-Dogs, which have image sizes of 224x224).
• Insight for K: K is tuned empirically according to the complexity of the dataset. For challenging datasets with over-confident predictions, a lower K value is needed to increase the confidence penalty, with an empirical ratio of $B/K \le5$. Conversely, for simpler datasets, a higher K value is necessary to reduce the confidence penalty, supported by an empirical ratio of $B/K \ge10$. More analysis on the selection of K can be found in Sec. 4.3 of the revised paper.

Hidden units (H): The choice of 512-dim is commonly used in MLP, and provides a good balance between initialization complexity and performance, as Table W1-2 shows empirically.

Table W1-2: Influence of MLP Hidden layer size on clustering accuracy (ACC %) and expected calibration error (ECE %) across STL-10 and CIFAR-20 datasets.

Weakness 2 & Question 1: Variability in Results Across Multiple Runs

The experimental results in Table 2 of the original paper were obtained from a single run under a random seed of 1024. We have depicted the training process, including variance, for both our method and comparative methods in Figure 4 of the original paper. To further analyze the performance of the models across all datasets, we conducted three runs with different random seeds, and the results are presented in the Table W2-1 and Table W2-2 below.

From the Table W2-1 and Table W2-2, we can observe that our method has much smaller variance than most compared methods, while at the same time, produces the best clustering performance and calibration performance. This is because the compared methods usually employ the random initialization on the clustering head, making model training unstable. While our method proposes a novel initialization strategy, utilizing feature prototypes from the pretrained model to initialize the clustering head and the calibration head. This approach significantly reduces the variance in our method's performance as evidenced in Figure 4 of the original paper.

The results and analyses has been added in Appendix C.2 of the revised paper.

Weakness 3: Clarification of Abstract Claim Regarding 10x Improvement in ECE
Thanks for your suggestions, we will revise this statement to "by 5x on average" to avoid exaggeration.

Question 2: Subscripts in Theorem 1
Yes, the subscripts should be switched as follows:

• $\mathbb{E}\_F[Conf\_{cal}] \leq \mathbb{E}\_F[Conf\_{clu}]$
• $\mathbb{E}\_T[Conf\_{cal}] = \mathbb{E}\_T[Conf\_{clu}]$

Table W1-1: The clustering performance ACC, NMI, ARI (mean±std %) and calibration error ECE (mean±std %) of various deep clustering methods trained on CIFAR-10, CIFAR-20 and STL-10.

Table W1-2: The clustering performance ACC, NMI, ARI (mean±std %) and calibration error ECE (mean±std %) of various deep clustering methods trained on ImageNet-10, ImageNet-Dogs and Tiny-ImageNet.

Thank you for your valuable comments.

Weakness 1 & Question 1: Reporting of Experimental Results

The experimental results in Table 2 of the original paper were obtained from a single run under a random seed of 1024. We have depicted the training process, including variance, for both our method and comparative methods in Figure 4 of the original paper. To further analyze the performance of the models across all datasets, we conducted three runs with different random seeds, and the results are presented in the Table W1-1 and Table W1-2 below.

From the Table W1-1 and Table W1-2, we can observe that our method has much smaller variance than most compared methods, while at the same time, produces the best clustering performance and calibration performance. This is because the compared methods usually employ the random initialization on the clustering head, making model training unstable. While our method proposes a novel initialization strategy, utilizing feature prototypes from the pretrained model to initialize the clustering head and the calibration head. This approach significantly reduces the variance in our method's performance as evidenced in Figure 4 of the original paper.

The results and analyses has been added in Appendix C.2 of the revised paper.

Weakness 2 & Question 2: Hyperparameter Sensitivity and Selection of $K$

Insight for K: The influences of min-clusters K on the performance have been shown in the Appendix B.5 of the original paper. We have the following insight to tune the value of K. For challenging datasets with over-confident predictions, a lower K value is needed to increase the confidence penalty, with an empirical ratio of $B/K \le5$. Conversely, for simpler datasets, a higher K value is necessary to reduce the confidence penalty, supported by an empirical ratio of $B/K \ge10$.

See more analysis in Appendix B.5 of the original paper.

Batch size and Number of mini-clusters (B, K): As demonstrated in Table W2 below, our model exhibits considerable robustness to variations within a reasonable range for B and K, especially on challenging datasets such as CIFAR-20. Variations of K by ±20% and B by ±50% only result in the changes in overall accuracy of ±1.4%.

Table W2: Influence of (B, K) on clustering accuracy (ACC %) and expected calibration error (ECE %) across STL-10 and CIFAR-20 datasets.

Weakness 3 & Question 3 & Question 4: MoCo-v2 Pretraining Details

Details on how to train the backbone are provided in Appendix "B.1 IMPLEMENTATION DETAILS" (Lines 854-878) of the original paper. Moreover, the code has been submitted to the Supplementary Material, where all the configurations are specified.

MoCo-v2 Hyperparameters (Lines 854-865): The Appendix B.1 details settings applicable to all datasets used in this study. We also provide parameter script files for all training stages across six datasets in the code files, facilitating reproducibility. Furthermore, we plan to open-source both the MoCo-v2 pre-trained models and the trained models.

Usage of MoCo-v2 (Line 871): We clarify that MoCo-v2 pre-training is utilized in the initial phase of the CC, SCAN, and CDC (ours) methods but not in SeCu due to its distinct training approach.

Usage of Backbone (Lines 858, 866-869, 871-876): Unless otherwise specified, ResNet-34 is employed as the backbone across all stages of the CC, SCAN, SeCu, and CDC (ours), as well as in the comparative methods.

In the revised paper, we have listed the parameter settings in Appendix B.1 to enhance the readability and reproducibility of the paper.

Question 5: General Applicability of CDC as a Self-Labeling Stage Following your suggestions, we applied CDC as a post-clustering self-labeling stage to models like SeCu and SCAN. The results presented in Table Q5 demonstrate that incorporating CDC following SCAN and SeCu not only enhances clustering accuracy but also significantly decreases the Expected Calibration Error (ECE). This proves the general applicability of our method. Thanks again for your suggestions.

Table Q5: Changes in clustering performance metrics (ACC, NMI, ARI %) and Expected Calibration Error (ECE %) after 100 epochs of training across the CIFAR-10 and CIFAR-20 datasets.

Weakness 4: Insights from Dynamic Thresholding in SSL

Table W4-2: Clustering performance ACC, NMI, ARI(%) and calibration error ECE (%) of Semi-Supervised Learning methods integrated into CDC on STL-10 and CIFAR-20. Sel. represents the proportion of selected samples.

Integration:

• Settings. Consistent with the previous settings, only the threshold selection strategy of CDC is replaced by FlexMatch and FreeMatch.

• Analysis for Table W4-2. CDC outperforms FlexMatch and FreeMatch on both datasets. The reasons are as follows. Under the predictions of a well-calibrated model, FlexMatch requires a dataset-specific global threshold. When the global threshold for STL-10 and CIFAR-20 is set to 0.95 as recommended by [d], it results in significantly different sample selection scales (91.2% in STL-10 and 3.2% in CIFAR-20), affecting performance improvements. FreeMatch is more robust, similar to CDC, with both methods being free from manually setting thresholds. The difference between FreeMatch and CDC on CIFAR-20 lies in the handling of head classes (classes with a large number of selected samples):

  • FreeMatch uses local threshold adjustments to modify the overall threshold (the average of the maximum probability values for all samples), with local thresholds after maxNorm ranging from 0 to 1. Thus, the thresholds for head classes are less than the overall threshold. Accordingly, FreeMatch will introduce more samples for these classes (and more wrong pseudo-labels).
  • CDC directly selects reliable samples based on calibrated probabilities for each class, ensuring that the thresholds for head classes are not reduced by the overall threshold.

• Selected Samples Analysis in Figure 10.

  Figure 10 of the revised paper shows the category-specific view of selection preferences on CIFAR-20, where FreeMatch's lower thresholds in high-confidence areas introduce more wrong pseudo-labels, impacting performance improvements.

  Specifically, Among the top 5 classes with the highest number of correct predictions, FreeMatch selectes 12,671 samples with an accuracy of 89.8%, while CDC selectes 10,602 samples with an accuracy of 94.9%. In the 5 classes with the lowest number of correct predictions, FreeMatch selectes 5,833 samples with an accuracy of 38.87%, whereas CDC selectes 6,011 samples with an accuracy of 44.52%.

Given that the semi-supervised learning also encounters unlabeled samples, the applicability of semi-supervised methods to deep clustering warrants further investigation. Thanks again for your suggestion.

Weakness 4: Insights from Dynamic Thresholding in SSL

Thank you for pointing out relevant work such as FlexMatch [d], which proposes dynamic and class-wise thresholds for pseudo-label selection. Additionally, we incorporate subsequent developments like FreeMatch[R2], which adapts the predefined fixed global threshold of FlexMatch to be adaptive based on the learning status. We explore the application of the thresholding strategies from FlexMatch and FreeMatch in our CDC framework by two approaches (Apply Directly and Integration).

[R2] FreeMatch: Self-adaptive Thresholding for Semi-supervised Learning. ICLR, 2023.

Table W4-1: Clustering performance ACC, NMI, ARI(%) and calibration error ECE (%) of Semi-Supervised Learning methods applied directly to self-labeling on STL-10 and CIFAR-20. Sel. represents the proportion of selected samples.

Apply Directly:

• Settings. Consistent with CDC, after pre-training with MoCo-v2 and performing prototype-based initialization, we utilize pseudo-label learning, evaluating the learning state based on the model's output.
• Analysis for Table W4-1. CDC outperforms FlexMatch and FreeMatch on both datasets. The reasons are as follows. FlexMatch and FreeMatch rely on the prediction confidence estimated by a well-calibrated model, which is reasonable in semi-supervised learning (SSL) scenarios where some labels are known. In deep clustering, the model updates thresholds based on over-confident predictions, which can lead to the selection of more noisy labels, causing performance degradation—this effect is more pronounced in challenging datasets like CIFAR-20. Specifically, selecting 80.2% of the overall samples corresponds to an accuracy of 54.9% in FlexMatch, selecting 90.8% of the overall samples also corresponds to an accuracy of 48.8% in FreeMatch, whereas selecting 55.2% of the overall samples corresponds to an accuracy of 61.7% in CDC.

Weakness 3: Enhancing Self-Training with SSL Insights

We appreciate the suggestion to incorporate insights from semi-supervised learning (SSL) methods such as [c]. The mentioned work [c] explores more moderately confident samples via Gradient Synchronization Filter (GSF) and Prototype Proximity Filter (PPF) to boost learning, which is conceptually aligned with our approach to pseudo-labeling.

We applied this method to expand our selected sample set, and the experimental results are presented in Table W3 below.

Table W3: Clustering Performance (ACC, NMI, ARI %) on STL-10 and CIFAR-20.

From the above table, we find that GSF and PPF can only bring marginal improvements in performance. The reason is the highly overlap in sample selection between the CDC's strategy and those selected by GSF and PPF, the expansion of selected samples is less than 5%.

Nevertheless, related techniques hold potential to further enhance our model, which we intend to explore thoroughly in the future.

Weakness 3: Impact of Self Training Methods on Calibration

Calibration is employed to measure the gap between confidence and accuracy. Compared to supervised learning, Self-Training exhibits a more pronounced overconfidence—overfitting some inevitable incorrect pseudo labels—thereby increasing calibration errors, as evidenced in Fig.1 (a)-(c) of the original paper.

In deep clustering, self-supervised methods increase confidence, and at the same time introduce substantial supervised information, leading to improvements in accuracy. Changes in the Expected Calibration Error (ECE) are influenced by both confidence and accuracy. Our experiments reveal that Self-Training degrades calibration in two scenarios, exemplified by SCAN-3 (applying self-labeling to SCAN-2):

1. Accuracy increases slower than confidence, as demonstrated on CIFAR-20 (ACC +1.2%, CONF +3.1%, ECE +1.9%).
2. Accuracy decreases (due to fitting too many noisy labels), as shown on Tiny-ImageNet (ACC -1.8%, CONF +19.5%, ECE +21.4%).

However, improvements in ECE are also observed in some cases, as shown on CIFAR-10 (ACC +6.2%, CONF +2.0%, ECE -4.2%), where the overall confidence increase is limited (CONF 95.0%→97.0%).

The key to model calibration is narrowing the gap between confidence and accuracy. Our explored techniques include:

• Regularization loss (targeting confidence only). The methods based on regularization loss reduce overall confidence to improve calibration, where effective calibration does not enhance performance (as evidenced in Section 4.1 "Competitive Failure Rejection Ability" of the original paper).
• Our approach (targeting both confidence and accuracy). Our method controls the increase in confidence while maintaining accurate predictions in reliable regions, achieving simultaneous improvements in performance and calibration ability.

Therefore, calibration methods should consider relationship between confidence and accuracy to achieve improvements in model performance.

Thank you for your valuable comments.

Weakness 1: Comprehensive Literature Review

We appreciate the reviewer’s suggestion to expand the literature review and include discussions of related works addressing the over-confidence issue in supervised learning and unsupervised domain adaptation. The suggested references [a] and [b] are indeed relevant, and we will incorporate them into the revised paper.

[a] Revisiting the Calibration of Modern Neural Networks (NeurIPS, 2021): This paper examines calibration in supervised learning and provides a comprehensive empirical analyses of modern neural networks, exploring how architectural and training techniques impact calibration. While [a] primarily addresses calibration in supervised learning, our work extends the concept to unsupervised clustering, where calibration is inherently more challenging due to the absence of labeled data.

[b] Transferable Calibration with Lower Bias and Variance in Domain Adaptation (NeurIPS, 2020): This study tackles calibration in the domain adaptation setting, where labeled source data is available. [b] leverages labeled source data to achieve calibration, while our method operates in a fully unsupervised setting. We will highlight these distinctions while addressing the overconfidence issue through targeted calibration techniques.

These discussions will contextualize our contributions within existing researches and clarify the novelty of our approach.

Weakness 2: Determining the Model’s Learning Status in an Unsupervised Context

In our approach, the calibration head estimates the learning status by analyzing the confidence distribution of predictions given by the clustering head. Specifically:

• Intra cluster level. The calibration head are learned based on the average confidence within mini-clusters, providing a better approximation of the model’s learning status than the confidence of single sample.
• Inter cluster level. Some previous methods estimate the learning status using the average of maximum predicted values across the entire dataset, or employing class-wise averages (as seen in SSL approaches like FlexMatch[R1] and FreeMatch[R2]). Those learning status estimation methods are coarse-grained and may be misled by the possible over-confidence predictions. Our calibration method alleviates overconfidence and operates at a finer granularity, i.e., we divide embedding space into reliable regions and unreliable regions, and we only trust the confidence predictions in reliable regions to estimate learning status, and calibrate the confidence of the samples in unreliable regions. So our approach offers a more accurate estimation of the state.

[R1] FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling. NeurIPS, 2021.

[R2] FreeMatch: Self-adaptive Thresholding for Semi-supervised Learning. ICLR, 2023.

The learning status estimated by our method is not perfectly accurate, but it is effective in practice and aligns well with unsupervised settings. Furthermore, ablation study III in Section 4.2 from the original paper demonstrates that relying on over-confident predictions as a measure of learning state leads to performance degradation, indicating the effectiveness of our learning status estimation method.

I have reviewed the authors' rebuttal as well as the comments from the other reviewers. I appreciate the thorough discussions, analyses, and experiments presented, which effectively address many of my concerns.

However, the revised paper does not adequately incorporate the suggested works for discussions [a, b] or experiments [c, d]. Additionally, I agree with other reviewers that the examination of variations across more random runs and the transfer of hyperparameters across datasets may require further attention. Importantly, the proposed experiments should be included in the camera-ready version or added to the supplemental material.

In light of the above, I will maintain my current rating. However, I would be willing to increase my score if the authors adequately address my concerns in their revisions.

I appreciate the authors for conducting additional experiments that demonstrate the superiority of the proposed CDC. They have effectively addressed my concerns in their revisions. As a result, I will be increasing my score.