Concept Bottleneck Models under Label Noise

I have read all reviews and replies to it. It seems that there is some agreement of the reviewers, actually, which did not change much within the discussion

We appreciate the reviewer for acknowledging the clear motivation of our method, and giving us constructive feedback. While we respond to the reviewer’s specific comments as below, we would be keen to engage in any further discussion.

Beyond SAM?

Although the paper details the impact and offers an in-depth analysis of label noise, the methodology could be strengthened. The use of SAM, which is borrowed from prior works, is the sole approach for enhancing robustness.

Thank you for the constructive criticism. Our primary focus was to highlight the vulnerability of CBM to label noise and to propose an effective remedy to mitigate this problem. The reason for adopting SAM (and additionally, label smoothing, as discussed in the appendix) is not arbitrary; rather, it is based on the fact that these methods are among the most effective approaches for addressing label noise as supported by prior works [1, 2]. Through various experiments, we confirm their effectiveness, e.g., we observe that even slight improvements in concept prediction caused by SAM can lead to significant gains in target prediction performance. We recognize that a broader comparison with alternative methods would strengthen the study and plan to pursue this in future work.

Unrealistic noise setting

The emphasis on label noise impact might be overstated, especially as noise levels are extended to 40%, which is unconventional for real-world scenarios. Natural label noise is already present in datasets like CUB and AWA2.

Thank you for raising this concern. While we agree that a 40% noise level is quite high, our analysis is not confined to this level, and in fact, it covers a wide range of noise levels ranging from 0% to 40%. Therefore, we would respectfully disagree with the assertion that the impact of label noise is overstated.

Notably, we observe that even with a modest noise level of 10%—a more realistic scenario in practice—the target accuracy of CBMs drops significantly from 74.3% to 61.9%. This behavior is distinctly different from traditional end-to-end models, highlighting the unique vulnerability of CBM to concept noise. Therefore, our claims about the impact of noise are reasonable and well-supported..

Accuracy

In Table 1, the concept prediction accuracy is only 92.4% for joint CBM when the noise level is 0%, which does not correspond to 96.9% reported in the vanilla CBM literature, despite using the same Inception-V3 backbone. What might account for this discrepancy?

Thank you for pointing this out. In the joint model, the model minimizes the following objective:

$\hat{f}, \hat{g} = \arg\min_{f,g} \sum_i \left[ L_Y \left( f \left( g \left( x^{(i)} \right) \right), y^{(i)} \right) \right] + \sum_j \lambda \mathcal{L}_{C_j} \left( g \left( x^{(i)} \right); c^{(i)} \right) \quad \text{for some } \lambda > 0.$

Instead of directly adopting the $\lambda$ value from prior work, we tune hyperparameters for each clean and noise settings, with a primary focus on target accuracy. As a result of using different $\lambda$ value, the concept accuracy differs from that reported in the original work.

LLM/VLM-based concept

This paper lacks an alternative type of literature such as [2] that generates concepts using LLM/VLM. This category is highly relevant to the label noise setting.

Thank you for your suggestion. Previous studies, including [2], have shown that even when concepts are annotated using LLMs, incorrect or ambiguous labels remain unavoidable. This implies that noisy labels are quite inevitable even when using LLM-based annotation approaches.

Investigating different types of noise including ones created by LLMs is certainly an interesting direction to pursue. We will discuss this aspect in the revised version.

Thank you to the authors for their time and effort in addressing my initial concerns. However, after reviewing their responses, I must express that my concerns have not been fully addressed.

First, while I understand and acknowledge that the choice of SAM is not arbitrary, the methodology remains weak, and the theoretical contributions are limited, which undermines the overall impact of the work. Furthermore, the authors claim that “we observe that even slight improvements in concept prediction caused by SAM can lead to significant gains in target prediction performance,” but this phenomenon seems to only be evident in independent and sequential CBMs. Moreover, there is a lack of ablation studies to rule out the possibility that the observed improvements could be due to other factors, such as the learning rate.

Second, the target accuracy does not align with that of the vanilla CBM across all three models. The authors state that “we tune hyperparameters for each clean and noise setting, with a primary focus on target accuracy,” which could introduce bias and result in unfair comparisons. A particularly perplexing finding is that the target accuracy is lower than that of the vanilla CBM, even though the authors explicitly state that they prioritize target accuracy in their tuning process. Could the authors provide a clearer explanation for this discrepancy?

Third, the authors claim that “the target accuracy of CBMs drops significantly from 74.3% to 61.9%” with a modest noise level of 10%. However, this substantial drop is also observed in the SAM condition, although I do not see results reported for the 10% noise level specifically. Given that a similar issue arises for both SGD and SAM, I question whether SAM is truly beneficial under high noise levels. Could the authors provide more evidence or justification for the effectiveness of SAM in such settings?

Fourth, the authors mentioned that “We will discuss this aspect in the revised version,” but I have not been able to find the corresponding discussion. Could the authors point out where this is addressed? Specifically, aside from the discussion of LLM-based CBMs, the authors also indicated they would explore the correlation between interpretability and concept prediction accuracy, yet this discussion is not present in the revised version.

Best,
Reviewer kFmQ

We sincerely thank the reviewer for finding our work well-motivated and intuitive. We also appreciate the reviewer’s constructive feedback. While we make clarifications to specific comments below, please let us know if there is anything we need to address further. We would be keen to engage in any further discussion.

Summerization

While the experiments are comprehensive for Section 3 and 4. Some of the results are pushed to the appendix (which is fine), however it would have been nice to summarise them in brief in the text.

Thank you for your suggestion. We will revise the content to provide a concise summary in the final version.

Theoretical intuition on using SAM

I enjoyed reading up to Section 4. Thank you. However, I would have appreciated some theoretical intuition on why SAM works better (unless I missed it).

Thank you for your feedback. According to a previous study [1], the robustness of SAM to label noise can be explained theoretically. Specifically, SAM acts like an $L2$-norm penalty on intermediate activations and the weights in the last layer in a binary classification task using a 2-layer network.

In detail, assuming the function $f(x)$ is bounded by $\|f(x)\|_2 \leq C$, the loss $\ell(x)$ for a data point $x$ is bounded as $\log\big(1 + (K - 1) \exp(-C)\big) \leq \ell(x) \leq \log\big(1 + (K - 1) \exp(C)\big).$

As $C$ decreases, the loss for clean data remains at a non-negligible level, while the loss for noisy data is capped at $\log\big(1 + (K - 1) \exp(C)\big)$. This ensures that the model learns clean data stably while preventing the loss from noisy data from growing excessively.

Although this analysis is based on a simple 2-layer linear network, SAM is expected to exert even stronger regularization in deep neural networks due to their larger and more complex weight spaces, thereby enhancing robustness to label noise.

Correlation of concepts and target

The paper performs experiments with CUB and AwA2 datasets, popular benchmark datasets for CBMs. These datasets however have a strong correlation between concept labels and the target label. I can imagine label noise to be very detrimental (as observed from Figure 2). The potential observed effect due to label noise might be weaker in the case of diverse concepts for each target.

Thank you for suggesting these relevant papers. We will make sure to include these papers in our work.

Related Works

The paper claims to be first the paper to looking at label noise in CBMs, while I would not refute this, I would like to point out the authors to some very relevant papers - [2] (noise added to concept labels, similar to some of the exps in Sec3) [3]-(concept robustness and adv attacks)

We appreciate the reviewer for suggesting related works [2, 3]. We will include the suggested works in our final version.

We really appreciate the reviewer’s positive and constructive feedback. While we address the reviewer’s specific comments below, we would be keen to engage in any further discussion.

Novelty

It is unsurprising that injecting label noise reduces predictive performance, and it is known that sharpness-aware minimization improves performance when label noise is present.

We appreciate the reviewer’s comments and agree that label noise may intuitively be expected to degrade model performance. However, quite surprisingly, our findings reveal that Concept Bottleneck Models (CBMs) demonstrate robustness to target noise, which contradicts initial assumptions, but are highly vulnerable to concept noise. To the best of our knowledge, we are the first to explicitly identify this phenomenon and analyze it through extensive experiments.

Specifically, we uncover two key reasons for this vulnerability to concept noise:

• First, concept noise disrupts representation clustering by preventing the model from forming distinct and meaningful clusters in the representation space, which undermines its interpretability and performance.
• Second, concept noise distorts concept-target relationships by altering the learned correlations between concepts and the target variable, resulting in unreliable predictions.

Lastly, we explore whether noise mitigation methods could address these points and enhance the robustness of CBMs. Focusing primarily on Sharpness-Aware Minimization (SAM), we find that even modest improvements in concept prediction lead to significant enhancement in target prediction performance.

Represent real-world noise

The findings are purely empirical, and it is unclear how representative the noise models used in the paper are for the types of noise that occur in real-world settings.

Thank you for raising this point. To address concerns about practicality in our noise setting, we begin by describing the symmetric noise used in this work. This randomly flips labels within each class to any other class label with equal probability. It has been extensively studied in prior works as a representative proxy for real-world scenarios [1, 2, 3].

Additionally, the pairwise noise (i.e., asymmetric noise) used in our study provides a realistic representation of label noise by incorporating dependencies between labels. For instance, it accounts for the scenarios where confusing a dog with a cat is more likely than confusing a dog with a flower. This setting captures an additional aspect of real-world noise scenarios [4].

Effect of linear model

The classifier considered in the submission is a linear one, and it is unclear how the results are affected by the fact that this classifier is relatively insensitive to noise.

In our study, we use a linear model following the original research [5]. For the goal of CBMs, i.e., interpretability, linear models can offer a transparent decision-making process. In contrast, while non-linear models can capture more complex patterns, they are more sensitive to label noise, increasing the risk of learning incorrect patterns. This can lead to predictions that appear accurate but rely on unreliable or incorrect concepts, ultimately undermining the interpretability, a key advantage of CBMs.

We also experimented with a target classifier using non-linear models to evaluate the generality of our findings. The results demonstrated that the difference between linear and non-linear models does not significantly impact our discoveries. The vulnerability of CBMs to concept noise became even more pronounced with non-linear models, further strengthening our claims. The results are provided below.

We summarize our findings as follows:

• Under target noise conditions, non-linear classifiers demonstrate similar performance to single-layer linear models.
• Under concept noise, performance degradation became more evident with higher model complexity.

This finding underscores our claim that "CBMs are highly vulnerable to concept noise," and this vulnerability becomes even more pronounced with non-linear models, further supporting our argument.

Marginal issues

There is a section on label smoothing in the appendix. Label smoothing is not referred to in the main text.
A couple of other small issues: "need to be trained with noisy labels" - rephrase
Multiple duplicate references in the bibliography.

Thank you for your suggestion. We will reflect it to our final version.

[1] Wang, Yisen, et al. “Symmetric cross entropy for robust learning with noisy labels.” ICCV, 2019.
[2] Chen, Pengfei, et al. “Understanding and utilizing deep neural networks trained with noisy labels.” ICML, 2019.
[3] Lukasik, Michal, et al. “Does label smoothing mitigate label noise?” ICML, 2020.
[4] Scott, Clayton, et al. “Classification with asymmetric label noise: Consistency and maximal denoising.” JMLR, 2013.
[5] Koh, Pang Wei, et al. “Concept bottleneck models.” ICML, 2020.