An Analysis of Concept Bottleneck Models: Measuring, Understanding, and Mitigating the Impact of Noisy Annotations

We sincerely thank the reviewer for acknowledging our works as practical and well-organized, and for providing thoughtful feedback. Below, we address the specific comments and welcome any further discussion.

---

On the independence assumption

The assumptions regarding the noise generation process are arguably too simplistic. In Section 2.1, this process ignores potential dependencies between concepts and labels. The noise simulation process appears oversimplified. As described in the paper: “each binary concept label $c_i \in \{0, 1\}$ is independently flipped with probability $\gamma$ to simulate concept noise $\hat{c}$. For target noise $\hat{y}$, each class label $y$ is randomly changed to a different label, uniformly sampled from the remaining classes, also with probability $\gamma$. However, as the authors themselves note in lines 31-33, “changing concepts can shift the label.” This indicates that concept and target label noise are not necessarily independent in real-world scenarios. Given this, how do the authors justify the assumption of independence between noisy concepts and noisy labels in their experimental setup? Would the proposed methods remain effective under a more realistic, correlated noise model?

Thank you for this insightful comment. Our primary objective is to provide a systematic analysis of how noise affects the performance and interpretability of CBMs. To isolate basic effects and to evaluate mitigation strategies in a controlled setting, we sought an experimental design in which concept noise and target noise can be manipulated independently. This design allows us to vary each factor without confounding and to attribute observed changes to a single source of corruption. For this reason, we considered the independence assumption a reasonable and transparent starting point for an initial analysis across a range of noise levels.

We agree that in many practical settings concept and target noise can be correlated. To examine this case, we additionally simulated a correlated noise model and evaluated susceptibility and its connection to uncertainty. Concretely, we first corrupted the target label; only when the target changed did we inject concept noise, thereby inducing dependence between concept and target corruption. The results are summarized below.

Correlation between uncertainty and susceptibility on practical noise

Even under dependent noise, the positive association between uncertainty and susceptibility persists for susceptible concepts. This pattern is consistent across noise levels and indicates that our analysis and mitigation approach remain effective in a more realistic correlated noise setting.

We sincerely thank the reviewer for recognizing the practicality and timeliness of our work and providing thoughtful feedback. Below, we address the specific comments and welcome any further discussion.

---

Significance of our work

The paper does not introduce any fundamentally new methods. Both SAM and uncertainty-based interventions are existing techniques that are simply applied to this setting. The novelty is limited to empirical observations and analysis, which–though valuable–do not represent a significant conceptual advance.

We appreciate the constructive comment. The significance of our study can be summarized in three parts.

• To our knowledge, this is the first systematic analysis of the noisy concept annotation problem in CBMs, a practical issue that can arise from human error and subjective judgment yet has remained largely unattended. We show, through multi-faceted analyses, that such noise negatively affects all core CBM properties: interpretability, intervention effectiveness, and predictive performance.

• We find that degradation is not uniform across concepts. Rather, it concentrates on a subset that is susceptible to noise. This small set explains most of the accuracy decline and therefore drives the overall performance drop.

• To mitigate the risks posed by susceptible concepts, we propose to use sharpness-aware minimization during training and uncertainty-guided intervention at inference. We provide broad empirical results and theoretical support demonstrating that these strategies are effective at restoring the reliability of CBMs.

---

Unexpected findings

Several results–such as the degradation of interpretability and performance under noise, or the benefits of targeting uncertain predictions–are somewhat expected. While clearly and rigorously shown, these insights are incremental.

We appreciate the reviewer’s point. This work, however, presents results that were not previously documented and are difficult to anticipate.

First, it is commonly believed that CBMs allow one to correct the final prediction via intervention even when a concept is mispredicted. From this view, one might expect that models trained with noisy concepts would readily recover performance through intervention. Our experiments contradict this expectation. As shown in Section 2.4, when CBMs are trained on datasets with concept noise, the intervention effectiveness itself is substantially reduced, so the expected recovery does not materialize. Given that real deployments often rely on concept sources with nontrivial noise, this indicates that intervention may fail to operate as intended and can lead to unexpected failure.

Second, CBMs degrade in a qualitatively different manner from standard DNNs under the same noise level. For example, on the CUB classification task with 0%, 20%, and 40% noise, a DNN exhibits a relatively moderate decline (79.7 → 65.9 → 53.0), whereas a CBM shows an abrupt collapse (74.3 → 50.3 → 4.0). This occurs because the two-stage structure of CBM propagates errors in concept prediction directly to the target predictor. The evidence shows that CBMs are considerably more vulnerable to concept noise than is typically assumed.

These findings reveal an inherent sensitivity of CBMs to concept noise and underscore the need to recognize this vulnerability and develop appropriate mitigation in order to build reliable CBMs.

---

Positioning relative to prior work

how do the authors position their findings in light of prior work (e.g., Shin et al., Penaloza et al., Sheth & Kahou), which also studied intervention strategies and robustness in CBMs? What distinguishes this work?

We appreciate the question.

Shin et al. [1]: The focus is different from ours. They propose and evaluate various intervention strategies and analyze their effects. Our work systematically analyzes how noise degrades CBM behavior and proposes practical mitigation for this noise setting. We further show, both theoretically and empirically, that uncertainty-guided intervention approximates a susceptibility-based intervention, supporting its utility under noisy supervision.

Penaloza et al. [2]: They share the motivation that CBMs can be sensitive to label noise and propose model improvements, assuming they are vulnerable to noise. In contrast, our contribution is a comprehensive analysis of how concept noise affects interpretability, intervention effectiveness, and predictive performance, together with an integrated mitigation framework that operates at both training and inference.

Sheth and Ebrahimi Kahou [3]: Their goal is different. They investigate that CBMs can learn irrelevant concept representations and target robustness under distribution shift. We study concept noise that arises naturally in practice, identify the key drivers of failure through a systematic analysis, and propose simple and effective strategies to mitigate the resulting risks.

---

Broader range of settings

For a paper whose main contributions are empirical, the use of only two datasets (CUB and AwA2) is limiting. The findings would be more convincing if demonstrated on a broader range of settings.

We appreciate this concern and will include additional results in the final version. We conducted two complementary experiments in the tables below.

Although the main paper analyzes susceptibility primarily on CUB, we replicated the analysis on AwA2. We observe that the positive correlation between uncertainty and susceptibility persists. While the non-susceptible set shows a more negative correlation in AwA2, this does not weaken the positive correlation within the susceptible set.

To better reflect practical conditions where concept and target may be dependent, we designed a dependent noise model. We first corrupt target labels, then add concept noise only when the target changes. The positive correlation between susceptibility and uncertainty remains under this more realistic setting.

Correlation between uncertainty and susceptibility on AwA2

Correlation between uncertainty and susceptibility on practical noise

---

Linking theory to training noise

The theory in Appendix B compares $f(\hat{c}_i)$ with $f(c^*_i)$, focusing on the error in predicting concept $i$. However, how does this prediction error relate to the training noise rate, which is the central focus of this paper? The setup seems to conflate inference-time uncertainty with training-time corruption–could the authors clarify this connection?

Thank you for raising this point. In Appendix B we intentionally define the intervention effect at inference as $\delta_i := E_{(x, y)} [\ell(f(\hat{c}), y) - \ell(f(\hat{c}_{-i}), y)]$ to quantify how much the loss decreases when intervening on concept $i$. The key point is that $\hat{c}$ is produced by a predictor trained on noisy concept labels. Hence $\hat{c}$ already reflects the training noise rate, and so does $\delta_i$.

Although the analysis is formulated at inference, it is grounded in predictions from a model affected by training-time corruption, which connects inference-time uncertainty to the training noise rate. We will make this linkage explicit in the revision.

---

Concept susceptibility and relevance

The definition of susceptible concepts is based on a greater-than-average drop in accuracy under noise. But what if a concept suffers a large decline yet contributes negligibly to the final prediction (e.g., in a linear bottleneck model, if its weight is close to zero)? Shouldn’t the uncertainty-aware intervention take into account concept relevance?

We appreciate the insightful observation. We agree that some concepts may suffer large accuracy declines yet have limited impact on the target. In our experiments, such cases were not frequent, but accounting for this possibility would improve diagnostic precision. As a direction for future work, we plan to incorporate measures of concept relevance into the evaluation. This extension would better capture rare cases where accuracy decline does not translate into downstream impact.

---

References
[1] Shin, Sungbin, et al. "A closer look at the intervention procedure of concept bottleneck models." ICML, 2023.
[2] Penaloza, Emiliano, et al. "Preference optimization for concept bottleneck models." ICLR Workshop on Human-AI Coevolution, 2025.
[3] Sheth, Ivaxi, and Samira Ebrahimi Kahou. "Auxiliary losses for learning generalizable concept-based models." NeurIPS, 2023.

We sincerely thank the reviewer for recognizing our work as insightful and comprehensive, and for providing thoughtful feedback. Below, we address the specific comments and welcome any further discussion.

---

Why SAM improves robustness

How does SAM improve the robustness of CBMs compared to other noise mitigation techniques?

Thank you for the insightful question. SAM optimizes by attenuating the sharpness of the loss landscape, which reduces the model’s sensitivity to noise and small perturbations. This, in turn, curbs unnecessary overfitting and promotes more stable learning under label noise. In addition, we observe that SAM tends to reduce errors specifically on susceptible concepts, providing a selective and structured mitigation where noise-induced errors concentrate.

Furthermore, our theoretical analysis shows that, within the CBM structure, SAM implicitly induces an $\ell_2$-regularization effect on the final layer weights and intermediate activations, thereby lowering the sensitivity of the model to label corruption. This analysis explains why SAM yields stronger robustness in CBMs than alternatives that do not influence these internal mechanisms as directly.

---

Predictive entropy as a proxy for susceptibility

Why do you think predictive entropy is a good surrogate for susceptibility? Can you explain the intuition?

Thank you for the thoughtful question. The intuition is as follows. When a concept label is noisy, the corresponding concept does not present a consistent pattern in the data and thus is not learned reliably during training. The model then struggles to form a confident decision for that concept, which manifests as high predictive entropy. In contrast, informative and consistent concepts typically yield low entropy because the model can make confident predictions. Based on this intuition, we regard predictive entropy as a practical indicator of concept susceptibility.

---

Generalization across modalities

How do you think your results might generalize to other modalities?

Thank you for the question. CBMs are now being explored beyond vision, including settings that combine CBMs with large language models [1, 2, 3]. The core CBM process first predicts concepts and then predicts the final target, which does not depend on a specific modality. Recent LLM based CBM studies employ a similar two stage design, so we expect our main findings to transfer to these settings.

That said, concept annotated datasets for LLM are not yet systematically established. An important next step is to curate well-defined concept annotated datasets suitable for LLM applications and to analyze how label noise in such datasets affects CBMs.

---

Real world noise

How do you think real world noise might differ from the simulated noise in this study, and how might that alter your results?

Thank you for the important question. In the main paper, we focus on uniform noise, but Appendix D.4 includes additional experiments under more realistic conditions (e.g., asymmetric, grouped noise). These experiments indicate that our core findings remain qualitatively stable.

Because real world noise often depends on particular concepts and on their relation to the target, we also simulate noise with dependencies between them. Specifically, we first corrupt the target label and then add concept noise only when the target has been changed. The results below summarize the correlation between uncertainty and susceptibility under this practical setting.

Correlation between uncertainty and susceptibility on practical noise

Even when noise exhibits such dependence, the positive correlation between uncertainty and susceptibility persists. We agree that considering diverse noise types is essential for broad applicability, and we appreciate the suggestion.

---

Additional factors behind susceptibility

What are the main factors beyond ambiguity and low prevalence in the dataset that make certain concepts more susceptible?

Thank you for the valuable question. Beyond ambiguity and low prevalence, we believe two additional properties can increase susceptibility: (1) Context dependence. Even for the same concept, its visual realization can vary with class, pose, or background, creating polymorphism. In such cases, the mapping from input $x$ to concept $c_i$ can become locally nonstationary, which destabilizes the decision boundary and leads to large performance drops even under mild noise; (2) Inter-concept correlation. When concepts are correlated, or when concept predictor $g$ shares internal representations across concepts, noise can interfere through these shared components, indirectly perturbing neighboring concepts and making their decision boundaries less stable.

---

References
[1] Tan, Zhen, et al. “Interpreting pretrained language models via concept bottlenecks.” Pacific-Asia Conference on Knowledge Discovery and Data Mining, 2024.
[2] Sun, Chung-En, et al. “Concept bottleneck large language models.” ICLR, 2025.
[3] Bhan, Milan, et al. “Towards Achieving Concept Completeness for Textual Concept Bottleneck Models.” arXiv preprint arXiv:2502.11100 (2025).

We sincerely thank the reviewer for the thoughtful, constructive, and detailed feedback. We address the reviewer’s specific comments below and welcome any further discussion.

---

Unexpected findings

We appreciate the reviewer’s point. This work, however, presents results that were not previously documented and are difficult to anticipate.

First, it is commonly believed that CBMs allow one to correct the final prediction via intervention even when a concept is mispredicted. From this view, one might expect that models trained with noisy concepts would readily recover performance through intervention. Our experiments contradict this expectation. As shown in Section 2.4, when CBMs are trained on datasets with concept noise, the intervention effectiveness itself is substantially reduced, so the expected recovery does not materialize. Given that real deployments often rely on concept sources with nontrivial noise, this indicates that intervention may fail to operate as intended and can lead to unexpected failure.

Second, CBMs degrade in a qualitatively different manner from standard DNNs under the same noise level. For example, on the CUB classification task with 0%, 20%, and 40% noise, a DNN exhibits a relatively moderate decline (79.7 → 65.9 → 53.0), whereas a CBM shows an abrupt collapse (74.3 → 50.3 → 4.0). This occurs because the two-stage structure of CBM propagates errors in concept prediction directly to the target predictor. The evidence shows that CBMs are considerably more vulnerable to concept noise than is typically assumed.

These findings reveal an inherent sensitivity of CBMs to concept noise and underscore the need to recognize this vulnerability and develop appropriate mitigation techniques in order to build reliable CBMs.

---

Significance of our work

We appreciate the constructive comment. The significance of our study can be summarized in three parts.

• To our knowledge, this is the first systematic analysis of the noisy concept annotation problem in CBMs, a practical issue that can arise from human error and subjective judgment yet has remained largely unattended. We show, through multi-faceted analyses, that such noise negatively affects all core CBM properties: interpretability, intervention effectiveness, and predictive performance.

• We find that degradation is not uniform across concepts. Rather, it concentrates on a subset that is susceptible to noise. This small set explains most of the accuracy decline and therefore drives the overall performance drop.

• To mitigate the risks posed by susceptible concepts, we propose sharpness-aware minimization during training and uncertainty-guided intervention at inference. We provide broad empirical results and theoretical support demonstrating that these strategies are effective at restoring the reliability of CBMs.

---

Why SAM

We understand the reviewer’s concerns. We adopted SAM because prior work by Baek et al. [1] indicates that SAM is a representative and often effective approach for alleviating label noise, and it can outperform alternative methods [2, 3, 4]. In our experiments, SAM also handled noisy concept annotations in CBMs, especially for susceptible concepts. As noted in Appendix D.3, we do not claim SAM is the only solution; rather, for the purposes of this work, we present it as one effective approach to the observed problem.

---

Compare to [5]

Thank you for the comment. We clarify the differences from the concurrent work [5]. While [5] shares the motivation that CBMs are sensitive to label noise, it primarily focuses on improving the learning objective, assuming this sensitivity without further investigation. It does not provide a comprehensive analysis of how noise affects all key CBM components (e.g., interpretability, intervention effectiveness, and predictive performance). In contrast, our work quantifies noise effects across these core properties, identifies their causes, and proposes an integrated mitigation framework spanning both training and inference. We will make this distinction clearer in the main text.

We also conducted additional comparisons between SAM and CPO from [5], which modifies the loss from a preference optimization perspective and focuses primarily on joint CBM training, with sequential training as a secondary case. We report comparisons between them in the table below.

Concept Accuracy

Task Accuracy

The main takeaways are: (1) In joint training, CPO tends to yield higher target accuracy, while SAM tends to preserve concept accuracy better; (2) In sequential training, SAM generally outperforms CPO, and combining the two can sometimes achieve the highest performance. These results suggest that two approaches are complementary and potentially usable together.

---

On Appendix materials

Thank you for the helpful comments. We placed the label noise mitigation experiments in the Appendix because the alternative methods did not achieve improvements comparable to SAM, so we treated them as supporting results. We referenced these analyses in the main conclusion under “further analyses”, and in the final version we will make the connection to Section 4.1 explicit.

We also placed SAM on CBM variants experiments in the Appendix because they showed trends similar to the main results and due to space constraints. In light of your comments, we revisited these analyses and added further evaluations across CBM variants. In particular, we examined the effect of SAM on susceptible concepts. For SCBM, SAM showed clear improvements over SGD: at 20% noise, 75.8 vs. 74.9 (+0.9), and at 40% noise, 60.1 vs. 56.4 (+3.7). For AR-CBM and CEM, the gains were smaller: AR-CBM at 20% noise 71.9 vs. 71.3; at 40%, both 53.9; CEM at 20%, 73.3 vs. 73.1; at 40%, 54.6 vs. 54.5.

These results indicate that the effect of SAM differs across CBM variants, and that resilience on susceptible concepts may depend on model design. While our study centers on CBMs, we agree that a systematic analysis across variants is important; we will reflect these points in the main text and consider a broader framework that covers such generalization in future work.

---

Uncertainty-guided intervention

Thank you for the comment. There seems to be some confusion regarding Figure 11. Our goal there is not to newly propose uncertainty guided intervention, but to analyze why this strategy becomes more effective when label noise is present. We state this explicitly in the paper. Our new observation is that the relative improvement of uncertainty-guided intervention increases as noise grows, a point that was not detailed in [6].

The core message of Figure 9 is that prioritizing interventions by susceptibility is effective. To make this clear, we compare the susceptibility-based policy not only with a random policy but also with stronger baselines. The comparative tables are included below.

Noise 10%

Noise 20%

Noise 30%

These results show that prioritizing susceptible concepts remains an efficient intervention policy even against non‑trivial strategies such as UCP and CCTP. Accordingly, we believe the validity of our core intervention strategy is well supported.

---

Minor points

We appreciate the careful, detailed feedback. We will correct the noted typographical issues and revise figures to improve readability in the final version. The embeddings in Figure 4 refer to the representations produced by the concept predictor $g$. Interventions are conducted at the individual concept level to analyze the effect of each concept on prediction through controlled intervention. If group-level evaluations are preferred, we are happy to add those experiments upon request.

---

References
[1] Baek, Christina, Zico Kolter, and Aditi Raghunathan. "Why is SAM robust to label noise?" ICLR, 2024.
[2] Zhang, Hongyi, et al. "mixup: Beyond empirical risk minimization." ICLR, 2018.
[3] Arazo, Eric, et al. "Unsupervised label noise modeling and loss correction." ICML, 2019.
[4] Jiang, Lu, et al. "Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels." ICML, 2018.
[5] Penaloza, Emiliano, et al. "Preference optimization for concept bottleneck models." ICLR Workshop on HAIC, 2025.
[6] Shin, Sungbin, et al. "A closer look at the intervention procedure of concept bottleneck models." ICML, 2023.

We sincerely thank the reviewer for recognizing the value of our work and providing constructive feedback. We address the reviewer’s specific comments below and welcome any further discussion.

---

Role of SAM under clean and noisy conditions

Since SAM improves performance even on clean data, its specific role in mitigating noisy impacts remains unclear.

Thank you for this observation. SAM is known to enhance generalization even in clean settings, and we observe a similar effect for CBMs. Our primary focus, however, is not merely the generalization benefit of SAM but how SAM specifically mitigates label noise within CBMs. In particular, as shown in Table 2, we find that SAM tends to reduce prediction errors concentrated on susceptible concepts, i.e., those most sensitive to annotation errors. This indicates a selective, structural mitigation effect beyond an overall accuracy increase.

---

Reproducibility

The noisy impact is highly dependent on hyperparameters and noise type (e.g., epochs, learning rate). These settings should be explicitly clarified in Sections 2-3 for reproducibility. Experiments were run across three different GPUs, making replication difficult. At minimum, the same hardware should be used for comparable experiments, and results should specify the GPU used.

Thank you for the thoughtful feedback. Most hyperparameters (e.g., epochs, learning rates) are specified in Appendix C.3. We agree that placing the essential settings in the main text would aid readers, and we will move the relevant details into Sections 2-3 in the final version.

Regarding hardware, the majority of experiments were run on NVIDIA RTX 3090, with some on A6000 and A100 due to compute availability. Although not all comparisons were executed on identical hardware, all comparative experiments were performed under same conditions (e.g., random seeds and hyperparameters), so we believe that hardware differences do not materially affect the conclusions. To further strengthen reproducibility, we will re-run the experiments that were not on the 3090 so that comparable results are consolidated on the same GPU.

---

Comparison with SDN

The observed noise effects resemble Subclass-Dominant Label Noise (SDN), where intra-class attributes influence robustness. A direct comparison would strength the analysis.

We appreciate the pointer to this closely related work. There are clear parallels. For example, our t-SNE visualizations show entanglement across classes under noise, qualitatively similar to the observations of SDN. In addition, the noise setting of SDN is related to our grouped noise setting (Appendix C.4 and D.4). Interpreting the subclasses of SDN as units analogous to our concepts makes the similarity more apparent.

A key difference is scope and objective. SDN focuses on a case where early stopping fails to alleviate label noise in DNNs and proposes a remedy. Our study quantitatively analyzes how concept noise affects interpretability, intervention effectiveness, and prediction accuracy in CBMs, and proposes an integrated mitigation framework spanning both training (SAM) and inference (uncertainty-guided intervention). We will incorporate an explicit discussion of SDN in the final version. Thank you for the valuable suggestion.

---

Impact of label noise on susceptible sets

If only a small subset of concepts is susceptible noise, does this limit the practical impact of label noise on CBMs if they do not occur on susceptible noise?

Thank you for the question. If noise does not occur on the susceptible set, the overall impact on a CBM can indeed be limited. While non-susceptible concepts may also experience some degradation, our results indicate that the aggregate performance drop is largely driven by noise on the susceptible concepts. Thus, we expect the effect of noise is substantially reduced when it does not target them.

---

Susceptibility across datasets

Figure 10 shows unclear separation between susceptible and non-susceptible concepts. Are these patterns consistent across different datasets. How many datasets have the authors evaluate?

Thank you for raising this point. While the main paper analyzes susceptibility on CUB, we agree it is important to examine whether the pattern generalizes. We therefore conducted the same analysis on AwA2 as an additional experiment; results are summarized below.

Correlation between uncertainty and susceptibility on AwA2

These results suggest that a positive correlation between uncertainty and susceptibility persists across different datasets. While the AwA2 dataset exhibits a clear negative correlation within the non-susceptible set, this does not undermine the positive relationship observed in the susceptible set. This extension supports that our analysis captures a more general pattern.

---

Noise injection protocol

Is noise applied independently to each concept or the entire dataset? This significantly affects interpretation of the results. For globally applied noise, does the model consistently identify the same concepts as susceptible across different noise instantiations, or does susceptibility vary randomly? I suggest the authors track these susceptible concepts.

We appreciate the question and suggestion. In the main paper, noise is applied at the concept level. Also, the susceptible concepts identified for each random seed are reported below.

Susceptible concepts across seeds

We observe that similar concepts are repeatedly identified as susceptible across seeds, suggesting structural sensitivity rather than random fluctuation.

---

Why target noise appears benign

Target noise does not affect the performance, even at 40% corruption, which seems odd. Why?

Thank you for raising this. In our CBM configuration, the target predictor $f$ follows prior work [1] and is a single linear layer from concepts to the target. Its limited capacity makes it less able to overfit noisy target labels, reducing sensitivity to target corruption. Moreover, CBM training primarily burdens the concept predictor $g$. When $g$ produces stable, accurate concept representations, $f$ operates on relatively clean inputs, so even substantial target noise has a muted effect on final accuracy. We will expand our discussion of this phenomenon in the final version.

---

Concept frequency shift under noise

In Figure 7, why do many initially low frequency concepts show frequency rise (2000x+) after noise injection?

Thank you for the question. Before noise injection, the dataset has a favorable signal-to-noise ratio and some concepts appear very rarely. Under uniform concept noise, each concept value is flipped with a fixed probability, which increases the chance that originally rare concepts appear due to random perturbations. As a result, low-frequency concepts can experience sharp relative increases in frequency. We view this as a distributional shift induced at the data-level by the noise process rather than a change in the model’s intrinsic prediction tendency.

---

References
[1] Koh, Pang Wei, et al. "Concept bottleneck models." ICML, 2020.