Concepts' Information Bottleneck Models

The performance gains brought by the IB regularization appear to be marginal.

The results in concept prediction appear marginal due to the saturation of the baselines. However, we observe an improvement in class prediction with our proposed IB regularizer compared to the base models. Additionally, the test-time interventions demonstrate clear advantages of the regularized methods over their base counterparts. Furthermore, our information-plane dynamics analysis indicates that our regularization enhances the information content of the representations.

Thus, when considering the overall set of results rather than individual ones, the significance of our findings becomes evident.

Some reported results are inconsistent with prior work. For example, the label accuracy of CBM (SJ) reported in the original CBM paper (ICML 2020, https://arxiv.org/pdf/2007.04612) is around 80% (see Table 1 and Fig. 2), whereas this paper reports a significantly lower value of 70.8%.

We utilized the ProbCBM official repository to run CBM and ProbCBM, incorporating our regularizers on top of them. Therefore, the difference in CBM results stems from this specific implementation. We chose this implementation to ensure a fair comparison, as it maintains the same base architecture, hyperparameters, and learning framework across CBMs. Additionally, we opted to use the implementation of a more advanced CBM, like ProbCBM, rather than starting with a simpler one and constructing the more complex versions on top of it.

As illustrated in Table 1, recent additions to CBMs, such as AR-CBM, CEM, and ProbCBM, yield stronger results compared to the original CBM. Hence, we did not consider this performance difference a major issue, given the fairness of the setup.

Furthermore, we highlight that inconsistencies exist in reported models across papers, as some utilize different versions of CBM models without clearly stating which version is used or detailing changes in the architecture. Therefore, we decided to create our own baselines to ensure fair and comparable models.

The authors should provide more detailed evidence to demonstrate how the proposed methods enhance the interpretability of CBMs. The numerical results in Table 2 are insufficient, as it is unclear what the reduction in OIS from 4 to 3 actually implies. For instance, the experiments in https://arxiv.org/pdf/2106.13314 offer a more intuitive demonstration of concept leakage in CBMs. It would be helpful to see a direct comparison between CBM and the proposed CIBM under such experimental setups.

Our experiments are multi-faceted and intuitive, as evidenced by our results of the reduction in concept leakage. Firstly, we employ state-of-the-art quantitative metrics to measure and compare concept leakage (Section 4.2). Our results show that models regularized with the Information Bottleneck (IB) framework exhibit a significant reduction in concept leakage compared to their original counterparts. Next, we conducted intervention experiments (Section 4.3), which revealed that correcting concept groups at test time resulted in CIBMs displaying significantly smoother, monotonic increases in accuracy. As well as the concept quality within the models (Section 4.4). Additionally, we analyzed information plane dynamics (Section 4.5), illustrating how CIBMs maintain higher mutual information with labels (good information) and lower mutual information with raw inputs (irrelevant information), aligning precisely with our theoretical goals.

This multi-faceted strategy (combining rigorous quantitative metrics, practical intervention experiments, and theoretical information analyses) is thoughtfully designed to provide comprehensive and intuitive evidence of leakage reduction from all critical perspectives.

Due to the limited time available during the rebuttal, we were unable to perform the requested experiments from "Promises and Pitfalls of Black-Box Concept Learning Models." However, while these experiments may enhance the existing evidence, we believe they are not crucial given the ample evidence we have already provided. Nonetheless, we will conduct these experiments and include the results in the final version of the manuscript.

The rationale for why leakage can be mitigated through IB regularization appears to be heuristic, lacking a formal theoretical justification. However, I do not consider such a theoretical proof to be strictly necessary for the contribution of this paper.

Our approach begins with a principled problem that compresses information in the latents concerning the data while maintaining expressiveness regarding the labels. We believe this establishes a meaningful connection between leakage and IB regularization through compression, which is stronger than a mere heuristic. However, we acknowledge that this link is not fully formalized.

While we agree that a comprehensive theoretical proof is valuable, we believe our empirical results and practical implications significantly contribute to the field. Our experiments show substantial improvements in reducing concept leakage, supporting the validity of our approach. We welcome further discussion and recognize the importance of formalizing these connections in future work.

I remain confused by your response, so I re-checked both the original CBM and ProbCBM papers. The original CBM paper (ICML 2020) uses InceptionV3 as the backbone (“by fine-tuning an Inception-v3 network”), while ProbCBM uses ResNet18 (“ResNet18 is used as a backbone in all models”). I believe the difference in backbones is the main reason for the discrepancy between the two papers, which is also consistent with my own experimental experience. However, in your paper, you clearly state that InceptionV3 is used for CUB (Line 535). This implies your backbone should be aligned with the original CBM paper, and thus should not result in such a large performance drop.

I don’t think I’m questioning the importance of your work, but at this point, I’m more concerned about whether the reported results are solid.

Thank you for the clarification. However, I still feel confused about the results reported in the paper:

1. CBM Accuracy Too Low

I have personally reproduced the results of CBM (Joint-CBM on CUB) following the original ICML 2020 paper, and achieving 78–80% label accuracy is quite straightforward. The reported 70.8% accuracy for CBM is much lower than prior work (78–80%). Even with different hyperparameters, it’s easy to reach above 75%. This result is hard to accept as a valid baseline.

2. Lack of Detailed Explanation

You mention using a different implementation, but do not specify what changed—backbone, loss weights, training setup, etc. Without details, the cause of the performance drop remains unclear.

Thank you for your response. However, I still find it difficult to believe that your current experimental results are valid and solid. Based on our previous exchanges, it has become clear that you trained the InceptionV3 backbone for only 100 epochs, which demonstrably leads to insufficient training. I believe models need to be sufficiently trained. Since there's a 10-point difference between 100 and 1000 epochs, you shouldn't just report the results from 100 epochs. ProbCBM uses ResNet18, and I suspect it can converge within 100 epochs, which is why the authors in ProbCBM only trained for 100 epochs. However, you're currently using InceptionV3, and it's been demonstrated that models trained for 100 epochs still have significant room for improvement.

Furthermore, your experimental setup neither aligns with the original CBM paper nor with ProbCBM as you initially claimed. Your statement in the first rebuttal, "we opted to use the implementation of a more advanced CBM, like ProbCBM," was incorrect, as you used a different backbone (InceptionV3) than ProbCBM (ResNet18). While you've provided new experiments for CBM, it raises a crucial question: did you also use InceptionV3 as the backbone and train for only 100 epochs for the other models reported on the CUB dataset?

Given our discussions, it appears that the authors may not have a thorough understanding of their own experimental settings, nor how they relate to prior work, when the paper was initially written. I respectfully believe it's imperative that the authors comprehensively re-examine all their experimental settings to ensure the reliability and consistency of their reported results.

Therefore, please forgive me that I cannot recommend this paper for publication at NeurIPS at this time.

Why are your results using InceptionV3 completely identical to those in the original ProbCBM paper using ResNet18, including CBM, ProbCBM, and CEM, across both concept and label accuracy and standard deviation?

The results in Table 1 of the ProbCBM paper (https://arxiv.org/pdf/2306.01574) are identical to the ones in Table 1 of your paper. But different backbones are used.

My main concern is that it's not entirely clear what is the big-picture contribution and/or deep new insight that this method can offer, beyond the technical results that seem somewhat incremental. If the authors can clarify this matter, that would be extremely helpful for assessing whether this paper merits publication at a top-tier ML venue like NeurIPS.

The significance of our CIBM framework goes beyond incremental technical contributions and represents a fundamental conceptual advance (as we detail below).

Traditional CBMs introduced intermediate concepts primarily as interpretable components; however, they lacked any rigorous mechanism to guarantee that these concepts would contain only meaningful, minimal, and task-relevant information. In contrast, our work explicitly integrates the IB principle directly into the concept layer. This represents a substantial, principled, theoretical reframing: moving interpretability from a heuristic add-on toward a structurally enforced, theoretically grounded property of the model itself.

From the perspective of information leakage, previous research primarily viewed leakage as an empirical inconvenience or a purely technical issue to be solved by modifying architectures or concept definitions. Our work offers a clear conceptual shift: we recognize and address leakage explicitly as a fundamental information-theoretic issue. Specifically, leakage arises because traditional CBMs allow irrelevant information from inputs to be encoded unintentionally into the concept layer. Instead of incremental architectural tweaks, we demonstrate that explicitly constraining mutual information between inputs and concepts through IB regularization solves this leakage problem at its root. This approach yields better concept representations, substantially improving their reliability for real-world interpretability and intervention tasks, as our empirical results convincingly show.

Moreover, the IB-based approach we introduce is broadly generalizable and not limited to a single CBM architecture or variant. Our empirical evaluations demonstrate consistent improvements across numerous concept-based frameworks, including CEM, ProbCBM, and AR-CBM. Therefore, our approach does not merely offer incremental improvement on a single model class; instead, it provides a robust, foundational methodology that can be adopted broadly across the interpretability community.

While the paper presents a substantial set of experiments, many of them are not clearly motivated or insightful. For example, the paper considers 12 variants -- two bounds and 6 training settings (HJ / HI / SJ / Prob / CEM / AR ) -- but it's unclear why they are all equally interesting or useful. The current presentation of the paper lacks proper motivation of all these variants, giving an impression they are all considered simply because it's possible, not necessarily because they give unique insight into the problem/solution.

While the paper presents a substantial set of experiments, we would like to clarify that each of these methods serves as contemporary and meaningful baselines to contextualize our contributions. The inclusion of the variants was essential to evaluate our method against commonly used foundational CBMs and methods specifically designed to reduce information leakage.

By conducting a comprehensive comparison, we aimed to establish clear benchmarks and quantify the degree of improvement our method offers. We included traditional CBMs (CBM-HJ, CBM-HI, CBM-SJ) as foundational baselines to demonstrate effectiveness, even though these models were not explicitly designed to address leakage. In addition, we compared our method to Auto-Regressive CBMs (AR-CBM), which specifically aim to minimize information leakage, making this a critical comparison for directly showcasing our approach's effectiveness. Moreover, we considered Probabilistic CBMs (ProbCBM) and Concept Embedding Models (CEM), which address leakage indirectly through enhanced concept representations, quality, or uncertainty modeling. These comparisons provide a thorough evaluation against state-of-the-art methods and demonstrate the generalization capabilities of our proposal.

In summary, our choice of variants was driven by the need to present meaningful insights rather than merely the possibility of including them.

Relatedly, the E-CIB bound assumes that the concepts entropy H(C) is constant, while this entropy is not constant given the implementation details in the appendix. Thus, this bound, even though it performs better than the SUB-CIB bound, does not seem to be well motivated, or at least this issue is not addressed in the main text.

In deriving the E-CIB bound, we simplified the formulation by treating the entropy of the concepts as approximately constant. On one hand, we assume that the concepts entropy was constant since it depends on the prior distribution of the concepts. On the other hand, this simplification is also empirically supported by our results (see below), which confirm that concept entropy indeed rapidly stabilizes early in training, exhibiting minimal fluctuation thereafter.

We computed the concepts entropy over the epochs and see that the entropy quickly stabilizes (after epoch 3) and doesn't significantly change, as shown below:

In the final version, we will add a plot showing the curve and more epochs. However, due to the text restrictions of this rebuttal, we cannot show the figure fully.

The choice of $\beta$ values seems quite arbitrary as well. Why these two particular values? The paper seems to incorrectly treat $\beta$ as a constant rather than a free (hyper)parameter.

As noted by the reviewer, $\beta$ is a hyperparameter. In our paper, we reported two values derived from initial experiments. Below, we provide additional values illustrating the behavior of CBM SJ + IB_E. As shown, $\beta = 0.5$ achieves the highest class accuracy, which supports our choice. While one might argue for exploring different values of $\beta$, this approach could lead to unfair comparisons between experiments and increase costs. Therefore, we decided to use a single value for all experiments. We will report these extended experiments in the final version of the paper.

The main text is not self-contained and relies too much on the appendix for key details and results. For example, the dataset are not explained at all in the main text, the main results for sections 4.5 and 4.6 are only in the appendix.

Due to space constraints in the paper, we had to move details into the appendix. While we agree with the reviewer that these details will be better in the main paper, we compromised by having the main discussion in the paper while having additional details in the appendix.

I think the main text would benefit from a concrete example to motivate and illustrate the method.

We thank the reviewer for the suggestion. We will review the final version to add an example that illustrates the method that is tied to Fig. 1.

Table 1 is pretty hard to digest, I would consider visualizing these results with bar plots and then maybe moving the detailed table to the appendix.

We thank the reviewer for the suggestion. We will consider it for the final version of the paper.

Typos: lines 56-57: use either K or k in both instances; Eq 3: missing E_p(c) for the third term

We thank the reviewer for the detailed comments. We will fixed them in the final version of the paper.

Typos: line 125: shouldn't this be I(X; Z) as in Eq.1?

In this line, we refer to our final objective in Eq. 2. But we can see the confusion since this could refer to the original objective (1) as well. We will review our writing to make this clear in the final version of the paper.

Thank you for your feedback on the rebuttal. However, we respectfully disagree with the notion that applying the IB framework in a new setting lacks value.

[...] however it's not new or unique to this work.

We acknowledge that the IB framework itself is not novel; this is neither our contribution nor our claim.

For this work to be insightful, I was looking for some interesting observations as to the nature of the solution compared to traditional CBMs, beyond marginal quantitative improvements.

We agree that exploring interesting insights from our framework compared to traditional CBMs would be beneficial for the community. However, our work represents a first step in this direction. To our knowledge, there are no other studies investigating the use of IB in CBMs, making the exploration of IB regularization and its effects on current models noteworthy.

In the IB framework, such insight typically arises from varying $\beta$ [...]

We agree that varying $\beta$ often yields insights, and we demonstrated that the changes are minimal in this case. Furthermore, we provided the information plane dynamics to illustrate the differences in training between methods. As mentioned, this is ongoing work that extends beyond the scope of our current proposal. Our study serves as a foundational effort for future research, further showcasing its significance and relevance.

While the method demonstrates effectiveness when applied to basic CBM families, there is no experimental comparison with CBMs that use other forms of information bottleneck techniques. Although the differences in approach are discussed conceptually, empirical comparisons would strengthen the evaluation of information leakage mitigation.

To the best of our knowledge, at the submission of this work, there was no evidence explicitly integrating an Information Bottleneck loss into a concept bottleneck model's training objective. A recent search revealed the pre-print titled "There Was Never a Bottleneck in Concept Bottleneck Models," uploaded in June 2025, indicating that it is concurrent work according to NeurIPS regulations. Furthermore, this pre-print does not conduct traditional experiments that would enable direct comparison.

Can the authors provide a comparison with previous methods that also aim to reduce information leakage in CBMs or similar models?

We would like to emphasize that we already conducted a comprehensive comparison, including methods that aim to reduce leakage, both directly and indirectly. Specifically, we compared our method against traditional CBMs (CBM-HJ, CBM-SJ, CBM-HI) as foundational baselines to establish clear benchmarks and quantify the improvement our method offers, despite these models not being explicitly designed to address leakage. Additionally, our method was compared to Auto-Regressive CBMs (AR-CBM), which specifically target minimizing information leakage, directly demonstrating the effectiveness of our approach. Furthermore, we considered Probabilistic CBMs (ProbCBM) and Concept Embedding Models (CEM), which address leakage indirectly through improved concept representations, quality, or uncertainty modeling.

These serve as contemporary and meaningful baselines to further contextualize our contributions. Therefore, while additional experiments involving other methods that aim to reduce information leakage would be valuable, they are not strictly necessary. Our current experiments effectively demonstrate the information leakage associated with our proposal relative to other methods that also strive to mitigate it.

In Figure 3, the improved TTI performance due to the IB regularizer is clearly visible for CEM, but less so for other methods. Consider revising the visualization to make this improvement more apparent across models.

We thank the reviewer for the suggestion. We will review the visualization to follow the advice.

In Figure E.1, the fact that the first and second columns correspond to different datasets is only indicated in the last row, which may be confusing. It would help to clarify this more explicitly.

We thank the reviewer for spotting this mistake. We will fix this issue in the final version of the paper.

Thank the authors for addressing my concerns.

While I acknowledge that the models primarily compared in the paper—CBM-HJ, CBM-SJ, and CBM-HI—are indeed among the most representative approaches proposed to reduce information leakage in concept bottleneck models, there are other relevant works that have also addressed this issue as follows:

[1] Sun, Ao, et al. "Eliminating information leakage in hard concept bottleneck models with supervised, hierarchical concept learning." arXiv preprint arXiv:2402.05945 (2024).

[2] Parisini, Enrico, et al. "Leakage and interpretability in concept-based models." arXiv preprint arXiv:2504.14094 (2025).

Even if [2] does not introduce an explicit mitigation strategy, but it clearly highlights the direction and importance of reducing leakage in concept-based models.

In addition, although the authors claim to have performed a comprehensive comparison with CBM-HJ, CBM-SJ, and CBM-HI, the key evaluation related to the paper’s central claim—i.e., reduction in concept leakage, as shown in Table 2—only includes CBM-SJ (among three) as a baseline. Even if the other methods do not show high performance compared to CBM-SJ, it is important to evaluate all three models side-by-side in this table to convincingly demonstrate the effectiveness of the proposed approach in reducing information leakage (given that CBM-HI was specifically proposed as the most conservative approach to prevent information leakage).

Some weirdness in experimental results (see Questions), for example baseline CBM performance is lower than reported on the original CBM paper [12].

Why is the black-box accuracy so high on CUB? This seems higher than ones reported in other papers, for example the original CBM[12] paper reports 82.5% class accuracy. This leaves a large gap between CBM methods and black-box methods.

The black-box accuracy comes from our own execution of the original CBM model [12] minus the concept losses trained supervisedly. We report this as a black-box baseline since the original CBM paper [12] uses a different backbone and thus it is not fully comparable.

Similarly, why are the CBM results reported (SJ) much lower than original CBM paper (70.8% vs 80.1%)?

We utilized the ProbCBM official repository to run CBM and ProbCBM, incorporating our regularizers on top of them. Therefore, the difference in CBM results stems from this specific implementation. We chose this implementation to ensure a fair comparison, as it maintains the same base architecture, hyperparameters, and learning framework across CBMs. Additionally, we opted to use the implementation of a more advanced CBM, like ProbCBM, rather than starting with a simpler one and constructing the more complex versions on top of it.

As illustrated in Table 1, recent additions to CBMs, such as AR-CBM, CEM, and ProbCBM, yield stronger results compared to the original CBM. Hence, we did not consider this performance difference a major issue, given the fairness of the setup.

Furthermore, we highlight that inconsistencies exist in reported models across papers, as some utilize different versions of CBM models without clearly stating which version is used or detailing changes in the architecture. Therefore, we decided to create our own baselines to ensure fair and comparable models.

How are the results on sequential CBMs? I think the most commonly used CBM type is soft sequential but I don’t see any results on that.

We thank the reviewer for the suggestion. We trained the CBMs on CUB using the soft sequential (SS) and soft independent (SI) versions, and we present the results below. As demonstrated, our regularizers perform as expected and continue to outperform the corresponding baselines.

We will include these results in the final version of the paper. However, we would like to highlight that, based on our experience, most papers report the soft joint version of CBM, with only the original paper presenting the soft sequential versions.

What are the leakage results (Table 2) of IB_E?

We report the missing IB_E values on Table 2 below. We will add them to the final version of the paper.

Is the number for Table 3 AUC IB_B with 8 corrupt correct? Seems not in line with the trends?

We checked the results again and the reported values are correct. We think that what the reviewer observed is an outlier. We actually think that the outlier result is the one for 16 concepts instead.

Not convinced by the measure of concept-set goodness(4.4), seems like this is closely connected to CBM quality and in the paper it is used to compare different CBMs with the same concept sets, not different concept sets. If the goal is to measure the concept set quality I think the metric should not depend on a particular CBM model.

We acknowledge that the intervention-based metrics introduced in Section 4.4 inherently depend on the specific CBMs used for evaluation, thus reflecting both concept set quality and model-specific interpretability. It is true that an ideal 'intrinsic' measure of concept set quality should be independent of modeling choices; our metrics, however, explicitly prioritize measuring the practical interpretability and operational utility of concept sets in realistic intervention scenarios by a given model.

More specifically, our metrics capture how effectively corrections to concept predictions translate into improved downstream task performance in a given model, providing critical insights into real-world usage. For tasks involving interpretability and human-in-the-loop interactions, this practical aspect of concept quality (captured by model-specific metrics) is crucial.

Nevertheless, we fully agree that future research should develop complementary metrics aimed explicitly at intrinsic concept set quality, independently of particular CBMs or modeling choices. Such metrics could further enrich our understanding and assessment of concept sets, helping researchers select or refine concept sets independent of the model.

Brief introduction in main text would be useful for several things introduced such as the datasets and the Concept Leakage metrics would be helpful.

We thank the reviewer for the suggestion. However, due to limited space we had to move these definitions into the appendix. We will review the text to make it clearer that they are in the appendix to help the reader.

The introduction categorization into four groups is confusing, what exactly are the four groups? Also seems like these groupings are not mutually exclusive, one method could belong to multiple of these groups, for example post-hoc explanations can be model-agnostic or model-specific.

These groups were our attempt to create a taxonomy to explain the state of the art and to simplify the comparison of our method within the existing ones. We are aware that this taxonomy is not exhaustive and is polythetic. We will review our description to ensure that the reader is aware of this limitation.

Fig 3 has too many plots in one small fig making it impossible to see all the lines

We agree with the reviewer and have therefore included the individual plots in the appendix (Fig. C.1). However, due to page limitations in the main paper, we had to strike a balance between comprehensively presenting the results and maintaining readability.

Some qualititative examples showing interpretable decisions on different datasets would be helpful

We will present qualitative examples from our experiments to illustrate the predicted concepts in the final version of the paper.

In eq. 5 I don’t see any term corresponding to the I(X; C) term, is this correct? If yes why is this the case?

Our objective is to transform the mutual information from the information bottleneck (2) into entropies instead (see Eq. A.1). The idea in this version of the bound is to understand the relation of the mutual information between the data and the concepts through the latent representations and its entropies instead, thus the lack of the term.

Dear Reviewer,

The deadline for the author-reviewer discussion is approaching (Aug 8, 11.59pm AoE). Please read carefully the authors' rebuttal and engage in meaningful discussion.

Thank you, Your AC

The specific details for each model are:

For the CBM family, the original paper used Inception, and we did the same, although our code is sourced from the ProbCBM paper. ProbCBM employed ResNet18, and we utilized their configuration (and backbone) and code for our experiments. CEM used ResNet34 in their paper, which we also adopted, using code from the CEM repository. AR-CBM used Inception, which we followed, using code from the AR-CBM repository.

The major concern for me is that the ProbCBM paper clearly states that all experiments were conducted using ResNet18 (including CBM, CEM, and ProbCBM), yet the results in your paper are exactly identical to theirs (including CBM, CEM, and ProbCBM). There are three possible explanations as I see it:

1. You may have directly copied the results from the ProbCBM paper.

2. The ProbCBM paper might have actually used three different backbones (e.g., ResNet18, ResNet34, Inception) when conducting experiments, but mistakenly claimed in the text that all experiments were conducted using ResNet18.

3. A very remarkable coincidence happened. The CBM results you obtained using Inception are exactly identical to the CBM results reported in the ProbCBM paper with ResNet18 backbone, and the CEM results you obtained using ResNet34 are exactly identical to the CEM results reported in the ProbCBM paper (using ResNet18).

If the authors are not intentionally misleading, then only the second and third possibilities remain. I would like to ask the authors what they think is the actual situation. Of course, I also welcome the authors to propose any other possible explanations.

Finally, I want to make it clear that I am not intending to give a negative evaluation. I truly want to understand what happened because the current results are quite confusing to me. If it turns out that your experiments are indeed correct and properly conducted, I am willing to revise my score accordingly.