Energy-Based Concept Bottleneck Models: Unifying Prediction, Concept Intervention, and Probabilistic Interpretations

Q6. For the gradient-based inference Algorithm 1 line 14,15,16 since this is done during inference the algorithm shouldn't have access to $\mathbf{c}, \mathbf{y}$ ? Is this a typo?

We are sorry for the typo. During inference, the algorithm does not have access to $c, y$. In the revision, we have corrected this by changing them to $\widehat{c}, \widehat{y}$ (i.e., the estimated concepts and class labels in the current iteration) in the algorithm.

Thank you for your valuable reviews. We are glad that you acknowledged that our ECBM is "the first work that captures concept interaction during intervention", and found our novelty "excellent" and our idea "original". Below we address your questions one by one.

For your general questions on clarity, significance, and code release:

• Clarity: Final Linear Layer of CBM as Conditional Interpretation. We apologize if our presentation has led to confusion. By 'intricate', we were referring to complex conditional dependencies such as $p(c|y)$, $p(c_{k}|y,c_{k^{\prime}})$, and $p(c_{-k}, y|x,c_{k})$. We agree that Post-hoc CBMs and vanilla CBMs, with their interpretative linear layers, provide a way to understand concept-label relationships. In fact, in PCBMs, the weights can indicate each concept's importance for a given label ($p(c_k|y)$). However, these existing CBM variants cannot provide more complex conditional dependencies such as $p(c_{k}|y,c_{k^{\prime}})$ and $p(c_{-k}, y|x,c_{k})$. In contrast, our model, which relies on the Energy-Based Model (EBM) structure, can naturally provide such comprehensive conditional interpretations.
• Significance: Training Cost. Training the ECBMs is indeed more complex than training regular CBMs. To manage the training cost, we employ a negative sampling strategy, similar to traditional CBMs' cost.
• Significance: Inference Cost. Indeed, due to the energy-based nature of our model, our ECBM's inference requires multiple optimization iterations rather than a single feedforward pass. There are strategies to expedite this inference process. One such approach is to use the predictions from a vanilla CBM as an initialization, thereby 'jump-starting' the optimization and substantially accelerating inference.
• Usage and Reproducibility: Code Release. Thank you for your encouraging comments and interest in our work. We understand the importance of reproducibility in scientific research. So far, we have finished cleaning up the source code and will release it if the paper is accepted.

Questions

Q1. What is the accuracy of a black-box model for datasets in table 1?

Thank you for mentioning this. For CBMs, the standard practice is to take the vanilla CBM as the baseline. However, in response to your suggestion and for the sake of a more comprehensive evaluation, we have also tested black-box models and included the results in our revised Table 4 in Appendix C. These results show that

• our ECBM achieves performance very close to the black-box model on the CUB and AWA2 datasets; for example, on CUB, the accuracy of the black-box model and our ECBM is $0.826$ and $0.812$, respectively.
• In some cases, such as with the CelebA dataset, it even improves upon the black-box performance ($0.343$ versus $0.291$). This can be attributed to our joint inference process of concepts and class labels.
• our ECBM outperforms all baseline models, including CBM and CEM.

Q2. Results in Figure 2 are from which dataset?

Our apologies for the oversight. The results displayed in Figure 2 are from the CUB dataset. We have updated the figure caption in the revised manuscript to include this information. Thank you for pointing this out.

Q3. Aren't $E_{\theta}^{class}$, $E_{\theta}^{concept}$, $E_{\theta}^{global}$ neural networks? If so why don't they have trainable or frozen signs in figure 1?

Thanks for mentioning this. Indeed, $E_{\theta}^{class}$, $E_{\theta}^{concept}$, and $E_{\theta}^{global}$ are neural networks. The absence of trainable or frozen signs in Figure 1 was purely for aesthetic and clarity purposes. We apologize for any confusion this may have caused. According to your suggestion, we have updated Figure 1 to include these signs.

Q4. Equation 9 is $[v_k]_{k=1}^{k}$ this the concatenation of all concepts embeddings?

Yes. In Equation 9, $[v_k]_{k=1}^{k}$ denotes the concatenation of all concept embeddings. Thank you for pointing this out, and we have clarified this in our revision accordingly.

Q5. Please clearly state the output dimensions of each energy model.

We are sorry for the confusion. Each energy model (e.g., $E_{\theta}^{concept}$) outputs a scalar value, which measures the compatibility between the variables. We hope this addresses your question. If you require further clarification or have additional questions, we will be more than happy to follow up with any details needed.

I would like to thank the authors for the clarifications. The changes made in the manuscript made it much clearer! My acceptance recommendation remains unchanged.

Questions

Q1.1. Could you please elaborate on the seemingly contradicting baseline results discussed in the weaknesses?

Thank you for mentioning this. Indeed, in our initial experiments with the CEM interventions, we used the vanilla CEM without RandInt for training. In response to your feedback, we have supplemented the experiments where CEMs are trained with RandInt. The new results can be found in Figure 12 of Appendix C.5. This should provide a more accurate reflection of the CEM intervention performance.

(1) Our ECBM underperforms CEM with RandInt in terms of class accuracy. This is expected since CEM is a strong, state-of-the-art baseline with RandInt particularly to improve intervention accuracy. In contrast, our ECBM did not use RandInt. This demonstrates the effectiveness of the RandInt technique that the CEM authors proposed; we are in the process of incorporating it into our ECBM to see whether it would improve ECBM's class accuracy. We will update the results in the author feedback if we could make it before the discussion deadline (Nov 22).

(2) Even without RandInt, our ECBM can outperform both CBM and CEM in terms of "concept accuracy" and "overall concept accuracy", demonstrating the effectiveness of our ECBM when it comes to concept prediction and interpretation.

(3) We would like to reiterate that ECBM's main focus is not to improve class accuracy, but to provide "complex" conditional interpretation (conditional dependencies) such as $p(c|y)$, $p(c_{k}|y,c_{k^{\prime}})$, and $p(c_{-k}, y|x,c_{k})$. (Please see the response to W6 for more details on the definition of "complex".) Therefore, our ECBM is actually complementary to seminal works such as CBM and CEM which focus more on class accuracy and intervention accuracy.

We have included the discussion above in our revision as suggested.

Q1.2. ...look for code...

Thank you for your interest in our work. We understand the importance of transparency in research and are committed to making our work as accessible and replicable as possible. For the code release, so far, we have finished cleaning up the source code and will release it if the paper is accepted.

Q2.1. Related to the question above, which dataset is used in the evaluation of Figure 2?

Our apologies for the oversight. The results displayed in Figure 2 are from the CUB dataset. We have updated the figure caption in the revised manuscript to include this information. Thank you for pointing this out.

Q2.2. ...are interventions performed on groups of concepts or on individual concepts at a time?...

Thank you for pointing this out. Indeed, we performed interventions on individual concepts. Following your suggestion, we have also added group-level interventions and updated our results accordingly in Figure 12 of Appendix C.5. This additional data provides a more comprehensive view of our model's efficacy in different intervention scenarios. Please also see the response to Q1.1 above for details observations from the new results in Figure 12.

W5. Related to the point above, there are no ablation studies showing the sensitivity of the proposed method to its hyperparameters.

In response to your feedback, we have conducted ablation studies to demonstrate the sensitivity of our proposed method to its hyperparameters. The results on the CUB dataset are available in the table below. These studies provide insights into the hyperparameter search process and show that our ECBM is stable over a wide range of hyperparameters. We appreciate your valuable suggestion and have included all additional results on CUB, AWA2, and CelebA datasets in our revised paper (Table 5 of Appendix C.2).

W6. The claim that previous concept-based methods do not allow one to understand concept to label relationships is not entirely true.

We apologize if our presentation has led to confusion. We meant to say that previous concept-based methods do not allow one to understand "complex" conditional dependencies (as mentioned in the abstract) such as $p(c|y)$, $p(c_{k}|y,c_{k^{\prime}})$, and $p(c_{-k}, y|x,c_{k})$. We agree that Post-hoc CBMs and vanilla CBMs, with their interpretative linear layers, provide a way to understand concept-label relationships. In fact, in PCBMs, the weights can indicate each concept's importance for a given label ($p(c_k|y)$). However, these existing CBM variants cannot provide more complex conditional dependencies such as $p(c_{k}|y,c_{k^{\prime}})$ and $p(c_{-k}, y|x,c_{k})$. In contrast, our model, which relies on the Energy-Based Model (EBM) structure, can naturally provide such comprehensive conditional interpretations. Thank you for pointing this out, and we have clarified this in the revision as suggested.

W7. The proposed evaluation is limited in the number and quality of the competing baselines.

Thank you for mentioning this. According to your feedback, we have updated Table 1 to provide a more comprehensive and fair comparison (see more discussion in the response to W1-W3 above). We appreciate your suggestion, which has helped improve the quality of our evaluation.

W8. Inference on ECBMs requires one to solve an optimization problem via a relaxation that is solved with gradient descent + back-propagation.

Indeed, due to the energy-based nature of our model, our ECBM's inference requires multiple optimization iterations rather than a single feedforward pass. There are strategies to expedite this inference process. One such approach is to use the predictions from a vanilla CBM as an initialization, thereby 'jump-starting' the optimization and substantially accelerating inference. We believe further research into accelerating energy-based CBMs would be interesting future work. We have included the discussion above in the revision.

W9. The paper lacks any discussion of the limitations of its method.

We appreciate your feedback regarding the omission of a discussion on the limitations of our method in the last section of the paper. We are sorry for the oversight and agree that such a discussion is crucial for a balanced presentation of our work. In response to your observation, we will add a comprehensive discussion of the limitations of ECBMs to the final section.

Your feedback has helped us identify several potential limitations, including the need for better solutions in the optimization process of our ECBM and the design of efficient initialization procedures. Additionally, our current model requires concept labeling, a process that could be improved with label-free approaches. As a potential future improvement, our model could be adapted to work with label-free CBMs. Thank you for your constructive feedback, it has been instrumental in helping us improve our work.

Thank you for your insightful and constructive feedback. We are glad that you found the problem we sovle "novel and significant"/"interesting", our paper "easy to follow"/"clear". Below we address your questions one by one.

W1. ...In my opinion, the biggest weakness of this paper is its evaluation and the fairness of the evaluation itself...ResNet18...ResNet101...

Thank you for mentioning this. Our preliminary results show that ProbCBM with ResNet101 underperforms ProbCBM with ResNet18. We suspect that probCBM may exhibit instability as the dimensionality of the feature embedding increases. Hence we have chosen to report the highest scores in the paper. The table below shows the results.

Furthermore, we expanded our evaluation to include more intervention experiments and adopted interventions across all datasets. This approach allows us to demonstrate the intervention performance under varying ratios (more details in the response to Q1-Q3 below). We believe your suggestions have significantly enhanced the comprehensiveness of our experiment. Thank you for your valuable input.

W2. Against common practices in the field, none of the results in the evaluation section include error bars.

This is a good suggestion. Accordingly, we have revised Table 1 to include standard deviations over five runs with different random seeds. These will help present a clearer perspective on the statistical significance of our results. Our standard deviations range from 0.000 to 0.006, except for the overall concept accuracy for CUB, which is 0.009. All values are represented up to three decimal digits. These results confirm the significance of the improvement brought by our ECBMs.

W3. Some of the key baseline results in this paper seem to contradict the results presented in previous literature.

We are sorry for the confusion. It seems the inconsistency with prior intervention performance might be due to our use of individual-level concept intervention. In light of your feedback, we re-conducted the intervention experiment using group-level intervention (more details in the response to Q1-Q3 below).

For the results of ProbCBM and Post-hoc CBM (PCBM), we acknowledge the misunderstandings regarding their requirements and capabilities. We revisited these models and retrained them accordingly. These revised results are now included in Table 1 of our revised paper. Your suggestions and guidance have been invaluable in addressing these issues. Thank you for drawing our attention to these points. The new results confirm that our ECBM outperforms these baselines (ProbCBM and PCBM) in both CeleA and AWA2 too.

W4. A lot of details required for reproducibility are missing.

Thank you for mentioning this. We understand the importance of transparency in research and are committed to making our work as accessible and replicable as possible.

For the code release, so far, we have finished cleaning up the source code and will release it if the paper is accepted.

For the baselines, we assure you that we have strictly adhered to the original papers and their provided codebases for reproduction.

Regarding hyperparameter $\lambda_{c}=0.49$, this is because in our implementation, we introduce $\lambda_l$ to Eqn. (12) in the paper, resulting in: $\lambda_l E_{\theta}^{class}(x,y) + \lambda_c E_{\theta}^{concept}(x,c) + \lambda_g E_{\theta}^{global}(c,y)$

We then set $\lambda_l=1$, $\lambda_c=1$ and $\lambda_g=0.01$.

This is therefore equivalent to setting $\lambda_c=1/(1+1+0.01)=0.49$ for the original Eqn. (12) without $\lambda_l$: $E_{\theta}^{class}(x,y) + \lambda_c E_{\theta}^{concept}(x,c) + \lambda_g E_{\theta}^{global}(c,y)$.

We are grateful for your attention to these details and have revised the paper accordingly.

Q6. The outer summation of Preposition 3.2 is very unclear.

Thank you for mentioning this, and we are sorry for the confusion. In Proposition 3.2, $c$ represents the full vector of concepts and can be broken down into $[c_k, c_{-k}]$, where $c_k$ contain one entry and $c_{-k}$ contains $K-1$ entries. The outer summation is indeed "iterating" over all concept vectors while excluding the $k$-th dimension. We have revised our paper accordingly to clarify this.

Q7. When learning the positive and negative embeddings for each concept, I was wondering if you have an intuition as to what inductive bias in your model is responsible for forcing the distribution of positive and negative embeddings to be easily separable?

Our strategy is unique and helps separate positive and negative embeddings. Unlike traditional methods, we enforce the distribution of positive and negative embeddings from the perspective of the input. In our ECBM, we focus on minimizing a specific target.

• When a concept is active, we only input the positive concept embedding and therefore only the positive embedding is trained. During training, minimization of the global energy term $E^{global}(c,y)$ in Eqn. 12 encourages the positive concept embedding to contain more information about the corresponding class label $y$. Similarly, minimization of the concept energy term $E^{concept}(x,c)$ in Eqn. 12 encourages the positive concept embedding to contain more information about the corresponding input $x$.
• On the other hand, when a concept is inactive, we only input the negative concept embedding and therefore only the negative embedding is trained. During training, minimization of the global energy term $E^{global}(c,y)$ and the concept energy term $E^{concept}(x,c)$ encourages the negative concept embedding to contain the corresponding class label $y$ and input $x$.
• Empirically we also observed that the positive and negative concept embeddings are sufficiently separable.

In summary, our approach ensures that the distribution of positive and negative embeddings is easily separable, as they are trained under different conditions based on the ground-truth activity of the concept in the training set. Thank you for your insightful question, which has helped us elucidate the unique aspects of our model.

At the end of this author response, we would like to express our sincere gratitude for your keen eye and attention to detail, which have helped improve the quality of our manuscript. We have addressed the following typos and minor errors you have pointed out. All the changes have been highlighted in blue in our revised version. Your review has been instrumental in enhancing the overall quality of our paper, both in terms of content and presentation. Once again, we appreciate your invaluable input.

Q3. Similarly, could you please elaborate on how ECBMs fare against competing baselines when intervened on in the other datasets?

This is a good suggestion. We have followed your suggestion to include results on concept intervention for other datasets, i.e., CeleA and AWA2. The results have been incorporated into our revised paper and can be found in Figure 13 and 14 of Appendix C.5. Note that CeleA and AWA2 do not have group assignment, therefore we perform individual-level intervention for these two datasets. Similar to the results for CUB, we have the following observations on how ECBMs fare against baselines:

(1) Our ECBM underperforms CEM with RandInt in terms of class accuracy. This is expected since CEM is a strong, state-of-the-art baseline with RandInt particularly to improve intervention accuracy. In contrast, our ECBM did not use RandInt. This demonstrates the effectiveness of the RandInt technique that the CEM authors proposed; we are in the process of incorporating it into our ECBM to see whether it would improve ECBM's class accuracy. We will update the results in the author feedback if we could make it before the discussion deadline (Nov 22).

(2) Even without RandInt, our ECBM can outperform both CBM and CEM in terms of "concept accuracy" and "overall concept accuracy", demonstrating the effectiveness of our ECBM when it comes to concept prediction and interpretation.

(3) We would like to reiterate that ECBM's main focus is not to improve class accuracy, but to provide "complex" conditional interpretation (conditional dependencies) such as $p(c|y)$, $p(c_{k}|y,c_{k^{\prime}})$, and $p(c_{-k}, y|x,c_{k})$. (Please see the response to W6 for more details on the definition of "complex".) Therefore, our ECBM is actually complementary to seminal works such as CBM and CEM which focus more on class accuracy and intervention accuracy.

Q4. Could you please elaborate on the lack of results for PCBMs and ProbCBMs given my comments on the weaknesses indicating that they are in fact baselines that could be evaluated in the setups used in this paper?

Thank you for your insightful comments. We acknowledge the misunderstandings regarding their requirements and capabilities. We revisited these models (PCBMs and ProbCBMs) and retrained them accordingly. The table below shows the revised results and is now incorporated into the revised Table 1 of our main paper.

Your suggestions and guidance have been invaluable in addressing these issues. Thank you for drawing our attention to these points. The new results confirm that our ECBM outperforms these baselines (ProbCBM and PCBM) in both CeleA and AWA2 too.

Q5. Could you please elaborate more on the specifics of the negative sampling strategy used to learn global $E_{\theta}^{global}$?

This is a good question. In our current implementation, during each iteration, we randomly select 20% of the samples as negative samples. This strategy has been effective for our experiments, but we acknowledge that more elaborate strategies might be required for larger concept spaces.

In terms of reproducibility and transparency, we commit to providing clear and comprehensive documentation of our method and all related elements upon acceptance of our paper. For the code release, so far, we have finished cleaning up the source code and will release it if the paper is accepted.

Q5. Many of the existing explanation methods aren't robust in practice. Are ECBMs robust to adversarial attacks?

This is a good point. We believe our ECBMs do potentially enjoy stronger robustness to adversarial attacks compared to existing CBM variants. Specifically, our ECBMs are designed to understand the relationships between different concepts, as well as the relationships between concepts and labels. As a result, during inference, ECBMs can leverage these relationships to automatically correct concepts that may be influenced by adversarial attacks. Our preliminary results (e.g., the new experiments on background shifts for Q2) suggest that our ECBM can potentially improve the robustness against adversarial attacks compared to existing CBM variants. We believe this is interesting future work; we appreciate your comments and have included the discussion above in the revision.

Thank you for your valuable comments. We are glad that you found our model "addressing the shortcomings", our paper "well-written"/"easy to follow", and our experiments show that our ECBM "outperforms the CBMs on real-world datasets".

Q1. Does the architecture of the network affect the performance? Does it need to be shallow or deeper? What are the design considerations for these networks?

Indeed, the architecture of the network plays a role in the model's performance. In our case, we've followed the approach implemented by all the baselines, utilizing the ResNet architecture. It is preferable for the network to be deeper. Our experiments show that the fusion of Energy-Based Models (EBM) and Concept Bottleneck Models (CBM) necessitates a deeper network structure; this allows for the extraction of higher-quality feature embeddings, which significantly enhances the model's performance. We also include more details on network architecture in Table 2 of the Appendix.

Q2. Is the model robust towards background shifts (ref. experiments in [1])

This is a good question. In response to your comment, we conducted an additional experiment, and the results are included in the table below. These additional results show that on background shift datasets derived from [1], our ECBM achieved a Class accuracy of 0.584 and a Concept accuracy of 0.945. These results are higher than the best Concept Bottleneck Model (CBM) results presented in Table 3 of [1] (i.e., Class accuracy of 0.518 and Concept accuracy of 0.931). We have included the discussion above in our revised paper (e.g., Table 6 of Appendix C) as suggested.

This suggests that our model demonstrates considerable robustness towards background shifts.

[1] Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, Percy Liang: Concept Bottleneck Models. ICML 2020

Q3. What is the compute overhead in comparison to CBMs?

During the training phase, the computational complexity of our energy-based model (ECBM) is similar to traditional CBMs. During inference, due to the energy-based nature of our model, it requires several iterations, typically between 20 and 100. There are strategies to expedite this inference process. One such approach is to use the predictions from a vanilla CBM as an initialization, thereby 'jump-starting' the optimization and substantially accelerating inference. We believe further research into accelerating energy-based CBMs would be interesting future work. We have included the discussion above in the revision.

Q4. One of the main shortcomings of CBMs is the need for concept annotations.

This is a good point. Indeed, the requirement for concept annotations is a known limitation of Concept Bottleneck Models (CBMs).

• We chose to build upon the vanilla CBM as our goal was to delve into the relationships between concepts and between concepts and labels.
• Moreover, the baseline models we used for comparisons, including PCBM [1], CEM [2], and ProbCBM [3], are also based on vanilla CBM. To ensure a fair comparison, we chose this approach.
• Our model is compatible with label-free CBMs. Therefore one could potentially use label-free CBMs to generate concept annotations automatically, and train our ECBM using these concepts, thereby addressing this limitation of concept annotation. This way, our ECBM can simultaneously enhance the interpretability, intervention capabilities, and performance of CBM, without manual concept annotations.

Self-Explaining Neural Networks (SENNs) and their variants efficiently encode relationships between concepts (as mentioned in [4]) and between classes and concepts. However, they are not supervised concept bottleneck models and do not allow for direct concept intervention. Therefore they are not applicable to our setting. This is also why previous CBM papers (e.g., our baselines CEM and PCBM) do not include SENNs as baselines. Nonetheless, we acknowledge the importance of these models and have cited and discussed them accordingly in the revision.

[1] Post-hoc Concept Bottleneck Models. ICLR, 2023.

[2] Concept Embedding Models. NIPS, 2022.

[3] Probabilistic Concept Bottleneck Models. ICML 2023.

[4] A Framework for Learning Ante-hoc Explainable Models via Concepts. CVPR 2022.

Q7. One limitation of concept bottleneck models is the limited applicability in a real-life scenario...shortcut identification...

Thank you for pointing us to these interesting papers. We have carefully gone through them and find that our model is actually equipped to carry out such tasks.

Specifically, $p(c|y)$ generated by our ECBM shows the importance of each concept for each class. By comparing our ECBM's estimated $p(c|y)$ with the ground-truth $p(c|y)$, we can identify any significant differences in concept importance to shed light on any spurious feature learned in the model. For example, if our ECBM estimates that $p(c_1=1|y=1) = 0.8$ while the groud-truth $p(c_1=1|y=1) = 0$, this indicates that the model learns concept $c_1$ as a spurious feature.

Our experiments show that our ECBM is robust against spurious correlations. For example, Figure 3 in the paper shows that our ECBM's estimated $p(c|y)$ is very close to the ground truth. This is possibly due to our ECBM's joint inference of class labels and concepts, which helps mitigate spurious correlation. Furthermore, we conducted additional experiments on the TravelingBirds dataset following the robustness experiments of CBM concerning background shifts. The results (Appendix C's Table 6) reveal that our ECBM outperforms CBMs in this regard. These findings underscore our model’s superior robustness to spurious correlations.

We have included the discussion and the references in our revision as suggested.

Q8. Also, there is a problem of leakage in CBM as in [3]. Will the ECBM be helpful in that regard?

This is an insightful question, and thanks for pointing us to [3]. We believe that our ECBM model could indeed offer a solution to this problem. In response to your suggestion, we conducted an experiment similar to the ones depicted in Figures 2 and 3 in [3], but using our ECBM model. We found that when only a few concepts were present, the accuracy of our method did not significantly increase. However, there was a noticeable increase in the performance of the baseline model(like CEM) at this stage. Our model's accuracy gradually increased when more concepts were given during training. This suggests that our ECBM model can help mitigate the leakage problem; in contrast, baselines such as CBM and CEM still suffer from some degree of leakage due to their soft concepts. We have included these additional empirical results in Figure 11 of Appendix C.5 of our revised manuscript. Thank you for your valuable suggestion, which has helped us strengthen our evaluation of ECBM.

[3] Addressing Leakage in Concept Bottleneck Models. Havasi et. al., Neurips 2022.

Q9. Two setbacks of concept bottleneck models.

Your question highlights some significant concerns prevalent among the CBM community.

(1) The cost of obtaining ground truth concepts: This issue can be mitigated using label-free CBM. As mentioned in response to Q4, our model is compatible with label-free CBM. We can employ label-free methods to generate the concepts, which can then be used to train our ECBM.

(2) The non-causal correlation of concepts: Our paper focuses on inherently interpretable models rather than the post hoc setup. Therefore we do not claim that ECBM learns causal relation. The goal of our ECBM is to learn a concept-based model that

• uses different conditional (not causal) probabilities to provide a comprehensive concept-based interpretation of the prediction model and
• improve prediction accuracy by considering compatibility among different concepts and class labels. It is worth noting that the vanilla CBM also differs from causal settings, as it does not require concepts to directly cause labels but instead requires that concepts are highly predictive of labels.

We hope this addresses your concerns and provides clarity on how ECBMs can potentially bridge these research gaps.

We value the feedback you have provided. We are glad that you found the problem we solve "novel", our presentation "good", and our experiments "comprehensive". We will address each of your comments in turn below.

Q1. The second gap: Given the class label and one concept, the probability of predicting another concept? is not clear

We appreciate your question about the interpretation of the second gap. This scenario is an example of conditional interpretation in our model. In this context, given a class label $y$ and one concept $c$, the model predicts the probability of another concept $c'$. This process provides insights such as: "from the model's perspective, how relevant concepts $c$ and $c'$ are for images in class $y$".

We understand that this explanation might not fully address your comment. If you could provide further details or specify the aspects that are unclear, we would be very glad to offer a more targeted explanation.

Q2. Incomplete literature review.

We appreciate your feedback about the comprehensiveness of our literature review. Your suggestions have indeed enhanced the breadth of our research and have been incorporated into the 'Related Work' section of our revised paper.

Q3. How does each of three conditional probabilities - p(y|x), p(c-k|x, ck), and p(ck|y, ck′)—mitigate the 3 limitations - Interpretability, Intervention, Performance?

Each of the three conditional probabilities addresses different limitations related to Interpretability, Intervention, and Performance:

• $p(c, y|x)$ contributes to performance improvment. Unlike original CBMs, which predict concepts first and then predict labels using these predicted concepts, our model predicts concepts $c$ and class labels $y$ simultaneously while considering their compatibility. Our global energy network measures the dependency between $c$ and $y$, thereby improving the prediction accuracy.
• $p(c_{-k}|x,c_k)$ is designed for intervention. For instance, given the input $x$, if an expert knows the correct concept $c_k$ and performs concept intervention on concept $k$, this can help correct other concepts $c_{-k}$. This is equivalent to computing the conditional probability $p(c_{-k}, y|x,c_k)$.
• $p(c|y)$ and $p(c_{k}|y,c_{k^{\prime}})$ are used for interpretability. $p(c_{k}|y,c_{k^{\prime}})$ indicates the strength of the connection between $c_k$ and $c_{k^{\prime}}$ for a certain class $y$ of images, providing conditional interpretation. $p(c|y)$ highlights which concepts are important to the given class.

We hope that these explanations provide clarity on your question, and we are open to further discussions on this.

Q4. How does this method impact label free CBM or Language in a bottle kind of framework?

This is a good question. Indeed, our ECBM is compatible with the label-free CBM method. One possible approach is to initially extract concepts using the label-free method and subsequently train our ECBM. This would be interesting future work. We appreciate your valuable insights and have incorporated this comment into the revision.

Q5. I believe this is an inherently interpretable method. How to integrate it in a posthoc setting like PCBM?

Yes, ECBM is an inherently interpretable method, not a post-hoc one. In this paper, we concentrate on inherent interpretability. Although integrating our method into a post hoc setting like PCBM is outside the scope of this study, we agree that it would be interesting and nontrivial future work to explore. We are grateful that you have accurately grasped the nature of our method, and will make this point clearer in the revision as suggested.

Q6. There is a new notion of difficulty estimation for samples using concept bottleneck models in .

We apologize for any confusion caused, but there seems to be a typo in your question. We are uncertain about the specific paper in which the notion of difficulty estimation is discussed. We examined the nine papers you provided, but could not locate this term. If you could clarify or provide more information regarding this, we would greatly appreciate it. Thank you for your understanding.

Dear Reviewer tSsK,

Thank you for your time and effort in reviewing our paper.

We firmly believe that our response and revisions can fully address your concerns. We are open to discussion (before Nov 22 AOE, after which we will not be able to respond to your comments unfortunately) if you have any additional questions or concerns, and if not, we will be immensely grateful if you could reevaluate your score.

Thank you again for your reviews which helped to improve our paper!

Best regards,

ECBM Authors

W2. "...missing baseline models that are not based on CBM in the experiments."

Thank you for mentioning this. Our current focus is primarily on concept-based models and, as such, our comparisons are chiefly made with standard CBMs, with an emphasis on both interpretability and model performance. In line with similar studies, such as those on PCBM [1] and CEM [2], we've chosen to include only CBM variants as baselines. This decision is based on the fact that non-CBM models do not possess both concept interpretability and direct concept intervention capabilities, which are crucial for our line of investigation.

Nevertheless, we understand the potential value of incorporating traditional black-box models as baselines to provide a more comprehensive perspective. In addition to our primary findings, we have conducted tests on a black-box model with the same network architecture. Please kindly find these results below and in Table 4 of Appendix C in our revised paper. These results show that

• our ECBM achieves performance very close to the black-box model on the CUB and AWA2 datasets; for example, on CUB, the accuracy of the black-box model and our ECBM is $0.826$ and $0.812$, respectively.
• In some cases, such as with the CelebA dataset, it even improves upon the black-box performance ($0.343$ versus $0.291$). This can be attributed to our joint inference process of concepts and class labels.
• our ECBM outperforms all baseline models, including CBM and CEM.

If you have any specific suggestions for such baselines, please do not hesitate to share them. We would be more than willing to conduct additional comparative experiments accordingly.

[1] Post-hoc Concept Bottleneck Models. ICLR, 2023.

[2] Concept Embedding Models. NIPS, 2022.

Q2. How can the model ensure a stable solution during inference while searching for the optimal values of 'c' and 'y'? Is there any complexity analysis in B.1.

As delineated in Appendix B, Algorithm 1, the computational complexity of our proposed method is $O(TN)$, where $T$ denotes the number of iterations, and $N$ denotes the number of parameters in the neural networks. Thus, the complexity scales linearly with both the number of iterations and parameters.

Empirically, we have observed that the inference process is generally stable. For the gradient-based inference process, we employed the Adam optimizer, which has shown consistent and reliable performance.

As shown in the updated Table 1 in the main paper, we ran our ECBM with 5 different random seeds and report the means and standard deviations. Across all datasets and all metrics, our standard deviations range from 0.000 to 0.006, except for the overall concept accuracy for CUB, which is 0.009. All values are represented up to three decimal digits. These results confirm the stability of our ECBMs.

Q3. Is it possible for the model to detect undefined 'concepts,' or is it capable of exploring new 'concepts'?

Thank you for this insightful question. Indeed, our model does have the potential to explore and identify new 'concepts'. Specifically, we can "reserve" additional "undefined" concept embeddings to allow the model to learn them, and introduce an inductive bias to ensure these embeddings are as distinct as possible from each other and from existing concepts. We believe this capability for concept discovery presents a promising direction for future research. In response to your question, we have incorporated a discussion on this aspect into the revision.

We hope this response addresses your concerns adequately. We appreciate your insightful comments and look forward to further discussions on improving our work.

W3. "There missing ablation study on the proposed method with only a single branch, i.e, the 'concepts' branch and the 'class' branch as shown in Figure 1."

Thank you for your astute observation regarding the absence of an ablation study focused on a single branch method. In response to your comment, we have conducted an additional ablation study and incorporated the results into our work.

Specifically, we include two baselines for this ablation study:

• x-y: This baseline involves a single branch that utilizes the class energy network to directly predict the class label.

• x-c-y: This baseline, on the other hand, employs a single branch that first uses the concept energy network to predict the concepts. Following this, it uses the global energy network to predict the class label.

The findings are as follows:

We have also included this table detailing these results in Table 4 of Appendix C for further reference.

From this new ablation study, we have the following observations:

• Our full model demonstrates superior performance when tested on the CelebA and AWA2 datasets, outperforming all other baseline models, and verifying the effectiveness of each component of our ECBM.

• For the CUB dataset, our ECBM $x-y$ single branch surprisingly approaches the performance of the black-box model. However, it lacks concept interpretability. In contrast, our full ECBM can provide such concept interpretability, albeit with a slight decrease in class accuracy. Note that our full ECBM still outperforms all baseline models, including "x-y", "x-c-y", CBM, and CEM, verifying the effectiveness of each component of our ECBM.

W4. The "related work" section about energy-based models is not complete.

We appreciate your insightful feedback on the 'Related Work' section of our paper. We agree that a more comprehensive review of the historical development and pioneering works of Energy-Based Models (EBMs) would enhance our discussion. We recognize the importance of the works you have cited, [2], [3], and [4], in the field of EBMs. Particularly, the use of an EBM for the distribution of labels (or other information) and data in [4] aligns closely with our research as it also captures high-level concepts and data. In light of your suggestions, we have updated our paper to include these seminal works.

[2] A Theory of Generative ConvNet. ICML 2016

[3] Cooperative Training of Descriptor and Generator Networks. TPAMI 2018.

[4] Cooperative Training of Fast Thinking Initializer and Slow Thinking Solver for Conditional Learning. TPAMI 2021.

For Questions:

Q1. As demonstrated in Table 1, the gap in performance between CBM and ECBM is noticeably smaller on the AWA2 dataset compared to that observed on the CUB and CelebA datasets. Is there any explanation for this phenomenon?

This is a good question. As indicated in Table 1, the CBM already achieves high performance on the AWA2 dataset, with Concept accuracy at 0.979, Overall concept accuracy at 0.803, and Class accuracy at 0.907. Given such high accuracy, there is very limited room for further improvement. It is also worth noting that other baseline models, such as ProbCBM and CEM, do not even surpass the performance of the original CBM on the AWA2 dataset. Despite the narrower performance gap, our ECBM model still achieves the state-of-the-art results on this dataset, underscoring its effectiveness.

Thank you for your valuable reviews. We are glad that you found our model "enhances the state-of-the-art". Below we address your questions one by one:

W1. "... due to the absence of a clear definition of the term "concepts" ... neglects the acquisition of features that can be explicitly identified."

This is a good question.

The Advantages of Concept Bottleneck Models (CBMs): We agree that "concepts" can be seen as "explicit features that can be readily discerned by humans". However, we would like to emphasize that by using these concepts as the bottleneck, Concept Bottleneck Models (CBMs) and their variants have additional advantages:

• Humans can intervene in these predicted concepts, thereby influencing final predictions. This is particularly beneficial in real-world settings, where it is crucial not only to provide interpretability but also to incorporate human participation into the model's decision-making process, known as the human-in-the-loop process.

• They predict interpretations that can be presented to humans for verification, creating a feedback loop for accuracy.

Addressing Performance Decline: Thanks for pointing us to the interesting paper [1], which we have cited and discussed in the revision. We agree that maintaining concept-based interpretability while ensuring model performance is an open question in the field. The baselines we compare in our work all strive to address this issue.

Our proposed ECBM further pushes the limit by utilizing an energy network. Specifically, by enabling the learning of compatibility between input $x$, concept $c$, and class label $y$, we aim to create a network that can capture features that are not readily identifiable. Looking at the empirical results (refer to Appendix C, Table 4), our ECBM achieves performance very close to the black-box model on the CUB and AWA2 datasets; for example, on CUB, the accuracy of the black-box model and our ECBM is $0.826$ and $0.812$, respectively. In some cases, such as with the CelebA dataset, it even improves upon the black-box performance ($0.343$ versus $0.291$). This can be attributed to our joint inference process of concepts and class labels.

[1] Progressive co-attention network for fine-grained visual classification. 2021 International Conference on Visual Communications and Image Processing (VCIP). IEEE, 2021.

Dear Reviewer XFNY,

Thank you for your time and effort in reviewing our paper.

We firmly believe that our response and revisions can fully address your concerns. We are open to discussion (before Nov 22 AOE, after which we will not be able to respond to your comments unfortunately) if you have any additional questions or concerns, and if not, we will be immensely grateful if you could reevaluate your score.

Thank you again for your reviews which helped to improve our paper!

Best regards,

ECBM Authors