Boosting Concept Bottleneck Models with Supervised, Hierarchical Concept Learning

W1.1: The definitions of unintended and additional information

Thank you for your constructive question. In Definition 2.1, we closely adhere to the frameworks established by [b1] and [b2]. We define "unintended" and "additional" information within the context of informational leakage as referring to the extra data present in the soft concept probabilities output by the concept predictor.

W1.2: How do we ensure LLM concepts match human intentions?

To guarantee that intended concepts are also the intention of the human users, we have outlined a thorough validation process in the main text, specifically in Section 5.3. Our study involved the use of Amazon Mechanical Turk to conduct human expert validation.

W1.3: The definitions of perceptual and descriptive concepts

We clarify that "perceptual concepts" are basic, observable features of objects identified, which are described using nouns without any qualifiers (e.g., legs, heads, wings). The "descriptive level" involves detailed interpretations of these concepts, adding adjectives for richer descriptions (e.g., "small, red wing"). This distinction emphasizes the difference between observable features and detailed attributes. Our framework highlights the recognizability of perceptual concepts, despite individual differences in interpretation.

W2: Statistical significance

Thank you for the suggestion. We have revised the “should be” to “is” to avoid ambiguity. We conducted a standard $t$ test and our null hypothesis is that “users randomly choose the results”. Our $p$-value, which is 0.0003 and is significantly lower than 0.05, suggests that we can reject the null hypothesis.

W3: Evaluation metric

Thank you for your question. We apologize for not explicitly stating the evaluation metric in our original text. In Table 1, we utilized accuracy as the primary evaluation metric to measure performance. Accuracy was chosen for its straightforward interpretation and relevance to classification tasks. The vanilla model achieves 88.8% accuracy on CIFAR-10 using the non-finetuned CLIP visual extractor. This choice ensures fair and consistent comparisons, as per reference [b3,b4,b5]. The non-finetuned CLIP excels in zero-shot learning and high-quality feature representation.

Q1: VLM case study

Thank you for your great question. Based on your suggestion, we compared our method with two advanced fine-tuned VLM baselines and added this analysis to appendix section E. The comparison highlights that while VLMs offer vivid descriptions, their susceptibility to hallucinations make them less reliable. SupCBM ensures accuracy and efficiency, making it a robust choice for reliable model explanations.

Q2: Extendability

Regarding extendability, our approach efficiently accommodates new classes without impacting existing values. Assuming the original intervention matrix has the shape $num\_{con} \times num\_{cls}$, where $num\_{con}$ is the total number of concepts in the dataset. When a new class is arrived, we obtain its two-level description (with a total number of $p \times q$ concepts). We then extend the original intervention matrix to $[num\_{con} + (p \times q)] \times [num\_{cls} + 1]$. In the extended matrix, we only update the values for the new concepts and the new class. This update specifically affects the section starting from the row index $num\_{con}+1$ and at the new class column. The rest of the matrix remains unchanged. Therefore, adding a new class to the matrix does not affect the values for existing classes. Our SupCBM supports the claim of extendability even when the backbone models do not inherently support new classes, highlighting the robustness of our approach.

Q3: Target relevant concept focus

As detailed in our main text, when the i-th concept is relevant to predicting the j-th class, we assign a value of 1 to the I_{ij} position in our intervention matrix. This ensures that SupCBM focuses exclusively on the relevant concepts, otherwise, the importance of non-relevant concepts is set to 0.

Q4: Oracle concept sets

SupCBM is a post-hoc Concept Bottleneck Model (CBM) that uses LLMs to generate textual concepts from images without manual annotations, similar to the methods in [b3,b4,b5]. In this approach, LLMs act as the Oracle, creating "ground-truth" concepts due to the absence of predefined ones in the dataset. We emphasize the need for human expert validation of LLM-generated concepts and propose a human study evaluation in Section 5.3 to assess the interpretability and understandability of SUPCBM and other post-hoc techniques' predictions by their Oracles.

[b1]: Promises and pitfalls of black-box concept learning models

[b2]: Addressing leakage in concept bottleneck models, NeurIPS ‘22

[b3]: Post-hoc Concept Bottleneck Models, ICLR ‘23

[b4]: Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification, CVPR ‘23

[b5]: Label-Free Concept Bottleneck Models, ICLR ‘23

I would like to thank the authors for their response. Some of my concerns are resolved. However, I still have serious concerns about the statistical rigours of this work.

In the author's response, they stated that a standard t-test is conducted, and the null hypothesis is that "users randomly choose the results". I am confused about how this is conducted, as the hypothesis of t-tests revolves around the mean of a population.

Q1: How does a two-level concept set affect SupCBM?

The two-level architecture is designed to enhance the expressiveness of the current CBM model. Using the CUB-Bird dataset as an example, each bird is characterized by common perceptual features like the head, beak, wing, and leg. If we only include perceptual concepts, the total number of concepts would be limited to around dozens, making it challenging to identify the specific bird types without descriptive concepts to capture the unique semantics of each bird. To illustrate this, we constructed a SupCBM variant using only perceptual concepts (thirty-five in total), which resulted in a poor accuracy of just under 30%. This demonstrates that relying solely on perceptual concepts is insufficient for identifying the unique differences among birds, thus underscoring the necessity of our two-level approach.

Q2: Addressing LLM limitations for consistent and reliable concept generation

Thank you for your insightful question. To address the limitations of using LLMs' concept generations on consistency and reliability, we conduct an ablation study on selecting kernel size q and stride p for LLM concept generation in Appendix F.1 which shows that LLM-generated knowledge remains consistent and reliable up to a certain concept threshold, indicating a saturation point in their conceptual expansion capability. Therefore, selecting appropriate parameters is crucial to ensuring the continued reliability and consistency of LLM-generated concepts.

Q3: Altering p and q?

Thank you for providing the detailed analysis. We clarify that the changes observed in performance due to altering the parameters 'p' and 'q' (p=7 and q=8) fall within the acceptable variance range noted in our randomness tests (Table 1). This indicates that the variations in performance are not substantial, reflecting low sensitivity to parameter adjustments and suggesting robust performance for SupCBM.

Q4: Injecting domain knowledge into the intervention matrix

We clarify that the construction of our intervention matrix enables embedding domain knowledge into our SupCBM. This process involves first extracting domain knowledge through a dual-level description prompting framework. Subsequently, we integrate these binary weights into the matrix, denoted as either 0 or 1 for the i-th concept in the j-th class.

To further show the effectiveness, in section H the Appendix, we utilize MedGPT, a medical GPT fine-tuned from ChatGPT, to enhance SupCBM’s domain specific capability. The results show that the performance of SupCBM can be further improved by domain-specific LLMs. Hence, when generating domain-specific concepts (e.g., in medical applications) for SupCBM, we recommend using domain-specific LLMs for related tasks in future work to ensure a high level of safety.

W1.1: Concept sizes and CBM performance

Thank you for your insightful question, we would like to clarify a few points regarding concept sizes and their performance. Firstly, more concepts do not necessarily result in better performance. In fact, even if they do, they often come at the cost of significant information leakage. For example, LaBo, the second-best baseline in our table, maintains around 10,000+ concepts across all four datasets. Although it performs relatively well compared to other baselines, it suffers greatly from information leakage.

On the other hand, SupCBM maintains approximately 2-3k concepts per dataset, which is a size similar to other baselines' concept sets. Despite having fewer concepts, SupCBM achieves the highest performance while suffering the least from information leakage. This indicates that SupCBM can maintain a moderate concept size not only enhances performance but also minimizes the risk of information leakage.

W1.2: What if the intervention matrix is replaced by a random matrix?

To demonstrate the role of our matrix, we conducted an intervention experiment (as described in Appendix C.1) to assess its model interpretability. Following your suggestion, we also included SupCBM with a random binary matrix in the table (dubbed $SupCBM\_{rand}$). Our results show that SupCBM achieves the highest interpretability by successfully fixing the most labels correctly, however, the random matrix performs the worst in this task, displaying unpredictable behavior.

W2: Overemphasizing the information leakage issue?

We would like to clarify that this work does not overemphasize the issue of information leakage. Rather, our work highlights how leakage undermines the trustworthiness and interpretability of the model. The findings in Table 3, located in Appendix A.2, demonstrate that the leakage issue can cause unstable CBM behavior, even with minimal perturbations to the visual input. Furthermore, when comparing the leakage results with the intervention experiments described in Appendix C.1, it becomes evident that models with higher resilience to leakage exhibit stronger intervenability.

W3: Limitations and insights of LLMs with diverse datatypes

To further discuss the limitation regarding the performance of off-the-shelf LLMs on diverse datasets, we conducted a semantic comparison, evaluating the reliability by comparing GPT-generated and expert-defined concepts in the HAM10000 dataset, as updated in Appendix H. The result indicates that while GPT-4 provides comparable results, fine-tuned domain-specific LLMs like MedGPT can offer more accurate descriptions. Therefore, in terms of data diversity limitations, we recommend utilizing expert domain large language models (LLMs) for generating domain-specific concepts for SupCBM in future work. This approach is particularly important to ensure a high level of safety, especially in medical applications.

W4: Why SupCBM is not sensitive?

As demonstrated in Table 8 of Appendix G, SupCBM exhibits robust performance across various types of LLMs. This robustness is attributed to SupCBM’s highly stable two-layer hierarchical concept layer construction, which ensures LLMs are confined within the range of simple perceptual and detailed descriptive levels of visual understanding, thereby stabilizing their performance both in construction and execution.

C/S1-4: Concept generation in safety-critical domains

Based on your kindest suggestion, we used the HAM10000 dataset [a1] to evaluate GPT-generated concepts against expert-defined concepts in the medical domain. This analysis is updated in Section H of the Appendix. SupCBM shows promising alignment with expert-defined knowledge, but results could improve with domain-specific LLMs. For medical applications, using domain-specific LLMs is recommended to ensure safety. We have also conducted a stability test across four datasets, running GPT three times with the same prompts. The results were identical across all runs. In terms of human expert validation, section 5.3 presents a human validation process with 100 questions involving the CUB-Bird dataset and concept predictions from various CBMs, including SupCBM.

We recommend the following guidelines:

• General Recognition Datasets (e.g., CIFAR): When dealing with general recognition datasets, the use of general large language models (LLMs) is sufficient to extract useful generalized feature information.
• Sensitive Fields (e.g., Medicine): For sensitive areas such as medicine, we strongly recommend employing LLMs with expert-defined knowledge. To ensure safety and accuracy, it is advocated to utilize LLMs that have been trained with expert-defined knowledge when generating concepts.

C/S5-7: Scalability

We clarify that the computational complexity of SupCBM during inference is equivalent to that of a standard matrix multiplication in a single fully-connect layer, namely, $O(num\_{con} \times num\_{cls})$, where $num\_{con}$ represents the size of the concept set and $num\_{cls}$ denotes the number of classes. As the concept set size increases, this remains computationally efficient, similar to standard matrix multiplication.

We clarify that, when the size of the concept set grows, conventional CBMs require re-training the entire model (the CB layer + the label predictor) to adjust their fully connected layers for inference. However, due to our intervention matrix design, SupCBM bypasses the need to re-train the FC layer. Specifically, upon the arrival of a new $i$-th concept, SupCBM enhances its intervention matrix by introducing a corresponding new ith row. In this row, the weight for the $j$-th class is iteratively set to either 0 or 1, based on the relevance of the $i$-th concept to the $j$-th class.

C/S8-10: The concept of bottleneck models

Our SupCBM assigns either 0 or 1 (i.e., only determining whether a concept is involved in label prediction without weighting) to the concepts for every class. This mechanism ensures that the intervention matrix is utilized to determine the final label prediction solely based on the presence or absence of concepts, which is the design principle of CBM. That said, instead of assigning relative importance in the intervention matrix, SupCBM does this in the concept bottleneck (CB) layer. For example, if an image shows an animal with prominent wings, the CB would assign a score of 0.9 to the “wing” concept to heavily represent the degree of birdness in the image.

C/S11-13: The model's design

First, we would like to clarify that the scenario where the case A = B = ∅ is extremely rare. We listed this in the paper to ensure that our analysis covers all possible cases. All CBMs, including SupCBM, by design, are expected not to distinguish classes if their corresponding concepts are indistinguishable. During rebuttal, we thoroughly checked our codebase across all four datasets used in our paper, and none of the classes are completely conceptually identical to the others.

C/S14-17: The impact of meaningless hard concepts

[a2] pointed out that performance gains in hard CBMs from adding meaningless concepts may stem from a sufficiently informative set. Similarly, [a3] observed a significant performance boost when the concept set was doubled with meaningless hard concepts. In our comparative study, we compare the behavior of SupCBM to that of hard CBMs. We construct the following models: SupCBM_hard is a hard CBM modification of SupCBM. They were all trained using two concept sets: the original set and an expanded double set that includes meaningless concepts. We tested these models using the CUB-Bird dataset, and the table below shows their accuracy. The SupCBM still outperforms, while the hard model shows a performance boost with doubled concepts, confirming previous findings.

[a1]:The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions, Nature ‘18

[a2]: Promises and pitfalls of black-box concept learning models

[a3]: Addressing leakage in concept bottleneck models, NeurIPS ‘22