A theoretical design of concept sets: improving the predictability of concept bottleneck models

Thank you for your in-depth review and for highlighting this theoretical work's high relevance to the NeurIPS community.

New experiments

The purpose of the experiments in our paper is to illustrate the theoretical insights, and we believe the current experiments serve that purpose well. However, to validate our insights further, and as per your comment, we have run experiments on additional datasets as suggested, including CINIC-10 for domain generalization. We refer to the 'Global Response' for the new results.

Questions

We emphasize our gratitude for the careful review shown by the questions posed, notably since each led to valuable improvements. Next, we indicate the resulting improvements to the paper.

Clarity

• We address the ambiguity regarding the meaning of $P$ in L99 with the following modification: "[...] $\\mathcal{D}=\\{(x_i, y_i)\\sim_{iid} \\mathbb{P}(\\mathcal{X}, \\mathcal{Y})\\}$ is a dataset drawn from the data-generating process $\\mathbb{P}(\\mathcal{X},\\mathcal{Y})$ [...]".
• Thanks for flagging the model typo. We use ViT B-32; we have updated it in the text in L275-276.

Linearity

Indeed, CLIP embeddings of images $CLIP:\\mathcal{X}\\rightarrow \\bar{\\mathcal{X}}$ are non-linear and result from a hierarchical composition of neurons. Our analysis does not assume linearity of such a mapping, but rather linearity of the mapping from the CLIP embedding $\\bar{\\mathcal{X}}$ to the concept representation, which is consistent with the CLIP objective [2] as stated in L150 ("where it assumed that concepts can be related to directions"), as well as linearity of the output layers, a standard practice for CBMs [1] and when using CLIP embeddings [2].

We understand our usage of $\\mathcal{X}$ to denote inputs from this embedding space might confuse readers, so we will denote the CLIP embedding space as $\\bar{\\mathcal{X}}$ throughout the paper and modify L149 to "To that end, we consider the input space $\\bar{\\mathcal{X}}$ as the joint embedding space of inputs and concepts from a foundational model [...]".

Redundant concepts

We agree that a more nuanced discussion would benefit L248.

As per Theorem 1, having a concept that does not decrease the misspecification $\\varepsilon$, which can happen either if it is spanned by the other concepts or orthogonal to the target function, will be detrimental to the risk since it increases the first term of the risk (see L177-178). This is what we mean by a redundant concept.

However, there will be many cases where the given concept will not be entirely redundant (i.e., only slightly affect $\\varepsilon$), and in these cases, the choice of keeping or discarding it will depend on the number of samples we have. Both our theoretical and empirical insights (see L281-293) conclude that in lower sample regimes, we should be more aggressive at discarding slightly redundant concepts, and as $n$ grows, we will benefit more from preserving them, as you point out.

We appreciate that you flagged the importance of providing a more comprehensive discussion of when to "remove redundant concepts" in L248 and connecting it to the theoretical insights. We have correspondingly included the above discussion as a footnote.

Examples

Motivation for CBMs

Your provided example nicely illustrates how CBMs can exploit the composition of concepts to perform tasks beyond CLIP's training distribution. We have correspondingly added a version of it in line 24. We believe it best to keep the presentation of CBMs in the first paragraph agnostic to the identification method (CLIP) for generality, so we slightly modified it to the following phrasing without discussing CLIP: "Indeed, despite never having seen a Tesla Model X, someone could reasonably classify them provided they can identify an SUV, a T logo, gull-wing doors, and the absence of exhaust pipes."

Non-Concepts

Let us start with a specific example and then move into a more formal characterization.

• We humans do not develop some concepts for fields outside our areas of interest (our interest is informative of our underlying task $\\mathcal{T}$) despite experiencing occurrences of such concepts. Consider, for example, filmography concepts such as "Jump cut," where two sequential shots of the same subject are taken from camera positions that vary only slightly (or a "Dutch tilt"). According to Def. 1, for people not interested in filmography, these characteristic functions would not be included in the concept set because they add negligible expressiveness to their current concept set, i.e., $N_0\\rightarrow \\infty$ without significantly contributing to the model-aware inductive bias, so $N_1 < N_0$, failing the definition.
• More abstractly, a characteristic function $c$ that does not provide additional valuable information on top of the existing set $\\mathcal{C}$ for $\\mathcal{T}$ will have low expressiveness (i.e., large $N_0$) without increasing $N_1$. This can occur when targets are independent of $c$ conditioned on $\\mathcal{C}$: e.g., concepts irrelevant to the task (e.g., tinted glass when predicting a Model X) or completely redundant with $\\mathcal{C}$. It can also happen when the output model cannot capture the correlation, e.g., $c$ might help cluster the input distribution into two concentrical spheres, but a linear model would not benefit from this. Further still, even if a characteristic function $c$ provides information that can be captured, if the inductive bias of the method disprefers the optimal hypothesis given this representation, once again, $N_0$ will be large.

We believe this discussion can be valuable to some readers, so we incorporate it in a new appendix and reference it in L109.

References

[1] P. W. Koh et al., "Concept Bottleneck Models".

[2] A. Radford et al., "Learning Transferable Visual Models From Natural Language Supervision".

Thank you for the response! I think the additional studies address my main concern regarding evaluation setup. I'm increasing my score to recommend Acceptance.

We thank you for your thoughtful evaluation and appreciate your recognition of our theory's insightfulness and clarity, as well as our results' unique perspective.

The use of LLMs for concept generation

The core objective of our method and experiments is to illustrate and validate our theoretical insights, which are agnostic to how concepts are sampled. In fact, we note that the first mention of an LLM as a generator happens in L229 in section 5.2. Consequently, using an LLM for sampling concepts in our experiments does not affect the practicality or generality of our theoretical insights. We employ LLMs to reduce the required human expertise and flexibly supply task-specific concepts, decreasing the barrier to adopting CBMs for new tasks.

As mentioned, our theory encompasses concept sets that are not dependent on LLMs. Options include human-generated concepts, concept graphs like ConceptNet, or leveraging correlations within text corpora like Wikipedia. Even hybrid approaches could be considered, with the goal of enhancing LLM generation by conditioning their context on external knowledge sources like ConceptNet or Wikipedia, as mentioned in L229-232.

Orthogonally, we also note that the method in section 5.2 can be regarded as a 'rejection-sampling loop' that partially palliates the practical limitations of relying on an LLM. This loop controls the quality of LLM generation through steps 2 and 3 by rejecting concepts that do not align with our theoretical criteria, thereby encouraging only high-quality concepts to be incorporated into the concept set.

To convey more effectively that this choice does not limit the practicality and generality of our theory and insights, we will include a version of the first two paragraphs above following L232 and a version of the third preceding L254.

New experiments

The purpose of the experiments in our paper is to illustrate the theoretical insights, and we believe the current experiments serve that purpose well. However, to further validate our insights and in agreement with your comment, we have run experiments on additional datasets and a new baseline. We refer to the 'Global Response' for the new results.

Once again, we express our gratitude for your comments. We hope that the new experiments and the discussion we incorporate about how our theory is agnostic of the generation method have addressed all your comments, and we look forward to your continued engagement with our research.

Thank you for your thoughtful review. We are glad you find the research question important and the insights interesting.

Notation

We agree with all the suggestions and have incorporated them. Specifically:

• Changes in the introduction:
  • We remove $\\theta$.
  • L22: we introduce $\\hat{g}$ with "[...] a CBM first identifies key concepts such as the presence of a semaphore or a pedestrian crossing the road ($C: \\mathcal{X}\\rightarrow\\mathcal{C}$), and learns $\\hat{g}:\\mathcal{C}\\rightarrow\\ \\mathcal{Y}$ to act based on them $(\\hat{g}\\circ C)(x)$".
  • L33: we define $\\mathcal{H},\\mathcal{H_C}$: "[...] the degree of misspecification $\\varepsilon$ as key drivers of the CBM performance, where $\\varepsilon=|| \\text{arg min}\_{f\\in\\mathcal{H}} \\ell(f) - \\text{arg min}\_{g\\in\\mathcal{H_C}} \\ell(g\\circ C)||$, with $\\mathcal{H}\\subseteq \\{f:\\mathcal{X}\\rightarrow\\mathcal{C}\\}, \\mathcal{H_C}\\subseteq \\{g:\\mathcal{C}\\rightarrow\\mathcal{Y}\\}$ the hypotheses spaces."
  • L48: the additions "concept-based models ($\\mathcal{R}(\\hat{g}\\circ C)$, $\\hat{g}\\in \\text{arg min}\_{g\\in\\mathcal{H_C}} \\ell(g\\circ C)$)" and "baseline counterpart ($\\mathcal{R}(\\hat{f}), \\hat{f}\\in\\text{arg min}\_{f\\in\\mathcal{H}} \\ell(f)$)" further clarify $\\hat{g}$ and $\\hat{f}$.
• L104: we thank you for pointing out this detail since there was a technical condition missing: either (1) if concept sets are unordered, then $\\mathcal{H_C}$ must be a complete set of maps under permutation symmetry, and the learning method invariant under feature permutations, which holds for the output methods considered in the paper; or (2) $\\mathcal{C}$ is a concept sequence, not a set. In the first case, the composition is well-defined since it is equivalent across concept reorderings; in the second, $\\cup$ should be substituted by $\\times$. We make this explicit in L101 by adding, "Consider the output hypothesis spaces $\\mathcal{H_C}\\subseteq\\{g:\\mathcal{C}\\rightarrow \\mathcal{Y}\\}$ complete under permutation symmetry and the learning algorithm invariant under feature permutations. We refer to Appendix B for a technical discussion of this assumption.". We correspondingly added Appendix B with an extension of the above discussion, including the full alternative definition and what methods satisfy assumption (1).
• We update Thm 1's statement to "Under the above setting and assuming that $\\mathbb{P}$ is an isotropic distribution on the $d-$dimensional input" to introduce $d$ unambiguously.
• To avoid confusion, we will denote the CLIP embedding space as $\\bar{\\mathcal{X}}$, and we modify L149 to "[...] input space $\\bar{\\mathcal{X}}$ as the joint embedding space of inputs and concepts from a foundational model [...]".
• We increased the figures' fonts. This is shown in the new figures; previous ones have been modified correspondingly.

New experiments

The experiments in our paper aim to illustrate the theoretical insights, and we believe the current experiments serve that purpose well. However, in agreement with your comment, we have run experiments on additional datasets and a new baseline to further validate our insights. We refer to the "Global Response" for details. We also included throughout appendices C.4-6 filtered concept sets from our experiments.

Concepts as a stable interface

This flexibility means we can replace the underlying concept identification model with a more advanced version in the future while keeping the output model unchanged. The concept set serves as a stable communication interface with fixed dimensionality and functionality. Additionally, concepts can act as common representations across different feature sets, potentially harmonizing datasets and modalities.

We welcome the opportunity to expand on these observations, which cover exciting properties that could lead to impactful future works.

Suppose we aim to learn $f^*:\\mathcal{X}\\rightarrow \\mathcal{Y}$, and let us consider a CBM counterpart, with the concept representation $\\mathfrak{C}^*:\\mathcal{X}\\rightarrow \\mathcal{R}\_{\\mathcal{C}}$, followed by a mapping $f\_\\mathcal{C}^*:\\mathcal{R}\_{\\mathcal{C}}\\rightarrow \\mathcal{Y}$ from concepts to targets. By keeping the concept set $\\mathcal{C}$ fixed (and thus the representation $\\mathcal{R}\_{\\mathcal{C}}$), we can replace an estimator of the concept mapping $\\hat{\\mathfrak{C}}$ by another $\\tilde{\\mathfrak{C}}$ while the probe $\\hat{f}\_\\mathcal{C}$ remains effective, only affected by a distribution shift. E.g., if we train the probe on a CLIP-based concept representation and migrate to a new CLIP2, our probe will remain applicable to the updated and more precise embedding. However, a probe cannot be swapped from the CLIP raw representations to the ones from CLIP2 since they will have different metrics or even dimensionality. That is, the concept representation $\\mathcal{R}\_{\\mathcal{C}}$ is a stable interface with fixed dimensionality and meaning.

Furthermore, suppose we have datasets with different features or modalities $\\mathcal{X}_1, \\dots,\\mathcal{X}_k$ (e.g., images from different sensors or angles). In that case, we can use a common concept set $\\mathcal{R}\_{\\mathcal{C}}$ which has fixed semantic meaning, and project all inputs there $\\mathfrak{C}^*_i:\\mathcal{X}_i\\rightarrow \\mathcal{R}\_{\\mathcal{C}}$, thereby harmonizing them.

Some readers might be interested in delving deeper into these ideas, so we include a version of the above text in the appendix and refer to it in L139.

Thresholds

We keep a concept if it improves the estimated risk, i.e., $R_m=0$, or $\\hat{\\mathcal{R}}(\\hat{f}\_{C_i}) - \\hat{\\mathcal{R}}(\\hat{f}\_{C_{i-1}})>0$, and we set the entropy threshold proportionally to the number of buckets $n_b$ used to estimate $\\Omega$ to ensure not all samples fall into the same bucket (i.e. $\\Omega_m=\\frac{1}{n_b}$). We will specify this as a footnote in L253.