From Causal to Concept-Based Representation Learning

We thank the reviewer for their review. We address their comments below.

This approach sacrifices causal semantics. This can be particularly problematic in situations where a deep understanding of causality is crucial

In this work, we focus on the many applications where a relaxation of causal semantics is allowed or necessary, such as when interventional data is out of reach (L46-50, L118-124). While a deep understanding of causality would be ideal, concept learning is itself a mature field independent of causality and we present a middle ground that is useful in contemporary applications.

The author has made numerous assumptions within the article, which could potentially impact the universal applicability of the theory presented.

We would like to highlight that without any assumptions, we cannot have identifiability (L29-31), therefore we make natural assumptions to prove our results. However, we tried to keep the assumptions as general as possible, e.g. we allow non-linear mixing (Assumption 1) and use Gaussian noise (Assumption 3).

We welcome additional suggestions to improve the work.

We thank the reviewer for their careful review and are glad that they appreciate our contributions towards the important open problem of bridging causal representation learning and concept based representation learning.

The experiments on CLIP and LLMs seem to me unrelated to the theory...How is this related to the theory the authors propose?

The CLIP experiments showcase that the learned representations of concepts indeed lie approximately within an affine hyperplane which is an illustration of our key hypothesis. Moreover, the training data approximately satisfies our assumptions if we view the caption as the environment, i.e., images with similar/equal captions form a concept conditional distribution and the joint image distribution is the observational distribution. Our experiments also show that the representations of different CLIP models agree up to linear transformation. As noted by the reviewer, a more rigorous connection would be to establish if CLIP models learn a representation such that concepts are represented by affine hyperplanes and whether concept conditional distributions are approximately given by eq. (2). In such a case, we can apply our identifiability theory to conclude that different CLIP models necessarily learn approximately the same representation up to linear transformation. Proving these facts rigorously for the representation based on the CLIP-loss is a challenging problem we leave for future work.

The LLM experiment is a bit more exploratory and the connection to our theory is via our conceptualisation of concepts. Indeed, this helps us to obtain intuition what the right steering vector should look like and guided our construction.

It would have been more useful to provide experimental evidence ... on semi-synthetic datasets like MPI3D or Shapes3D. Some real-world datasets ... like CUB200 and OAI [5], CelebA [6], and many more [7,8].

We appreciate the nice suggestions. Since the primary thrust of the work is theoretical and it is currently quite challenging to scale these type of methods to large datasets, we leave this interesting direction for future work.

The synthetic experiment is impossible to understand from the main text and many details ... are confined to appendices...How do you evaluate $A^e$?

We apologize that this part is difficult to understand as details were deferred due to lack of space. In order to evaluate $A^e$ in our synthetic experiments, we compute the linear correlation metrics $R^2$ and MCC between the ground truth $Z$ (restricted to the projection space) and the predicted $Z$ (from our model), which we report in our tables. As outlined to other reviewers, we will use additional space to add experimental details to the main text.

About assumptions in general, we believe our assumptions could likely be further relaxed at the expense of more technical work, which we leave for future work.

Assumption 1 is fine and is common in studying identifiability. One could hope to extend the results also to non-invertible functions. However, this is not a serious limitation given the novelty of the results.

We agree.

Assumption 2 takes the concepts to be linearly independent, what happens if for some of them is not the case?

Then we can in general not identify the concept matrix, think, e.g., of the case where two concepts are collinear.

Assumption 3 requires a Gaussian distribution for the noise, it is remarked that other distributions work as well (which distributions?) but there is no citation.

The result could be extended to exponential families, we will clarify this and add a reference (see e.g. Khemakhem et al. [50]).

How should Assumptions 4 and 5 be understood, what data are expected to be collected?...how data should be gathered for the theory to work, and thus the scaling of the proposed method.

Assumptions 4 and 5 state that the environments need to be sufficiently diverse. We do not think that the approach should be used to collect data in practice because this would indeed require a lot of prior knowledge which could probably be directly used to learn the concepts. Instead we think that often real-world data comes with subtle heterogeneity that allows us to identify concepts.

Causal Representation Learning seems more of an inspiration to the paper... not being essential to support their claims and the connection.

We generally agree with the reviewer and refer, e.g., to the discussions in L43-50, 60-63, for this remark. We're happy to add more details to clarify this.

I was also expecting to see a comparison to Concept-based models' current practices

We appreciate the relevant references and are happy to include them in the paper. In short, some of these works study very useful empirical methods whereas we focus on rigorous theoretical contributions, whereas the others differ in the kind of assumptions and settings studied, i.e. they're related but not directly comparable at a technical level.

Recovering the concepts without post-hoc analysis?...One in practice has still to find the matrix $A$ related to the concept, and that cannot be done without concept annotation. Is that the case?

Yes, our identifiability is only upto linear transformations. After having learned the nonlinearity, in order to recover the transformation itself, additional information such as concept annotation may be utilized in practice.

It seems that $d_Z$ and $\tilde{d_Z}$ have to be the same to guarantee the inverse exists (and how it is used in the proofs). Is that the case? Would the theory hold also for models with different latent dimensions $d_Z \neq \tilde{d_Z}$?

Yes, for technical reasons this is the case. To consider $d_Z\neq \tilde{d_Z}$ we need to relax the injectivity assumptions otherwise $d_Z$ has to correspond to the dimension of the data manifold.

We hope the rebuttal answered the questions raised, and are also happy to take additional feedback to improve the writing.

Thank you for your clarifications. After reading the reply and other reviews I feel some parts of the paper still present the limitations we pointed out (experimental section). I have some comments for your rebuttal:

We appreciate the nice suggestions. Since the primary thrust of the work is theoretical and it is currently quite challenging to scale these type of methods to large datasets, we leave this interesting direction for future work.

I understand that scaling to real-world datasets is challenging and a bit out of scope. However, it should be shown that the method can applied to semisynthetic datasets (even MNIST, which is 28x28 dimensional), otherwise leaving me puzzled about the practical utility of the theory and method proposed. Given the theoretical focus of the authors, I accept their decision to do that in future work.

The CLIP experiments showcase that the learned representations of concepts indeed lie approximately within an affine hyperplane which is an illustration of our key hypothesis.

This is ok. It seems to me more of a verification of an interesting empirical fact rather than a consequence or an explanation that is already captured by the theory.

In such a case, we can apply our identifiability theory to conclude that different CLIP models necessarily learn approximately the same representation up to linear transformation. Proving these facts rigorously for the representation based on the CLIP-loss is a challenging problem we leave for future work.

That would be interesting, but I would expect the theory would require several adjustments to consider the captions to train visual representations.

The LLM experiment is a bit more exploratory and the connection to our theory is via our conceptualisation of concepts. Indeed, this helps us to obtain intuition what the right steering vector should look like and guided our construction.

Similarly, this is an interesting empirical fact, but not explained by the theory.

As outlined to other reviewers, we will use additional space to add experimental details to the main text.

Thank you.

We do not think that the approach should be used to collect data in practice because this would indeed require a lot of prior knowledge which could probably be directly used to learn the concepts. Instead we think that often real-world data comes with subtle heterogeneity that allows us to identify concepts.

What do you mean by real-world data having this heterogeneity, is it something that could be verified? It seems to me that you have first to assume what are your concepts of interest (being linearly related to generative variables) and then to have those environments. Both points seem challenging.

We generally agree with the reviewer and refer, e.g., to the discussions in L43-50, 60-63, for this remark. We're happy to add more details to clarify this.

Thank you.

In short, some of these works study very useful empirical methods whereas we focus on rigorous theoretical contributions, whereas the others differ in the kind of assumptions and settings studied, i.e. they're related but not directly comparable at a technical level.

I also agree with this. Common practice has been shifting towards using concept-supervised examples (which trivially address the identifiability of concepts) which is a limitation and your theory is very relevant in relaxing it.

Yes, our identifiability is only up to linear transformations. After having learned the nonlinearity, in order to recover the transformation itself, additional information such as concept annotation may be utilized in practice.

Thank you for the clarification. It would be interesting to connect to [1] to see if the same can be done without concept supervision.

[1] When are Post-hoc Conceptual Explanations Identifiable? Leeman, UAI 2023.

We thank the reviewer for their detailed review and nice summary of the paper. We also appreciate their insightful comments on the importance of this work.

In the synthetic experiments, I was expecting to see large $d_z$ and small $n$

Thanks for the suggestion, we ran additional experiments and included the results in the global response. We kept $n$ as $4$ and increased $d_z$. While the metrics naturally degrade a bit, they're still comparable to other nonlinear CRL works.

Could the authors think of any experiment to contrast concept learning to CRL?

This is an interesting suggestion and would potentially add to the empirical evidence. However, it is beyond the scope of our work to compare against CRL comprehensively, partly because we're studying settings where CRL is not directly applicable (e.g., due to lack of interventional data). Moreover, our work is primarily theoretical and we leave for future work to extensively compare concept learning methods to CRL techniques empirically.

I'm also a bit unclear about where this work is taking us, and would have liked it if it was explained better. In particular, are we hoping to change our representation learning towards learning concepts?

Our hope with this work is to provide concept learning a rigorous footing via the theory of identifiability. We do not focus on a specific task and instead build a conceptual bridge between causal representations and concept learning. While causal representations may be ideal or even necessary for some tasks, our work presents a middle ground for many contemporary settings where this may not be possible (please see also L43-50). As reviewer ohha points out, "Bridging identifiability in (causal) representation learning and concept-based learning is an open problem" and this is one of the main motivations of our work. We will include this discussion in the paper.

The notion of environment in the context of concepts appeared out of the blue on page 7

Thank you, we will make sure to introduce this terminology carefully.

Not a weakness of the method - but the learning method seems like an important component of the paper which is deferred altogether to the appendix

We agree that the contrastive learning method is an important component of the work, however, we have regrettably deferred the details to Appendix F due to lack of space. With the additional page available in the final version, we are happy to include this in the main paper.

We next address the questions in order.

Could you connect/contrast this work the recent advances in sparse autoencoders (SAEs)?

We're briefly aware of works on SAEs that learn interpretable features in models. However, to the best of our knowledge, they do not provide theoretical identifiability guarantees for learning concepts, which our work endeavors.

If we care about a few features, why not use CRL methods that guarantee weaker identification requiring much less interventions?...the multi-node intervention line of work (cited by the authors) alleviates the challenge of perfect interventions and relaxes the requirement of learning all of $Z$

Could the reviewer please clarify which specific work they mean? We're more than happy to comment further and have a proper discussion on this in the paper as well.

Does equation 2 come from the independence of concepts. Why is not $k = dim(C)$?

Yes, equation 2 arises if the noise for the noisy estimates is independent for all atomic concepts. The product runs from 1 to $\mathrm{dim}(C)$ so our expression is the same as $\prod_{k=1}^{\mathrm{dim}(C)}$.

Line 294 onwards, is $e$ properly defined before and used here?

We will clarify that $e$ is just an environment label for a concept conditional distribution.

Line 304, is $S_n$ a typo? $n$ was used as superscript before that, here it's used as a subscript.

No, here $S_n$ corresponds to the permutation group on $n$ elements, we will use a different font to make the distinction clear.

Could assumption 4 be explained in words as well? Where does it come from? When/in what situations does it break?

This condition is similar to other diversity conditions in identifiability theory. It ensures that there are sufficiently many, non-redundant datasets. It breaks, e.g., if there are fewer datasets than atomic concepts of interest or when several of the concept conditional datasets disagree.

How should one interpret table 2 of the appendix? Is there a reason why it's not plotted, and presented as numbers? Shouldn't we expect a linear plot?

Note that for the hue variables the different numbers correspond to different colors but the numbers do not correspond to meaningful concept valuations, but are just discrete labels for the different colors. Therefore, we do not expect to recover a linear relation between the evaluated concept valuations and the label index. This is different for attributes such as, e.g., size or orientation where the value should correspond approximately to the valuation (potentially up to a non-linear transformation). Note that it is moreover not clear whether color is an atomic concept as we assume here (e.g., standard color representations are 2 dimensional). We nevertheless observe a high correlation coefficient between the representations learned by different models (see Table 5).

Line 165, what do you mean by entangled concepts...Related to independence of concepts: Can the theory say anything about hierarchical concepts similar to hierarchical representations?

We would like to clarify that in assumption 2, we talk about linear independence (and not statistical independence) of atomic concepts. However, the concepts we actually allow can each consist of multiple atomic concepts and different non-atomic concepts can overlap (not superposition), which can be interpreted as hierarchical or entangled concepts as well.

We thank the reviewer for their suggestions and welcome additional feedback to improve the text.

The works that I can recall are: https://proceedings.mlr.press/v238/ahuja24a/ahuja24a.pdf, https://arxiv.org/pdf/2311.12267, https://arxiv.org/pdf/2406.05937 (there are probably more)

Thanks for clarifying. We have taken another look at these papers, and to the best of our knowledge, these papers still require a number of environments that is lower bounded by the dimension of the latent space $d_z$ (with few notable exceptions with purely observational data, on which we comment below, please see also the footnote in page 3 where we state this). Thus, these papers do not seem to support the claim that there are "CRL methods that guarantee weaker identification requiring much less interventions". If we are mistaken, and the reviewer can pinpoint a specific result in one of these papers that accomplishes this, we would be happy to discuss further.

It is true that the work https://proceedings.mlr.press/v238/ahuja24a/ahuja24a.pdf (reference [2] in the paper) and similarly [54, 40] show identifiability from just a single environment as we state in the footnote in the paper. Note, however, that they all make restrictive assumptions on the mixing function and on the distribution of all variables of the latent space, e.g., in the case of [2] it is assumed that the mixing function is polynomial and the support of the latent variables is the Cartesian product of bounded intervals. Therefore, neither these results nor their techniques extend to general mixing functions or settings where we only make assumptions on the distribution on some of the latent variables. This contrasts with our work, where we make minimal assumptions on the mixing function and only make assumptions on the latent distributions with respect to the concepts of interest.

However, since the paper contrasts itself a number of times to CRL requiring many interventions, I don't see it totally beyond the scope of this work to make a minimal effort at contrasting to basic CRL methods. I agree with the authors that the motivation for concept-based learning is to go beyond where CRL is not applicable, but still would find it compelling if there could be a setup where the advantage (in terms of the fewer number of domains required) could be showcased.

Thank you for the question. Making an apples-to-apples comparison to CRL is not straightforward because the goals are different. However, we can say the following:

• Experimentally, we would like to clarify that in cases where CRL is also applicable, our algorithm, which is similar to Buchholz et al. [13], can always be applied. Indeed we clarify in the paper that our algorithm is inspired by their works, please see L370-371, 1451. However, in the case of sublinear number of environments in our setting, standard CRL techniques are not applicable whereas our work is applicable, and this is the main motivation for our paper.
• Theoretically, if we can intervene as per the concept distribution we work with (as also described in L129-134), then our results show that $2n$ concept interventions suffice to learn $n\ll d_z$ concepts, and existing methods would not handle this setting to the best of our knowledge (please see also Appendix C for alternate technicalities). Thus, in our setting, current CRL results require $\Omega(d_z)$ interventions, whereas we only need $o(d_z)$ interventions.

Of course, part of our contribution is to propose a different goal (learning concepts) under different assumptions (conditioning vs. intervention), which makes such comparisons difficult. Nonetheless, we hope this offers some insight to understand how our work compares.

We thank the reviewer for their positive review and are glad they find the paper novel, well-written and rigorous. We address the weakness below.

In other words, the requirement of data partition $X^0, \ldots, X^m$ goes against the motivation of the proposed framework of handling continuous concept valuations.

Apologies for the confusion, but we would like to clarify that our setting indeed handles continuous concepts (as claimed) as follows. The actual valuations could be continuous (e.g. we allow Gaussian noise) and our identifiability results do hold in such settings. In other words, the concept conditional distribution do not condition on a fixed value but rather allow for noise. Let us consider this using the example of intensity which was brought up in the review. Assume that we have datasets consisting of images taken at different times of the day. Then the intensity of the colors will fluctuate within each dataset due to slight variations of the time or weather conditions, but they will fluctuate around different mean valuations for each dataset. This matches our assumptions for this concept with continuous valuations.

We also note that the experiments we conduct also involve some continuous valuations, since the latent points lie in only an approximate hyperplane. However we acknowledge that our methods may have not been fully probed in complex non-discrete settings and we leave this exciting direction for future work.

We now address the questions raised.

(line 208) Can you elaborate? I mean isn’t $A$ a projector matrix?

The formal definition 1 allows $A$ to be any linear transformation (e.g. it can be scaled), but as the reviewer noticed, there is no loss in generality in assuming it's a projector matrix, and we choose this definition for technical convenience.

Is there any way to quantitatively measure concepts learned by two different models are how much linearly-related to each other?

Yes, one option are the $R^2$ metric and the Mean Correlation Coefficient (L373-375) that we report in our synthetic experiments (while we compute them against the ground truth since we know it in the case of synthetic data, these can also be computed across models). Indeed, in our CLIP experiments, we use the correlation coefficient to measure to what degree the learned concepts for two different models are linearly related (L1177-1188).

The proposed method using contrastive learning should be described in the main section in more detail. Currently, it appears at the appendix, but I think that the algorithm is a key part of the paper which illustrate the practical utility of the proposed framework.

We appreciate the reviewer's acknowledgment of the contrastive learning method. However, we have regrettably deferred the details to Appendix F due to lack of space. With the additional page available in the final version, we are happy to include this in the main paper.

We thank the reviewer for the additional typographic suggestions and welcome additional feedback to improve the paper.