Concept Bottleneck Generative Models

Thank you for the feedback. We each of your key points here.

Originality

Thank you for raising this point, we take it to mean that we should make the case for concept-bottleneck generative models more explicit in the paper. We addressed your originality point in the general comment, and have updated the first paragraph of the introduction to reflect the change. However, we make some additional points here.

• VLMs: Current large-scale vision language models do not include intrinsically interpretable components that allow one to control and interpret/debug them. Importantly, when these models make mistakes, there is no way to identify the factors upon which they base generation, which makes them challenging to fix. As we show in Appendix C, a concept-bottleneck layer can be added to a VLM to better understand and fix the model’s mistakes. Our proposal is not to replace VLMs/LLMs with CBGMs; to the contrary, we actually see our work as an initial step towards augmenting these models with intrinsically interpretable components that will make them easier to debug. We return to this point again in response to your concern about significance.

• Conditional Generation: Our goal is not solely conditional generation. We seek a generative model with an intrinsically interpretable component that can be used to simultaneously explain, debug, and steer the model. Current conditional generation approaches do not come with the ability to explain the model.

• Glancenets (VAEs in concept-bottleneck): As you mentioned, Glancenets are not model-agnostic (i.e., they are designed specifically for VAEs) and were not designed to support steerability.

Quality

Thank you for your feedback on the experiment section. We clarify here the goal of these sections.

• Steerability Metric: The metric is exactly what you described. We start with a high-accuracy classifier trained on real data; we forgot to mention that we ensure that the minimum accuracy of any concept classifier is at least 98% on a held-out test set from the real data. We sample noise latent vectors and then use it to generate images from the model; for each concept we pass the generated image to the concept classifier; we save the latent vector if the classifier predicts the concept to be absent (i.e., the probability of the concept present is less than 0.5) we continue this process until we have 1K samples. For each conditional generation approach (including CB layer), we intervene on a concept and generate new images. We then pass the newly generated images to the classifier again and measure the fraction of inputs for which the indicates greater than 0.5 probability of concept presence, which we report in Table 1. The classifiers were trained on real data, but the steerability metric is on synthetically generated data, which might cause an adversarial effect or the classifier might consider the samples out-of-distribution causing the accuracy to be low. However, since we use the same classifier across all of the models and steerability approaches, we believe that the metric is fair i.e. the model ranking is reliable. The steerability test and metric that we measured is designed to be a difficult test that aligns with what people typically seek in practice.

• Intrinsic Interpretability and Debuggable Components: We now clarify the interpretability and debugging experiments. First, the goal of the experiments that we show in the interpretability and debugging section is to demonstrate capabilities that were previously not possible for other generative and conditioning approaches. Specifically, we demonstrate that the output of the CB layer can be used to determine the features that the model relied on. We note that in the literature, quantifying interpretability and debuggability is an open problem that is often assessed via randomized control experiments. Here we argue that such an approach is not needed. The reason is that the output of the concept bottleneck is not a post-hoc explanation of the model, like feature attributions or linear probing approaches. We explicitly design the rest of the generative model to rely on the output of the concept bottleneck. One can think of the output of the CBM as analogous to the weights of a linear regression model. By design, these weights directly reveal feature importance.

• Assessing Interpretability: To address your concern, we conduct a correspondence experiment. We generate 50 random samples and intervene on a subset of 2 concepts for these samples. We then inspect these 50 samples to assess whether the intervened samples are reflected in the generated output. This assessment communicates the extent to which the probability histogram can be used as a reliable interpretation. On the CB-VAE trained on Celeb-A 64 by 64, we find an 87 percent correspondence, and 96 percent for a CB-DDPM trained on Color-MNIST. We added this experiment to appendix Section B.4.

Dear Reviewer 5NVe,

Given that the end of the discussion period is approaching, we would like to ask if you have any further concerns or questions, particularly as a follow-up to our response?

Thank you in advance!

Thank you for the detailed reading and feedback. First, we address the weaknesses in the following comments.

Re: Weaknesses

• Ground-truth Annotations: Indeed, as currently formulated, the CB-generative models require annotations for the entire training set. However, this requirement can be easily relaxed. To directly address your concern, we performed a new set of experiments where we varied the percentage of concept annotation available for CBGMs. We added this experiment in Appendix B.3.3. We find that with about 20% of annotations, we can directly match steerability metrics of a fully annotated training set. This result is not surprising; in the interpretability literature, concept bottleneck models for classification papers (Kim et al. (2018) and Yuksekgonul et al. (2022)) have shown that it is possible to learn a high-performing concept classifier with as few as 200 samples. This experiment suggests that those results translate to the generative model setting as well.

• Scaling Concept Layer: Increasing the number of concepts does increase the model parameters as in any concept bottleneck model. However we show in the paper, we can scale the number of concepts from 8 to 40 on the Celeb-A 64 by 64 dataset, and on the LAION dataset, we used 155 concepts. To address your concern we scaled LAION to 750 concepts which we were able to train using only 4 V100 GPUs. To clarify how the CB layer affects the number of parameters of a model we now discuss the total number of parameters that the CB layer adds. A standard CB layer consists of: $2k$ concept networks ($k$ for the positive concept states, and $k$ for the negative concept states), $k$ probability networks, and 1 unknown concept network. Similar to Espinosa et. al. (2022), we parameterize each concept network as a fully connected module; hence, each concept network has $am$ parameters where $a$ is the input dimension to the concept layer for a given instance, and $m$, is the concept embedding size. Consequently, for $2k$ networks, we have $2kam$ parameters for $2k$ concepts and $am$ parameters for the unknown concept network giving: $(2k+1)am$ parameters. Each probability network has $2m$ parameters leading to $2mk$ parameters for $k$ concept vectors, which in total means the CB layer adds: $(2k+1)am + 2km$ parameters to a given architecture if the networks are instantiated as fully connected layers. In practice, we share a single probability network across all concepts, which reduces the total number of parameters required as done by Espinosa et. al. (2022). For example, training CB diffusion models with 155 concepts required 120 million parameters scaling to 750 concepts increased the number of parameters to 250 million. Current hardware allows us to scale to multiple billion parameter models.

• Unsupervised Concepts: We have two categories of concepts: known and unknown concepts. First, for the concepts that we want to be able to interpret and control, we need labels. However, we allow for unknown concepts that are learned completely unsupervised except with the constraint that these concepts be orthogonal to the known concepts. Supervision for the known concepts is important. As we referenced in the paper, Locatello et. al. (2019) show that one cannot learn, unsupervised, representations that are also disentangled. In addition, there is no guarantee that these representations should be human-understandable. This means that simply allowing of an entirely unsupervised representation will limit both interpretability and the ability to control the model. In fact, this challenge is why approaches like GANSpace resort to a PCA of the linear representations to ‘search’ for semantically meaningful and disentangled features. Consequently, our approach allows for both known and unsupervised concepts.

Dear Reviewer YbNz,

Given that the end of the discussion period is approaching, we would like to ask if you have any further concerns or questions, particularly as a follow-up to our response?

Thank you in advance!

Thank you for the detailed reading and feedback. First, we address the weaknesses and then respond to your questions point-by-point.

Re: Weaknesses

• Comparison to SOTA baselines: As we stated in the general comment, our goal is to use an intrinsically interpretable component of a generative model to both steer, interpret, and debug the model. As it stands, none of the baselines (stable diffusion, GANSpace, etc) accomplishes these goals. We now discuss each baseline in turn.

  • Text-to-image models (e.g. stable diffusion): These models allow for fine-grained control with text inputs. However, given a trained text-to-image model, it is still unclear what features the model relies on to generate its output. While these models allow for text-based control, they do not provide any insight into the models; hence, they are difficult to debug. As an example, in recent work, Tong and Jones et. al. (2023) show that text-to-image models can easily ignore critical words in an input prompt. For example, a Stable Diffusion XL model asked to generate: “a woman proposing to a man” instead produces an image of a man proposing to a woman. As we show in Figure 18, a concept bottleneck model inserted in a diffusion model can help identify the features the model bases its generation on, and hence can immediately point to how to address deficiencies with the model.

  • GANSpace: As we stated, our goal is not to do post-hoc interpretation but to insert an intrinsically interpretable layer into the generative model. GANSpace uses PCA to identify the key directions/features in the representations of a GAN. The approach can then vary the representations along these directions to vary the output of the model. Our approach differs from theirs in several ways: 1) since we insert an interpretable layer into the model, we do not need to search for the representations in order to control them. 2) It is possible that the model representations might not encode for a feature that one might want to control. For example, in the Celeb-A setting, we might be interested in controlling the presence of Glasses. However, it is possible that none of the most important PCA directions correspond to Glasses. In contrast, our proposal directly encodes the `glasses' concept as part of the model’s latent representation, so we can immediately identify whether the model has been able to learn that concept.

  • StyleGAN: We agree that the StyleGAN model has been used to demonstrate impressive steering/control capabilities. Controlling styleGAN typically can be done in different ways, either by conditioning where the concepts are concatenated to the latent vector, classifier guidance, or model editing methods such as GANSpace as discussed previously. Such methods enable steerability (although we found that for steerability CB layer outperforms conditioning and classifier guided methods as shown in Table 1), but it does not allow for interpretability (i.e we do not understand which concepts the model relied on during generation) and debuggability (i.e we can not tell if the model actually learned a certain concept during training). An alternative approach is adding a CB layer to styleGAN, by doing this, we can explicitly learn the desired concept, we eliminate the need to search for weights correlated with concepts as we know ahead of time which parameters will encode for a particular concept, the concept layer allows us to debug and interpret the generative model during and after training. During training, we can identify which concepts the model is currently unable to encode, and post-training, we can use the output of the concept layer to identify the key features responsible for the model’s output. To address your concern, we added a section in the appendix explaining how one can create CB-StyleGAN; please find the full experiment and results in Appendix B.1.1 (in the updated paper).

Re: Questions

• Unseen Concepts: As we discuss in the paper, we partition concepts into two categories: known (pre-determined human understandable ones), and unknown concepts. The unknown concepts indeed directly encode for unseen concepts for which the model designer did not specify ahead of time in the known set. Your suggestion about open world concepts is interesting, we did not consider it in our initial formulation. However, we can easily adapt the current formulation to allow for a dynamic set of concepts. We leave an extension to open-world concepts for future work.

• Diffusion Model: Training diffusion on LAION required 240 V100 GPU hours, training diffusion on Celeb-A required 112 V100 GPU hours. We have updated the manuscript to include the compute time.

• Training from scratch: Yes all models are trained from scratch.

References

• Shengbang Tong, Erik Jones, and Jacob Steinhardt. Mass-producing failures of multimodal systems with language models. arXiv preprint arXiv:2306.12105, 2023.

Dear Reviewer 9WtF,

Given that the end of the discussion period is approaching, we would like to ask if you have any further concerns or questions, particularly as a follow-up to our response?

Thank you in advance!

Thank you for the detailed reading and feedback. First, we address the weaknesses and then respond to your questions point-by-point.

Re: Weaknesses

• Abstract, Introduction, and Normalization Flows: Thank you for raising this important point. Indeed, we tested three types of generative models in the draft. However, we found your suggestion of normalizing flows and invertible models intriguing, so we tested our proposed approach on a Normalizing flow model trained on Color MNIST, we added this experiment in Section B.1.2 of the Appendix. Our CB layer proposal translates, as is, to such models as well. We show examples generated from the model in Section B.1.2 of the Appendix. We expect future work to test our findings more exhaustively and extend them to other families of generative models. We take your feedback to mean that we should explicitly specify the class of models we focus on, which we have done in the updated manuscript.

• Not covering modern ones such as StyleGAN2: As per your request, we added a CB layer to StyleGAN2 in Appendix B.1.1 (in the updated paper). For StyleGAN, the CB layer was added after the mapping network in the generator, an encoder was also added before the generator to ensure that the generated image matches the concepts constrained in the loss, the exact architecture is shown in appendix B.1.1 Figure 7 (a). We trained StyleGAN2 on Celeb-A 64x64 and CUB 128x128. We added images generated by CB-styleGAN2 for both datasets in the appendix Figure 7 (b) and reported FIDs as well. We find that using more complicated architectures improves the quality of the generated image (as expected) and inserting a CB layer does not degrade the generation quality.

• GAN inversion baseline: As we mentioned in the general comment, while GAN inversion approaches enable a model to be steered, these approaches do not address the goals that we set forth to tackle in this work. To re-iterate: we seek an intrinsically interpretable component of a generative model that allows us to steer and interpret the output of the generative model simultaneously. Meanwhile, GAN inversion approaches do not provide any form of understanding or insight into which features the model bases its generation on. In addition, we require that the approach be intrinsic to the model and not a post-hoc approach. Critically, a GAN inversion approach cannot indicate the key features that a model relies on for its output. Similarly, to the best of our knowledge, most GAN inversion techniques do not explicitly insert an interpretable component into the model. We think the GAN inversion approaches address an important challenge; however, our goals are orthogonal to these challenges.

• Qualitative Results: As you requested, we updated the Appendix to include several additional qualitative steerability experiments from StyleGAN2; see Figure 11 in the appendix.

• Concept Orthogonality Loss: Thank you for noting this issue, $j$ is the sample in the mini-batch, we have updated the main text under Eq 5 to clarify this.

Re: Questions

• Optimization: Thank you for raising this important point. On the VAE model we compared Platt and Barr to a standard grid search/sweep over a hyper-parameter set. We didn’t observe a substantial improvement in model FID scores for the subset of models trained. We used an open-source Pytorch implementation of the Platt and Barr approach, which does not require any modification for the setting considered here. The approach minimizes a loss function that is subject to bound constraints with arbitrary parameters that weight the importance of these functions. We added this discussion in Section B.2 in the appendix; we also referred to the open-source implementation used.

• Orthogonality: We agree with your assessment that it is crucial for the unknown concept embedding to be orthogonal to known concept embeddings. Removing orthogonality between the known and unknown concepts causes steerability degradation, as shown in the ablation Table 3. Our current algorithm encourages orthogonality by adding the loss in equation 5 but does not enforce it. We tried forcing orthogonality during training (as you suggested) by projecting the concepts embedding to be orthogonal to non-concepts embedding during training using the Gram-Schmidt Process [1]; we found that this makes training unstable and harms generation. We note that there are other schemes that can achieve independence, including KL minimization, mutual information minimization, etc. We leave exploring different orthogonality schemes as future work.

[1] Schmidt E.Zur Theorie der linearen und nichtlinearen Integralgleichungen I. Teil: Entwicklung willkürlicher Funktionen nach Systemen vorgeschriebener. Mathematische Annalen 1907; 63: 433–476.

Dear Reviewer vC1Y,

Given that the end of the discussion period is approaching, we would like to ask if you have any further concerns or questions, particularly as a follow-up to our response?

Thank you in advance!

Other Concerns

Below we briefly discuss other cross-cutting concerns.

• Adapting the CB layer to StyleGAN (Reviewer vC1Y and 9WtF): We added section B.1.1 in the appendix showing how to add a CB layer to StyleGAN2.
• Ground-truth Concept Annotations (Reviewer YbNZ and 5NVe): We performed a new set of experiments where we varied the percentage of concept annotation available for CBGMs we added this experiment in Appendix B.3.3. We find that with about 20% of annotations, we can directly match steerability metrics of a fully annotated training set.
• Steerability metric (Reviewer YbNZ and 5NVe): To clarify, we calculate the steerability metric as follows: We start with a high-accuracy classifier trained on real data; we ensure that the minimum accuracy of any concept classifier is at least 98 % on a held-out test set from the real data. We sample noise latent vectors and then use it to generate images from the model; for each concept we pass the generated image to the concept classifier; we save the latent vector if the classifier predicts the concept to be absent (i.e., the probability of the concept present is less than 0.5) we continue this process until we have 1K samples. For each intervening approach (including CB layer), we intervene on a concept and generate new images. We then pass the newly generated images to the classifier again and measure the fraction of inputs for which the indicates greater than 0.5 probability of concept presence, which we report in Table 1.

For a detailed discussion, please refer to the individual point-by-point responses. Note that in the revised manuscript, changes are shown in orange. We look forward to hearing from you and will happily address any remaining feedback!