Neural Concept Binder

W1 (More background):

We agree that adding more information on Sysbinder and Slot Attention can help the reader and make the paper overall more self-contained. We have added an additional section with details for the camera-ready version and provide it in a comment below.

W2 (Clarity of Figure 2):

We agree and have modified the Figure: (i) we have updated the digits (that had previously represented the concept symbols) in the "Hard Binder", "Retrieval Corpus" and "Concept-slot encodings" components to the notation used throughout the experiments. Specifically, we now denote concepts in Figure 2 with a capital letter for the block and a natural number for the category ID, e.g., A3 for the third concept in the first block (A). (ii) Furthermore, we have added Roman numerals (I-VI) to each component in the figure and reference these specifically within the main text. (iii) We have updated the text accordingly for more clarity. Overall, we agree this greatly helps in understanding the individual components. Apologies for having missed that in the initial version.

W3 ($`enc`_l^j$):

Let us go through this question step by step. We base this on the more detailed training descriptions in A.2 (appendix). First, NCB infers the block-slot encodings of a set of input images. These are denoted as $\bar{Z}$ in Alg. 1 in the appendix. NCB then performs clustering of these encodings per block, i.e., it clusters the blocks j of each encoding (denoted $\bar{Z}^j$) and assigns each of them one cluster label, $v \in \{1, \cdots, N_C\}$. Per cluster, $v$, NCB next identifies exemplar and prototype encodings (i.e., representative encodings identified via the clustering method and cluster averages). Each of these exemplar and prototype encodings correspond to one $`enc`^j$ in the tuples, $(`enc`^j, v)$, that are stored in the retrieval corpus $\mathcal{R}^j$ of one block $j$. We now use the index $l$ to identify specific encodings out of $\mathcal{R}^j$, leading to $\mathcal{R}^j := \{(`enc`_l^j, v_l) : l \in \{1, ..., |R^j| \} \}$. In conclusion, $`enc`_l^j$ represents one block encoding that is stored in $\mathcal{R}^j$ and has been assigned to a specific cluster $v_l$. We agree that highlighing these additional steps benefit the understanding of the main text and have updated it accordingly. We apologize for the brevity in the initial submission.

W4 (Revision operators):

Indeed, we agree that the revision operators are somewhat confusing. It seems the attempt to formalize these steps has overcomplicated things. We have now removed the formal operations from section 3.3 and described these in words. E.g. we have posted the updated $`merge`$ description in a comment below.

W5 (Additional baselines):

We fully agree on the value of additional baseline models. We have, therefore, added the recent NLOTM model as a novel baseline for our evaluations. We refer here to our general response and hope the reviewer can agree on the relevance of NLOTM as a baseline model.

However, we politely disagree that the methods suggested by the reviewer represent valuable baselines. Specifically, [1] cannot handle object-level concept learning, nor are the concept assignments discrete (an image is represented as a continuous mixture of concepts). [2] on the other hand, focuses on image regions as concepts. Similar issues hold for [3], i.e., they consider concepts to represent segments of an image and do not provide object-level concepts. Moreover, [3] focuses on post-hoc learning of concepts, which makes potential model revisions tricky and the inspection of the model ambiguous. Overall, we fully agree that the references to unsupervised concept learning are related to our work and we have noted them accordingly in our paper. However, none of them tackle the same problem setting as NCB aims to, i.e., learning discrete, object-level concept representations without supervision. We therefore consider the benefit of running these particular baselines as minor.

Concerning supervised models ([4,5]): in fact, we had compared against supervised concept-based models in the context of Q2 and Q4 (denoted as SA (supervised), c.f. also F.7) whereby we observe that indeed NCBs unsupervised concepts are competitive in comparison to the GT supervised concepts of a slot attention-based CBM despite any supervision in training NCB.

W6 (Additional datasets):

We kindly refer here to the general response above.

W7 (Related work):

Agreed, we have added the missing references. We note that we had already referenced ProbCBM, though not in the context of discrete and continuous. We have now added this to this respective section.

W8 (Unclear sentence):

We apologize for the confusion here. We have updated the sentence (see comment below).

W9 (Inspection terminology):

We agree that connecting the terms example-based and counterfactual explanations can help to understand the outcome of the different inspection forms of NCB. We have now denoted comparative and interventional inspection as two ways to obtain forms of counterfactual explanations and implicit and similarity-based inspection as two ways to obtain forms of example-based explanations. Thanks for the hint!

Q1 ($v_l$→$v_m$):

We have removed this notation. It had previously described replacing $v_l$ with $v_m$. We refer here to the response above concerning revision operators.

Q2 (add operator):

We have updated the add revision as in a comment below. Overall, this operation can be used when the hard binder has missed concepts of the original encoding space. However, if NCB's soft binder has not learned to encode a specific concept at all (i.e., does not represent it in its block encodings) or the concept that a user wishes to add is not present in the dataset at all, one has to revert to the fourth type of revision, i.e., an additional finetuning of the soft binder is necessary.

Background

The binding mechanism (SysBinder) of Singh et al. (2024) allows images to be encoded into continuous block-slot representations and relies on the recently introduced slot attention mechanism [2]. In slot attention, so-called slots, $s \in R^{N_S \times N_B D_B}$ (each slot has dimension $N_B D_B$), compete for attending to parts of the input via a softmax-based attention. These slot encodings are iteratively updated and allow to capture distinct objects or image components. The result is an attention matrix $A \in R^{N_S \times D}$ for an input $x \in R^{D}$. Each entry $A_{i}$ corresponds to the attention weight of slot $i$ for the input $x$. Based on the attention matrix, the input is processed to read-out each object by multiplying $A$ with the input resulting in a matrix $U \in R^{N_S \times N_B D_B}$.

SysBinder now performs an additional factor binding on the vectors $u_i$ of $U$. The goal of this factor binding mechanism is to find a distribution over a codebook memory for each block in $u_i$, i.e., $u_{i}^j$. This codebook memory (one for each block), $M^j \in R^{K \times D_B}$, consists of a set of $K$ learnable codebook vectors. Specifically, for each block $j$ an RNN consisting of a GRU and MLP component iteratively updates the $j$-th block of slot $s_i$, $s_{i}^j$, based on $u_i^j$ and previous $s_{i}^{j}$. Finally, a soft information bottleneck is applied where each block $s_i^j$ performs dot-product attention over the codebook memory leading to the final block-slot representation:

This process is iteratively refined together with the refinement processes of slot attention. Overall, the encodings of SysBinder represent each object in an image by a slot with $N_B$ blocks where each block represents a factor of the object like shape or color.

Note that in the main text, the final $s_i^j$ is denoted as $z_i^j$.

[1] Singh, Gautam, Sungjin Ahn, and Yeongbin Kim. "Neural Systematic Binder." ICLR, 2023.

[2] Locatello, Francesco, et al. "Object-centric learning with slot attention." NeurIPS, 2020.

Briefly, given an image, $x$, NCB infers latent block-slot encodings, $z$, and performs a retrieval-based discretization step on $z$ to infer concept-slot encodings, $c$. These express the concepts of the objects in the image, i.e., object-factor level concepts.

(iii) Add Encodings or Concepts: If a specific concept is not sufficiently well captured via the existing encodings in $\mathcal{R}^j$, one can simply add a new encoding for the concept, $m$, to the corpus: $\hat{`enc`}_{l+1}^j$

This leads to an additional entry in the corpus: $(\hat{`enc`}_{l+1}^j, m)$

Accordingly, it is also possible to add encodings for an entire concept. Hereby, via the soft binder one infers block encodings of example objects that represent that novel concept, $b$, and adds these to the corpus as $(\hat{`enc`}_{l+1}^j, b)$ with $b = N_C+1$.

(i) Merge Concepts: In the case that $\mathcal{R}$ contains multiple concepts that represent a joint underlying concept (e.g., two concepts for purple in Fig.3 (right)) it is easy to update the model's internal representations by replacing the concept symbols of one concept with those of a second concept. Specifically, according to human or additional model feedback, if concept $m$ in block $j$ should be merged with concept $b$ ($m,b \in \{1, \cdots, N_C\}$) then for all corpus tuples, $(`enc`_l^j, v_l) \in R^j$, we replace $v_l$ with $b$ if $v_l = m$.

I thank the authors for their efforts in trying to address the suggested issues.

However, I will increase my score to 5 only since I still think is a borderline paper:

i) The method presentation should have been improved according to what the authors have reported, but I should review it again to assess it better.

ii) Testing only on toy datasets is still not acceptable for a neurips paper. Although I agree that better object centric learning will improve the performance of the model, how the current method behave on natural image dataset is of crucial importance to globally assess the model. Bad, but promising results would have still been appreciated.

W1.1 (NeSy contribution):

We agree that the field of neuro-symbolic AI is rapidly evolving with different focuses, e.g., visual reasoning in real-world images. However, works that are focussing on higher-level neuro-symbolic problems still heavily rely on mapping raw input images to symbolic representations, whether that is by using pre-trained concept extractors [1, 2, 3] or foundation models [4, 5, 6] or forms of weaker supervision [7].

However, how to obtain meaningful symbols from images in a fully unsupervised way remains a core challenge for the community. With our work, we address exactly this open problem, and to our knowledge, only [8] has additionally attempted to tackle this task. We hereby specifically focus on the importance of being able to inspect and effectively revise learned symbols, particularly due to the unsupervised nature of the learning setup. These aspects are most often overlooked in the majority of research that is in the pursuit of performance-driven models. Overall, we believe that our work offers a unique contribution to the field of concept learning, as also all other reviewers agree upon. If the reviewer is aware of specific works that are unknown to us yet relevant for this topic, we would be grateful if they could share them so we can better assess the reviewer's remarks.

W1.2 (utilizing SysBinder):

We strongly disagree that our work represents only an incremental improvement over SysBinder. While in this work, we have focused on utilizing the SysBinder encoder as NCB's soft binder, our method is not limited to it. Instead, this encoder can be replaced by any current or future models that process images into object-factor encodings (e.g., also models that can process natural images). With our framework, we propose a general method for discovering discrete symbols from continuous representations that offers inspection and revision possibilities for human stakeholders. This is not trivial and a very important aspect for unsupervised learning that, to our knowledge, has not been addressed by other works.

W2.1 (attribute classification as "toy task"):

We disagree and consider extracting symbols from (unlabeled) images as a cornerstone for a variety of neuro-symbolic methods. Many neuro-symbolic approaches rely on extracting symbolic representations from the input, whereas most recent works specifically rely on pre-trained models [1, 2, 3, 9]. NCB, on the other hand, allows to learn such symbols without supervision and a main goal of our work is to introduce NCB as a core module for other frameworks (as particularly reviewer VMaX has acknowledged), e.g., for more complex visual reasoning methods. With the evaluations in the context of Q2-Q4, we have highlighted this ability.

W2.2 (CLEVR Sudoku):

With CLEVR Sudoku, we are proposing a very relevant challenge that is combining complex visual inputs with explicit reasoning on a symbolic level. Specifically, the underlying reasoning skills required to solve CLEVR Sudoku are very much relevant for real-world scenarios. In fact, many works agree that reasoning with abstract representations (i.e., concepts) are essential for robust generalization [10, 11, 12]. Overall, there is a great deal of interest, and thus, a variety of benchmarks that investigate exactly this form of required intelligence [13, 14, 15, 16, 17]. These specifically exclude language priors and common knowledge priors of real-world images to asses a model's learning and reasoning abilities.

W2.3 (Relational tasks):

The goal of our NCB framework is to extract meaningful symbolic representations from the input, which can be beneficial for a variety of tasks. Hereby, NCB focuses on learning discrete object-level concept representations from images, i.e., focusing on unary relational concepts. The aim overall is not to propose a framework to solve relational tasks. However, NCB can be used as an easily integrable module for models that perform relational reasoning, e.g., which usually require pre-trained, (semi-)supervised concept extractors. Overall, we agree that it is important for future work to integrate NCB into more complex relational tasks, as we had noted in our conclusion.

Q1 (closed-loop reasoning):

Yes, exactly, NCB can be easily integrated into closed-loop reasoning. This is exactly the intended application of our method. E.g., NS-VQA had relied on the extraction of the scene representation via a supervised trained scene parser with ground truth object attributes. Instead of this scene parser, one can employ NCB, which discovers attributes without supervision and use the object’s attention masks (from the slot attention component) to determine their position. In NS-CL, the concepts are learned via the question-answering task. This is an alternative approach to concept discovery that, however, requires question-answer pairs for the image domain that target those concepts.

Q2 (relational concepts):

In principle, we do think it is possible to represent relational concepts via soft/hard binding, though we do not investigate this here. As mentioned above, our approach is intended to be integrated into other methods, such as visual programming or program synthesis, where the learned relations would be based on NCB’s concepts. Hereby, some current work focuses learning relations from images neurally, and others focus on symbolic approaches. Thus, whether to perform relational concept learning via neural, soft binding principles or symbolic, hard binding principles, or a combination of these is still up for investigation. We further refer to our response above concerning relational tasks.

[1] Yi et al. "Neural-symbolic vqa: Disentangling reasoning from vision and language understanding." NeurIPS, 2018.

[2] Koh et al. "Concept bottleneck models." ICML, 2020.

[3] Shindo et al. "α ILP: thinking visual scenes as differentiable logic programs." Machine Learning, 2023.

[4] Surís et al. "Vipergpt: Visual inference via python execution for reasoning." CVPR, 2023.

[5] Gupta et al. "Visual programming: Compositional visual reasoning without training." CVPR, 2023.

[6] Hsu et al. "What’s left? concept grounding with logic-enhanced foundation models." NeurIPS, 2024.

[7] Mao et al. "The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision." ICLR, 2019.

[8] Wu et al. "Neural Language of Thought Models." ICLR, 2024.

[9] Wüst et al. "Pix2Code: Learning to Compose Neural Visual Concepts as Programs." UAI, 2024.

[10] Mitchell, Melanie. "Abstraction and analogy‐making in artificial intelligence." Annals of the New York Academy of Sciences, 2021.

[11] Zhang et al. "How Far Are We from Intelligent Visual Deductive Reasoning?." ICLR 2024 Workshop: How Far Are We From AGI.

[12] Lake et al. "The Omniglot challenge: a 3-year progress report." Current Opinion in Behavioral Sciences, 2019.

[13] Chollet, François. "On the measure of intelligence." arXiv, 2019.

[14] Moskvichev et al. "The conceptarc benchmark: Evaluating understanding and generalization in the arc domain." arXiv, 2023.

[15] Raven, Jean. "Raven progressive matrices." Handbook of nonverbal assessment, 2003.

[16] Lake et al. "Human-level concept learning through probabilistic program induction." Science, 2015.

[17] Nie et al. "Bongard-logo: A new benchmark for human-level concept learning and reasoning." NeurIPS, 2020.

W1 (dependence on continuous encoder):

Indeed, the quality of the initial continuous concept encodings is important for the resulting discrete concept representation. We had remarked on this in the context of our ablations. We have now added an ablation study to highlight this empirically (c.f. Tab.1 (middle) in additional pdf). Specifically, via earlier checkpoints of the sysbinder encoder we observe that the weaker the encoder is, the less expressive are NCB's final discrete concept representations. However, as better models are being developed (e.g., that are shown to also process natural images), one can easily integrate these into the NCB framework and replace the SysBinder encoder that we used for our evaluations.

W2 (ablation clustering):

Thanks for the hint! We have added an ablation evaluation on the CLEVR dataset in the context of Q1 where we compare the expressiveness of concept representations when utilizing k-means rather than HDBSCAN (see corresponding table in additional pdf and general response above). We observe that for generalization, the HDBSCAN approach performs significantly better.

Q1 (causal representation learning):

Thanks for the pointer! This work is indeed interesting, as it connects the more “informal” object-centric detections with the guarantees (and also assumptions) of causal representation learning. While this work and NCB could be considered somewhat parallel, some possibilities exist to combine both. One could use the weakly supervised perturbations to finetune NCB's initial soft binder encoder if one wants to move toward weak supervision. On the other hand, one could also utilize the unsupervised trained NCB with its interventional inspection to generate sparse perturbation of images. This could potentially be possible without specific human supervision but might be limited to the concepts NCB discovers unsupervised. Overall, there are many possibilities for future research in this direction. We have added the reference accordingly into the main text.

Q2 (OC-learning and logic):

That's a very important question for the future development of object-centric and neuro-symbolic learning in general. In fact, one motivation for our proposed framework is to be able to extract unsupervised discrete representations from object-centric representations that can be integrated into symbolic AI approaches such as different logic systems. Specifically, NCB's concepts can already be utilized, to some extent, for first-order logic as properties about different objects are encoded into symbols, i.e., unary relations. Thus, first-order logic formulas containing quantors can be expressed out of the box. However, the NCB framework is currently not aimed at encoding arbitrary n-ary relations. We consider utilizing NCB in approaches like [1] could mitigate this issue to learn more complex logic programs based on the unsupervised learned concepts of NCB.

[1] Wüst et al. "Pix2Code: Learning to Compose Neural Visual Concepts as Programs." UAI, 2024.

W1 (regarding RQ2 (CLEVR Sudoku)):

A determining factor for the performance on CLEVR-Sudoku is the classification of the digits. We agree that this has, to some degree, already been investigated in the context of Q1, where we tested the suitability of NCB’s concept representations for few-shot GT attribute classification. However, in Q2 we wanted to extend these findings into a much more challenging downstream task (than just GT attribute prediction) and specifically focus on the integration into purely symbolic systems that require correct, task-specific symbolic mappings. Importantly, as NCB is trained unsupervised, there is no guarantee that the learned concept symbols will be directly translatable to the symbols required for a downstream task. It is, therefore, potentially necessary to learn a translation from NCB's concept symbols to the task symbols, e.g., individual object attributes to specific object compositions. Overall, this is a very valuable problem setting concerning real-world applications, e.g., in planning (e.g., navigating through a world of objects). In Q2, we investigate NCB's potential for integration in such settings and consider the evaluations on CLEVR-Sudoku to be a helpful illustrative example. Additionally, the evaluations in Q2 also serve as baselines for investigating the ability to revise NCB's concepts for such a complex downstream task in the context of Q3.

Nonetheless, we do acknowledge that Figure 8 also provides valuable insights that fall a bit short in the main paper (thanks for the hint!). We have updated the main text to put more focus on these results within the discussion of Q2.

W2 (additional citations and baseline):

Thank you for pointing out the related works, we were not aware of them. We consider Arefin et al. orthogonal to our work as they focus on learning object-level concepts in comparison to our object-factor level concepts and have included it in our related works section. Indeed, Wu et al. is relevant to our problem setting. Unfortunately, it was published so close to the NeurIPS deadline that we seem to have missed it; thank you for pointing it out! We have provided the results of this model in our general response and included it as an additional baseline in our paper. We agree this helps highlight our work.

W3 (Evidence for discussion 3.3):

Thank you for raising this point! We agree that the paper would definitely benefit from more illustrations of the results that can be obtained by inspection. We therefore added example visualizations (c.f. additional pdf). Further, in the context of Q3, we do investigate revising models. Specifically, we apply $`merge`$ and $`delete`$ revision; the corresponding models are denoted as "NCB revised (GPT-4)" and "NCB revised (human)". We also investigate the application of $`add`$ in the "left-right" experiment in the appendix (F4).

Q1 (Training SysBinder (hard) and (step)):

Good question! The SysBinder (hard) model did not train well, i.e., the reconstruction was terrible. The SysBinder (step) model, on the other hand, had stable training and provided reconstructions of the quality of the vanilla SysBinder. These results support recent findings [2] that training from the start with hard discretization results in badly performing models. I.e., it is very difficult for the model to learn good representations at all (e.g., for reconstruction) with such a strict bottleneck. On the other hand, learning step-wise can more easily lead to issues with local optima than for models without any bottleneck.

Q2 (RQ4 evaluations):

We are sorry for the confusion here. Within Q4, we investigate the popular setting of concept-bottleneck-like approaches [1,2] in which a model predicts (discrete) high-level concepts from an image and a second model makes the task prediction based on these concept activations. This leads to models with more transparent and high-level explainability properties. In principle, one can also utilize continuous concept encodings rather than fully discrete concept representations. However, this makes their understandability more ambiguous. In Q4, we investigate the case in which the prediction model is given discrete concept representations. We compare to supervised discrete concept representations (c.f. F7) and particularly investigate the ability to inspect and revise such models that utilize NCB's concepts. Thus, the used concept encodings for the evaluation are actually the discrete concept-slot encodings of NCB. Overall, interpreting and revising models that utilize continuous embeddings is not as straightforward, we have therefore omitted such comparisons here. We have further clarified this now in the text of Q4.

Q3 (importance of hdbscan):

Thanks for the valid questions! We refer here to the general response above with novel ablation evaluations in the additional pdf.

[1] Koh et al. "Concept bottleneck models." ICML, 2020.

[2] Stammer et al. "Right for the right concept: Revising neuro-symbolic concepts by interacting with their explanations." CVPR, 2021.