Right for the Right Reasons: Avoiding Reasoning Shortcuts via Prototypical Neurosymbolic AI

Firstly, we would like to thank the reviewer for their comments, which allowed us to highlight better both the novelty of our approach and its broad applicability. Additionally, we are thankful for pointing us towards the abductive learning literature, if we have missed some reference, please let us know.

[W1.a] Same assumption as SSL+Nesy (abductive learning) methods

Our method relies on a slightly different assumption: namely, that datapoints are similar—i.e., clusterable—in the learned embedding space. The embedding space is not fixed but is shaped dynamically during training via backpropagation from both the supervision provided by labelled datapoints and the neuro-symbolic (NeSy) loss. Consequently, similarity in this space does not necessarily correspond to similarity in the original input (feature) space. Rather, it reflects a form of functional similarity that integrates both the data and the symbolic knowledge expressed in the constraints. In Section 4, we formally derive the gradient of the total loss, showing that the embeddings are updated based on both (i) their proximity to class prototypes and (ii) the error signal stemming from the NeSy component. This confirms that the learned representation is shaped by the interplay between data-driven and knowledge-driven signals.

Whenever pseudo-labels are involved, it is indeed a core problem to decide how and when to commit to a label, as an early pseudo-labelling mistake might propagate further. This is why in the abductive learning community, many researchers have studied how to define consistency measures to update the candidate pseudolabels with the most similar ones which are also compliant with the background knowledge. For example, [1,2] take the consistency score as the size of the largest subset of unlabeled examples with abduced labels that are consistent with the knowledge base, while [4,5,6] measure the confidence of the predicted labels by the machine learning model.

The most similar work to ours in spirit is ABLSim [7], as they also use the embedding similarity to guide training. However, ABLSim constructs a similarity-based consistency score purely in the embedding space, encouraging samples with identical (pseudo-)labels to form tight clusters and those with differing labels to be well-separated. This score is used to select among logically consistent pseudo-label assignments, and the perception model is trained accordingly. In contrast, our method integrates symbolic knowledge directly into the loss function, such that the embedding space is shaped not only by class-based proximity (via prototype loss) but also by gradients arising from differentiable neuro-symbolic constraints. This results in an embedding space that reflects functional similarity: two points may be embedded nearby not because they share superficial input features, but because they induce similar outcomes with respect to both the downstream task and the symbolic knowledge. As shown in our gradient derivations (Section 4), this joint signal ensures that the learned representation is aligned with the structure of both the data and the knowledge base—beyond what similarity in feature space alone can capture. We will integrate this discussion in the paper.

[Q1+W1.b] Comparison with SSL+NeSy methods or intuitive strategy

We thank the reviewer for their suggestion, and to answer to their question we compare our method against one abductive learning method [1-7], that we believe is a much stronger baseline than the intuitive strategy.

In particular, we trained this model using the ABLKit package [3] on MNIST-EvenOdd, (i) first without giving the model any labelled datapoint (indicated with ABL in Table A below) and then (ii) giving the model access to exactly the same augmented labelled datasets as our models (indicated as ABL (Aug) in Table A below). In Table A we report the results. For comparison, we also report the performance of our DPL+PNet.

Table A. Average results obtained on MNIST-EvenOdd over 5 seeds.

As we can see from the results, the semi-supervised method is able to improve its own performance by using the augmented datasets, but is not able to perform as well as the prototypical networks. The much higher standard deviation presented by the abductive learning method is also something worth noticing: when performing pseudo-labelling, the methods have to “commit” to a choice (e.g., by classifying an image of the digit 7 as indeed 7 or as 9):, if the wrong decision is taken early on, then this error will propagate through training.

More in general, notice that ABLKit obtains very good performance on the standard MNIST-Add dataset, where it achieves Acc(C) = 0.98, thus outperforming both LTN and DeepProbLog. However, MNIST-EvenOdd is a particularly challenging scenario due to the high number of reasoning shortcuts it presents. Indeed, even assuming the model achieves perfect final label accuracy (Acc(Y) = 1.00), there remain 49 possible digit-level assignments it could have produced, of which only one corresponds to the correct underlying reasoning. This highlights the difficulty of the task: neither the knowledge nor the data alone are enough to learn the right concepts.

[W1.c] The method proposed is very similar to training with SSL or one-shot learning until convergence and then using this predicted label information to enhance NeSy models.

Our approach is similar in spirit to the method proposed by the reviewer but actually it very presents some fundamental differences highlighted by the following example. Suppose we have (as in our case) a single example for each concept (even augmented) and training a SSL or one-shot learning method until convergence. The probability of the network overfitting on the few available examples will be very high. If then we use the overfitted network to train with the NeSy methods, then each network update will only take into consideration the error with respect to the NeSy loss thus making it very likely that the model will again fall prey to the reasoning shortcuts problem. On the contrary, the weights update done in our method takes into account both embedding similarity and knowledge satisfaction for all datapoints (both labelled and unlabelled) -- see Theorem 4.1 and Corollary 4.2.

[W2] NeSy methods should perform well in noisy scenarios

We totally agree with the reviewer. Given your response and those of other reviewers, we realised though that we might have overstated the limitations of the approach. To obviate with such problem, we run the following models on the autonomous driving task (BDD-OIA) and report the results obtained in Table B:

1. A fully supervised model, which has been trained on all images and all their lab (Fully Supervised in the Table),
2. An augmented version of DeepProbLog, which has been trained with all the unsupervised images and the 128 labelled datapoints we used to train our model (DPL (Aug) in the Table),
3. A new version of our model, where we use prototypes for all concepts, and not just those that cause the action “Stop” (DPL+PNet in the Table).

For completeness, we also include the results with the standard DPL method, which has been trained with no labelled images.

Table B. Results on the autonomous driving dataset (BDD-OIA)

As we can see from the results, even though the improvements are smaller than in the MNIST-EvenOdd and Kand-Logic, using DeepProbLog and our PNets we manage to outperform even the fully supervised model, trained with all the supervised images available in the dataset. To train DPL+PNet on the other hand, we only had to label only 128 images, with no data augmentation performed.

If we compare the results concept by concept for the fully supervised model and DPL+PNets, we can see that for almost all the concepts DPL+PNet outperforms the fully supervised model.

Table C. Concept-level performance (in terms of F1) on BDD-OIA

This is indeed in line with the assumption of the reviewer, that in presence of noisy data a NeSy system with access to very limited data can perform better than a fully data-driven one.

References:

[1] Wang-Zhou Dai et al., Bridging Machine Learning and Logical Reasoning by Abductive Learning, Neurips, 2019

[2] Zhi-Hua ZHOU, Abductive learning: towards bridging machine learning and logical reasoning, 2019

[3] Yu-Xuan Huang et al., ABLkit: a Python toolkit for abductive learning, Frontiers of Computer Science, 2024

[4] Qing Li et al., Closed loop neural-symbolic learning via integrating neural perception, grammar parsing, and symbolic reasoning. ICML, 2020.

[5] Wang-Zhou Dai and Stephen H. Muggleton. Abductive knowledge induction from raw data. IJCAI, 2021.

[6] Le-Wen Cai et al. Abductive learning with ground knowledge base. IJCAI, 2021

[7] Yu-Xuan Huang et al., Fast Abductive Learning by Similarity-based Consistency Optimization, NeurIPS, 2024

Dear Reviewer,

The deadline for the author-reviewer discussion is approaching (Aug 8, 11.59pm AoE). Please read carefully the authors' rebuttal and engage in meaningful discussion.

Thank you, Your AC

We thank the Reviewer for all the comments, and especially for encouraging us to consider a theoretical analysis of how the use of prototypes impacts the number of reasoning shortcuts that can affect the model. We believe this will be a great contribution to the paper.

[Q1] Reliance on prototype initialization and consideration of scenarios where disentanglement does not hold in terms of required annotations

We believe there might have been a slight misunderstanding here. We will make sure the below will be made clear in the paper. Regarding the reliance on prototype initialization, our method relies on prototype initialization as much as standard networks rely on weights initialization. In the paper we have simply provided a smart initialization scheme which allows us to avoid the trivial satisfaction of the constraints. This initialization scheme can be applied in any scenario.

Additionally, please note that our method does not require an example for every possible configuration, but rather one example per concept. For example, in the case of MNIST-EvenOdd, for RQ1 we annotated 10 images—one for every digit).

If we have one labelled example for every concept, the prototypes get trained and our method is very stable wrt weights initialization (see, e.g., Table 1), and as expected, if we have 0 labelled examples for more than one concept then we can see in Figure 5 that the method becomes more noisy. (This chart also shows why it is important to pick $p=0.99$ in our initialization).

When compared to training standard networks with NeSy methods (see Figure 3) we can see that our approach is much more robust to the number of unsupervised datapoints during training, even though the NeSy methods have been trained with exactly the same amount of labelled datapoints.

Regarding the required annotations, in RQ4 we have studied what happens when we do not provide any labelled examples for a subset of the concepts. For example, in Figure 4 we show how the performance vary on the ground of how many classes we leave unspecified in the MNIST-EvenOdd dataset (i.e., how many classes we have in our dataset for which we have 0 labelled examples.) Notice that when we say that we have 8 specified classes, it means that we have only labelled two images and then we perform data augmentation on those images.

[Q2] Representation collapse

We have never experienced centroid collapse. We will provide the t-SNE plots in the paper in case of acceptance. In BDD-OIA though, we have seen the representations of all the datapoints clustering around the negative prototypes (and this is why we have concept collapse $\not = 0$).

[Q3] Dataset needs to be clusterable in the representation space

We thank the reviewer for this comment, which gave us some ideas for future development of this work. While it is true that prototypical networks assume that each class forms a cluster in the representation space, we emphasize that the model is explicitly trained to learn embeddings in which such clusterability emerges, and in our practical tasks this assumption is empirically well-supported. In general, this assumption is particularly suited for standard NeSy methods, which learn “basic concepts” and then use the logical constraints to infer the more complex ones.

However, we agree that for example in presence of highly multi-modal distributions one cluster might not be enough. To obviate this problem, we could study how to learn multiple prototypes for each class (as done in [1]) and then use the logical constraints.

[Q4] Theoretical analysis on the number of reasoning shortcuts

We totally agree that adding this an analysis on the number of reasoning shortcuts would be beneficial. Below we conduct such analysis and we will include it in the paper in case of acceptance.

For clarity, we follow the same notation as [2].

Suppose the conditional distribution $p_\theta(\mathbf{Y} \mid \mathbf{X}; K)$ is induced by a prototypical network with embedding function $f_\theta: \mathbb{R}^d \to \mathbb{R}^{m_1 \times m_2 \times \ldots \times m_k}$.

Assume that the datapoints are clusterable in the embedding space. Now, consider the subset of concepts indexed by $I \subseteq [k]$ for which we have at least one labeled datapoint available. Let $\mathcal{S} \subseteq \text{supp}(G)$ be the subset of concept assignments in the support of $G$ where concept supervision is available (at least one labeled example per concept $i \in I$).

By adding a cross-entropy loss for the concepts in $I$ to the training objective, given the clusterability assumption, and under the Assumption 1 from [2] the number of deterministic $p_\theta(\mathbf{C} \mid \mathbf{G})$ optima becomes:

$$\sum_{\alpha \in \mathcal{A}} \mathbb{1}\{ \bigwedge_{g \in \mathcal{S}} \bigwedge_{i \in I} \alpha_i(g) = g_i \}.$$

Notice that this is the same number of deterministic optima obtained by the data-based mitigation strategy in [2], which assumes that for every concept $i \in I$ and for every datapoint the concept supervision is provided. Our method achieves this equivalence by leveraging clusterability in embedding space, hence reducing the labeling effort required. Indeed, under our clusterability assumption, it suffices to have only a single labeled datapoint for each concept $i \in I$ to match the theoretical guarantees obtained by dense supervision in [2].

[Q5] Tested on totally unsupervised settings

One of the assumptions of this work is that we assume we have at least two datapoints belonging to two different classes (or concepts) labelled. Indeed, as shown above, without such supervision the number of reasoning shortcuts is equivalent to the number of reasoning shortcuts existing in any other model. Also, looking at the results shown in RQ4 and Figure 4 (as well as Figures 13 and 14 in the Appendix), it is clear that the more concepts have at least one labelled example, the better the results. This is a very lightweight assumption in many application domains, as randomly sampling two datapoints and labelling them is not costly. However, we agree that how to drop this assumption is an interesting avenue for the future.

[Q6] Solution proposed in [3] requires dense annotations

We believe there might have been a slight misunderstanding here.

We stated that one of the mitigation strategies proposed in [2] requires dense annotations. In [2], to conclude Proposition 5, the authors need to have dense annotations for the concepts they want to consider annotated, otherwise they cannot make use the fact that the deterministic concept distributions $p_\theta(\mathbf{C} |\mathbf{X})$ of NeSy predictors $p_\theta(\mathbf{Y} \mid \mathbf{X}; K)$, correspond one-to-one to the deterministic distributions $p_\theta(\mathbf{C} \mid \mathbf{G})$ yielding label distributions $p_\theta(\mathbf{Y} \mid \mathbf{G}; K)$.

In our experimental analysis (RQ2) we have compared with the mitigation strategies proposed so far in the literature and also with their combination. As we can see, using prototypical networks managed to outperform even the combination of the three mitigation strategies proposed so far.

Regarding a study on the efficacy of these methods, we will try to conduct one in the rebuttal time.

[Q7] Code unavailable

Our paper is not yet publicly available on arxiv. We will make the code and paper publicly available upon publication.

References:

[1] Allen et al. Infinite Mixture Prototypes for Few-Shot Learning, 2019.

[2] Marconato et al. Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcuts, 2023

[3] E. Marconato, BEARS Make Neuro-Symbolic Models Aware of their Reasoning Shortcuts, UAI (2024)

Thank you for the detailed response.

I have a few additional concerns. I may not have expressed myself clearly earlier, please feel free to ask for clarification if that is the case.

First, while I agree that only one labeled concept is needed under the disentanglement assumption, this assumption is critical. If it does not hold, you would probably need a labeled example for each possible concept configuration. For datasets like BDD-OIA, this could require up to $2^{21}$ labelled concept configurations, which seems impractical. Could elaborate on this?

Secondly, regarding your answers to Q1, Q3, and Q6: I believe the main text would benefit from clearer explanations on those points.

Lastly, regarding the answer for Q5, I found the BDD-OIA experiment in the appendix particularly compelling. This is just general curiosity, do you think it could fit into the main paper?

We thank the reviewer for their insightful comments, as we believe implementing them will strongly improve the narrative of the paper.

Due to space concerns, at times we had refer to answers to other Reviewers.

[Q1+W.Method1] Prototype proximity and logical consistency

Taking into account the proximity of the embeddings of known datapoints when updating the weights of the neural network does not help in better satisfying the constraints, but rather in better assigning the ground truth label to the unlabeled datapoints. Indeed, for many datasets satisfying the constraints is not enough to assign the correct labels.

To give a simple example, suppose we have an image multi-label classification problem with three classes [$Zebra, Mammal, Fish$] and the constraint $Zebra \to Mammal$, stating that if for an image the label $Zebra$ is predicted so should the label $Mammal$.

Suppose we have a dataset with 10 images, 7 portraying fish, 1 portraying a zebra (and hence a mammal) and 2 portraying mammals that are not zebras.

Suppose we have 0 labelled datapoints, then a model always predicting the labels [$Zebra, Mammal$] would achieve perfect logical consistency (and NeSy loss = 0) but it would get accuracy over the labels equal to:

Acc($Fish$) = 0.3, Acc($Zebra$) = 0.1 and Acc($Mammal$) = 0.3

Suppose now we have the same dataset with two labelled datapoints: one portraying a fish and one portraying a zebra (and the rest unlabelled). Then, a model always predicting [$Zebra, Mammal$] for all datapoints, but the one labelled [$Fish$]—for which it predicts [$Fish$]---would achieve loss = 0, but only get accuracy equal to:

Acc($Fish$) = 0.4, Acc($Zebra$) = 0.2 and Acc($Mammal$) = 0.4

Now, if we suppose that the datapoints are clusterable in the embedding space (for a formal analysis see answer to [Q4], Reviewer cxP2), then given the same dataset, a model like ours, if it achieves perfect loss, then it also achieves Acc($Fish$) = Acc($Zebra$) = Acc($Mammal$) = 1.0.

[Q2] Relationship between semantic alignment and symbolic satisfaction

By semantic alignment we mean associating to each datapoint the right set of labels. For symbolic satisfaction we mean that the set of labels associated to each datapoint satisfies the constraints. Hence, semantic alignment is a sufficient condition for symbolic satisfaction, and symbolic satisfaction is a necessary condition for semantic alignment. We will make this clearer in the final version of the paper.

[W.Intro1] Intuition lacks rigour

We thank the reviewer for raising this point.

We acknowledge that the first intuition, concerning the availability of randomly annotated datapoints, is indeed an assumption. However, we believe that it is realistically grounded in many real-world settings, where practitioners can easily annotate a few datapoints. Thus, our assumption reflects a standard partial supervision regime in which the user is asked to label in most cases just one or at most a handful of datapoints.

Regarding the second intuition, we emphasize that this is not merely a heuristic design choice but a key explanatory mechanism underlying the success of our method, particularly in contrast to prior NeSy approaches.

To illustrate this, consider the MNIST-EvenOdd task discussed in the paper. In this setting, standard NeSy techniques typically proceed as follows:

1. For datapoints labelled only with the value of the sum, the model is trained to ensure that the sum of the predicted digits return the correct value
2. For datapoints labelled with both individual digit values, the standard cross-entropy loss is applied

A core limitation of this approach lies in the nature of the constraint space. In MNIST-EvenOdd, given the pairs present in the dataset, there are 49 distinct combinations of digit values that satisfy the sum constraint (see answer to [Q4] to reviewer bWcR for a detailed explanation). Hence, for the unlabelled datapoints the model is not guided toward correct digit-level predictions, but is only trained to satisfy the constraint, allowing for many incorrect configurations that are constraint consistent.

The model should instead prefer constraint-satisfying configurations that are similar to those observed in the fully-labelled examples, but, more importantly, this notion of similarity cannot be easily defined in the raw input space, as it would require specifying a highly task-specific metric. Instead, we exploit the learned embedding space—induced by prototypical networks—where similarity to labelled examples emerges naturally through training.

This mechanism allows every datapoint to contribute to training in a way that is guided by proximity to the few annotated examples. As shown in Figure 3 (RQ3), this mitigates the sensitivity to the number of unlabelled datapoints.

We will include the above in the introduction.

[W.Intro2] Conceptual novelty

The mitigation strategies proposed so far and against which we compare are as follows:

1. In [1], the authors propose to densely annotate a subset of the labels, and they are able to theoretically prove that if some concepts are fully annotated then the number of reasoning shortcuts a perfectly trained model can take goes down, as expected.
2. Again in [1], the authors propose to use a reconstruction loss in addition to the cross-entropy loss. To use this loss, encoder-decoder architectures are needed: the encoder outputs both the concepts and a representation of the input, which is then used by the decoder to reconstruct the input.
3. In [2], the authors propose to use the Shannon entropy loss as an additional term in the loss function. Notice that this is equal to 0 only when each distribution $p_\theta(\mathcal{C^k})$ is uniform, which is often not the case.

When compared with the solutions above, ours:

1. Does not require dense annotations
2. Does not require an encoder-decoder architecture
3. Does not assume that the distribution over each $\mathcal{C}^k$ is uniform

Additionally, as shown in Table 2 of the paper, our method greatly outperforms all of the above.

We will make these aspects clearer in the introduction.

[W.Method1] Not clear how structures are retained in prototypical networks

In our approach, prototypes serve to anchor concept embeddings, while the NeSy loss $\mathcal{L}_{\text{NeSy}}$ encodes the hierarchical and compositional structure and drives the updates of both embeddings and prototypes accordingly.

Section 4 shows that the gradient of the NeSy loss with respect to an embedding $z_I^k$ (for the $k$-th group $\mathcal{C}_k$) is a weighted combination of prototype directions, where the weights correspond to logic-consistent error signals (Theorem 4.1, Corollary 4.2).

[W.Method2] Initialization is ad-hoc and lacks semantic grounding

Centroids are only initialized this way and then we train them. They will move according to the knowledge satisfaction: which will give them the semantic grounding.

[W3.Method3] Missing citation

We thank the reviewer for this citation, we will add it to our discussion. We will also take inspiration from it for our future work, where we will allow for more prototypes for each class.

[W.Method4] Reliance on data augmentation

Data augmentation is not a necessary aspect of our algorithm, just something that, when possible, leads to more stable centroids. To underline this, in the autonomous driving experiment, we have not used any data augmentation, still getting remarkable improvements. See answer to [Q4] of Reviewer bWcR for more details.

Please notice that data augmentation is a standard technique used in both semi-supervised learning and zero-shot learning, and it upholds the label efficiency narrative, as these annotations are not manually made but rather obtained automatically.

[W.Evaluation1] Perfomance gains too large, concerns on complexity of the task.

We would like to clarify that the MNIST-EvenOdd and Kand-Logic tasks (together with BDD-OIA) were not introduced by us; rather, they were recently proposed specifically to address the challenge we study. These tasks are part of the “reasoning shortcut benchmark” (rs-bench), introduced at NeurIPS 2024 (D&B track), with the goal of evaluating the ability of neurosymbolic methods to overcome the reasoning shortcuts problem. Please see answer to [Q4] for reviewer bWcR for more details.

[W.Evaluation2] More modest results on BDD-OIA

To better highlight how much we can improve even on this task we re-run the experiments on BDD-OIA, but this time we used prototypes for all concepts, and not only those that lead to the action Stop. Please see answer to [Q4] Reviewer bWcR for more details on this experiment. The results show that our method can outperform all the considered models.

[W.Evaluation3] Unclear how the method performs on tasks where concepts are not easily represented by one prototype.

This is surely an interesting question for future work. However, regarding the current paper, please take into account that we manage to significantly improve the results on the the most recent benchmark proposed to evaluate NeSy models, where current methods do not manage to disambiguate between concepts that can be represented as a single prototype.

[W.Evaluation4] Old baselines

The baselines still represent sota methods in NeSy, they are used in many papers studying the reasoning shortcuts problems (e.g., rs-bench), and we compared against mitigation strategies that are more recent [1,2].

In answer to [Q1+W1.b] to Reviewer CSxp we showed that our method is able to greatly outperform ABLKit as well.

References

[1] Marconato et al Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcuts, NeuIPS, 2023

[2] Manhaeve et al Neural probabilistic logic programming in deepproblog. Artificial Intelligence, 2021

[3] Yu-Xuan Huang et al., ABLkit: a Python toolkit for abductive learning, Frontiers of Computer Science, 2024

Dear Reviewer,

The deadline for the author-reviewer discussion is approaching (Aug 8, 11.59pm AoE). Please read carefully the authors' rebuttal and engage in meaningful discussion.

Thank you, Your AC

We would like to thank the reviewer for their thoughtful and constructive comments. We will incorporate the suggested discussions and clarifications in the revised manuscript, which will strengthen the overall contribution and improve the presentation and interpretation of the experimental results.

[Q1] Why randomness?

We chose to retain the well-known benefits of random weight initialization in deep learning—such as effective exploration of the parameter space and robustness across different seeds. In our setting, centroids act as learnable parameters (analogous to weights) and thus benefit from similar initialization strategies.

However, initialization is particularly important in our setting due to the presence of reasoning shortcuts. Without any control on their placement, a prototype may be placed far from the relevant embeddings, leading the model to ignore it and instead collapse multiple concepts onto a single, incorrect prototype that still satisfies the constraints. To mitigate this, we sample centroids within a hyperball centered on the mean of known prototypes (with $p=0.99$). This encourages separation among mutually exclusive classes and reduces shortcut-induced failures, while still preserving the flexibility and diversity of random initialization.

[Q2] Standard notation.

If we followed the standard notation used e.g., in [1] then we would have made the assumption that there are a set of concepts (or intermediate outputs that are not supervised) and a set of final labels (of final outputs which are fully supervised). However, this assumption is not necessary in many NeSy systems (like SL), as they can seamlessly handle scenarios where each datapoint can be partially labelled with any subset of the labels, and there is not inherent distinction between intermediate concepts and final labels.

Additionally, the current notation allows us to explicitly associate every atom in the formula with a specific input-output pair of the neural network, making the connection between the two more explicit.

[Q3] Unfair comparison

Regarding this point we would like to highlight that, in addition to the standard baselines (with no access to labelled datapoints)---indicated with LTN, DPL and SL, we have also compared with:

1. Fully supervised CNNs (the same we used as prototypical extractors) that have access to the same amount of supervised datapoints on the MNIST-EvenOdd task, as reported with “Standard NN” in Figure 3. We agree that we should have run this baseline also on Kand-Logic and we should have included these results in Table 1. We will do so in case of acceptance. Below, we report the performance on the MNIST-EvenOdd dataset in Table A and the performance on Kand-Logic in Table B. As we can see from the baseline, with the small augmented labelled dataset available to our PNets, the fully supervised baselines only managed to get mean accuracy equal to 0.59 on MNIST-EvenOdd and 0.34 on Kand-Logic. We have also run the fully supervised baseline on the autonomous driving dataset (BDD-OIA), see answer below.
2. Augmented neurosymbolic AI baselines which are trained with both labelled and unlabelled datapoints indicated with LTN(Aug), DPL(Aug) and SL(Aug) - these methods are trained with exactly the same dataset as our methods. As we can see from Table 1, adding the same amount of supervisions helps in improving their performance, but in a much more limited way than using the PNets. This is because, for each unlabeled datapoint, the weights are updated solely based on the degree of constraint satisfaction. In contrast, when using prototypical networks, the update also accounts for the proximity of the embedding to each prototype. This also explains why our method exhibits significantly greater robustness to the number of unlabeled datapoints used during training compared to standard architectures, as demonstrated in RQ3 of the paper.

Table A: Performance on MNIST-EvenOdd for (i) the fully supervised baseline, (ii) the augmented NeSy methods and (iii) our NeSy + prototypical networks. For F1 the higher the score the better---indicated as ($\uparrow$)---for Cls the lower the score the better---indicated as ($\downarrow$). Best results are always in bold

Table B: Performance on Kand-Logic for (i) the fully supervised baseline, (ii) the augmented NeSy methods and (iii) our NeSy + prototypical networks

We believe this comparison is fair, as every model has access to the same amount of data.

[Q4] Simple experiments

We would like to clarify that the MNIST-EvenOdd and Kand-Logic tasks were not introduced by us; rather, they were recently proposed specifically to address the challenge we study. These tasks are part of the “reasoning shortcut benchmark” (rs-bench), introduced at NeurIPS 2024 (D&B track), with the goal of evaluating the ability of neuro-symbolic methods to overcome the reasoning shortcuts problem.

These tasks are particularly challenging due to the large number of possible reasoning shortcuts a model can exploit. For example, in the MNIST-EvenOdd task, there are 49 different digit-to-label assignments that perfectly satisfy the logical constraints, yet only one of them corresponds to the correct digit labeling. This arises because only a limited set of digit pairs appears in the training set. In Table C below we give the list of the 8 possible input image pairs together with their associated label (their sum) appearing in the dataset:

Table C. Input image pairs and their labels in MNIST-EvenOdd dataset

As a result, the model may converge to a spurious mapping that satisfies the sum constraint on all training examples but mislabels every digit. For instance, interpreting each 0 image as 4 and 6 image as 2 leads to correctly infer the label 6 for every image pair (0,6). Similarly, the interpretation in Table D achieves perfect constraint satisfaction on the training set while failing to correctly classify every single digit image:

Table D. Possible digits assignment for MNIST-EvenOdd

Hence, even though MNIST-EvenOdd and Kand-Logic might appear simple datasets, they are actually not for NeSy models, as the knowledge is not enough to correctly identify the concepts. Additionally, as we have seen in the previous question, few labelled datapoints are not enough to achieve ~0.90 accuracy.

However, we agree that real world scenarios might present noisier data. To better highlight how our model would perform in noisy and real world scenarios, we re-run the experiments on the autonomous driving dataset (BDD-OIA), but this time we used prototypes for all concepts, and not only those that lead to the action Stop.

To have a fair comparison, we also

1. trained DPL not only with the unlabelled data but also with the 128 datapoints available to our method (indicated as DPL (Aug)), and
2. the model in a fully-supervised way (using all the 16k datapoints), thus highlighting the difficulty of the task.

We report the overall results in Table E.

Table E. Results on the autonomous driving dataset (BDD-OIA)

Finally, we report the results when stratifying the concepts according to the resulting action in Table F. For example, F1(Forward) is calculated as the mean over the F1 of all the concepts that cause the action ''Forward'', i.e., "Green Light", "Follow" and "Road Clear".

Table F: Results on BDD-OIA stratified by the observed action

Our original aim in including the BDD-OIA experiment (over just some concepts) was to demonstrate that our approach remains effective even in more realistic and noisy scenarios, where perfect accuracy—as seen in MNIST-EvenOdd and Kand-Logic—is not attainable. However, we acknowledge that the current presentation may have overemphasized the limitations without sufficiently highlighting the strengths. To address this, we will include the above additional results covering all concepts as well as comparisons with the fully-supervised baseline, which will better showcase the model's robustness and generalization in this setting.

References

[1] Marconato et al. Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcuts, NeurIPS, 2023

Dear Reviewer,

The deadline for the author-reviewer discussion is approaching (Aug 8, 11.59pm AoE). Please read carefully the authors' rebuttal and engage in meaningful discussion.

Thank you, Your AC

Dear Reviewer,

We thank for engaging with us.

We would like to point out that all the LTN(Aug), SL(Aug) and DPL(Aug) are trained with all the available supervised data and unsupervised datapoints. The difference in performance between our method and these methods shows that the presence of the labelled datapoints does not make the task easy.

We understood now that the reviewer meant that we should have pre-trained until convergence on the supervised data and then fine-tune on the unsupervised. We are happy to provide this experiment. Just allow us some time to run it.

Thanks a lot for your time.