Learning Reliable Rules by Re-generating Deep Features

We thank the reviewer for the time and effort put into evaluating our work. We address each of the concerns in detail below.

Q1: understanding logical reasoning for real-life images.

A1: To understand logical reasoning for real life images, we need two critical specifications: 1. a proper abstraction (or representation) over the information encoded in a given image (i.e. the construction of predicates and their meaning), and 2. logical rules on top of a given abstraction.

In this work, we mainly focus on the second point and learns propositional rules on top of the abstraction of feature vectors (since classifiers rely on the feature maps to perform predictions). In fact, it is possible to fit other abstractions into our Vi-SATNet models since they are designed to be a drop-in layer given any abstraction (in this work, feature maps) paired with a reasoner (in this work, a classifier).

Q2: performance peak at 30% masking ratio.

A2: We thank the reviewer for the detailed observation in our results. The difference in the performance of the model from 10% masking to 30% masking is marginal. We have evaluated the model again across ten runs and the result is summarized in Table 4 (we have updated these results accordingly in the revised pdf). Increasing the mask ratio makes the reasoning task more difficult and hence results in lower accuracy. We can see that with re-generation of feature vectors given by our model, the classification accuracy maintained above 90% even when 80% of the values are missing, significantly outperforming the vanilla VGG-19.

Table 4: Classification accuracy with different blur mask ratio on Imagenette. Mean accuracy and error bar reported on 10 runs for each mask ratio.

We appreciate the reviewer's constructive feedback and concise summary of our contributions. Please find our detailed responses to each of the comments in the following.

Q1: Vi-SATNet training process and hyperparameters.

A1: We thank the reviewer for pointing out the confusion on the hyperparameter selection process. In Table 3 we include explanations for each of the hyperparameters used during training (taking the Imagenette as an example). It is worth noticing that the Vi-SATNet model is not very sensitive to the values of hyperparameters, which is a nice property inherited from SATNet[4]. During training, we did minimum ablation on the hyperparameters and found insignificant difference in performance. Hence, we present the performance of simple models for evaluation. More results on ablation studies can be included in future work to assess the robustness of Vi-SATNet models.

Table 3: Vi-SATNet hyperparameters for Imagenette.

Q2: Interpretation and significance of the learned rules; explanation of MSFs with respect to the rules.

A2: The detailed explanation of the correlation between MSFs and the learned rules can be illustrated by the following examples.

Consider the following example of an image from the "golf ball" class (this image was included in the paper pdf, but you can also find it in the supplementary material, Figure 1b):

There are around 12K rules extracted from the weight matrix learned for the class "golf ball" in the form of weighted MaxSAT. This set of rules is a discrete representation of what a Vi-SATNet has learned. The rule extraction procedure follows the one presented in [1]. The first few lines of the extracted rules are shown in Table 1. Given a target feature vector and its MSFs, we can pinpoint the rules that are related to variables representing the target feature vector and its MSFs, which on average results in a set of less than 100 rules (depending on the size of MSF). The first few lines of the subset for image 1b with f12 as the target feature is shown in Table 2.

Table 1: Some rules from the extracted weighted MaxSAT rules for class "golf ball".

Table 2: Some rules that are related to the target feature (f12).

Now, each feature vector can be mapped back to the discrete (boolean) space by randomized rounding [2]. Hence, we can deploy an external SAT solver[3] to compute the boolean value of any missing feature vector, given the learned rules (which constraint the relations between the features) and the MSFs (which can be discretized and plugged into the formula as known values). By solely inputting the boolean values of the MSFs, the solver is able to correctly output the boolean value for the target feature (in this example, the target feature is f12, and the MSFs are $<$f11, f13, f14, f19, f26$>$, the solver assigns f12 the value of 1). The ground truth boolean value of the target feature is obtained by applying randomized rounding to the feature vector itself (in this example, the ground truth of f12 is also 1).

The exact same procedure can be applied to a different example from the "parachute" class (please kindly find this image in the supplementary material, Figure 1c), with target feature being f13, and MSF indices being $<$12, 14, 20, 28, 47$<$. The solved value for f13 is 1 and the ground truth value for f13 is 1 as well.

We thank the reviewer for the nice summary and the thoughtful questions. Below, we address the comments point-by-point.

Q1.1: MSFs are surrounding features.

A1.1: We completely agree with the reviewer that CNN features are inherently closely related to the surrounding features. However, we would like to point out that not all surrounding features are included in the minimal significant feature set (MSFs) found. Specifically, consider the four direct neighbors of a target feature (up, down, left, right): in the MSFs shown in the results, we can see a clear inclusion of neighboring features that are in the foreground (if target feature is in the foreground) and exclusion of neighboring features that are in the background. This observation precisely shows the meaning of an MSF: only the features that are significant to the generation of the target feature are identified.

Please kindly find a new example in the supplementary material (Figure 1a), which has an MSF that includes a top left feature located at position (0,0) for the target feature that is in the top right corner located at position (6,0). The two features included in the MSF are both background features (since the target feature is also in the background) and one of them is not a neighboring feature of the target feature.

We hope that this example clarifies the concern of MSFs only containing neighboring features.

Q1.2: Rule interpretation from other perspectives.

A1.2: We thank the reviewer for asking this question. As mentioned in the paper, we chose to present the learned rules via visualization of the receptive fields of MSFs because it is the most straight forward interpretation method. A different method to present the learned rules is by directly showing the extracted propositional formulas on the boolean variables, where each variable represents an abstracted state of the corresponding feature vectors. Consider the following example of an image from the "golf ball" class (this image was included in the paper pdf, but you can also find it in the supplementary material, Figure 1b):

There are around 12K rules extracted from the weight matrix learned for the class "golf ball" in the form of weighted MaxSAT. This set of rules is a discrete representation of what a Vi-SATNet has learned. The rule extraction procedure follows the one presented in [1]. The first few lines of the extracted rules are shown in Table 1. Given a target feature vector and its MSFs, we can pinpoint the rules that are related to variables representing the target feature vector and its MSFs, which on average results in a set of less than 100 rules (depending on the size of MSF). The first few lines of the subset for image 1b with f12 as the target feature is shown in Table 2.

Table 1: Some rules from the extracted weighted MaxSAT rules for class "golf ball".

Table 2: Some rules that are related to the target feature (f12).

Now, each feature vector can be mapped back to the discrete (boolean) space by randomized rounding[2]. Hence, we can deploy an external SAT solver[3] to compute the boolean value of any missing feature vector, given the learned rules (which constraint the relations between the features) and the MSFs (which can be discretized and plugged into the formula as known values). By solely inputting the boolean values of the MSFs, the solver is able to correctly output the boolean value for the target feature (in this example, the target feature is f12, and the MSFs are $<$f11, f13, f14, f19, f26$>$, the solver assigns f12 the value of 1). The ground truth boolean value of the target feature is obtained by applying randomized rounding to the feature vector itself (in this example, the ground truth of f12 is also 1).