Understanding and Improving Adversarial Robustness of Neural Probabilistic Circuits

We sincerely thank the reviewer for the thoughtful feedback! We're glad you found the paper well-structured and easy to follow, with comprehensive theoretical analysis and extensive experiments!

Q1: The model partitions the attribute space into $V\_y$ for each class, assuming these are non-overlapping. But some attribute combinations may correspond to multiple classes. Can authors clarify this?

A1: We answer this question with the following three steps.

• Clarify why the sets $\\{ V\_y \\}$ are non-overlapping.

  We first review the construction process of $\\{ V\_y \\}$. Specifically, for any attribute assignment $a\_{1:K}$, we assign it to the class $y^\*=\arg\max\_{y\in\mathcal{Y}} \mathbb{P}\_D(Y=y\mid A\_{1:K}=a\_{1:K})$. That is, $a\_{1:K}\in V\_{y^\*}$. In the event of a tie, we can break it arbitrarily (e.g., by coin flipping), ensuring that each $a\_{1:K}$ is assigned to a unique $y^\*$. Thus, the sets $\\{V_y\\}$ are non-overlapping by construction.

• Clarify why the sets $\\{ \mathcal{N}(y, r) \\}$ are non-overlapping.

  We further interpret the reviewer's concern as questioning the non-overlap of the neighborhoods $\\{\mathcal{N}(y, r)\\}$. This property holds under a mild and implicit assumption: the radius $r$ of the attribute set is non-negative ($r \geq 0$). This corresponds to a minimum Hamming distance of at least 1 ($d\_{min} \geq 1$), which generally holds in the context of concept bottleneck models, where each class is associated with distinct concept combinations. In cases where this does not hold, we propose the following strategy.

• Propose a strategy for overlapping scenarios.

  One potential approach is to incorporate additional unsupervised attributes [1] during training to help capture fine-grained distinctions between classes sharing the same observed attribute values.

Thanks for this question. It helps us clarify the scenarios considered in our work and point out potential scenarios. We will include this discussion in the paper!

[1] "Concept bottleneck model with additional unsupervised concepts." IEEE Access. 2022.

Q2: Would the authors clarify that the RNPC's prediction is a weighted sum over classes of their contributions to the predicted class distribution, rather than to the variable $Y$ itself? Please consider rephrasing this statement for precision.

A2: Thanks for the suggestion. Below is our rephrased version that improves the clarity of Eq. 2:

Compared to the original Eq.2, we add $Y=y$ and $X=x$, making it clearer that RNPC's prediction is a weighted sum over classes of their contributions to the predicted class—$y$.

Q3: How does RNPC perform when confronted with adaptive attacks that craft adversarial perturbations capable of crossing inter-class? If authors could include an analysis or experiments to measure RNPC's resilience to such attacks, it would prove the robustness and versatility of RNPC against different types of attacks.

A3: We answer this question with the following three steps.

• Clarification on the role of the radius $r$.

  First, we'd like to clarify the relationship between RNPC's performance and the radius of the attribute set, denoted by $r := \lfloor\frac{d\_{\min}-1}{2}\rfloor$. As stated in Lines 167-170, when the number of attacked attributes is not greater than $r$, the perturbed probabilities are transferred to nodes within $\mathcal{N}(y^\*, r)$, where $y^\*$ denotes the ground-truth class. RNPC is designed to aggregate these probabilities via Eq. 2, thereby tolerating such perturbations. Theoretically, Lem. 4.6 ensures that the prediction perturbation $\Delta\_{\theta, w}^{RNPC}$ is small in this case. Empirically, Fig. 2 demonstrates that RNPC achieves high adversarial accuracy on MNIST-Add3 ($r = 1$), MNIST-Add5 ($r = 2$), and CelebA-Syn ($r = 1$) datasets when a single attribute is attacked. These results validate RNPC’s robustness when the number of attacked attributes is within $r$.

• Define adaptive attacks for RNPC.

  Based on the understanding of the role of $r$, we can now define adaptive attacks for RNPC. Specifically, knowing the value of $r$, an adaptive attacker for RNPC will deliberately generate adversarial perturbations that target more than $r$ attributes. Then, how does RNPC perform against such adaptive attacks?

• Evaluation against adaptive attacks.

  In fact, this question has been answered in our paper. Specifically, Fig. 3 (a-b) in Sec 5.3 illustrate the performance of RNPC under varying numbers of attacked attributes. For your convenience, we summarize the results in the following tables. As shown in Table 14 and Table 15, RNPC consistently outperforms baseline models. In particular, when the number of attacked attributes is within $r$, the performance gap is significant. For instance, RNPC achieves up to 45% higher adversarial accuracy than the best-performing baseline on MNIST-Add3 when one attribute is attacked. As the number of attacked attributes exceeds $r$, RNPC’s advantage narrows, yet it still maintains the highest robustness among all models.

Overall, on these datasets, compared to baseline models, RNPC exhibits strong robustness against the adaptive attacks that target more than $r$ attributes.

Table 14: Adversarial accuracy on MNIST-Add3 under the PGD attack with varying numbers of attacked attributes.

Table 15: Adversarial accuracy on MNIST-Add5 under the PGD attack with varying numbers of attacked attributes.

Q4: Would it be feasible to consider weighting the Hamming distance to reflect the relative importance or semantic impact of different attributes?

A4: Yes, our framework could definitely be extended to incorporate weights associated to each attribute when defining the distance between two attribute assignments.

Formally, let each attribute $A\_i$ be assigned with a weight $w\_i$, such that $\sum\_i w\_i = K$. We can then define a weighted Hamming distance between two attribute assignments $a\_{1:K}$ and $a\_{1:K}^{\prime}$ as:

Based on this distance, we can define a new neighborhood:

This formulation allows certain attributes to be emphasized by assigning them higher weights, while others can be downweighted with smaller values of $w\_i$. However, determining appropriate weights would still require some prior domain knowledge specified by humans. Thanks for this suggestion, which presents an interesting direction to extend in future work.

Once again, thank you for your insightful questions, which help us clarify our work more thoroughly and explore promising future directions!

Thanks for your constructive comments! Given the approaching end of the discussion phase, we wonder if the provided responses have addressed your questions? — Actually, we greatly value your feedback since it

• allows us to explain our method with better clarity,
• gives us the opportunity to demonstrate the robustness of RNPC against adaptive attacks,
• helps us exploit new potential directions.

We would appreciate it if these responses have satisfactorily addressed your concerns. Also, your acknowledgement of the comprehensiveness of our theoretical and empirical results has encouraged us a lot! Thanks again!

Dear Authors, thank you for the detailed answers. Your explanations addressed my main concerns about how the attribute partitions and neighborhoods are constructed to be non-overlapping, and detailing how RNPC’s robustness behavior is well-characterized both within and beyond the radius $r$. This addresses my concern about neighborhood-crossing perturbations.

We sincerely thank the reviewer for the comprehensive feedback! We're glad you found our robustness analysis insightful (model robustness only depends on robustness of the attribute recognition model) and appreciated the efficiency and effectiveness of RNPC (only requiring inference-time changes)!

Q1: Can authors include robustness results against corruptions and $l_1$, $l_2$-norm-bounded attacks?

A1: We answer this question in two steps.

• Point out existing results against $l\_2$-norm-bounded attacks

  The evaluations in this paper are not limited to $l\_\infty$-norm-bounded attacks. We performed extensive evaluations using both $l\_2$-norm-bounded PGD and CW attacks with varying budgets. Results are presented in Appendix G.2 (Fig. 6&7).

• Explore new results against $l\_1$-norm-bounded attacks and corruptions

  • Setting: We adopt the EADL1 attack [1] with varying values of $\beta$, which controls the $l\_1$ regularization strength with a larger value yielding a sparser perturbation, and use different types of corruptions, respectively.
  • Results: As shown in Table 9, RNPC consistently outperforms other baselines across EADL1 attacks with different $\beta$ values, demonstrating robustness of RNPC against $l_1$-norm-bounded attacks. For corruptions, as shown in Table 10, all models appear robust to defocus blur and Gaussian noise. CBM and DCR are notably vulnerable to fog, whereas RNPC still exhibits high robustness.

Overall, these additional results for corruptions and $l\_1$-norm-bounded attacks, together with the results for $l\_\infty$ and $l\_2$-norm-bounded attacks provided in the paper, validate the robustness of RNPC under various types of perturbations.

[1] "Ead: elastic-net attacks to deep neural networks via adversarial examples." AAAI.

Table 9: Adversarial accuracy against EADL1 on MNIST-Add3.

Table 10: Performance against corruption on MNIST-Add3.

Q2: How sensitive is RNPC to the parameter $r$ used for partitioning the attribute space?

A2: We answer this question in two steps.

• Clarification on radius. We clarify that, as defined in Def. 4.2, $r$ denotes the radius of the dataset, which is determined by its intrinsic structure. This value is fixed unless the dataset itself is modified. To avoid confusion, we explicitly denote this radius as $r^\*$. In contrast, it's possible to adjust the value of $r$ used in RNPC (see Eq. 2).
• Explore the effect of changing $r$ on the performance of RNPC
  • Setting: We conduct this analysis using MNIST-Add5, which has 5 attributes and a radius of $r^\* = 2$. We vary $r$ in the range [0, 5] and evaluate performance of RNPC under an $l\_\infty$-norm-bounded PGD attack with a perturbation budget of 0.11.
  • Benign Accuracy: As $r$ increases from 0 to 4, we observe a decreasing logit gap between the top-1 and top-2 predicted classes. Nevertheless, as shown in Table 10, RNPC consistently achieves near-perfect accuracy. When $r$ reaches 5, the accuracy drops to 29.9%. This is expected because, at $r = 5$, $\mathbb{P}\_\theta(A\_{1:K} \in \mathcal{N}(\tilde{y},r) | X) = \mathbb{P}\_\theta(A\_{1:K} \in \Omega | X) = 1$, meaning Eq. 2 no longer contains any information from the input $X$.
  • Adversarial Accuracy: When $r$ deviates from $r^\*$, the adversarial accuracy declines. Despite this drop, RNPC ($r\neq 5$) still outperforms other baseline models, which achieve below 40% accuracy (see Fig. 2 (middle)). Notably, when $r = 5$, adversarial accuracy drops to 29.9%, matching the benign accuracy and thus indicating a prediction perturbation of $\Delta\_{\theta, w}^{RNPC}=0$. This aligns with theoretical results, because in this case, the right-hand side of Lem. 4.6 becomes zero.

Overall, when $r \neq$ #attributes, RNPC demonstrates strong resilience to changes in $r$ under benign settings; while adversarial accuracy declines when $r$ deviates from $r^\*$, RNPC remains superior to baseline models.

Table 10: Performance of RNPC with various $r$ on MNIST-Add5

Q3: The theoretical analysis depends on the dataset having a sufficiently large minimum hamming distance. If the distance is small and the number of attributes is large, the number of partitions can be exponential. How does the proposed approach scale to tasks with many attributes?

A3: We answer this question in three steps.

• Clarifications

  First, we clarify that the theoretical results do not assume a specific radius $r$ or the minimum Hamming distance $d\_{\min}$. The results hold for any non-negative radius $r \geq 0$, which corresponds to $d\_{\min} \geq 1$. But technically, larger values of $r$ (and thus larger $d\_{\min}$) are expected to provide higher robustness.

  Second, the number of partitions is determined solely by the number of classes in the dataset, i.e., $|\\{V\_y\\}| = |\mathcal{Y}|$, and is independent of both $d\_{\min}$ and the number of attributes $K$.

• Scaling to large $K$

  We interpret the reviewer’s concern as follows: When $K$ increases, the size of the attribute space $\prod\_{k=1}^K |\mathcal{A}\_k|$ grows exponentially. How does RNPC handle this scaling?

  As shown in Prop. 4.4, the computational complexity of RNPC is:

Here, $|\mathcal{Y}| \leq |V| = \sum\_{y\in\mathcal{Y}} |V\_y| \leq \prod\_{k=1}^K |\mathcal{A}\_k|$. Therefore, although $|V|$ can be exponential in $K$ in the worst case, it can also be as small as $|\mathcal{Y}|$ (when every class only has one high-probability attribute assignment), depending on the dataset's structure. In contrast, the computational complexity of NPC is always exponential in $K$.

• More discussion

  In fact, scaling remains an open challenge in neuro-symbolic models, especially those built upon graphical models. Some recent work explores approximation-based solutions. We will include this point in the discussion section.

Nevertheless, as demonstrated in Prop. 4.4, RNPC significantly reduces computational complexity compared to NPC, with improved inference time shown in Table 2 of Appendix E.

Q4: Authors discuss the vulnerability of RNPC to attack propagation in Limitations Section but didn't propose ways to mitigate it.

A4: In Appendix C, we provide a detailed discussion of strategies to mitigate the attack propagation effect. Here, we also introduce an adversarial training–based approach for mitigation and evaluate its effectiveness on GTSRB-Sub, where spurious correlations between attributes are strong and result in significant attack propagation (see Fig. 3(d)).

• Intuition

  Suppose a model learns a spurious correlation between two attribute values, e.g., red color and circle shape. That is, the model may rely on features of "red" to identify "circle". To mitigate this correlation, we need to disentangle their features. Specifically, by generating adversarial examples that perturb the color attribute, we can shift the feature representation of "red" toward another color, thereby weakening its connection with "circle".

• Method and Evaluation

  Following this idea, during training, we generate adversarial samples that target a random attribute and train the model on both these samples and benign inputs. Table 13 reports performance of resulting models. Compared to models trained without adversarial training (Table 12), the adversarially trained models show significantly smaller drops in accuracy of different attributes when color is attacked, which indicates a reduction in the attack propagation effect. Consequently, the advantage of RNPC over NPC becomes evident again.

Overall, the adversarial-training-based approach leverages adversarial perturbations during training to disentangle spurious correlations between attributes, which effectively mitigates the attack propagation effect and enhances the robustness of RNPC on real-world datasets.

Table 12: Col 2-4: Relative accuacy drop (%) of different attributes when color is attacked by PGD. Col 5-6: Task accuracy (%) of NPC and RNPC on GTSRB-Sub with a standard attribute recognition model

Table 13: Results with an adversarially trained attribute recognition model

Once again, thanks for your questions, which lead us to understand and evaluate our method more comprehensively!

Thanks for your constructive comments! Given the approaching end of the discussion phase, we wonder if the provided responses have addressed your questions? — Actually, we greatly value your feedback since it gives us the opportunity to validate the robustness of RNPC under various types of perturbations! Also, we greatly appreciate your acknowledgment of the novelty of our theoretical analysis in robustness. Thanks again!!

We sincerely thank the reviewer for the constructive feedback! We're glad that you found the paper well-written with clear motivation and sound theoretical analysis, the empirical results compelling, and the ablation studies effective in validating our hypothesis and highlighting meaningful future directions!

Q1: The proposed inference is based on some neighbourhoods of radius $r$. It would be interesting to see how the performance of RNPC varies with this radius.

A1: We answer this question with the following steps.

• Clarification on radius. First, we'd like to clarify that, as defined in Def. 4.2, $r$ denotes the radius of the dataset, which is determined by its intrinsic structure. This value is fixed unless the dataset itself is modified. To avoid confusion, we explicitly denote this radius as $r^\*$. In contrast, it's possible to adjust the value of $r$ used in RNPC (see Eq. 2).
• Explore the effect of changing $r$ on the performance of RNPC.
  • Setting: We conduct this analysis using the MNIST-Add5 dataset, which has 5 attributes and a radius of $r^\* = 2$. We vary $r$ in the range [0, 5] and evaluate both benign accuracy and adversarial accuracy of RNPC under an $l\_\infty$-norm-bounded PGD attack with a perturbation budget of 0.11. Results are provided in Table 8.
  • Benign Accuracy: As $r$ increases from 0 to 4, we observe a decreasing logit gap between the top-1 and top-2 predicted classes. Nevertheless, as shown in Table 8, RNPC consistently achieves near-perfect accuracy. However, when $r$ reaches 5, the accuracy drops to 29.9%. This is expected because, at $r = 5$, we have $\mathbb{P}\_\theta(A\_{1:K} \in \mathcal{N}(\tilde{y},r) | X) = \mathbb{P}\_\theta(A\_{1:K} \in \Omega | X) = 1$, meaning Eq. 2 no longer contains any information from the input $X$.
  • Adversarial Accuracy: When $r\leq r^\*$ ($r\geq r^\*$), reducing $r$ (increasing $r$) results in worse performance. That is, deviating from the dataset radius $r^\*$ leads to a drop in adversarial accuracy. Despite this drop, RNPC ($r\neq 5$) still outperforms other baseline models, which achieve below 40% accuracy (see Fig. 2 (middle)). Notably, when $r = 5$, adversarial accuracy drops to 29.9%, matching the benign accuracy and thus indicating a prediction perturbation of $\Delta\_{\theta, w}^{RNPC}=0$. This aligns with our theoretical result, because in this case, the right-hand side of Lem. 4.6 becomes zero.

Overall, when $r \neq$ #attributes, RNPC demonstrates strong resilience to changes in $r$ under benign settings; while adversarial accuracy declines when $r$ deviates from $r^\*$, RNPC remains superior to baseline models.

Table 8: Performance of RNPC with various $r$ on MNIST-Add5.

Q2: Under which conditions will Eq. 2 be identical to Eq. 1? Is RNPC a generalized version of NPC?

A2: We will answer this question using two steps. First, we will directly compare $\mathbb{P}\_{\theta, w}(Y|X)$ (Eq. 1) with $\Phi\_{\theta, w}(Y|X)$ (Eq. 2), showing why RNPC is not a generalized version of NPC. Second, we will provide a more accurate comparison, comparing $\mathbb{P}\_{\theta, w}(Y|X)$ (Eq. 1) with the normalized probability distribution $\hat{\Phi}\_{\theta, w}(Y|X)$ (Line 183), showing when these two distributions are the same.

Step 1: Under Assump. 3.2 and the assumption that $\Omega = \bigcup\_{\tilde{y}}\mathcal{N}(\tilde{y}, r)$, we can write Eq. 1 as follows:

Meanwhile, Eq. 2 is:

These two equations only differ in the final terms. Specifically, the final term in NPC is independent of the class $\tilde{y}$, while the final term in RNPC is dependent on$\tilde{y}$, suggesting that it is doing the class-wise integration.

Therefore, RNPC is not a generalized version of NPC; they represent fundamentally different ways of integrating over attribute probabilities.

Step 2: Consider a scenario where we learn an optimal attribute recognition model and an optimal probabilistic circuit, i.e., $\mathbb{P}\_{\theta}(A\_{1:K}|X) = \mathbb{P}^\*(A\_{1:K}|X)$ and $\mathbb{P}\_{w}(Y|A\_{1:K}) = \mathbb{P}^\*(Y|A\_{1:K})$. Consequently, we have: $\mathbb{P}\_{\theta, w}(Y|X) = \mathbb{P}^\*(Y|X)$ and $\hat{\Phi}\_{\theta, w}(Y|X) = \hat{\Phi}^\*(Y|X)$ (Refer to Prop. 4.8).

Then, the comparison between $\mathbb{P}\_{\theta, w}(Y|X)$ and $\hat{\Phi}\_{\theta, w}(Y|X)$ reduces to evaluating the difference between $\mathbb{P}^\*(Y|X)$ and $\hat{\Phi}^\*(Y|X)$.

As shown in Thm. 4.11,the expected total variance distance between these two distributions is upper bounded by:

The detailed proof is provided in Appendix H.4, specifically Lines 1033-1038.

Hence, when this bound is zero, and both modules are optimally learned, we have: $\mathbb{P}\_{\theta, w}(Y|X) = \hat{\Phi}\_{\theta, w}(Y|X)$, i.e, the predicted distributions of NPC and RNPC are the same.

Once again, thanks for your questions, which lead us to better illustrate our method and intepret our theoretical results!

Thanks for your constructive comments! Given the approaching end of the discussion phase, we wonder if the provided responses have addressed your questions? — Actually, we greatly value your feedback since

• your first question gives us the opportunity to show that the value of $r$ used in our method turns out to be optimal,
• your second question allows us to use our theoretical results to gain a deeper understanding of the relationship between NPC and RNPC.

We would appreciate it if these responses have satisfactorily addressed your concerns. Also, we’re greatly encouraged by your acknowledgement of our writing ("Motivations, intuitions and theoretical discussions are well-crafted and aptly put in the paper.")! Thanks again!

We sincerely thank the reviewer for your thoughtful feedback! We're glad that you found our paper well-written, the theoretical analysis sound, the proposed method effective (only requiring inference-time changes), and the ablation studies insightful!

Q1: How does training the recognition module adversarially affect the robustness of NPC and RNPC?

A1: To explore the impact of adversarial training, during training, we use the PGD attack with an $l\_\infty$ bound of 0.11 to generate adversarial samples that perturb the recognition module’s predictions, and train the recognition module using both adversarial samples and benign samples.

Denote the NPC (RNPC) with an adversarially trained recognition module as NPC-AT (RNPC-AT). We evaluate performance on MNIST-Add3 under PGD attacks targeting the first attribute. Results are provided in Table 3.

• AT significantly improves the robustness of the recognition module.

  Without AT, the accuracy of predicting the first attribute declines from 99.3% to 2.2% after the attack. In contrast, with AT, this accuracy merely drops to 85%.

• AT also improves the robustness of both NPC and RNPC.

  Without AT, the task accuracy of NPC and RNPC declines to 19% and 64% after the attack. In contrast, with AT, their accuracy remains as high as 85.8% and 86.3%, respectively. This is attributed to the fact (Thm. 3.4 and Lem. 4.6) that the robustness of NPC and RNPC is only bounded by the robustness of the recognition module.

• AT leads to a slight performance drop in the benign setting.

  When using AT, the model's benign task accuracy drops by less than 0.5%.

Overall, adversarial training significantly strengthens the robustness of both NPC and RNPC by improving the robustness of their recognition module, and RNPC is still more robust than NPC thanks to the class-wise integration.

Table 3: Accuracy of predicting the attacked attribute ("Attr1 Acc") and class labels ("Task Acc").

Q2: How do NPC and RNPC compare against each other when attacking the end-to-end prediction pipeline?

A2: We begin by clarifying our threat model. NPC is a compositional model that integrates a graphical model, specifically a probabilistic circuit, and the only component with a black-box nature (which may be pretrained and obtained from a third-party source) is the attribute recognition model. Therefore, we consider a threat model in which the adversary targets the attribute recognition model, rather than the entire pipeline.

Next, to explore the impact of attacking the entire pipeline, we apply the PGD attack with varying $l_\infty$ bounds on the end-to-end prediction pipelines of NPC and RNPC. In addition to these compositional models, we also train an end-to-end ViT-B/32 model and attack it for comparison. Results are provided in Table 4.

Similar performance is observed on NPC and RNPC, which is reasonable since now the adversarial samples are crafted to perturb final class predictions. This setting doesn't align with RNPC's design objective, i.e., to robustly integrate incorrect attribute predictions with outputs of probabilistic circuit. Despite the performance drop, both NPC and RNPC perform much better than ViT, highlighting the advantage of compositional models. This finding also aligns with the one in [1].

Overall, both NPC and RNPC are vulnerable to attacks targeting the entire pipelines; however, they exhibit much higher robustness than the end-to-end black-box model.

[1] Exploring the impact of conceptual bottlenecks on adversarial robustness of deep neural networks. IEEE Access. 2024.

Table 4: Task accuracy against attacks targeting the end-to-end pipelines.

Q3: Can we predict which attributes will be attacked together based on how often they co-occur in the training data?

A3: We answer the question using three steps.

• Quantify the co-occurrence between attributes

  We use the mutual information (MI) metric, defined as $MI(X, Y) = \sum\_{x \in X} \sum\_{y \in Y} p(x, y) \cdot \log \left( \frac{p(x, y)}{p(x) p(y)} \right)$. A higher MI indicates a stronger dependency between two attributes. Table 5 displays the MI between color and other attributes. The attribute with the highest MI is symbol.

• Measure the influence on other attributes when a specific attribute is attacked

  Table 5 also reports the relative accuracy drop of other attributes when color is attacked. The attribute with the largest accuracy drop is also symbol.

• Assess the correlation between these metrics

  To statistically assess this correlation, we use the Pearson Correlation Coefficient (PCC), defined as $\rho = \frac{\mathrm{cov}(X, Y)}{\sigma\_X \sigma\_Y}$. We observe a PCC of 0.92, indicating a strong positive linear correlation between MI and accuracy drop.

Overall, these results suggest that it's possible to utilize the mutual information in the training data to find out which attributes are more likely to be affected when a specific attribute is attacked.

Table 5: Mutual information between color and other attributes, and relative accuracy drop when color is attacked on GTSRB-Sub.

Q4: Authors discuss remedies to mitigate the attack propagation effect in appendices. Is there any empirical evidence?

A4: In Appendix C, we provide a detailed discussion of strategies to mitigate the attack propagation effect. Here, inspired by your previous question, we introduce an adversarial training–based approach for mitigation and evaluate its effectiveness on GTSRB-Sub, where spurious correlations between attributes are strong and result in significant attack propagation (see Fig. 3(d)).

• Intuition

  Suppose a model learns a spurious correlation between two attribute values, e.g., red color and circle shape. That is, the model may rely on features of "red" to identify "circle". To mitigate this correlation, we need to disentangle their features. Specifically, by generating adversarial examples that perturb the color attribute, we can shift the feature representation of "red" toward another color, thereby weakening its connection with "circle".

• Method and Evaluation

  Following this idea, during training, we generate adversarial samples that target a random attribute and train the model on both these samples and benign inputs. Table 7 reports the performance of the resulting models. Compared to models trained without adversarial training (Table 6), the adversarially trained models show significantly smaller drops in accuracy of different attributes when color is attacked, which indicates a reduction in the attack propagation effect. Consequently, the advantage of RNPC over NPC becomes evident again.

Overall, the adversarial-training-based approach leverages adversarial perturbations during training to disentangle spurious correlations between attributes, which effectively mitigates the attack propagation effect and enhances the robustness of RNPC on real-world datasets.

Table 6: Col2-4: Relative accuacy drop (%) of different attributes when color is attacked by the PGD attack. Col5-6: Task accuracy (%) of NPC and RNPC on GTSRB-Sub with a standard attribute recognition model.

Table 7: Col2-4: Relative accuacy drop (%) of different attributes when color is attacked by the PGD attack. Col5-6: Task accuracy (%) of NPC and RNPC on GTSRB-Sub with an adversarially trained attribute recognition model.

Q5: The authors use a small adversarial budget ($\epsilon = 0.11$) for MNIST datasets compared to a standard value of 3.0.

A5: We'd like to clarify that the commonly cited value of 3.0 applies to $l_2$-norm-bounded attacks, not $l_\infty$-norm-bounded ones. Fig. 2 in Sec. 5.2 corresponds to the latter. In fact, we did use $l_2$-norm-bounded PGD attacks with $\epsilon$ ranging from 3.0 to 11.0 on MNIST datasets. Please refer to Fig. 6 of Appendix G.2 for detailed results.

Once again, thanks for your insightful questions, which inspire us to better understand our method and propose effective solutions to potential limitations!

Thanks for your constructive comments! Given the approaching end of the discussion phase, we wonder if the provided responses have addressed your questions? — Actually, we greatly value your feedback since it

• helps us demonstrate that the robustness of RNPC can be further enhanced through adversarial training,
• motivates us to investigate the mechanism behind attack propagation and explore ways to mitigate it.

We are inspired a lot by your questions, and we would appreciate it if our responses have satisfactorily addressed your concerns.

Also, your acknowledgement of the soundness of our theoretical results (“All theorems are adequately defined and backed up with proofs. All assumptions and limitations are clearly stated.”) has encouraged us a lot! Thanks again!