DAG-SHAP: Feature Attribution in DAG based on Edge Intervention

We are very grateful to the reviewer for the insightful comments. To address the concerns, we provide detailed point-to-point responses as follows.

Response to Weakness and Question 1. In Theorem 4, we demonstrate that the sampling error increases non-linearly (with a complexity exceeding linear growth) as the number of edges in the DAG increases, requiring a higher number of samples to achieve a given error threshold. Since the total sampling time is equal to the number of samples multiplied by the computation time of the model for each sample, the sampling time scales linearly with the model's computational complexity. Therefore, the primary factor affecting sampling time is the number of edges in the DAG. DAGs in feature attribution are unlikely to appear in extremely high-dimensional data like images. Instead, they are typically found in tabular data. In such feature attribution scenarios, the goal is to help users understand the contributions and relationships between features. Users are typically interested in the contributions of a small number of key features, as these features are more relevant to the core of the problem and help avoid being overwhelmed by irrelevant or less significant information. Therefore, users tend to focus on a limited set of features to clearly understand their impact on the output. This is also why the datasets used in related works such as causal Shapley value and Asymmetry Shapley value experiments have not adopted large datasets.

When dealing with large datasets, parallel computing can effectively accelerate the feature attribution calculations. Feature attribution for different data points can be executed in parallel. Additionally, within the feature attribution process for a single data point, sampling different permutations can also be executed in parallel. We extended synthetic dataset (a) such that each data point had 100 features, and the DAG contained 200 edges. Specifically, we replicate $X_1$, $X_2$, $X_3$, and $X_4$ from the synthetic dataset (a) 25 times, resulting in 100 features in total. Each replicated feature maintains its causal relationship with $Y$, ensuring that the collective contribution of all features to $Y$ remains consistent with the contribution in the original structure. We use the same neural network structure as in the experiments in Section 5.1 and use a server equipped with two AMD EPYC 9754 128-Core Processors, providing a total of 512 logical processors. For each data point, we conduct two independent rounds of sampling and compare the mean absolute error (MAE) between the sampling results. The experimental results show that with 128 sampling permutations per data point, the MAE is 5.17%, which is significantly smaller than the errors caused by the different feature attribution algorithms. For instance, the absolute error between off-SHAP which has the smallest MAE in baseline and the benchmark was 17.24%. Using 128 threads for parallel sampling of 128 permutations, the computation time was only 57.5 seconds. Additionally, we conducted experiments with 256 and 384 permutations, resulting in mean absolute errors of 4.23% and 3.71%, respectively. We added the experiment analysis in Appendix Section D.3.

We conclude that the computational cost of DAG-SHAP is acceptable for most real-world applications.

Response to Q2. A real-world example can be found in medical diagnosis, where X1 represents a patient’s genetic mutation (e.g., a BRCA1 mutation), X2 represents the level of a protein biomarker detected in blood tests (e.g., the overexpression of a specific protein), and Y represents the predicted cancer risk. In this causal structure, X1 directly affects X2, and X2 acts as a mediator that transmits the effect of X1 to Y. Here, X2 has no independent exogenous contribution, and its impact on Y is entirely derived from X1. Traditional feature attribution methods may fail to capture this structure correctly, as they often ignore dependencies in causal pathways. These methods might assign a non-zero independent contribution to X2 or fail to distinguish between the indirect and direct contributions of X2, leading to incorrect attributions. In contrast, DAG-SHAP leverages the causal graph to accurately identify X2 as a variable that solely transmits the effect of X1. It assigns zero exogenous contribution to X2 and attributes the effect appropriately to X1. This precise attribution is critical in fields such as medical diagnosis and genetic research, where misjudging the importance of mediator variables can have significant implications.

We are very grateful to the reviewer for the insightful comments. To address the concerns, we provide detailed point-to-point responses as follows.

Response to W1. We acknowledge that in certain applications, the DAG cannot be directly identified. However, many approaches in the field of causal inference, such as the Additive Noise Model, Post Nonlinear Model, Peter-Clark Algorithm, and Inductive Causality, can assist in constructing the DAG. DAGs are particularly suitable for modeling feature relationships in tabular data, and since we only require the causal directions between features, the task becomes significantly simpler. Moreover, DAGs are a fundamental and widely used structure applicable to numerous scenarios, making it crucial to ensure reasonable attributions within this framework.

Response to the Difference to Average Causal Effect. DAG-SHAP values are inherently local, meaning they explain feature contributions for individual predictions rather than providing a global average effect. This locality refers to the specific interaction of a feature with other features in a single data point, capturing how the combination of features contributes to the model’s output for that specific instance. In contrast, measures like Average Causal Effect (ACE) are global and reflect the average impact of a feature across the entire population, without considering specific feature combinations or the behavior of the model on individual predictions. By integrating causal knowledge into SHAP, DAG-SHAP retains this local perspective, making it particularly suitable for analyzing feature interactions and their contributions to a model’s prediction for individual data points.

suppose we have a model that predicts the probability of a heart attack (Y) based on two features: X1 (BMI) and X2 (average sleep duration). Consider Sample A as the baseline (BMI = 25, Sleep Duration = 7 hours) and Sample B (BMI = 30, Sleep Duration = 5 hours) as the target case. Using ACE, we determine that relative to the baseline, a 1-unit increase in BMI raises the probability of a heart attack by 1%, while reducing sleep duration by 1 hour raises it by 2%. Based on these average effects, the total increase in probability for Sample B would be estimated at 9% (5 BMI units × 1% + 2 hours less sleep × 2%). However, because BMI and sleep duration may have synergistic effects, the true probability increase for Sample B could be higher due to their interaction. For instance, their combined influence might lead to an 11% increase in probability, rather than the additive 9% suggested by ACE. DAG-SHAP captures this interaction by distributing the contributions among the features based on their individual and joint effects. In this case, DAG-SHAP might allocate 6.5% to BMI and 4.5% to sleep duration, accurately reflecting the combined and interactive effects of these features on the prediction. This ability to provide local explanations for individual predictions is a key advantage of DAG-SHAP. It allows the method to account for interactions between features, distinguishing it from ACE, which provides only global average effects. This makes DAG-SHAP particularly effective for interpreting machine learning models with complex feature interactions.

Response to Additional Concern 1. In causal inference, traditional node intervention $\text{do}(X=x)$ applies an external intervention to a variable $X$, severing its causal relationships with all parent nodes and forcing its value to be fixed at $x$. In contrast, edge intervention $\text{do}(\text{edge}=e)$takes a different approach. Its goal is to intervene on a specific edge $e = (P \to C)$ in a DAG, affecting only the causal transmission from the parent node $P$ to the specified child node $C$, without altering $P$'s influence on other child nodes. We clarify the definition of Edge Intervention in Section 3.2. DAG-SHAP focuses on explaining the causal attribution for individual samples, evaluating how each feature contributes to the model output, considering interactions and dependencies among features from a cooperative perspective. Its uniqueness lies in the way it allocates causal effects, which can be referenced in the response to the differences with the Average Causal Effect (ACE).

Thank you for your time and valuable feedback. Regarding your concern that DAGs are not identifiable from data, we would like to address your concerns with the following points:

Identifying DAG from data. In the field of causal inference, many methods have been proposed to identify and estimate DAG structures from data. For example, the Peter-Clark algorithm and Inductive Causality Algorithm build DAGs by testing conditional independence relationships in the data. Other methods, such as the Greedy Inference from Graph-based models (GIE) algorithm, use a likelihood maximization approach to infer causal structures. These methods have been successfully applied in various practical scenarios, demonstrating high accuracy and reliability. Additionally, there are causal function-based models, such as the Additive Noise Model (ANM) and Post-Nonlinear Model (PNL). The core principle of these models assumes that there is a functional relationship between the cause (X) and the effect (Y). Specifically, if X positively regresses Y and the noise term is independent of X, and if Y negatively regresses X and the noise term is not independent of Y, then X is inferred to be the cause of Y. Furthermore, prior knowledge and hybrid methods are also commonly used for DAG identification from data.

DAG in Feature Attribution. We would like to point out that there has been some existing research on DAGs applied to feature attribution, such as Shapley Flow[1], Recursive Shapley Value[2], Shapley ICC[3], and PWSHAP[4]. We believe that while these methods address the basic structure of a DAG between data features, existing feature attribution techniques still face challenges related to exogeneity, externalities, and causality. In response to these issues, we propose a DAG-SHAP method based on edge interventions. As a fundamental causal structure, addressing attribution issues in DAGs is meaningful. Moreover, our method does not require a specific causal structural equation, but only the directional relationships between features, making it a practical solution.

Challenges in DAG Identification. We understand your concerns about the difficulties in identifying DAGs, as there are cases where identifying a DAG from data may be challenging or even infeasible. However, we do not think that the research based on DAGs is completely impractical. Denying the possibility of identifying feature relationships in a DAG would, in fact, negate the progress made in the causal inference field, even though identifying DAGs falls outside the scope of our paper.

We hope that our response can alleviate some of your concerns regarding the prior knowledge of our paper. We are happy to continue the discussion if you have any further questions.

[1]Wang J, Wiens J, Lundberg S. Shapley flow: A graph-based approach to interpreting model predictions[C]//International Conference on Artificial Intelligence and Statistics. 2021
[2]Singal R, Michailidis G, Ng H. Flow-based attribution in graphical models: A recursive shapley approach[C]//International Conference on Machine Learning. 2021
[3]Janzing D, Blöbaum P, Mastakouri A A, et al. Quantifying intrinsic causal contributions via structure preserving interventions[C]//International Conference on Artificial Intelligence and Statistics. 2024
[4]Ter-Minassian L, Clivio O, Diazordaz K, et al. PWSHAP: a path-wise explanation model for targeted variables[C]//International Conference on Machine Learning. 2023

We are very grateful to the reviewer for the insightful comments. To address the concerns, we provide detailed point-to-point responses as follows.

Response to W1. We provide the proof that DAG-SHAP fulfills the desired properties in Appendix Section B.2, please refer to lines 786-917.

Response to W2. The computational complexity of DAG-SHAP is acceptable for most practical feature attribution tasks involving DAGs in real-world applications. DAGs in feature attribution are unlikely to appear in extremely high-dimensional data like images. Instead, they are typically found in tabular data. In such feature attribution scenarios, the goal is to help users understand the contributions and relationships between features. Users are typically interested in the contributions of a small number of key features, as these features are more relevant to the core of the problem and help avoid being overwhelmed by irrelevant or less significant information. Therefore, users tend to focus on a limited set of features to clearly understand their impact on the output. This is also why the related works such as causal Shapley value and Asymmetry Shapley value experiments have not adopt large datasets. To illustrate that DAG-SHAP support larger dataset, we conduct feature attribution in a synthetic dataset with 100 features and 200 edges, the detailed experimental results, experimental setups, models used, and the computational hardware are all provided in Appendix Section D.3.

Response to W3. We acknowledge that there are applications where the DAG cannot be directly identified. However, many approaches in the field of causal inference, such as the Additive Noise Model, Post Nonlinear Model, Peter-Clark algorithm, and Inductive Causality, can assist in constructing the DAG. DAGs are particularly suitable for modeling feature relationships in tabular data, and since we only require the causal directions between features, the task becomes significantly simpler. Moreover, as a fundamental and widely used structure, DAGs are applicable to numerous scenarios, making it important to ensure reasonable attributions within this framework.

Response to W4. Please see detailed response to Q1.

Response to Q1. The paper "Feature Relevance Quantification in Explainable AI: A Causal Problem" primarily focuses on the distinction between calculating Shapley values using observational versus interventional conditional distributions. This distinction corresponds to our discussions on the On-manifold Shapley value and the Causal Shapley value in our work. Note that the interventional Shapley value discussed in their paper differs slightly from the Causal Shapley value. They do not explicitly highlight direct versus indirect contributions and the role of symmetric versus asymmetric sampling in Causal Shapley value. However, the total attribution values for each feature remain consistent between the interventional Shapley value and the symmetric causal Shapley value. As a result, the comparisons in our work with the Causal Shapley value inherently encompass the interventional Shapley value approach outlined in the referenced paper. Nevertheless, it is one of the first to highlight the role of causality in feature attribution.

Regarding the paper "Quantifying Intrinsic Causal Contributions via Structure Preserving Interventions", we appreciate its relevance and foundational contributions to the field. Below, we outline the similarities and differences between our work and theirs, which we have addressed in the revised manuscript. Our work focuses on attributing the contributions of features to a model's output, similar to the Causal Shapley value and On-manifold Shapley value. In contrast, their work aims to attribute uncertainty (e.g., variance or Shannon entropy) in the target to its influencing features. This distinction makes the objectives of the two approaches fundamentally different. Both approaches emphasize isolating the contributions of features independently of others. They describe these as "intrinsic" contributions, while we frame them as "exogenous contributions." However, our approach uses value-based interventions (manipulating feature values directly), while theirs employs structure-preserving interventions aimed at measuring uncertainty reduction. From our view, their approach can be seen as focusing on feature interventions that assess changes in uncertainty metrics. Their definition of Shapley-based ICC does not account for the sequence of interventions (e.g., parent-to-child order), which we discuss as potentially causing causal inversion in Structural Causal Models (SCMs). Thank you for highlighting these two important related papers. We have added discussions of them in the revised manuscript.

For edge interventions in DAG-SHAP, the edges for the instance $x^*$ are denoted as $e_1^*, e_2^*, e_3^*$, and the marginal contributions of each edge in each arrangement are as follows:

Therefore, the attribution value for $x_1^*$ in DAG-SHAP is $(3/2 + 3/2 + 1/4) / 3 + (0 + 0 + 5/4) / 3 = 3/2$, and the attribution value for $x_2^*$ is $(1/2 + 1/2 + 1/2) / 3 = 1/2$. Similarly, since $x_1^* = 1$ influences $X_2$ and it directly influences $Y$ , it is clear that $x_1^* = 1$ is more important than $x_2^*=2$ as half of the value of $x_2^*=2$ is dependent on $x_1^*=1$. Therefore, the attribution value provided by symmetric node interventions is misleading, because it includes the contribution of $x_1^*$ in the marginal contribution of $X_2$ with an empty set, i.e., $\mathbb{E}[Y | \text{do}(X_2 = x_2^*)] - \mathbb{E}[Y | \text{do}(\emptyset)]=2$.

Structure-preserving interventions are based on the assumption that the distribution of exogenous variables for each feature is known, and then intervene on the exogenous variable of each feature accordingly. However, in reality, the distribution of exogenous variables is unknown. The authors use an additive structural causal model to fit the data generation process. By subtracting the distribution of the parent node from the distribution of the child node, they obtain the exogenous variable distribution for the child node. However, most real-world data cannot be represented by an additive structural causal model. For example, in this case $X_1 = \mathbf{x}_1$, where $\mathbf{x}_1$ is a random variable uniformly distributed on $[0, 1]$, representing the exogenous influence of $X_1$ and $X_2 = X_1 \cdot \mathbf{x}_2$, where $\mathbf{x}_2$ is another random variable uniformly distributed on $[0, 1]$. In the generation of the $X_2$, the exogenous variable $\mathbf{x}_2$ acts multiplicatively. Thus, fit the data using an additive causal model, we cannot get the right causal structure, so the attribution values calculated using this method would not be accurate. Consider a special case where when $x_1^* = 1$ and $x_2^* = 1$, the resulting exogenous variable for $x_2$ is 0. This implies that $x_2$ has no effect on the data generation process, which clearly contradicts the true data generation process. DAG-SHAP does not require calculating the distribution of exogenous variables, nor does it assume the influence of exogenous variables in the data generation process is linear. It only need the causal direction between features. We think this is a significant distinction.

Thank you very much for your time and valuable feedback. We truly appreciate your thoughtful comments. If you have any further questions or need additional clarification, please don't hesitate to reach out.

Thank you for your response. We completely agree that using examples to clarify the distinction between our paper and several key related works will help to highlight the novelty. To this end, we have used a few simple examples to explain why their methods may fail while DAG-SHAP remains reasonable attribution. We have discussed this in the revised paper, specifically in the remark on lines 249-256. Due to space limitations, the detailed comparison is provided in Appendix Section D, lines 938-1036. Below is our specific comparison.

Regarding do-Shapley(On Measuring Causal Contributions via do-interventions) and Causal Shapley(Causal Shapley Values: Exploiting Causal Knowledge to Explain Individual Predictions of Complex Models), both use node interventions, and their fundamental distinction lies in whether the goal is to explain the impact of features on the data generation process of $Y$ or on a given predictive model $f$. Therefore, the utilities in their definitions are $v(S) = \mathbb{E}\left[\mathbf{Y} \mid \text{do}(\mathbf{x}_S = x_S)\right]$ and $v(S) = \mathbb{E}\left[f(\mathbf{x}) \mid \text{do}(\mathbf{x}_S = x_S)\right]$, respectively. Shapley-ICC(Quantifying intrinsic causal contributions via structure preserving interventions) employs structure-preserving interventions to measure the contribution of nodes to the reduction of uncertainty in the generation of $Y$. Structure-preserving interventions involves separating the exogenous variables within a feature from the influences of parent nodes. Interventions are then applied to each node's exogenous variables, where the intervention is still node-based. A key prior for this method is that the distribution of the exogenous variables must be known. We think that it's important to highlight the difference lies in the techniques of node intervention, structure-preserving interventions, and the edge intervention proposed in our paper, rather than in the attribution targets of these works (since these targets themselves already differ). Thus, we simplify the comparision by assuming that the model $f$ has perfectly learned the data generation relationship, i.e., the function of model $f$ is consistent with the generation function of $Y$. This allows us to avoid considering the distinction between the feature inference targets of do-Shapley and Causal Shapley, treating both as node interventions. We also apply structure-preserving interventions in the context of explaining the model $f$'s output or the generation of $Y$, making the goals of all methods consistent and facilitating comparison. For node interventions, we consider two cases: symmetric sampling node interventions and asymmetric sampling node interventions. Although do-Shapley does not discuss sampling order, Causal Shapley does.

Asymmetric sampling node intervention. We use the following example to explain why it may fail. Let $X_1 = \mathbf{x}_1$, where $\mathbf{x}_1$ is a random variable uniformly distributed on $[0,1]$, representing the exogenous influence of $X_1$; $X_2 = X_1 \cdot \mathbf{x}_2$, where $\mathbf{x}_2$ is another random variable uniformly distributed on $[0,1]$, representing the exogenous influence of $X_2$. The generation of $Y$ follows $Y = X_1 \cdot X_2$. In summary, $X_1$ directly influences $Y$ and indirectly influences $Y$ through $X_2$. $X_2$ influences $Y$ with its own exogenous influence $\mathbf{x}_2$ and transfers the indirect influence of $X_1$. We aim to attribute values to each feature of a specific explained input $x^* = [x_1^*, x_2^*] = [1, 1]$ with respect to the baseline [0, 0].

For asymmetric sampling node interventions, the only valid sample permutation is $(x_1^*, x_2^*)$. The marginal contributions of $x_1^*$ and $x_2^*$ in the permutation $(x_1^*, x_2^*)$ are shown in the table below:

Thus, the attribution value assigned to $x_1^*$ by the asymmetric sampling node intervention is $1/2$, and the attribution value assigned to $x_2^*$ is also $1/2$.

For edge intervention, we denote the edge $X_1 \to X_2$ as $\mathbf{e}_1$, $X_1 \to Y$ as $\mathbf{e}_2$, and $X_2 \to Y$ as $\mathbf{e}_3$. We denote the edges of the instance $x^*$ as $e_1^*, e_2^*, e_3^*$. According to the definition of DAG-SHAP , there are three valid edge permutations: $(e_1^*, e_2^*, e_3^*)$, $(e_1^*, e_3^*, e_2^*)$, and $(e_2^*, e_1^*, e_3^*)$. The marginal contributions of each edge are as follows:

Thus, the attribution value assigned to $x_1^*$ by DAG-SHAP is $(0+0+1/2)/3 + (1/2+1+0)/3 = 2/3$,and the attribution value assigned to $x_2^*$ is $(1/2+0+1/2)/3 = 1/3$. As $x_1^*=1$ directly influences $Y$ and it also indirectly influence $Y$ through $X_2$, it is clear that $x_1^*=1$ is more important than $x_2^*=1$. Therefore, the asymmetric sampling node intervention provides unreasonable attribution, as it fails to account for the external contribution of $x_1^* = 1$.

Symmetric sampling node intervention.

We use $X_1=\mathbf{x}_1$, where $\mathbf{x}_1$ is a random variable uniformly distributed on $[0,1]$, representing the exogenous influence of $X_1$; $X_2=X_1 + \mathbf{x}_2$, where $\mathbf{x}_2$ is another random variable uniformly distributed on $[0,1]$, representing the exogenous influence of $X_2$. The generation of $Y$ follows $Y=max(X_1, X_2)$. We want to attribute a specific explained input $x^*=[x_1^*,x_2^*]=[1,2]$ with respect to the baseline set as [0,0].

For symmetric node intervention, both the permutations $(x_1^*,x_2^*)$ and $(x_2^*,x_1^*)$ are valid, and the marginal contributions of $x_1^*,x_2^*$ in the permutations $(x_1^*,x_2^*)$ and $(x_2^*,x_1^*)$ are shown in the following table:

Thus, the attribution value for $x_1^*$ is $(3/2+0)/2=3/4$, while the attribution value for $x_2^*$ is $(2+1/2)/2=5/4$.

Thank you for your timely and insightful response.

We think that feature attribution methods, which require DAGs as prior knowledge, are generally designed to support explanations for both the data generation process and a given predictive model from a technical perspective. For example, in [1], it is assumed that the data generation function cannot be explicitly obtained, and thus the data generation process is modeled by training a machine learning model that fits the $Y$ generation function. The utility $\mathbb{E}\left[\mathbf{Y} \mid \text{do}(\mathbf{x}_S = x_S)\right]$ is approximated by $\mathbb{E}\left[\hat{f}(\mathbf{x}) \mid \text{do}(\mathbf{x}_S = x_S)\right]$, where $\hat{f}(\mathbf{x})$ is a model specifically trained to fit $Y$ rather than a model optimized purely for prediction accuracy or similar metrics. However, the attribution calculation process in both cases does not differ significantly. In [2], the calculation of $\mathbb{E}\left[\mathbf{Y} \mid \text{do}(\mathbf{x}_S = x_S)\right]$ does not rely on an accessible model, but the authors also point out that if $Y$ is a deterministic function, their method reduces to measuring the contribution of features to the model $f$. Therefore, if the attribution method has accounted for the clear causal relationships between features, the primary difference between explaining the data generation process and the given predictive model lies in the distinction between the calculations of $\mathbb{E}\left[\mathbf{Y} \mid \text{do}(\mathbf{x}_S = x_S)\right]$ and $\mathbb{E}\left[f(\mathbf{x}) \mid \text{do}(\mathbf{x}_S = x_S)\right]$. However, if a feature attribution method does not rely on the causal graph structure between features, it cannot accurately determine the impact of removing certain features and their contribution to the label $Y$. This is because they are unable to compute $\text{do}(\mathbf{x}_S = x_S)$ according to the causal data generation process. Therefore, it cannot attribute from the perspective of data generation. This is why methods that support both types of explanations require an understanding of the causal relationships between features.

Once again, thank you for your response. If our clarification has not addressed your concern, please let us know, and we will clarify further.

[1]Sun Q, Xia H, Liu J. Data-Faithful Feature Attribution: Mitigating Unobservable Confounders via Instrumental Variables[C]//The Thirty-eighth Annual Conference on Neural Information Processing Systems. 2024.
[2]Jung Y, Kasiviswanathan S, Tian J, et al. On measuring causal contributions via do-interventions[C]//International Conference on Machine Learning. PMLR, 2022: 10476-10501.

Thank you for your time and valuable feedback, especially for helping us improve the proof of the properties satisfied by DAG-SHAP and for pointing out related papers we hadn't compared with.

As the discussion period is nearing its end, we want to check if our responses have addressed your concerns. Would you be willing to adjust the score based on the clarifications and additional comparisons provided?

Once again, thank you very much for your valuable insights and feedback.

Thank you for your response. Regarding how line 11 in Algorithm 1 is handled, we did not assume a structural causal model or use re-weighting methods. In fact, we employed the Mixture Density Network (MDN) to model the distribution of child node feature values after intervention on parent node feature values. We then sample from the trained model, which requires fewer assumptions about the generative relationships between features. As such, our DAG-SHAP method only needs to know the causal directions between features in the DAG. In the experimental section, specifically between lines 377-393, we explain the technique used in the practical computation: "We utilize a Mixture to predict the distribution of child vertices after intervention on parent vertices within the input features, consistent with the approach used in causal Shapley value (Heskes et al., 2020)." Once we have the estimated values of all features after the intervention subset of edges, we can input these into the explained model to estimate the utility, supporting the use of Monte Carlo methods for estimating attribution values for all edges. In future versions of the paper, we will add this specific technique to the description of Algorithm 1 for further clarification.

Thank you for reviewing our paper. As the discussion period is drawing to a close, we would like to confirm whether our responses have addressed your concerns. Would you be open to adjusting your score based on the clarifications and additional comparisons we've provided?

Thank you once again for your valuable insights and feedback.

We are very grateful to the reviewer for the insightful comments. To address the concerns, we provide detailed point-to-point responses as follows.

Response to Assumptions and Comparisons in DAG-SHAP. We agree with the reviewer's observation that there exist methods that do not rely on knowledge of the causal graph, such as MCI or UMFI. However, due to the "no free lunch" theorem, these methods lack certain properties inherent to our approach. For example, MCI measures feature contribution based on the maximum marginal contribution a feature can bring which cannot capture the right causal contributin. For the example in [1]: there is an example $f(x_1, x_2) = x_1$ with probability distribution for binary features $X1$ and $X2$ where the $p(x_1, x_2) = 1/2$ for $x_1 = x_2$. The conditional expectations for all subsets are $f_{\emptyset}(x) = \mathbb{E}[f(X_1, X_2)] = 1/2$, $f_{\{1\}}(x) = \mathbb{E}[f(X_1, X_2) | x_1] = x_1$, $f_{\{2\}}(x) = \mathbb{E}[f(X_1, X_2) | x_2] = x_2$ and $f_{\{1,2\}}(x) = f(x_1, x_2) = x_1$. In this case, feature $X_2$ is irrelevant since it does not influence the function $f(x_1, x_2)$, and any meaningful measure of feature contribution should reflect this irrelevance. Methods like MCI, which evaluate based on maximum marginal contribution, would give $x_2$ the attribution $f_{\{2\}}(x)-f_{\emptyset}(x)=1/2$ if $x_1=x_2=1$. Therefore, MCI cannot capture such dependencies. In fact, if $X_2$ is an intermediate node between $X_1$ and Y, MCI would yield the same result. Additionally, as stated in the properties provided in the MCI paper, MCI satisfies Super-efficiency and Sub-additivity but does not satisfy Efficiency and Additivity in the context of feature attribution. This indicates that the attribution values assigned by MCI to all features do not sum up to the total effect contributed by all features collaboratively. Furthermore, when a feature is involved in multiple attribution tasks, its attribution value is not equal to the sum of its attribution values in each task.

The key difference between UMFI and MCI lies in the way marginal contributions are computed. UMFI removes the dependencies of the feature being attributed from other features before calculating its marginal contribution. As a result, it requires a causal graph to identify these dependencies. This is because, without such a graph, determining the dependencies would not be possible. Additionally, since UMFI relies on an approximately optimally preprocessed feature set for its computations, it does not satisfy properties like efficiency and additivity, which are inherent to the original Shapley value.

Our method can be used for explaining data, and whether a feature attribution method is employed to explain the data or the model output depends on the user's objective[2]. When our method is used for data explanation, the model $f$ is trained to approximate the unknown data-generating function from reality, serving as a tool for interpreting the data generation process. In attributing feature contributions to the generation of data labels, our method incorporates edge-based interventions. This ensures properties such as causality, exogeneity, and the efficiency and additivity axioms we discussed earlier. These properties, which are crucial for robust feature attribution, are not satisfied by methods like MCI and UMFI. We have added the discussion in the revised paper.

Response to Discussions on Externality and Exogeneity. We move the definitions to the beginning of section 3 for clarity.

Thank you for the response, it has addressed some of my concerns. The provided example demonstrates a pitfall of MCI for causality-based feature importance. Although UMFI does not fail in this example (the dependency removal step enables the blood relation axiom), the authors correctly point out that the causal structure remains unknown without prior knowledge. So the requirement for prior knowledge on the causal graph is well motivated. I think that this is good discussion to include in the paper in order to clarify the need for a strong prior.

I still struggle a bit to see the merit of the axioms. For externality, the argument is that we also want to model indirect effects to the response. But if the causal graph is assumed to be known, why not just have a method that models direct effects, and if one wishes to study the compounded causal effects along a causal chain, just re-define the response along the chain? I agree with exogeneity as an axiom. In terms of causal interpretation, I don't see why efficiency and additivity are necessary or even desirable properties.

I still have conerns about causal interpretations under externality. Since the mean feature importance score does not distinguish external influences (it is rather the variance), interpreting the variance of scores is impractical for numerous reasons. What if the numerical scales of the variables are inherent different? What if the quality of data is hampered, for example due to small samples?

Thank you for your response. We are glad to have addressed some of your concerns. We think that there are situations where the mean feature importance scores can distinguish the effect of external influences while variance may not. For example, consider the following case: $X_1 = \mathbf{x}_1$, where $\mathbf{x}_1$ is a random variable uniformly distributed on $[0,100]$, representing the exogenous influence of $X_1$; $X_2 = X_1 + \mathbf{x}_2$, where $\mathbf{x}_2$ is another random variable uniformly distributed on $[0,100]$, representing the exogenous influence of $X_2$. The generation of $Y$ follows $Y = X_1 \cdot X_2$. $X_1$ directly influences $Y$ and indirectly influences $Y$ through $X_2$. $X_2$ influences $Y$ both with its own exogenous influence $\mathbf{x}_2$ and by transferring the indirect influence of $X_1$. We aim to attribute values to each feature of a given input with respect to the baseline [0, 0]. The $x_1^*$ and $x_2^*$ values of the input to be explained are both greater than zero, since we set the baseline to the smallest possible values, the feature contributions of the input will always be positive. In this case, the mean feature importance scores of feature $X_1$ can be used to distinguish the effect of external influences. Because the externality effect is positive, and the contribution from the part outside the externality effect is also positive, this results in a larger overall attribution value for methods that can accurately measure the externality effect. However, if the input data we are explaining is very similar, such as [85.0, 77], [85.1, 77.2], [84.9, 76.9], then their attribution values will be close. In such cases, analyzing externality through variance becomes unreasonable. Here, our method will show that the attribution values are large due to external influences, which distinguishes it from other methods. In our original experiment, the larger variance was a result of sampling a large amount of data from the entire distribution (some of the explained data feature values were greater than the baseline, while others were smaller), which made it possible to demonstrate the presence of externality through the variance. We believe that using a range threshold to detect externality effect can be feasible under certain conditions. First, we should normalize the range obtained from our method against the range from other baseline methods to address potential issues with numerical scale differences across attribution tasks. Second, we need to assess whether the data being explained is overly concentrated. If the data is too concentrated, the range may lose significance, as the SHAP value calculation from sampling introduces some error. Therefore, we think that whether to use range or mean feature scores should be decided based on the specific context. If you have any further thoughts, we would be very happy to continue the discussion with you.