Data-faithful Feature Attribution: Mitigating Unobservable Confounders via Instrumental Variables

Thank you for your encouraging and insightful comments. Please find our response below.

Response to Weaknesses

• W1: We will follow your suggestions to enhance the readability.

• Minor comments: Fixed. Thanks!

Response to Questions

• Q1: In our paper, when addressing the impact of unobserved confounders, it is not necessary to identify what these confounders are. Instead, we only need to determine their existence and, if present, employ our method to mitigate their effect on feature attribution. For example, we can use potential instrumental variables for detection. When we intervene on an instrumental variable, we can then analyze whether the correlation between variable $\tilde{\boldsymbol{x}}$ and the target feature aligns with the change of the target variable caused by the instrumental variable. If the correlation is strong, but the change in the target variable caused by the instrumental variable is small, this suggests the presence of unobserved confounders which increases the correlation between $\tilde{\boldsymbol{x}}$ and the target feature [1].

• Q2: Feature attribution indeed involves explaining a model output $f(x)$ by assigning attribution scores to the input instance $x$. However, our focus is on data-faithful feature attribution, i.e., we are not trying to explain the output of a specific model but trying to explain the target feature $y$ through a model. For example, consider a medical scenario where a patient is interested in understanding the impact of all features on a disease, rather than just the prediction output of a particular diagnostic model. In data-faithful attribution, the trained model is used to estimate feature subset utilities because the real-world data generation equation is unknown. Thus, the model serves as a tool for data-faithful attribution. We propose a training method to reduce confounder influence and enhance data-faithful attribution. We will clarify them in the paper.

• Q3: Our experiments suggest that the model's accuracy might be affected, indicating a trade-off in most scenarios. When we train the model using features that have been re-estimated to remove the influence of confounders, the model may lose the correlation between the predicted output and the confounders, which also affects the target feature $y$. We compared the Mean Squared Error (MSE) of our model with a predictive model in the experiment and found our loss to be slightly higher. We note that our goal is not to maximize the predictive accuracy of the model but rather to ensure that the attribution results are more faithful to data. We will clarify it in the paper.

• Q4: In our experiments, the average change rates between IV-SHAP and IV-IG scores in both synthetic and real-world datasets are below 10%. SHAP and IG are inspired by the Shapley and Aumann-Shapley values [2], respectively, and share many common properties [3]. In contrast, methods like LIME and DeepLIFT follow different principles and produce different values. Thus, merely removing confounders is unlikely to resolve inconsistencies between LIME/DeepLIFT and IV-SHAP/IV-IG. Discrepancies among attribution methods also arise from how they handle correlations between input features. For example, on-manifold Shapley [4] uses conditional expectations to supplement unselected features, while causal Shapley [5] uses interventions. These methods can be combined with ours, but addressing confounders alone cannot resolve differences due to varying approaches to feature correlation. Our paper focuses on the impact of unobserved confounders, and we suggest choosing the appropriate method for handling feature correlations based on the specific context.

• Q5: The ranking of scores obtained from our methods and existing methods are different. We statistically analyze the proportion of attribution score ranking changes for IV-SHAP and IV-IG compared to SHAP and IG on Dataset A when $\rho=0.2$. The results are shown in the table below.

  Feature Deviation	0.125	0.250	0.375	0.500
  IV-SHAP	82.7%	87.1%	92.3%	94.2%
  IV-IG	58.4%	69.6%	77.1%	86.3%

  In the Griliches76 dataset experiments, 57.1% of the attribution rankings changed. Even when the rankings remain the same, changes in attribution scores can offer valuable insights into feature contributions. For example, if the attribution score for education decreases, even without a change in ranking, it could indicate that the return on education for income is lower than initially expected.

• Q6: The proposed feature attribution approach may be feasible for image and text classification. Images often encounter confounders like lighting. If a model is trained where most cat images are in strong lighting and dog images in weak lighting, the attribution for a cat image taken in weak lighting may be misleading. Identifying suitable instrumental variables to mitigate the confounders can make feature attribution more reliable and intuitive. This is an insightful extension, and we will explore it further.

References

[1] Baiocchi M, Cheng J, Small D S. Instrumental variable methods for causal inference[J]. Statistics in medicine, 2014.

[2] Tauman Y. The Aumann-Shapley prices: a survey[J]. The shapley value, 1988.

[3] Sundararajan M, Najmi A. The many Shapley values for model explanation[C]//(ICLR), 2020.

[4] Frye C, de Mijolla D, Begley T, et al. Shapley explainability on the data manifold[C]//(ICLR), 2020.

[5] Heskes T, Sijben E, Bucur I G, et al. Causal shapley values: Exploiting causal knowledge to explain individual predictions of complex models[J]. (NeurIPS), 2020.

Thank you so much for your insightful follow-up questions and comments.

• Q1: Yes, your understanding is correct.

• Q2: Yes, your understanding is correct. Our proposed approach can be used as a tool for analyzing phenomena, making it particularly suitable in medical and financial fields.

• Q6: Yes, data-faithful concept attribution is a suitable scenario for applying the approach of confounders, and we appreciate this idea. The example mentioned in the rebuttal can be effectively modeled by treating the concept of lighting conditions as an unobservable confounder.

  Besides, we also think that data-faithful feature attribution is meaningful in the presence of pixel-level confounders. In adversarial attacks, a well-known example is adding invisible pixel-level noise to a panda image, leading to the model misclassifying the image as a gibbon [1]. When we conduct feature attribution on the image, those noisy pixels appear important, though humans cannot see their significance. If we could train a model to eliminate the influence of such noise, the attribution would better match the judgment of a human on the panda image and the shape of the panda. We think that there might be similar noise confounders existing in image classification. If we remove these confounders, the feature attribution results of the model would be more data-faithful, which might imply better model robustness. Of course, the example of data-faithful attribution in the image we mentioned above has some differences from our paper's focus. Firstly, the method for removing pixel-level confounders in images might not use instrumental variables. Secondly, data-faithful feature attribution, as we mentioned here, aims to help humans judge the reliability of a model, as in image classification, where model performance is usually the main concern. The performance-oriented goal is different from the goal of our paper on understanding the relationship between input features and target feature values. We believe this is an interesting topic that deserves more detailed thought in terms of motivation and methodology.

We hope our responses have clarified your questions and helped improve your opinion of our work.

Reference

[1] Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and Harnessing Adversarial Examples. ICLR 2015.

Thank you for your encouraging and insightful comments. Please find our response below.

Response to Weaknesses

• W1: The implementation and computation complexity of the two-stage training process is similar to the model training in the model-faithful feature attribution. In the first stage, we need to train a DNN classifier $\hat{M_\phi}$ if $\tilde{\boldsymbol{x}}$ is discrete (or mixture density network $\hat{M_\phi}$ if $\tilde{\boldsymbol{x}}$ is continuous) to approximate the distribution of unconfounded $\tilde{\boldsymbol{x}}$. Thus, the implementation is easy and computation complexity does not require significant computation resources in the first stage. The second stage of model training is the same as the training of the predictive model in model-faithful feature attribution, except that the feature values of $\tilde{\boldsymbol{x}}$ are replaced with the re-estimated unconfounded values. The re-estimated unconfounded values are sampled from the first-stage trained model, the sampling has little influence on the implementation and computation complexity of the second-stage model training. Therefore, the two-stage training process does not limit the method's practical usability. We will clarify it in the paper.

• W2: We conduct the theoretical derivations based on the assumption that the influence of unobserved confounders on the target features is linear. We can give an accurate formula for the error in the linear assumption while other types of influence are difficult to decouple in theoretical analysis. However, our method can still work well in the practical scenario when the effect of the confounders is not linear. In the real datasets Griliches76 dataset and the Angrist and Krueger dataset we used in Section 5.2, the confounder of Ability has a non-linear impact on income. The experimental results still demonstrate significant effectiveness.

Response to Questions

• Q1: To identify suitable instrumental variables (IVs), we can first conduct a preliminary screening, such as LASSO regression, to assess the correlation between potential IVs and confounded features. Variables with non-zero coefficients may be correlated with $\tilde{\boldsymbol{x}}$ and are considered potential IVs. Next, we can use causal interventions to empirically verify these IVs, ensuring their effectiveness through statistical tests, such as checking if the F-statistic exceeds a threshold or using Hansen tests for validity. For a more in-depth study on IV identification, we recommend the seminal works [1][2][3]. Although identifying IVs is beyond this paper's scope, we will add a discussion to make this paper more self-contained and accessible to readers.

• Q2: Our method can effectively address the non-linear effects of unobserved confounders. In experiments with real datasets, such as Griliches76 and Angrist and Krueger, the confounder Ability has a non-linear impact on income. For example, while factors like IQ positively correlate with income, the relationship is not linear. Although we assume linear effects of confounders for the convenience of theoretical analysis, our method remains effective for non-linear effects. This is because we re-estimate $\tilde{\boldsymbol{x}}$ using instrumental variables in the first stage, allowing us to obtain unconfounded $\tilde{\boldsymbol{x}}$ for attribution, regardless of how unobserved confounders influence $Y$.

• Q3: In Section 5.1, we examined distribution shifts using synthetic datasets because real datasets do not allow control over external confounders and instrumental variables (IVs). In our synthetic datasets, we use $\rho$ to control the magnitude of unobserved confounders, $\tilde{\boldsymbol{x}}$ and the unobserved confounders are correlated. Consequently, the distribution of $\tilde{\boldsymbol{x}}$ also changes. Figures 1 and 2 demonstrate that our method's effectiveness increases with confounder influence. Notably, even at $\rho=0.2$, our method is effective. Additionally, experiments show that higher-quality IVs reduce confounder-induced errors more effectively. For example, we test the average error of IV-SHAP on Dataset A when $\rho=0.2$ with $\psi \sim (0,0.5), \psi \sim (0,1), \psi \sim (0,2)$ where $\psi$ controls the correlation of IVs with $\tilde{\boldsymbol{x}}$. The results are shown in the table below.

  Feature Deviation	0.125	0.250	0.375	0.500
  $\psi \sim (0,0.5)$	0.034	0.174	0.268	0.397
  $\psi \sim (0,1)$	0.018	0.057	0.166	0.280
  $\psi \sim (0,2)$	0.012	0.039	0.0127	0.204

• Q4: Our method scales well with larger datasets and higher-dimensional data. While larger datasets increase the training cost of the two-stage model, they do not complicate implementation. The training method costs twice as much as a typical predictive model, but remains manageable. Besides, the larger datasets increase the robustness and stability of the two-stage model. For higher-dimensional data, the computational demand increases, so we introduced approximation methods for IV-SHAP and IV-IG (see Appendix Section F), which we validated with the Spambase dataset. Extending our experiments to a 100-dimensional synthetic dataset, we found our method remained effective. Since data-faithful feature attribution is used mainly for tabular data to help understand feature relationships, and typically does not involve large dimensions. The average attribution ratio can still be used to evaluate scalability in the experimental section.

References

[1]Angrist, J. D., & Imbens, G. W. Identification and estimation of local average treatment effects. Econometrica, 1994.

[2]Baiocchi, M., Cheng, J., & Small, D. S. Instrumental variable methods for causal inference. Statistics in Medicine, 2014.

[3]Murray, M. P. Avoiding invalid instruments and coping with weak instruments. Journal of Economic Perspective, 2006.

Thank you for your encouraging and insightful comments. Please find our response below.

Response to Weaknesses

• W1: For the case of discrete $\tilde{\boldsymbol{x}}$, $\hat{M_\phi}$ is trained as a DNN classifier with softmax output, where the output element represents the probability of the corresponding category. We can enumerate the probability of each category and compute the weighted sum of all categories when we need to compute the expectation term condition on $\hat{M_\phi}\left(\tilde{\boldsymbol{x}} \mid \overline{\boldsymbol{x_t}}, \psi_t \right)$. The computation complexity is $O(c*n)$, where $c$ is the number of categories and $n$ is the number of sampled data tuples in model training. For the case of continuous $\tilde{\boldsymbol{x}}$, we implement a mixture density network to fit the distribution of $\hat{M_\phi}\left(\tilde{\boldsymbol{x}} \mid \overline{\boldsymbol{x_t}}, \psi_t\right)$. We use 20 random samples from the distribution output by the mixture density network to approximate the expectation, which has demonstrated great stability in our experiments. Thus, the computation complexity is $O(n)$. Both cases can be effectively computed given typical computational resources. We will add the details in the paper.

• W2: The effectiveness of IV-SHAP and IV-IG may be reduced when the three assumptions of instrumental variables are violated. However, the extent of this reduction depends on how severely the assumptions are violated. To mitigate the influence of unobservable confounders, the instrumental variables need to have a more direct influence on $\tilde{X}$ while having less correlation with other features. Additionally, the instrumental variables should have little or no direct influence on $Y$. The instrumental variables are still effective at reducing the impact of unobservable confounders if the assumptions are only mildly violated. For example, in our experiment with the Griliches76 dataset, the chosen instrumental variable, Parental Education, has a weak correlation with the unobservable confounder, Ability, as we analyzed in Appendix G.2. However, the influence of parental education on child education is much stronger, as demonstrated by the statistical characteristics of the real dataset in Appendix G.2. Despite this mild violation, the effectiveness of IV-SHAP and IV-IG remains significant. We will add a discussion in the paper/

• W3: The correlation of input features may affect the data-faithfulness of IV-SHAP and IV-IG. We can combine methods which deal with the correlated input features to the two-stage model to better capture these correlations. For example, on-manifold Shapley [1] can be used to account for feature correlations, while causal Shapley [2] can be applied if the causal structure of the input features is known. We note that our primary focus in this paper is on addressing the influence of unobservable confounders. Therefore, we chose the two most representative feature attribution methods, SHAP and IG, to formulate the problem. We will include this discussion in the paper.

Response to Questions

• Q1: Other feature attribution methods can be applied according to the demands. For example, when considering the correlation between input features, we can apply on-manifold Shapley or causal Shapley to our two-stage model if sufficient prior knowledge is available. Our methods focus on addressing unobservable confounders by re-estimating $\tilde{\boldsymbol{x}}$. Therefore, we can combine other feature attribution methods to handle the input features after re-estimating $\tilde{\boldsymbol{x}}$ with the two-stage model.

• Q2: We experimented with multiple baseline values by subtracting specific values from the features $\tilde{\boldsymbol{x}}$ and $\overline{\boldsymbol{x}}$ in both synthetic and real datasets. Lines 283-286 and 311-312 in Section 5 provide the details of experimental setting. The removed features are replaced with the average values from the dataset in IV-SHAP, which is consistent with the approach in SHAP.

Response to Minor comments

• M1: Fixed. Thanks!

• M2: We have omitted the $D$ following your suggestion.

• M3: $h(u_i)$ is the weighted gradients of a deep neural network, which is continuously differentiable and bounded in our setting. We will clarify it in the paper.

References

[1]Frye C, de Mijolla D, Begley T, et al. Shapley explainability on the data manifold[C]. International Conference on Learning Representations (ICLR).

[2]Heskes T, Sijben E, Bucur I G, et al. Causal Shapley values: Exploiting causal knowledge to explain individual predictions of complex models[J]. Advances in neural information processing systems (NeurIPS).

Thank you very much for your valuable comments. We appreciate your emphasis on the assumptions of instrumental variables and correlations of input features, which we agree are crucial factors for the effectiveness of IV-SHAP and IV-IG. We will make sure to highlight these points in the final version.