GInX-Eval: Towards In-Distribution Evaluation of Graph Neural Networks Explanations

We thank Reviewer DCst for his/her insightful comments.

One major comment of the reviewer concerns the limited applicability of the method to white-box models. Now we propose a fine-tuning strategy that enables both to overcome the OOD problem and to be generalizable to any black-box models only accessible through API calls. See general comments to all reviewers.

Although the methodology is not novel, the whole purpose of this paper is to bring a new evaluation tool to the xAI and GNN community to avoid common mistakes and get meaningful insights into the true informative power of explainability methods as well as validate ground-truth explanations, something that is currently missing in the community!

This paper doesn’t claim to solve the OOD problem, it shows first the limitation of a popular evaluation metric in xAI that is not robust to the OOD problem as well as the consequences (observation1,2,3) and proposes a novel evaluation strategy that overcomes this.

1. “The highlighted explanation subgraph’s distribution is different from the training data.” Why is different and what’s the nature where this difference comes from?

We have added Figure 2 to illustrate the OOD problem.

2. Why did the explanations’ distribution shift to a better side but not a worse side? ... (section 1 paragraph 1)”.

Since explanations are out-of-distribution, there is no general rule beyond the observed faithfulness scores. The idea here is that the GNN is fooled and therefore has not the capacity to correctly assess the importance of the remaining edges. Figure 2 very well illustrates that explanations are closer to non-toxic molecules and will therefore be misclassified even though they are good explanations for toxic molecules (if we look at the GT or GinX-Eval results).

3. There are two removal strategies: “hard” and “soft” removal strategies. I wonder is there any difference between them toward the GNN output? If the outputs f(G_e_hard) and f(G_e_soft) are different, what’s the reason for that?

We refer the reviewer to the Appendix where he will find a more detailed analysis of the differences in the GNN output between hard and soft edge removal strategies. In Appendix A.3 we analyze the differences in the GNN output between hard and soft edge removal and in Appendix C.2 we also provide the results for the soft edge removal strategy and describe the similarities and differences with the results with hard selection.

4. There should be many hyper-parameters to tune ...

We have included the details of hyperparameters for GNNEXplainer and PGExplainer in the Appendix. We fully agree with the reviewer that PGExplainer is extremely sensitive to the choice of hyper-parameters and our final choice is the result of hyper-parameter tuning.

5. In Figure 1, what’s the random seed for the random baseline...

For each experiment (the random baseline included), we have run the experiments on five different seeds (0,1,2,3,4). The random baseline estimates the edge importance sampling from uniform distribution values between 0 and 1. The weight of the edge for the random explanation is therefore a random value between 0 and 1. To compute the AUC score, we simply compare this random edge importance assignment to the ground-truth one. We have added a sentence in section 4.1 to clarify the AUC computation. “The AUC score is computed between the explanatory weighted edge mask and the ground-truth edge mask with binary values in {0,1}.”

6. The GInX-Eval is computined via retraining, and finally evaluating the quality of the explanation of the original on the original pretrained GNN model. However, the GNN behavior... clarify.

As mentioned in the general comment to all reviewers, we now propose a fine-tuning strategy. When removing t% of the informative edges, f_b(fine-tuned on the reduced train dataset) will not find the important information in the reduced test set - therefore we expect f_b to make wrong predictions and the test accuracy to drop and the GinX score to increase.

We thank Reviewer qa24 for his/her insightful comments.

What is the difficulty of applying the idea of retraining to the evaluation of explanatory techniques for GNNs compared to those for CNNs?

Thanks for the question. Theoretically, GInX-Eval should be easily transferable to CNN models as well. However, the difficulty with CNNs comes from the nature of the inputs themselves, i.e. images. The challenges arise when removing pixels from images. The removal strategy of pixels when using GInX-Eval with CNNs has to be well justified. Two main challenges should be addressed:

• As mentioned by Hsieh et al. [1], assigning features with the baseline value might lead to a bias toward features that are far from the baseline. For instance, with images the baseline is black color. If we set the pixels to black in RGB images, this introduces a bias favoring bright pixels. Therefore, CNN fine-tuned on those modified images will often omit important dark objects.
• In addition, removing pixels by setting their value to zero (black pixel) might lead to the CNN confusion between no signal composing the black background and pertinent negative pixels, which are relevant values that are dismissed [2].

Therefore, applying GInX-Eval for CNN models requires defining an appropriate pixel removal strategy to overcome those two difficulties.

[1] Hsieh, Cheng-Yu, Chih-Kuan Yeh, Xuanqing Liu, Pradeep Ravikumar, Seungyeon Kim, Sanjiv Kumar, and Cho-Jui Hsieh. n.d. “Evaluations and Methods for Explanation through Robustness Analysis.” https://openreview.net/pdf?id=4dXmpCDGNp7.

[2] Dhurandhar, Amit, Pin-Yu Chen, Ronny Luss, Chun-Chen Tu, Paishun Ting, Karthikeyan Shanmugam, and Payel Das. 2018. “Explanations Based on the Missing: Towards Contrastive Explanations with Pertinent Negatives.” arXiv [cs.AI]. arXiv. https://proceedings.neurips.cc/paper_files/paper/2018/file/c5ff2543b53f4cc0ad3819a36752467b-Paper.pdf.

We have addressed the concerns of the reviewer in the updated version of the paper. As mentioned in the general comment for all reviewers, we have modified section 3 with more consistent mathematical notations and this also modified Equation 3. In 3.3.1, we have corrected the “top-k edges” and clarified that t is the fraction/ratio of edges. We have also completed the incomplete references when papers were published at conferences and added the year for all of them. Concerning the colors, we have chosen colorings to reflect the similarity of methods (green colors for gradient-based methods, warm colors for non-genative ones, and cold colors for generative ones).

As mentioned in the general comment to all reviewers, we now proposed a fine-tuning strategy rather than re-training from scratch the GNN model. Therefore, the GInX-Eval method can now be applied to any black-box models, including language models and CNNs.

We thank Reviewer EeHX for his/her insightful comments and address them one by one.

1. Certain aspects of the design are not intuitively clear. Specifically, the rationale behind Equation 4 is not well-explained. Elaborating on the underlying intuition would aid in understanding its relevance and function within the model.

As explained in comment 5. below, we have re-formulated the EdgeRank score to the HomophilicRank score and elaborated more on the meaning of this score in the new version of the paper.

2. The terms "hard selection" and "soft selection" are used without formal definitions. Providing precise mathematical formulas for these concepts would clarify their meaning and implementation in the context of the proposed method.

We have added a mathematical definition of the hard and soft edge selection functions using the notations of the paper to bring more clarity to the paper.

3. A major concern with GINX-EVAL is that it necessitates the re-training of the evaluated model. This process alters the original model, potentially leading to explanations that do not accurately reflect the model's decision-making process in its original state.

We have addressed this major issue by proposing instead a fine-tuning strategy that enables both to overcome the OOD problem and to have an evaluation method that can be widely used for black-box models.

4. The utility of edge ranking as a metric is questionable. It assumes that the importance of individual edges correlates directly with subgraph importance, an assumption that may not hold true in all cases. Further justification or alternative metrics should be considered.

The EdgeRank score does not evaluate the importance of individual edges but the importance of the first fraction $t\in[0.1,0.2,…0.9]$ of edges. We agree with the reviewer that the 10% most important edges might be correlated with the 10-20% edges, leading to an increase in the GInX score only after removing the full 20% edges. However, observing the size of ground-truth explanations for the 6 datasets (between 10 and 30%), we expect the sharpest drop in test accuracy in the first 3 removal steps (t=0.1, t=0.2, t=0.3). Therefore assigning more importance to these first steps makes sense. Now, it is true that within the 3 first steps, important edges might be correlated.

We follow the reviewer’s suggestion and re-formulate the EdgeRank score to the HomophilicRank score. The HomophilicRank score measures the capacity of a method to rank edges by their correct importance ordering while assigning similar importance weights to correlated edges. This score penalizes methods that for instance only discover a subset of important edges and do not account for the edges correlated to those. The HomophilicRank score favors methods that treat correlated edges on equal footing. It measures the capacity to uniformly assign importance to redundant information rather than only putting importance to a single representative edge and none to the correlated edges.

5. The range of GNN backbones tested is somewhat limited. Incorporating more diverse architectures, such as GCN, would provide a more comprehensive evaluation of the proposed method's effectiveness across different models.

We followed the reviewer’s suggestion and added the GCN model. Since the GCN does not take edge features into account we only run GInX-Eval with GCN on the four datasets: BA-2Motifs, BA-HouseGrid, MUTAG, and MNIST_BIN where edge features could be removed without or with small loss of information.