LLM Enhancers for GNNs: An Analysis from the Perspective of Causal Mechanism Identification

Thank you very much for your analysis and suggestions for revising our paper, and your support for our paper. We greatly appreciate your feedback. Below, we will address each of the raised concerns.

Methods And Evaluation Criteria: In the appendix, we provide a detailed description of the composition of our dataset. In fact, the semantic information in the Controlled Causal-Semantic Graph dataset is primarily reflected in the semantic richness of its node features. The relationships and labels in the graph are determined based on the entry categories and associations provided by Wikipedia, following a deterministic approach to ensure causality.

Experimental Designs Or Analyses： Our proposed method can be applied to any graph dataset capable of abstracting higher-order causal models, a point we will clearly mention in the paper. Achieving this currently requires synthetic graph datasets with controllable causal relationships, where node features must contain rich semantic information to demonstrate the effectiveness of LLMs. Due to the limited availability of such datasets, we built our own CCSG dataset. To validate the generalizability of our method, we incorporated node data from the ogbn-arxiv dataset and reconducted the node-level experiments, we also conduct the experiments using random data, with the results as follows:

Table 1: The results of node-level experiments, where $Z^{h}$ corresponds to the output variables $\Psi_1$,$\Psi_2$, and$\Psi_3$ of $h^{\text{node,1}}(\cdot)$, using OGBN-Arxiv data and random data as node features, are presented. These results are consistent with those in the paper. Bold values indicate the minimum $\mathcal{L}_{\text{II}}$.

Table 2: The results of node-level experiments, where $Z^{h}$ corresponds to the output variables $\Phi_1$ and $\Phi_2$ of $h^{\text{node,2}}(\cdot)$, using random data as node features, are presented. These results are consistent with those in the paper. Bold values indicate the minimum $\mathcal{L}_{\text{II}}$.

We will add these results to the paper as well.

Thank you very much for your analysis and suggestions for revising our paper, and your support for our paper. We greatly appreciate your feedback. Below, we will address each of the raised concerns.

Methods And Evaluation Criteria:

1. Thank you for your suggestions. We have incorporated data from the ogbn-arxiv dataset and conducted additional experiments. The results are as follows:

Table 1: The results of node-level experiments, where $Z^{h}$ corresponds to the output variables $\Psi_1$,$\Psi_2$, and$\Psi_3$ of $h^{\text{node,1}}(\cdot)$, using OGBN-Arxiv data and random data as node features, are presented. These results are consistent with those in the paper. Bold values indicate the minimum $\mathcal{L}_{\text{II}}$.

We also conducted experiments using random data, and the results are as follows:

Table 2: The results of node-level experiments, where $Z^{h}$ corresponds to the output variables $\Phi_1$ and $\Phi_2$ of $h^{\text{node,2}}(\cdot)$, using random data as node features, are presented. These results are consistent with those in the paper. Bold values indicate the minimum $\mathcal{L}_{\text{II}}$.

We will include these results in our paper.

2. We compared our results with TAPE, and the outcomes are as follows:

Table 1: The results of node-level experiments, where $Z^{h}$ corresponds to the output variables $\Psi_1$,$\Psi_2$, and$\Psi_3$ of $h^{\text{node,1}}(\cdot)$, using TAPE as baseline, are presented. The results show that, due to its complexity, the TAPE method aligns with later layers of the neural network compared to those reported in the paper. We will include the relevant analysis results in our paper. Bold values indicate the minimum $\mathcal{L}_{\text{II}}$.

Experimental Designs Or Analyses:

We have included the experimental results for this hyperparameter and will incorporate these results into our paper:

Table 1: Accuracy corresponding to different q values, using different numbers of prompts. Llama3 was used as the enhancer. The standard deviation of results over 10 trials.

We will also provide a detailed analysis of the hyperparameters. Thank you for your suggestion.

Thank you very much for your analysis and suggestions for revising our paper. We greatly appreciate your feedback. Below, we will address each of the raised concerns.

Methods And Evaluation Criteria:

Regarding the AT module, we conducted experimental analysis using several public datasets. To demonstrate the generalizability of the proposed analytical method, we first note that it can be applied to any graph dataset capable of representing higher-order causal models. Currently, validating this requires synthetic graph datasets with controllable causal relationships, where node features are rich in semantic information to effectively showcase the power of large language models. However, due to the limited availability of such datasets, we constructed our own CCSG dataset. To further establish the robustness of our approach, we expanded our evaluation by incorporating node data from the ogbn-arxiv dataset and re-executing the node-level experiments. Additionally, we conducted experiments using random data. The results are as follows:

Table 1: The results of node-level experiments, where $Z^{h}$ corresponds to the output variables $\Psi_1$,$\Psi_2$, and$\Psi_3$ of $h^{\text{node,1}}(\cdot)$, using OGBN-Arxiv data and random data as node features, are presented. These results are consistent with those in the paper. Bold values indicate the minimum $\mathcal{L}_{\text{II}}$.

Table 2: The results of node-level experiments, where $Z^{h}$ corresponds to the output variables $\Phi_1$ and $\Phi_2$ of $h^{\text{node,2}}(\cdot)$, using random data as node features, are presented. These results are consistent with those in the paper. Bold values indicate the minimum $\mathcal{L}_{\text{II}}$.

We will add these results to the paper as well.