Causal Order: The Key to Leveraging Imperfect Experts in Causal Inference

>4.2 - Another question regarding this theorem: It seems that ... to resolve the key problem of this paper.

Response: The key difference is between local acyclicity for a triplet (assumption) and global acyclicity for the graph (empirical result). Based on our experiments with LLMs such as GPT-3.5 and 4, we find that they can obey the acyclicity constraint over 3 variables when specifically instructed to do so. Therefore, we assume that imperfect expert's output over 3 variables will not contain a cycle. Based on this, Prop 4.1 shows that error in predicting causal relationship between pairs of nodes is lower for the triplet prompt compared to the pairwise prompt. (For context, in the triplet method, all nodes are divided into all possible groups of three, and the expert (like LLMs) is specifically prompted to create a Directed Acyclic Graph which models the causal relationship between the nodes for each subgroup (refer Table A24 for the prompt template used for triplet subgraph)).

This result, however, does not ensure that the global graph has no cycles. This is what the triplet method achieves, through aggregation over triplets and further steps. Overall, since the triplet method ensures a higher quality prediction of the edge direction, we believe this leads to lesser cycle formations empirically as compared to pairwise.

My main concern is the lack of comparison with existing LLM-based methods. ... against existing baselines, such as those mentioned in Section 2?

Response:

We thank the reviewer for this point. Our work focused on tackling the shortcomings of the standard pairwise LLM-based approaches for causal discovery. We compare our method against the pairwise prompting strategy proposed by (Kiciman et al., 2022). For a fair comparison, we further propose stronger variants of the pairwise method based on additional context and advances in LLM prompting (All Directed Edges, One Hop Iteration and Chain of Thought, refer Table A5).

As an additional baseline reflecting the state-of-the-art in LLM-based methods, we have implemented two more methods from Jiralerspong et al., 2024 ("Efficient Causal Graph Discovery using Large Language Models") on 4 complex graphs used in our analysis: Covid, Alzheimers, Child and Asia. We evaluate the methods over two LLMs: GPT-3.5-Turbo and GPT-4. This paper presents an efficient breadth-first approach (BFS) to causal discovery using LLMs. The pipeline operates level-wise, starting by querying the LLM to identify all nodes independent of others in the graph. From these independent nodes, the LLM is queried to determine dependent nodes, adding edges accordingly. The dependent nodes are then added to a queue, and the process is repeated iteratively for each node in the queue until it is empty. For BFS+Stats method, the pearson corelation coefficient for the independent nodes to all the other nodes is added in the prompt as additional context for graph discovery.

As the results below show, BFS and BFS-Stats methods obtain lower accuracy than the Triplet method. In particular, except BFS with GPT-4, all configurations lead to cycles in Child and/or Covid datasets. SHD and Dtop for BFS and BFS-Stats methods (especially with GPT-3.5) are also higher than the Triplet method.

BFS (GPT-3.5-turbo)

BFS (GPT-4)

BFS + Statistics (GPT-3.5-Turbo)

BFS + Statistics (GPT-4)

For reference these are the results of triplet with GPT-3.5-turbo

We thank the reviewer for their insightful comments, we have tried our best to incorporate their suggestions and answer their queries.

Response to weaknesses:

>3.1 - While I agree with the intuition that identifying causal .... chance of errors still preserves the usefulness of the method.

Response: The key result of this section is that even with a Perfect Expert (e.g., human domain expert), the inferred graph using pairwise queries can be incorrect (Prop 3.2). Given that inferring edges using pairwise queries is the dominant method in the LLM-based discovery literature, we believe that this is result of significance.

>3.2 - This section should have provided evidence addressing the ..... this condition broadly to include everything that satisfies it seems impractical.

Response: Section 3.2 provides 3 key results showing the usefulness of causal order.

1. Prop 3.1 shows that causal order can be used to find a valid, unbiased backdoor set. For example, in Table A14, we compare the estimation error when using the minimal backdoor set obtained using the full graph, to the "maximal" backdoor sets obtained using the causal order. Across sample sizes from 250-10000 for the Asia dataset, we find the difference in estimation error between the two kinds of adjustment sets is minimal (see also Table 5). However, as Cinelli et al. (2022) argue, such a maximal set may have high variance. Note that we consider the causal order as output only because it is a structure that we can obtain accurately from an expert. To obtain both unbiased estimation and low variance, we can use the causal order as input to a data-based graph discovery algorithm (as described in pt 3 below) and then derive the optimal backdoor set using the obtained graph.

2. In addition to Prop 3.2, the key result of this section is that the causal order metric, $D_{top}$ is a more accurate measure of effect estimation error than SHD. This result assumes significance since SHD is a popular method for evaluating graph quality especially for LLM-based methods; we show that if the downstream task is effect inference, measuring $D_{top}$ is more suitable.

3. We also provide algorithms on how causal order can be used to improve accuracy of data-based graph discovery algorithms. Empirically, we show that the accuracy of incorporating (expert-predicted) causal order in discovery algorithms is higher than that of the discovery algorithms alone.

>4.1 - Should these be considered as contributions of the paper?

Response: Yes, our extensions of pairwise strategies can be considered as contributions of the paper. To evaluate the benefit of the Triplet method, we create stronger versions of the standard pairwise method that utilize additional graph context and the latest advances in LLM prompting. The objective is to make sure that we have explored possible extensions of the pairwise method as much as possible, before proposing the Triplet method. One of these methods, Pairwise (CoT), obtains significantly better results than the standard pairwise method used in the literature (see Table 3), but is still less accurate than the Triplet method.

>4.2 - The authors try to justify why triplet prompt is ... would become an ϵ′-expert where ϵ′>ϵ? If so, would this become a trade-off and how should we interpret this tradeoff?

Response: Yes, as we increase the number of nodes in the prompt, there is a tradeoff: Adding more nodes provides more context and thus is beneficial, but more nodes in the LLM's prompt can also lead to higher error (Levy et al., 2024) and higher computational cost. Therefore, we tackled this question empirically by comparing pairwise, triplet, and quadruplet-based prompts. As Table A9 shows, using a quadruplet prompt slightly increases accuracy but leads to a significant increase in the number of LLM calls. In contrast, the increase in accuracy (especially cycle avoidance) is substantial when moving from pairwise to the triplet method.

Given these considerations, we decided to go with the Triplet prompt, as it allows for adding more context with minimal increase in prompt complexity and total number of LLM calls. Note that future iterations of language models might be able to handle longer context better with more improvements, therefore the ϵ′ will vary with model size, architecture and data the model is trained on. Since we don't have the information about this, it will be difficult to model ϵ′ accurately. However, with the LLMs that we have tried (GPT-4, GPT-3.5, Phi-3 and LLama3), we do not see an increased error when using the triplet prompt compared to the pairwise prompt.

Thanks for engaging on the rebuttal. We clarify on the importance and usecases of causal order below.

W2 (regarding 3.2) - Could the authors answer my original question that "Proposition 3.2 offers a sufficient condition for backdoor adjustment, but applying this condition broadly to include everything that satisfies it seems impractical."?

Including variables appearing before treatment (in the causal order) is actually a widespread practice in biomedical and social science empirical studies. In these studies, such variables are called "pre-treatment variables" and a common practice is to condition on all of them. For this reason, we do not think that our proposal is impractical. The importance of Prop 3.2 is to show the utility of the causal order to identify such a commonly used adjustment set.

For example, refer to the Covariate selection chapter [1] by Sauer, Brookhart, Roy and Vanderwheele in a User Guide ("Developing a Protocol for Observational Comparative Effectiveness Research"). In the section on "Adjustment for all observed pre-treatment covariates", they mention the widely used propensity score adjustment and write, "The greatest importance is often placed on balancing all pretreatment covariates." They also add that while theoretically colliders can bias the result, "in practice, pretreatment colliders are likely rarer than ordinary confounding variables.".

Further, when unobserved confounding cannot be ruled out (as is the case with most observational studies), evidence is not clear on whether we should include all pre-treatment covariates or select a few, especially because the true graph may be unknown. "Strong arguments exist for error on the side of overadjustment (adjusting for instruments and colliders) rather than failing to adjust for measured confounders (underadjustment). Nevertheless, adjustments for instrumental variables have been found to amplify bias in practice". As the last sentence suggests, note that we are not claiming that adjusting for all pre-treatment variables (variables before treatment in causal order) is always the correct approach; but rather showing that it can be practical in many situations.

Theoretically, of course, improvements to this causal order criterion are possible. Vanderweele and Shpitser (2011) [2] cite the popular practice of using "all pre-treatment variables" and propose the Disjunctive Cause criterion as an improvement. This criterion states that if a pre-treatment variable causes the treatment, outcome, or both; then it should be included in the adjustment set. Note that this criterion---effectively including all pre-treatment ancestors of treatment and/or outcome---is quite close to the causal order-based criterion in our paper. Except for possibly conditioning on a collider in cases where there are unobserved variables in the graph (see Fig. 1 from [2]), additional variables in the causal order adjustment superset will not have a significant effect on the estimate.

[1] Sauer, Brockhart, Roy, Vanderweele. Chapter 7: Covariate Selection https://www.ncbi.nlm.nih.gov/books/NBK126194/ in Developing a Protocol for Observational Comparative Effectiveness Research: A User's Guide (2013).

[2] Vanderweele, Shpitser (2011). A new criterion for confounder selection. Biometrics. https://pmc.ncbi.nlm.nih.gov/articles/PMC3166439/

>1. The current method produces a causal order rather than a full causal graph .... would be valuable to assess the practicality and risks of this trade-off. Response:

This is a great question. Contrary to the intuition above, we find that the triplet order method obtains more accurate effect estimates than the pairwise method that outputs a graph. Below we show an analysis using the Survey dataset and five combinations of treatment and outcome. The backdoor set computed from the pairwise method's graph results in a higher error than the "maximal" backdoor set computed from the triplet method's order.

The result can also be understood analytically. In Proposition 3.3, we show that causal order($D_{top}$ metric) can be a suitable measure for the downstream error in causal effect estimation. Recall that from the Table 3 of main paper, for survey causal graph, the causal order obtained by pairwise prompt gets $D_{top}=3$ and triplet prompt gets $D_{top}=0$. This difference in $D_{top}$ directly impacts the estimated causal effects as shown in the table.

That said, the triplet method produces a causal order only because of the limitation of expert knowledge extraction through prompts. In practice, the triplet method can be combined with a data-based discovery algorithm (such as PC or CaMML) to obtain a causal graph and then compute the optimal backdoor set for causal effect inference. We use DoWhy libray for estimating the causal effects and linear regression as the estimator, and the sample size is 1000.

>2. In the triplet method, the authors employ a four-step process to ..... differences between the pairwise approach under this four-step method and the current approach.

Response: If we consider only the prompts provided to an LLM, then yes, pairwise approach can be considered as a special case of the triplet approach (with auxiliary variables being the null set). However, the inclusion of auxiliary variables is the key ingredient that makes the triplet method substantially more accurate. Specifically, the auxiliary variable provides context for determining the relationship between a variable pair; and enables querying the LLM multiple times for the same variable pair, leading to a more robust answer. Also, selective use of GPT-4, for resolving clashes ensures further robustness for node pairs where the model might face difficulty. With reference to the four steps of Triplet method, the second and the third step are the key for the accuracy of the Triplet method. In comparison, for the pairwise method, the third step is absent and the second step does not provide additional context.

As a result, even using smaller models such as Llama3 or Phi-3 with the Triplet method leads to better accuracy than the Pairwise method with GPT-4.

We thank the reviewer for their insightful comments, we have tried our best to incorporate their suggestions and answer their queries.

Response to weaknesses

>1. Typo error: Lines 333-334 seem to have missed a closing parenthesis in O(V^3).

Response: Thanks for pointing it out. We will fix it.

>2. Complexity considerations: According to the paper, the triplet-based method ... methods poses significant limitations for causal discovery.

Response: For larger graphs, we propose a $O(kV^2)$ variant of the triplet method below. Before that, however, we want to highlight the significant increase in both accuracy and efficiency that the triplet method provides compared to the pairwise method.

With the triplet method, our main aim was to develop a method that significantly improves the accuracy of LLM-based causal discovery. A second aim was to build a robust method such that even smaller LMs can be used and they provide better accuracy than the pairwise method. As a result, the triplet method leads to a higher accuracy and (cost and time) efficiency compared to the pairwise method, even though its theoretical complexity is $O(V^3)$. For a large graph with many nodes, using the triplet method with GPT-3.5 can obtain significantly higher accuracy than pairwise method using GPT-4, and costs significantly less: based on OpenAI's pricing if we want to query for a 100 node graph, pairwise (GPT-4) would cost $574 while the triplet method costs $55 (see Appendix E, line 1371). Further, for larger graphs, we can even run Triplet method with much smaller models such as Phi-3 (3.8B params) and Llama3 8B, leading to further efficiency gains in both wall clock time and costs (while obtaining better accuracy than pairwise with GPT-4, see Table 2).

That said, we believe modified approaches of triplet could be adopted for improved scalability. Rather than incorporating votes from all triplet subgraphs while deciding on edge direction between a pair of nodes, we can sample a fixed number of triplets per variable pair. So if number of triplets used per pair is k, then the overall time complexity becomes O(kV^2), where k can be constant. Identifying optimal ways of choosing the subset can be a future extension of the work.

> 3. The approach of using LLMs to simply determine ... understanding beyond general-purpose language models.

Response: We agree that using LLMs for causal discovery is challenging in real-world scenarios. However, many real-world graph discovery problems involve a combination of inferring novel and widely known relationships, and we believe that LLMs can save significant effort in extracting the known relationships.

Therefore, the focus of this work is to develop a robust method to extract causal relationships from LLMs, at least the ones that are known to a general-purpose language model. As can be seen from our experiments in highly domain-specific datasets such as Neuropathic and Covid-19, the triplet method substantially improves the accuracy of obtaining such causal relationships.

That said, for complex real-world scenarios, a combination of LLM and data-based algorithms is more suited. Therefore, we proposed two variants that combine LLMs with data-based algorithms such as PC and CaMML (for example, LLMs may be used for providing a prior on the known relationships, which may help algorithms to learn the remaining novel relationships). In a similar direction, a key benefit of the triplet prompt is that it can also provide a measure of LLM's uncertainty (for each variable pair $<A, B>$, fraction of triplets that predict A causes B, B causes A, or no relationship) that can be useful for weighting the LLM-based prior for data-based algorithms.

We thank the reviewer for their insightful comments, we have tried our best to incorporate their suggestions and answer their queries.

Reply to weaknesses:

> 1. It is hard to determine whether triplet prompting is effective for high-performance LLMs ..... LLMs in Table 2, to clarify this matter?

Response: We thank the reviewer for this point. We did not run Triplet with GPT-4 due to efficiency reasons. We have now run the experiments with GPT-4 as expert for orienting subgraphs and then re-using GPT-4 for resolving clashes during merging phase. Following are the results for graph discovery on Asia, Alzheimers and Child graphs. Upgrading to a superior model (GPT-4) leads to better results for all three graphs.

> 2. In the experimental design related to the downstream ..... experiment with a very small dataset as a showcase for a data scarcity regime? It would be very interesting.

Response: We thank the reviewer for their insightful question. As shown in Table 4 (main paper) and Table A3 (appendix), we analyzed the impact of observational dataset sizes on performance, ranging from data-scarce settings (250 instances) to data-rich ones (up to 10,000 instances) covering varying sample sizes (N=250, 500, 5000, 10000). Our results reveal that incorporating causal order using the triplet method consistently improves causal discovery, regardless of dataset size. These results are across graphs of varying sizes (4–5 nodes to 24+ nodes), including domain-specific contexts like healthcare.

Based on your suggestion, we also ran an experiment for even more data-scarce setting, N = 100. We observe that incorporating triplet prior has a stronger positive impact on graph discovery performance for data scarce settings. In particular, for the medium-size graphs, at N=100, $D_{top}$ for PC reduces from 6.3 to 2.3 for Child, and $D_{top}$ for CaMML reduces from 12.5 to 5, a significant improvement.

N=100

We plan to extend the results in paper to show how LLM priors help in data scarce settings for causal discovery.

We thank the reviewer for their insightful comments, we try our best to incorporate their suggestions and answer their queries. We will update our final version to reflect the same as well.

Response to weaknesses

> Despite a sound experimental analysis, there a few details missing, i.e., how many repetitions were done in Tables 1 and 2

Response: Each LLM based experiment (Table 2) was run three times, and the average of the final score was reported. Similarly, for the human-based graph construction (Table 1), for each dataset, three human annotators were asked to annotate the final graph and the aggregate of that was reported. Each annotator was randomly allotted a graph for both pairwise and triplet query strategies while ensuring no annotator got the same graph to query with both strategies. To get an estimate of the upper bound of human performance, for resolving tie-breaking conflicts in the triplet method, we used a ground truth-based oracle (proxy for a human domain expert). We will make sure to add this detail in the final version of the paper.

Response to questions

> Can you clarify the experimental setup in Table 4? .... search space inside the causal discovery method.

Response: We thank the reviewer for this point. We agree, LLM + < causal discovery > is a more appropriate title for our CamML based hybrid method, since we use LLM triplet's output to reduce the discovery algorithm’s search space for getting to the most optimal causal graph. However for PC hybrid approach, we use LLM triplet's output to identify the most optimal graph from the MEC obtained from PC, therefore < causal discovery > + LLM might be more suitable for this case (that said, we also propose an LLM+PC algorithm at the end of this response that may be interesting!). We will make updates in the paper to make this more clear.

> Can you describe in more details the causal effect component? .... are the counterfactuals estimated?

Response: Following Proposition 3.2, we use all the variables that precede the treatment variable in the estimated topological order as the adjustment set. This set qualifies as a valid backdoor adjustment set. Once the adjustment set is identified, the causal effect is estimated using the DoWhy library and linear regeression as the estimator. Appendix Table A14 presents an analysis comparing the causal effects estimated using this approach to those obtained by adjusting for the (minimal) backdoor set in the Asia dataset. The results show almost no differences in the estimated causal effects.

> Another ablation study that would be helpful is to ... improve this current causal discovery bottleneck?

Response: Focusing on scalability for larger graphs would be a great direction of research---thanks for suggesting this. Below we discuss how incorporating LLM triplet-based order in causal discovery algorithms can reduce the search space over graphs.

LLM Triplet + Score-based methods: In the paper, we presented how causal order from the Triplet method can be used to provide a level order prior for the CaMML algorithm. In general, score-based methods sample different graphs, evaluate a score function for each graph, and iteratively select the graph with the best score value. Causal order from LLMs can be used to significantly decrease this search space. For instance, we can pre-compute the "forbidden" edges that violate the causal order (as a hard constraint); consequently graphs with those edges are no longer explored by a score-based discovery algorithm.

For example, the GES score-based algorithm runs in two phases, forward and backward, and uses a score function to greedily search for the best graph. In the forward phase, it starts with an empty graph. Next, it adds a single edge to the graph among all possible edges that could be added that maximizes the score of the new graph. This process is repeated multiple times until no additional edges can be added. To reduce the search space, we propose our modification: Instead of scoring every possible edge addition, the algorithm can rule out those edges that violate the LLM Triplet order (this can be precomputed), thus helping reduce the search space at each iteration.

LLM Triplet + Constraint-based methods: Constraint-based methods such as the PC algorithm depend on conditional independence tests. The PC algorithm starts with complete undirected graph and then for every pair of nodes connected by an edge, it removes the edge if the two nodes are independent or conditionally independent given other subsets of nodes. In the second stage, edges are oriented. The main computational burden in the PC algorithm is the skeleton building. However, as the skeleton is undirected, it is unclear how causal order-based constraints can help in this step. Therefore, in this work, once the skeleton is obtained, we propose using the Triplet method to help orient the edges (Sec. 3 (line 282)).

(continued...)

We thank the reviewer for their insightful comments, we have tried our best to incorporate their suggestions and answer their queries.

Reply to weaknesses:

1. "In the broader context of causal discovery ..... causal discovery problems."

Response: We agree that in truly novel situations, $\epsilon$ error of LLMs in causal discovery can be high. As in previous work on LLM-based discovery, we instead focus our attention on known relationships and LLM's capabilities on extracting them. This can be practically useful; in real-world problems, building a full causal graph often involves a combination of novel and previously known relationships, and we hope that LLMs can save significant effort in extracting the known relationships.

For the genuinely challenging causal discovery problems, we think a combination of LLM and data-based algorithms is more suited. That is why we proposed two variants that combine LLMs with data-based algorithms such as PC and CaMML (for example, LLMs may be used for providing a prior on the known relationships, which may help algorithms to learn the remaining novel relationships). In a similar direction, a key benefit of the triplet prompt is that it can also provide a measure of LLM's uncertainty (for each variable pair $<A, B>$, fraction of triplets that predict A causes B, B causes A, or no relationship) that can be useful for weighting the LLM-based prior for data-based algorithms.

2. "The premise in this paper is that querying experts with questions ......this is interpreted only as a causal ordering."

Response: That's a great point. Existing LLM-based methods (Kiciman et al. 2023, Long et al. 2022, and others) used pairwise queries to infer an edge (which motivated our story), but we agree that it is not a necessary feature of a pairwise method. We will refine our introduction and Section 3 to reflect this.

**3. "There is a bit of extra context that I would have liked in the empirical studies:

1. can you contextualize the datasets and graphs that are studied further and discuss whether they are semi-synthetic, purely realistic, etc.?"**

Response: All graphs are real-world graphs constructed by human experts. The data is generated using different methods. We summarize the datasets below (also see Table A15).