ICDA: Interactive Causal Discovery through Large Language Model Agents

Thank you for your review! We are glad you found our direction "interesting" and paper "easy to follow". We address some of your concerns below.

The experimental setup is quite simple, comparing only three basic methods: random selection, direct LLM, and static confidence selection.

We feel our analysis is fairly thorough, as we investigate in detail the performance of the following algorithms:

• simple random edge selection
• selection by directly promting an LLM
• selection via direct LLM prompting + online updates to the graph via binary feedback
• selection via static confidence
• selection via confidence + global updates to the graph via binary feedback
• ICDA: selection via confidence + local updates to the graph via binary feedback

So in total we compare the performance of six different algorithms, teasing out the importance of each component used in the final method. Additionally, we note that LLMs have never been applied to this task before. As a result, there are no previous baselines relying on semantic metadata and binary edge feedback.

Additionally, the comparison should include the performance of different language models, not just one.

In Figure 6 page 9 we do plot the performance of multiple models ranging from 8B - 70B parameters across two model families. These results demonstrate that smaller models struggle on this task.

In the main experimental section, it would be better to include a table for quantitative results alongside the graphs.

We would be happy to include a table of results in an updated version of the paper!

The experimental setup is overly simplistic, and for a conference like ICLR, the complexity of the method, theoretical analysis, and experimental thoroughness are insufficient.

Do you mind expanding on what you feel is overly simplistic about our setting, method and analysis? We are studying LLMs in a novel applicaton to interactive causal discovery and have provided an analysis which ablates each component of our method (resulting in five separate baselines). This anslysis is conducted across 8 real world causal graphs. In addition we investigate several other factors including model size/model family, and the impact of memorization on the discovery process.

Thank you for your review! We appreciate you found our work "interesting" and our experiments "extensive and exhaustive". We address some of your comments below.

Originalty/Signifiance/Quality : this point harms the correctness of the claims of the paper and is my main concern : it seems like the iterative updates do not satisfy the framework of LLMs as optimizers... The LLM is simply used on a post-hoc manner after having queried edges and read their associated output ground-truth labels.

As you say we do not directly compute the F1 score of intermediate graphs but instead receive partial feedback on the correctness of individual edge labels. Importantly, this partial feedback does still give some information about the F1 score of the hypothesized graph. We choose this type of feedback motivated by the experimental setting: for example biologists may be seeking to determine the causal effect of a regulatory gene R on the expression of a downstream target gene T. In practice a perturbation (knockout) can be applied to R and the resulting effect on T observed. If this knockout on R induces a statistically significant difference in the expression of T we may conclude the presence of an edge between R and T. This result could then be passed to our method and used to prioritize future experimentation.

Additionally, we note the selection of edges is not pre-determined by an initial set of confidences. This is because, after recieving edge feedback, the LLM may update the confidences of adjacent edges, directly affecting the edge selection policy. In fact, one of our key findings is that the initial set of LLM confidences become better calibrated as the LLM receives experimental feedback. For example, the last plot in Figure 6 demonstrates the majority of graph improvement early-on comes from updates to edges made without any direct interventional feedback. Recall that edges updated due to direct interventional feedback are selected for intervention for having low-confidence scores. However, the opposite quickly becomes true as the majority of correct graph updates begin to come from useful interventional feedback as more edges are probed. This shows the updated edge confidence scores (updated using results for neighboring nodes) are the main drivers of performance later on in the discovery process, which suggests the edge confidence values are improving when compared to the static baseline (where confidences are not updated).

Can you elaborate on : a) how your algorithm satisfies the LLMs as optimizers framework? ; b) why using the F1 metric specifically as a loss?, c) why these specific updates on parents of intervened edges as the choice of local updates?, d) how exactly the intervention is performed, at least in experiments?

a) The LLM as optimizer framework is defined as having an objective $f$ to maximize and a sequence of trials $x_1,...,x_N$ with associated values $f(x_1),...,f(x_N)$. The LLM must then propose a new value $x_{N+1}$ maximizing $f$ using the previous $N$ trials.

In our setting, the objective $f$ as the F1 score is not directly or globally observable for the entire graph. Instead we only observe the correctness of particular edges selected for intervention by the LLM. So now each trial $x_i = \{e_{i,1},...,e_{i,I}\}$ is a set of edges selected for intervention and feedback is a set of binary labels $f(x_i) = \{l_{i,1},...,l_{i,I}\}$. As mentioned previously, we make this design choice motivated by what is often done in experimental practice. However, importantly the feedback given to the LLM can still be used to implicitly maximize the hypothesized graph F1 score.

b) We choose F1 score simply as a way of measuring the overlap of the our hypothesized graph with the ground truth causal graph. We could choose any other overlap measure equivalently (e.g. SHD, NHD)

c) To review, given binary feedback $l_{ij}$ for an edge $e_{ij}$ we update all adjacent edges $e_{ik}$ and $e_{lj}$ (so both parents and children). We chose this local strategy for two reasons:

• We found the LLM struggled to reason about combination effects (e.g. if we put multiple sets of edge feedback and multiple edges for update in-context at once the result is significantly worse)
• It scales better to larger graphs. Again because we do not need to put the whole graph, or large subsets, in-context.

d) Because we already have access to the ground-truth graphs our intervention procedure is very simple: simply look-up the correct edge label. In practice determining this label would be done experimentally e.g. in a gene regulatory setting by introducing knockouts on the parent and observing the resulting effect on the child.

There seems to be a confusion in time here… wrong Google Scholar citation?

Thank you for catching this!

Can you increase the font of Figure 2?

We plan to improve the readability of figures in an update to the paper.

Thank you for your review! We are glad you found our paper "interesting" and that some of our experiments "demonstrate the effectiveness of our approach". We address some of your comments below.

The setting may not be realistic, since it is challenging to directly obtain the ground-truth causal edge label in each intervention; In the proposed setting, line 144, it is assumed that one could directly obtain the ground-truth causal edge label in each intervention, which is not realistic; The setting significantly differs from the standard practice in the literature of experimental design [1,2,3];

To summarize, our 'edge intervention' formulation assumes the access to a noiseless operation that reveals presence/absence of a causal edge between two variables. This design is motivated by experimental settings where for example biologists may be seeking to determine the causal effect of a regulatory gene R on the expression of a downstream target gene T. In practice a perturbation (knockout) can be applied to R and the resulting effect on T observed. If this knockout on R induces a statistically significant difference in the expression of T we may conclude the presence of an edge between R and T. This can be extended to the multi-edge setting where now we examine the effect of either a single gene knockout or multiple gene knockouts simultaneously on target downstream variables. See [1] for an example of this procedure in neuronal stimuli.

Further, we note that previous literature focuses exclusively on methods utilizing numerical observational and interventional data for causal discovery. In contrast, our method (and LLMs for causality in general) rely purely on pre-training, semantic metadata and experimental feedback. However, we believe both data sources (numerical and semantic) are complementary. Combining both into a single system for interactive causal discovery is promising future work.

There is no guarantee that LLMs could provide valid results; It is widely shown that LLMs can not provide faithful causal results [4,5], while the proposed framework heavily rely on the results of LLMs; Similarly, the uncertainty provided by LLMs is not warranted;

The initial confidence estimates we get from the LLM seem to be unreliable only on some graphs but more reliable on others. For example, compare the performance of the static confidence baseline on the Covid and Asphyxia graphs. The Covid graph is relatively simple and likely highly represented in pre-training data. As a result, we speculate this is why initial confidence estimates are well-calibrated for this graph. However, the Asphyxia graph is much larger and likely less well-represented in pre-training data. As a result, the initial LLM confidences aren't much better than the random baseline.

However, one of our key findings is that the initial set of LLM confidences become better calibrated as the LLM receives experimental feedback. For example, the last plot in Figure 6 demonstrates the majority of graph improvement early-on comes from updates to edges made without any direct interventional feedback. Recall that edges updated due to direct interventional feedback are selected for intervention for having low-confidence scores. However, the opposite quickly becomes true as the majority of correct graph updates begin to come from useful interventional feedback as more edges are probed. This shows the updated edge confidence scores (updated using results for neighboring nodes) are the main drivers of performance later on in the discovery process, which suggests the edge confidence values are improving when compared to the static baseline.

Previous baselines on experimental design are neglected, for example, previous works on intervention selection or experimental design [1,2,3].

An apples-to-apples comparison with statistical methods is somewhat difficult. This is because statistical methods use numerical observational/interventional data whereas our method (and LLMs for causality in general) rely purely on pre-training, semantic metadata and experimental feedback. So the data sources each method utilizes are entirely separate and potentially complementary. We believe combining these two methods for the task of interactive discovery would be interesting future work.

Additionally, for this reason we instead strive to benchmark our complete method against various weaker but natural LLM based methods (e.g. random selection, direct prompting, static confidence). We believe this benchmarking is thorough and demonstrates the necessity of our method over simply, for example, prompting a model with the entire graph.

[1] Single neurons may encode simultaneous stimuli by switching between activity patterns, https://pubmed.ncbi.nlm.nih.gov/30006598/

Thank you for your review! We are glad the reviewer found our application novel and well-presented! We address some of your comments below:

Lack of comparison with statistical methods.

Statistical methods use numerical observational/interventional data whereas our method (and LLMs for causality in general) rely purely on pre-training, semantic metadata and experimental feedback. So the data sources each method utilizes are entirely separate and potentially complementary. We believe combining these two methods for the task of interactive discovery would be interesting future work.

Additionally, for this reason we instead strive to benchmark our complete method against various weaker but natural LLM based methods (e.g. random selection, direct prompting, static confidence). We believe this benchmarking is thorough and demonstrates the necessity of our method over simply, for example, prompting a model with the entire graph.

There is a lack of comprehensive results across models. I acknowledge the authors have presented results in Figure 6, but comparing different ICDA variants and random agents for smaller models would have been interesting.

We agree a more thorough investigation across models (beyond what is already done in Figure 6) would be interesting. However we instead chose to prioritize other ablations investigating the impact of different parts of the edge selection scheme and the potential impact of memorization.

Some of the figures done have confidence bound (last subplot for Fig 6). Am I missing something?

Thanks for pointing this out! We can fix this in an updated version.

What does "simplicity" mean on L138?

We mean to express that the graphs are simple i.e. no nodes are self-causal.

Some works suggest that LLMs confidence might be unreliable, I wonder what the intuition on much better results with ICDA is in comparison to random agents?

The initial confidence estimates we get from the LLM seem to be unreliable only on some graphs but more reliable on others. For example, compare the performance of the static confidence baseline on the Covid and Asphyxia graphs. The Covid graph is relatively simple and likely highly represented in pre-training data. As a result, we speculate this is why initial confidence estimates are well-calibrated for this graph. However, the Asphyxia graph is much larger and likely less well-represented in pre-training data. As a result, the initial LLM confidences aren't much better than the random baseline.

However, one of our key findings is that the initial set of LLM confidences become better calibrated as the LLM receives experimental feedback. For example, the last plot in Figure 6 demonstrates the majority of graph improvement early-on comes from updates to edges made without any direct interventional feedback. Recall that edges updated due to direct interventional feedback are selected for intervention for having low-confidence scores. However, the opposite quickly becomes true as the majority of correct graph updates begin to come from useful interventional feedback as more edges are probed. This shows the updated edge confidence scores (updated using results for neighboring nodes) are the main drivers of performance later on in the discovery process, which suggests the edge confidence values are improving when compared to the static baseline.

Thank you as well! As we mentioned above, statistical methods rely on numerical observational/interventional data for training. As a result, their performance is sensitive to the amount of numerical data available (which can vary significantly for real world problems). LLMs do rely on numerical data but instead of variable semantic metadata and pre-training. Because there is no overlap in the data used to make edge predictions it is not clear how to even go about making a fair comparison (e.g. how much numerical data should be provided to statistical methods?). Instead, the intent of our paper is to demonstrate that LLMs, as general purpose models of language, can be effectively used for interactive causal discovery provided the right agent/system design. However, we agree combining these statistical methods and LLMs would make for interesting future work.

Thank you for the reivew! We are glad the reviewer found our paper "cleanly written" and our experiments "extensive". We address some of your comments below:

The definition of edge intervention feels unrealistic. Could the authors please provide an example of a causal operation that reveals the edge without additional knowledge or assumptions about the graph structure? It seems to me that such data might be extremely costly to obtain and the operation might in some cases be equivalent to revealing the whole graph, thus making the described approach impractical.

To summarize, our 'edge intervention' formulation assumes the access to a noiseless operation that reveals presence/absence of a causal edge between two variables. This design is motivated by experimental settings where for example biologists may be seeking to determine the causal effect of a regulatory gene R on the expression of a downstream target gene T. In practice a perturbation (knockout) can be applied to R and the resulting effect on T observed. If this knockout on R induces a statistically significant difference in the expression of T we may conclude the presence of an edge between R and T. This can be extended to the multi-edge setting where now we examine the effect of either a single gene knockout or multiple gene knockouts simultaneously on target downstream variables. See [1] for an example of this procedure in neuronal stimuli.

It's additionally worth noting that our current method does allow for the incorporation of noisy edge feedback (where now the confidence score assigned to a feedback edge may be < 100). However we did not evaluate this setting in any of our experiments.

The paper lacks a discussion about the limitations of the proposed approach.

One clear limitation is that cannot make use of numerical observational/interventional data when updating our graph prediction. This is because classical statistical methods use numerical observational/interventional data whereas our method (and LLMs for causality in general) rely purely on pre-training, semantic metadata and experimental feedback. However, we believe both data sources (numerical and semantic) are complementary. Combining both into a single system for interactive causal discovery is promising future work.

The plots in Figures 2 and 4 are very small and hard to read.

We apologize for this! We plan to improve the readability of figures in an updated version of the paper.

[1] Single neurons may encode simultaneous stimuli by switching between activity patterns, https://pubmed.ncbi.nlm.nih.gov/30006598/