The best of both worlds: Improved outcome prediction using causal structure learning

Given the paper’s significant shortcomings in empirical validation, clarity, and reproducibility, it is not suitable for acceptance in its current form. Below are detailed suggestions on how the authors could address the identified weaknesses:

For rebuttal, the authors should:

1. Improve Reproducibility (Experimental Setup): The authors should provide more comprehensive implementation details directly in the paper, not just in appendices or supplementary materials. For example, the hyperparameter tuning details, such as how the parameter κ in Equation (5) is selected and tuned, should be included in the Results section.

2. Enhance Comparisons to Baselines (Section 5): The authors should consider more advanced causal learning models and baseline prediction methods in Section 5.1. They should justify why CASTLE, MLP, and L2+ES are chosen as primary baselines and compare their method to more recent models. For instance, they should explore recent deep learning-based models, such as graph-based neural networks (GNNs), for causal discovery and models based on temporal attention for healthcare predictions.

3. Address Practical Challenges in Real-World Application (Section 5.5): The authors should address how their method would perform with incomplete, imbalanced, or noisy data—common issues in real-world healthcare. Furthermore, a detailed discussion of the computational cost and the resources required to run the proposed model in large-scale clinical settings should be added to the Scalability Analysis section. This would make the paper more practically relevant to real-world medical applications.

4. Conduct Ablation Studies (Section 5): To better understand the contribution of each component of the proposed method, the authors should perform ablation studies. For example, in Table 5, how does the model perform when the causal structure learning component is removed? These studies would clarify whether causal structure learning actually contributes significantly to outcome prediction.

5. Deepen Analysis of Results (Section 5): The Results section, especially Table 5 (Worcester heart attack study), should include more detailed discussions of why the method performs better than baselines. What specific features or causal relationships drive this improvement? A more nuanced interpretation of the results would strengthen the paper’s empirical contributions.

See Weaknesses.

The paper needs arguments and theorems to show theoretical soundness. Could you take this up?

One may get the impression that the authors are not familiar with causal theory more generally, outside of the soft acyclicity optimization ideas, and perhaps not even for that entire literature. Could you please add text to assure the readers that you have carefully considered alternative ways of proceeding and are offering your way as an improvement over previous techniques for this purpose?

The simulation section can be expanded to include a wider variety of nonlinear functions. One may get the impression that the simulations work just for these nonlinear functions and not for others. Could this be expanded?

• What does "shared representations of the data" mean?
• How the "visualisation of the learnt causal graph" is an improvement over the existing causal discovery techniques?
• Why is "outcome prediction" performed using causal graphs? It is known that causal graphs are not useful when it comes to prediction tasks: they are useful for causal inference.
• Where is the rho parameter in Equation 6?
• Why is there a task about classification? How should this prove that your technique generalize? This only shows that the learnt graphs are somewhat able to generalize, it tells nothing about your technique ability to generalize.
• Why the survival analysis case is discussed in light of association rather than causation if this paper is about causality?

In the abstract, the authors mentioned that “due to evolving conditions and treatment approaches, causal relationships between the variables change over time”, but in this work, why do the authors not give any solutions to solve this problem any more?

Eq.(2) is only a general latent representation of input data X. How can Eq.(2) restrict predicting Y using a non-linear function of its causal parents?

Eq.(3) only simply replaces the WX in Eq.(1) with a latent representation by neural networks, lots of work has used this idea, what are the novel contributions in the work?

In Eq.(5), the first two parts aims to learn a DAG, how can Eq.(5) focus on the causal variables or important ones for predictions?

In Eq.(5), how do the authors address the confounder problem?

In experiments, why do the authors not compare their algorithm with the latest out-of-sample generalisation with the widely used benchmard OOD datasets?

• Following Weakness point 1, why would including the prediction loss in equition (5) helps causal discovery? Can authors provide some intuitive explanation or theorems to explain the results?

• Will including outcome prediction components in model always helps causal discovery? Will including the outcome prediction component sometimes cause model to be biased to the prediction task, leading to worse causal graph learning result? If this is true, can authors show an ablation study of when outcome prediction component helps and when outcome prediction component doesn't help?

• It seems case 2 of simulation doesn't really represet a case with unmeasured confounder. In case 2 , authors claim that "the outcome is not dependent only on its causal parents" due to that $X^{non-pa(Y)}$ term. However, this is because authors still treat the weighted matrix W as the true causal graph. Actually, when authors generate the Y variable based on $X^{non-pa(Y)}$, the original $W$ is no longer represents the true underlying graph. Rather the weighted adjacency of the true causal graph should be updated to $W^{new}$ such that its entry corresponding to $X^{non-pa(Y)}, Y$ needs to be filled with $1$ rather than $0$. Still, $Y$ is generated by the observed variables in $X$, not some unobserved variables not provided to causal discovery. Therefore, I don't think this is a case of "unmeasured confounder", nor can it prove the proposed method's robustness to assumption violation

• In simulation setting case 1 and case 2, what is the distribution of noise vector Z? The settings of generating graphs and data are rather limited. Can authors includem more settings of functional type (such as linear model or other non-linear model in ANM), noise distribution, graph type, and edge density to demonstrate the proposed method can succeed not only in the limited cases presented in Section 5?

• In results of Section 5.2, can authors provide the standard error of the causal discovery evaluation metrics? Also, the proposed method seem to perform worse according to FDR and SHD, which contradicts the claim that the proposed method improves causal discovery, can authors explain that?

• In Section 5.2, can authors compare with more causal discovery methods (for example, some classific methods from the 3 classes reviewed in Section 2)?

• In Section 5.3, can authors provide intuition of why the proposed method is more efficient? In fact, the proposed method optimizes on both causal discovery loss and the prediction loss, but ends up with faster convergences, this could be counter intuitive. Is it possible to prove time complexity of proposed method follows smaller order compared to CASTLE?

• Minor points on paper writing: In Section 3, why would authors mention "linear structural equation model" while the simulation setting doesn't follow LSEM?