A Conditional Independence Test in the Presence of Discretization

We sincerely thank the reviewer for the detailed and thorough review, which has helped us realize that the organization of our original submission lacked clarity and may have caused confusion. In response, we have revised our paper with the following key improvements:

1. Notation Table: We have simplified the notation to avoid unnecessary complexity. Additionally, we have included a complete table of notations in Appendix A and provided definitions for the key notations at the beginning of Section 3 to enhance readability and accessibility.

2. Algorithm Pseudocode: To enhance clarity, we now include the pseudocode of our core algorithm, DCT, in Appendix~D, offering a clearer understanding of its workflow and implementation.

3. Reorganization and Intuition: We have reorganized the sections on the bridge equation, calculation of covariance, and independence tests. In these sections, we have introduced new notations where necessary and made an effort to provide intuitive explanations alongside the formal mathematical definitions.

4. We formulate the preliminary of nodewise regression as a lemma to highlight the conclusion it provides direct link to the regression parameter $\beta$ with the covariance $\mathbf{\Sigma}$ and precision matrix $\mathbf{\Omega}$.

We hope these changes address the reviewer’s concerns and significantly improve the paper’s clarity and structure.

Thank you for the insightful discussion, which has helped us better understand your concerns. Out of respect for the reviewers, we only used GPT to check the grammar.

Indeed, the experiment included in the main results (Section 4 in the revised version) investigates only sparse DAGs, while additional experiments with denser graphs are provided in Appendix E.2. However, we would like to emphasize that, as discussed in the Appendix, the primary focus of DCT is on cases where conditional independence (CI) is incorrectly judged as conditional dependence due to discretization. With an increase in edges, such cases of true CI become rarer, diminishing any advantages that DCT might have compared to other tests. Therefore, denser graphs are not the primary focus of our paper.

To alleviate your concerns, given the tight timeline, we conducted some experiments on denser graphs. Here are the results where the number of nodes $p=10$ and there are in total $2p$ edges. We use $n=500$ samples:

We can see that the precision and SHD of DCT are still better than other baselines, while the recall is lower due to the fact that discretization tends to keep all nodes connected. The experiment of other settings investigating denser graphs is going on and we will let you know the result once it is finished.

Regardless, we sincerely thank the reviewer for the dedicated review and insightful questions, which made us realize the shortcomings in the readability of the paper in the first submission. We have made every effort to improve the clarity and quality of the paper in its revised version.

Q. The discretization model (function) is unclear. How many discretization levels are used in this formulation?

A: Thanks for your question. We kindly note that we claimed in line108-109: " The cardinality of the domain after discretization is at least 2 and smaller than infinity."

Q:  Providing a step-by-step explanation of CDT in a numerical example would enhance understanding of the proposed method.

A: We sincerely thank the reviewer for the valuable feedback. We have included the Pseudo code of DCT in Appendix~D in the revised version to enhance the clarity of the paper.

Q: The proposed method has improved the accuracy of the CI tests. However, another significant challenge with CI tests is the computational burden, particularly when there are many conditioning variables and a large sample size.

A: While our proposed test exhibits higher computational complexity compared to traditional methods like the Fisher's z-test and the Chi-square test, we believe the benefits justify this trade-off. The main computational challenge arises in calculating the variance of the null distribution as outlined in Theorem 3.8.

Here's a breakdown of the computational steps:

1. Compute Inner Products: For each $i$ from 1 to $n$, we compute the inner product $\mathbf{a}[k]^\top \text{vec}(\mathbf{B}^i)$.

   • Cost per $i$: This operation requires $O(p^2)$ computations because $p(p-1) \approx p^2$ for large $p$.

2. Sum the Squares: We then sum all the computed inner product values.

   • Total Cost: This step has a computational cost of $O(n)$.

The total computational cost amounts to $O(np^2)$.

At the same time, our current method is well-suited for parallelization, as the calculations for estimating the variance in both unconditional and conditional independence tests are parameter-free and can be efficiently distributed across multiple processors.

Q.  Does not the number of discretizing levels affect the accuracy (Type I and Type II error) of the DCT method?

Thank you for highlighting the confusion. We would like to take this opportunity to clarify our approach. As explained in Section 3.1.1 of the revised version ( lines 174–180 of the original submission), to unify both discretized pairs and mixed pairs, we treat every discretized variable as a binary variable. Consequently, the level of discretization does not affect the performance of DCT.

Q. Using practical examples for the application of the CDT in causal discovery can help to indicate the effectiveness of the proposed method in practical and real problems.

We kindly note that the real-world experiment is already included in Appendix D.4 of the revised version (Appendix C.4 of the original submission). In this experiment, we perform causal discovery on the Big Five dataset, where each variable represents the level of agreement to a specific question with discrete choices. This case directly aligns with the causal graph introduced in the paper. Specifically, the variable "I worry about things" is conditional independent with "I seldom feel blue" given " I often feel blue". However, given their discretized observations, traditional tests fail to capture the true CI relationships while DCT successfully identifies them.

Q. Var(X2| \tilde{X2}) will never be zero since \tilde{X2} is the discretized version of X2 (information loss).

Thanks for your great question. We have included more detail in our revised version to clarify this point. The basic idea is as follows: The conditional variance $Var(X_2| \tilde{X}_2$ can be zero if and only if the $X_2$ is determined exactly by $\tilde{X}_2$ without any randomness, which clearly doesn't hold true in the presence of discretization.

Q. According to Figures 5, 6, and 8, CDT outperforms other methods in F1 score and Precision but generally performs worse in Recall.

Thank you for your question. We would like to clarify that this phenomenon is already explained in lines 528–530 of the revised version (lines 526–529 in the original submission). In essence, other tests tend to infer conditional dependence in most cases due to the effects of discretization, resulting in nearly all nodes being connected.

To elaborate, recall is calculated as:

$\text{Recall} = \frac{\text{TP}}{\text{TP} + \text{FN}},$

where:

• True Positives (TP): The number of edges correctly identified by the algorithm as existing in the ground truth graph.
• False Negatives (FN): The number of edges in the ground truth graph that the algorithm failed to identify.

In scenarios where the algorithm connects most nodes (traditional tests due to discretization), such as in the extreme case of a fully connected graph, the number of False Negatives (FN) approaches zero. As a result, the recall approaches one, regardless of whether the inferred graph accurately reflects the true causal relationships.

Dear Reviewer WY1e,

Thanks for the time and effort you dedicated to reviewing our submission. We hope your concerns have been properly addressed, and if you have further comments, we hope for the opportunity to respond to them. Your feedback would be appreciated.

Best wishes,

Authors of submission #6671

Thank you to the authors for their clear and comprehensive responses. Most of my major questions have been addressed, and the paper has been revised effectively. Regarding my comment on applying the proposed method to causal discovery, I was referring to using real-world structures like ASIA, SACHS, or ALARM. Based on the responses and modifications, I have updated my score from 3 to 6.

We sincerely thank the reviewer for the comprehensive and insightful feedback, which highlighted areas in our original submission that lacked clarity and may have led to confusion. In response, we have made the following key revisions to improve the organization and readability of our paper:

1. Notation Table: We have included a complete table of notations in Appendix A and provided definitions for key terms at the beginning of Section 3 to improve accessibility and understanding. The notations now are consistent and simplified.

2. Algorithm Pseudocode: To further enhance clarity, we have added the pseudocode for our core algorithm, DCT, in Appendix D. This addition provides a more straightforward explanation of the algorithm’s workflow and implementation.

3. Reorganization and Intuition: We have reorganized the sections discussing the bridge equation, calculation of covariance, and independence tests. Where necessary, we have introduced new notations and supplemented them with intuitive explanations to complement the formal mathematical definitions.

These changes aim to address the concerns raised and improve the overall clarity of our paper.

Dear Reviewer nGxm,

Thank you for having taken the time to provide us with your valuable comments, which helped improve our paper. This is a gentle reminder that the discussion period is nearing its conclusion, we hope you have taken the time to consider our responses to your review. If you have any additional questions or concerns, please let us know so we can resolve them before the discussion period concludes. If you feel our responses have satisfactorily addressed your concerns, it would be greatly appreciated if you could raise your score to show that the existing concerns have been addressed.

Thank you!

The Authors

Q. The authors primarily discretized variables into binary categories, which could potentially lead to some efficiency loss

A. We are grateful for the reviewer’s insightful observation. You are totally right. As we discussed in Appendix~H, in the current framework, the estimation of hidden covariance $\sigma_{j_1,j_2}$ is accomplished by solving a single bridge equation. This equation, can be interpreted as looking for the suitable $\sigma_{j_1,j_2}$ to match the "region" in continuous domains, linking the covariance with discretized observations. However, there might be multiple bridge equations available while the current framework only allows the usage of one of them. This is, as discussed in the paper, by far the largest limitation. In the future, we will try to make efforts towards this problem to solve more bridge equations thus improving the sample efficiency.

Q. If we can only observe $\tilde{X}_2$, why is it necessary to infer the original conditional relationships?

A. Thanks for your great question and we would like to take this chance to highlight our motivation of this paper. Traditionally, when we talk about CI test for discrete variables, we take it for granted that those variables are inherently discrete (variables like gender). However, numerous discrete variables, are inherently continuous but only discretized observations available. Just like the real-world experiment we have done in this paper, the variables representing the agreement to a certain question is categorized into five levels. In this case, we care about the CI relations among the original continuous variables rather than their discretized observations. However, as illustrated in the causal graph, this discretization leads to the possible failure conclusion of traditional tests.

Q. Could the authors provide practical examples to clarify this necessity?

A: One simple example could be the real-world experiment we included in this paper. The variable "I worry about things" is conditional independent with "I seldom feel blue" given " I often feel blue". However, given their discretized observations, traditional tests fail to capture the true CI relationships while DCT successfully identifies them.

Q. Multiple definitions for $h_j$ are possible, such as substituting $E[X_j]$ with other quantities.

A: Thanks for your insightful question. You are totally right that there are multiple ways to define $h_j$. We choose the mean of $E \tilde{X}_j$ as the quantity just for its simplicity. Currently, there is no theoretical progress to systematically determine the optimal $h_j$. However, we believe it is doable empirically. One naive solution would be to test $h_j$ with different quantiles ($E \tilde{X}_j$, $2E \tilde{X}_j$, etc.), calculate the corresponding variance using Theorem~3.3 and choose the one with the smallest variance.

Q: The method is developed under the assumption of Gaussianity, which is quite limiting.

A: Thank you for your insightful feedback. We acknowledge that the assumption of multivariate Gaussian distributions can limit the generality of the proposed test. However, we would like to share a few points regarding its reasonability:

1. Challenges in Conditional Independence: Inferring the conditional independence of latent variables based on their discretized values is indeed a complex problem. The discretization drastically reduces available information. Without mild assumptions, establishing the statistics that reflect the real conditional independence and inferring its null distribution by only using those discretized values is particularly challenging and could even be overly ambitious. In this paper, we rely on the property that (1). Gaussian assumption allows us to obtain a consistent estimator of latent covariance based on discretized observations thanks to its parametric form. (2) Gaussian model has the property to infer conditional independence based on the covariance only.
2. Empirical Performance: Although the theory requires that the latent continuous variables follow a multivariate Gaussian distribution, we empirically validate the effectiveness of DCT even when the assumptions are violated. As demonstrated in Appendix E.1 of the revised version (Appendix C.1 of the original submission) and Figures 5 and 6 where we present causal discovery results for various distributions including linear uniform, linear student distribution, linear exponential, and nonlinear Gaussian distributions. From the experiment, DCT still shows superior performance than other baselines.
3. Popularity of Copula Model: The assumption of multivariate Gaussian, also called Gaussian copula model, is well-studied and widely accepted in the community. There is a substantial body of work demonstrating the effectiveness of the copula model in various scenarios [1] [2]. On the other hand, we hope this paper can inspire the community to propose more general and powerful solution to handle this obvious but overlooked spurious conditional dependence caused by discretization.

Q. Additionally, Equation (5) lacks precision, as a Taylor expansion includes a residual term.

Thanks for raising the looseness. You are totally right we omitted the second-order terms and more. We have clarified this point in our revised submission.

Q. The empirical results are limited for showing the generality of the developed method for non-gaussian settings.

Thanks for your question. We kindly point out that the empirical results validating the robustness of DCT in non-Gaussian settings are already listed in Appendix E.1 of the revised version (Appendix C.1 of the original submission) and Figures 5 and 6 where we present causal discovery results for various distributions including linear uniform, linear student distribution, linear exponential, and nonlinear Gaussian distributions. The experiments demonstrate the superiority of DCT even when assumptions are violated.

[1] Fan, J., Liu, H., Ning, Y., and Zou, H. High dimensional semiparametric latent graphical model for mixed data. Journal of the Royal Statistical Society Series B: Statistical Methodology, 79(2):405–421, 2017.

[2] Zhang A, Fang J, Hu W, et al. A latent Gaussian copula model for mixed data analysis in brain imaging genetics[J]. IEEE/ACM transactions on computational biology and bioinformatics, 2019, 18(4): 1350-1360.