On the Recoverability of Causal Relations from Temporally Aggregated I.I.D Data

We are very grateful for the constructive comments and the time devoted to our work.

[W1] Although this paper is mostly of theoretical interest, it would have been good to have more experiments. In the main text, the author(s) only consider simulated experiments on conditional independence tests and LiNGAM.

Due to the page limit, most of the pages are dedicated to presenting theoretical results, leading to only a limited number of experimental results included in the main text. The methods we have tested encompass PC, FCI, GES, LiNGAM, and ANM. The settings cover different aggregation levels and scenarios, including linear/non-linear scenarios, with and without prior knowledge, and the aggregation of both time-delay and instantaneous models.

We have included 5 experiments in our original submission:

• applied widely-used causal discovery methods to aggregation data(Appendix C)
• functional consistency(Appendix D)
• conditional independence consistency(Appendix E)
• the impact of the k value(Appendix F)
• preliminary solution to the aggregation problem(Appendix G)

[Q1] In your submission, you discuss the relationship with faithfulness and conditional independence consistency. However, it can be argued that faithfulness is generally an unrealistic assumption in practice. Have you consider weaker assumptions, such as Causal Minimality (see i.e. Assumption 3 in [1])?

We completely agree that 'faithfulness is generally an unrealistic assumption in practice.' In fact, in our paper, we only use the faithfulness assumption as a starting point in Remark 1. The remainder of that section is dedicated to discussing how violating the faithfulness assumption can achieve conditional independence consistency.

Thank you once again for the time you've dedicated to reviewing our paper. The discussion period will end soon, and we would appreciate the reviewers' feedback. Specifically, if the reviewers have further questions or concerns after reading the rebuttal, we hope to have the opportunity to address them.

We are very grateful for the constructive comments and the time devoted to our work.

[W1,Q2] I doubt that there is a technical flaw in the proof of Theorem 3, which is described in the Questions section.

In the formula derivation, $s_Y$ is not a function of $y_1, \ldots, y_k$. $s_Y$ is a value given at the beginning of the derivation: $p_{S_{Z}|S_{Y},S_{X}}(s_{Z}|s_{Y},s_{X})$, which is completely independent of $y_1, \ldots, y_k$. For the relationship $S_Y = \sum_{i=1}^k Y_i$, yes, it does hold. It is naturally guaranteed by the integrands: $p_{S_{Y},S_{X},Y_{1:k}}(s_{Y},s_{X},y_{1:k})$ (the third equation) and $p_{Y_{1:k}|S_{Y},S_{X}}(y_{1:k}|s_{Y},s_{X})$ (the fourth equation): when $\sum_{i=1}^k y_i \neq s_Y$, both $p_{S_{Y},S_{X},Y_{1:k}}(s_{Y},s_{X},y_{1:k})$ and $p_{Y_{1:k}|S_{Y},S_{X}}(y_{1:k}|s_{Y},s_{X})$ are equal to $0$.

[W2,Q3] The credibility of Corollary 1 is also in question, as it follows Theorem 3. This is mentioned in the Questions section.

Given Fig.2, how can $S_X$ independent with $Y_{1:k}$ given $S_Y$?

Without the causal faithfulness assumption, it is possible to have conditional independence that does not correspond to d-separation in the DAG. For examples of violations of faithfulness, you can refer to [1].

In what situation does the "if" statement hold?

Corollary 2 then informs the reader under what conditions the 'if' statement holds.

The logic of Section 4.2 is as follows:

Remark 1: Given the structure of the DAG as shown in Figure 2, chain and fork structures cannot have conditional independence consistency unless there are coincidental circumstances (violations of faithfulness) in the causal functions within the DAG.

Then, where should the violations of faithfulness occur to gain consistency?

Theorem 2 and Corollary 1: The violations of faithfulness should appear in $\{S_X \perp\perp Y_{1:k}\mid S_Y\}$ or $\{S_Z \perp \perp Y_{1:k}\mid S_Y\}$ to gain the consistency.

Regarding the causal functions, how can such conditional independence be established?

Corollary 2: If the causal function between X and Y or the one between Y and Z is linear, then the corresponding conditional independence holds.

[W3,Q4] While the name of Theorem 3 is "Necessary and Sufficient Condition for Conditional Independence Consistency,"...

You may be referring to Theorem 2. Sorry to cause vagueness. The full name of Theorem 2 should be "Necessary and Sufficient Condition for Conditional Independence Consistency of Chain and Fork Structure." We have modified the name to avoid misunderstanding in the revised version. The logic related to this is as follows:

• We consider three fundamental cases: chain, fork, and collider structures.
• Under typical assumptions of constraint-based causal discovery, collider structures naturally have conditional independence consistency, but chain and fork structures do not. (Remark 1)
• Then, under what conditions do chain and fork structures have conditional independence consistency? (See Necessary and Sufficient Condition Theorem 1, Sufficient Conditions Corollaries 1 and 2)

According to the definition of conditional independence consistency, chain and fork structures are supposed to satisfy $S_{X} \perp\perp S_{Z} \mid S_{Y}$(condition (i)), and this is equivalent to conditions (ii) and (iii).

The significance of conditions (ii) and (iii): Although conditions (ii) and (iii) are derived from condition (i) through some simple transformations, they are more specific requirements compared to condition (i). The integrands in (ii) and (iii) are always the multiplication of two parts, one part only related to components X and Y, and the other part only related to components Y and Z. This motivates us to conclude that only by meeting requirements for the substructure can we obtain consistency for the full structure. From this, we claim: partial linearity is sufficient.

[1] Weinberger, Naftali. "Faithfulness, coordination and causal coincidences." Erkenntnis 83.2 (2018): 113-133.

Thank you once again for the time you've dedicated to reviewing our paper. The discussion period will end soon, and we would appreciate the reviewers' feedback. Specifically, if the reviewers have further questions or concerns after reading the rebuttal, we hope to have the opportunity to address them.

We are very grateful for the constructive comments and the time devoted to our work.

[W1] ...I had a hard time following. I especially noted this in the description of the intuition behind Definition 1 and Theorem 1.

Sorry to cause vagueness. We modified Definition 1 and Theorem 1 and rewrite the explanation for Definition 1 in revised version.

Definition 1:

Functional Consistency means that there exists a function $\hat{f}$ such that the temporally aggregated data $\overline{X}$ is described by $\overline{X} = \hat{f}(\overline{X}, e)$, where $e$ is a noise vector with independent components that are only related to the independent noises encountered in the process, denoted as $e_{2:k+1}$.

Explanation:

This definition implies that if functional consistency holds, then the aggregated data can at least be represented as a Structural Causal Model (SCM) in vector form, and the source of independent noise aligns with the underlying process. Please note that we allow the generative mechanism $\hat{f}$ to differ from the underlying causal function $f$. However, even with this allowance, achieving functional consistency in the nonlinear case remains challenging, as we will demonstrate in this section.

Theorem 1:

Consider a function $f(X, e)$ that is differentiable with respect to $X$. Define the following:

Statement 1: The function $f$ is of the form $f(x, e) = Ax + f_2(e)$ for some function $f_2$.

Statement 2: For any positive integer $k$, there exists a function $\hat{f}$ such that the functional equation $\frac{\sum_{i=1}^k f(X_{i}, e_{i+1})}{g(k)} = \hat{f}(\overline{X}, e)$ holds for any $X_i$, $e_i$, and any normalization factor $g(k)$, where $e$ is related only to $e_i$ for $i = 2, \dots, k+1$.

Statement 1 is a necessary condition for Statement 2.

Theorem 2 introduces a whole new set of variables (alpha, beta, and gamma) that don't appear to ever be defined

The definitions of the notations alpha, beta, and gamma have been provided within the content of Theorem 2 in the original manuscript.

[W2] In Section 3.1, after equation 1, you suggest that g(k) can be any normalizing function, such as g(k)=1 or g(k)=k ... but no discussion (that I noticed) is ever had about how large k needs to be for the results of this paper to hold. Especially since in the experiments, the largest k I see is 50, which doesn't feel large enough to assume that g(k) -> 0.

We have corrected this in the revised version. We do need $\lim_{k \to \infty} g(k) = +\infty$ for the approximation part. However, when our analysis begins with the aggregation of the instantaneous model and considers any finite $k$, the choice of $g(k)$ does not matter at all.

For how large $k$ needs to be, please note that in our paper, we only require a large $k$ to diminish unaligned terms for approximation/alignment. We need to approximate the aggregation of the time-delay model as the aggregation of the instantaneous model. Once we start the analysis from an aligned model, the value of $k$ is no longer a concern. $k$ can be any finite integer larger or equal to 2. In the experiment in Appendix F, we discuss how large a $k$ value is sufficient for approximation. The experiments show that when $k = 30$, the results of the aggregation of the time-delay model and the aggregation of the instantaneous model are very similar, which is sufficient to make the approximation reasonable.

[W3] While the authors discuss the dangers of temporal aggregation at a high level in the intro, the only example actually provided in the intro (the effect of temperature on ice cream sales, aggregated daily) is actually one where temporal aggregation doesn't seem to pose a problem. While the effects become instantaneous at the aggregate level, it's an effect that's consistent with the dynamics at the de-aggregated level and one that seems reasonable for causal discovery methods to detect...

“While the effects become instantaneous at the aggregate level, it's an effect that's consistent with the dynamics at the de-aggregated level and one that seems reasonable for causal discovery methods to detect.” No. The aggregation of an instantaneous causal model still suffers from the distortions caused by aggregation.

All theoretical results in our paper are about the aggregation of instantaneous effects. The damage to causal discovery performance is due not only to time-delay causal models but also to the aggregation itself. The primary focus of our paper is to investigate the effects caused by aggregation itself, underscoring how it can potentially distort causal relationships.

[W4] The experiments are quite weak...but the lack of any remotely realistic data, data with more than 2-3 variables or across more than 2 time steps, and structures with any amount of complexity leaves the evaluation feeling very weak.

Due to the page limit, most of the pages are dedicated to presenting theoretical results, leading to only a limited number of experimental results included in the main text. The methods we have tested encompass PC, FCI, GES, LiNGAM, and ANM. The settings cover different aggregation levels and scenarios, including linear/non-linear scenarios, with and without prior knowledge, and the aggregation of both time-delay and instantaneous models.

We have included 5 experiments in our original submission:

• applied widely-used causal discovery methods to aggregation data(Appendix C)
• functional consistency(Appendix D)
• conditional independence consistency(Appendix E)
• the impact of the k value(Appendix F)
• preliminary solution to the aggregation problem(Appendix G)

"doesn't actually employ a specific causal discovery algorithm": Please see Appendix C: we applied widely-used causal discovery methods(PC, FCI, GES) to aggregation data with 4 variables.

"data with more than 2-3 variables": Please see Appendix C,G

"across more than 2 time steps": Please see Appendix D,F

In the revised version, we have rewritten the first paragraph in the experiment section of the main text to clearly outline the experiments included, making it easier for readers to understand what experiments were conducted.

For the statement 'the second set of experiments doesn't actually contain any aggregation.': If the data represents the aggregation of an instantaneous model, the experimental results for different aggregated levels k large or equal to 2 will not show much difference. Moreover, an aggregation level of k=2 still qualifies as aggregation.

[Q1] In Section 4, why are only chain-like, fork-like, and collider-like models considered? Given that the focus is on temporal aggregation, cycle structures are...

We agree that the cyclic model is important in discussions about aggregation, and the analysis in Section 3 includes the cyclic model. However, Section 4, which focuses on conditional independence consistency, does not discuss the cyclic model. This is because, to our knowledge, the methods for discovering cyclic models are mainly FCM-based (discussed in Section 3) or score-based methods.

Regarding your example, when structure is cyclic, there is no conditional independence involving $\overline{X}, \overline{Y}, \overline{Z}$. Aggregation generally makes conditional independence conditionally dependent; it does not change a dependent relationship into an independent one. Therefore, we believe that aggregation has no influence on conditional independence in your example.

Thank you once again for the time you've dedicated to reviewing our paper. The discussion period will end soon, and we would appreciate the reviewers' feedback. Specifically, if the reviewers have further questions or concerns after reading the rebuttal, we hope to have the opportunity to address them.

We are very grateful for the constructive comments and the time devoted to our work.

[W1,Q1] Section 3 is confusing...

Sorry to cause vagueness. We modified Definition 1 and Theorem 1 and rewrite the explanation for Definition 1 in revised version.

Definition 1:

Functional Consistency means that there exists a function $\hat{f}$ such that the temporally aggregated data $\overline{X}$ is described by $\overline{X} = \hat{f}(\overline{X}, e)$, where $e$ is a noise vector with independent components that are only related to the independent noises encountered in the process, denoted as $e_{2:k+1}$.

Explanation:

This definition implies that if functional consistency holds, then the aggregated data can at least be represented as a Structural Causal Model (SCM) in vector form, and the source of independent noise aligns with the underlying process. Please note that we allow the generative mechanism $\hat{f}$ to differ from the underlying causal function $f$. However, even with this allowance, achieving functional consistency in the nonlinear case remains challenging, as we will demonstrate in this section.

Theorem 1:

Consider a function $f(X, e)$ that is differentiable with respect to $X$. Define the following:

Statement 1: The function $f$ is of the form $f(x, e) = Ax + f_2(e)$ for some function $f_2$.

Statement 2: For any positive integer $k$, there exists a function $\hat{f}$ such that the functional equation $\frac{\sum_{i=1}^k f(X_{i}, e_{i+1})}{g(k)} = \hat{f}(\overline{X}, e)$ holds for any $X_i$, $e_i$, and any normalization factor $g(k)$, where $e$ is related only to $e_i$ for $i = 2, \dots, k+1$.

Statement 1 is a necessary condition for Statement 2.

[W2] ...what about ANM? methods that do not make any functional assumptions?

We included experiments on Additive Noise Models (ANM) and methods that do not impose any functional assumptions, such as PC with Kernel-based Conditional Independence (KCI), in our original submission. Please refer to Appendices C, D, and E for details. Figure 8 presents the experimental results for ANM. In our experiments with non-linear models, we consistently employed general score functions for score-based methods and the Kernel Conditional Independence test for constraint-based methods, which are designed to be effective for any non-linear functions.

[Q2] Why does the infinite K only consider the linear model?

Why not consider infinite k for nonlinear model?

An infinite k introduces several mathematical challenges in terms of assumption or definition. For instance, the convergence of infinitely aggregated data heavily depends on functions f and g. If f is linear and g(k) = k, then the aggregated data loses randomness due to the law of large numbers. We require $g(k) \sim O(\sqrt{k})$ to maintain a converged variance. Additionally, we must assume data centralization to avoid infinite expectations. If f is nonlinear,(a) an appropriate $g$ that fits $f$ is necessary to maintain a well-defined variance; (b) different variables in the aggregated data may require different $g$ functions; and (c) for some nonlinear $f$, the variance of aggregated data may never converge. In practical scenarios, it is unrealistic to expect that observed data will perfectly align with these requirements of suitable $g$ and centralization. Furthermore, even if we overcome these assumption and definition issues, analyzing the central limit of non-independent and non-identically distributed variables in nonlinear case remains a complex problem in probability theory.

In summary, an infinite k is unnecessary and impractical in real-world scenarios, and addressing convergence issues detracts from our work's fundamental question: "Is aggregation of causality recoverable?".

Why consider an infinite k for the linear model?

The main purpose is to alert the reader to a special issue caused by aggregation: it transforms any distribution into a Gaussian. This issue causes non-Gaussian based causal discovery methods to collapse, even when the functional form itself is not disrupted by aggregation.

[Q3] ...assumptions in Theorem 2?

Theorem 2 itself does not introduce additional assumptions. If you are referring to the assumptions mentioned in Remark 1, the causal Markov condition and faithfulness assumption are common in causal discovery. To validate the faithfulness assumption, please refer to [1]. In our paper, this means that, given the structure of the DAG as shown in Figure 2, chain and fork structures cannot have conditional independence consistency unless there are coincidental circumstances in the causal functions within the DAG. Remarks 2 and 3 further explain under which specific coincidental conditions the faithfulness assumption might be violated and how chain and fork structures could become conditionally independence consistent.

[1] Zhang, Jiji, and Peter Spirtes. "Detection of unfaithfulness and robust causal inference." Minds and Machines 18 (2008): 239-271.

Thank you once again for the time you've dedicated to reviewing our paper. The discussion period will end soon, and we would appreciate the reviewers' feedback. Specifically, if the reviewers have further questions or concerns after reading the rebuttal, we hope to have the opportunity to address them.

Thanks for your valuable question. We realized that defining the specific mathematical object to which this consistency property applies is crucial. Thus we modified the definitions of functional consistency and conditional independence consistency as below:

Functional Consistency:

Consider an underlying causal process generating temporally aggregated data. This process is said to exhibit functional consistency if there exists a non-trivial function $\hat{f}$, distinct from identity functions, such that for any realization of the states $X_{1:k}$, and the independent noises encountered in the process $e_{2:k+1}$, the temporally aggregated data $\overline{X}$ satisfies the equation $\overline{X} = \hat{f}(\overline{X}, e)$. Here, $e$ denotes a noise vector comprising independent components only depend on $e_{2:k+1}$.

Conditional independence consistency:

Consider an underlying causal process generating temporally aggregated data. This process is said to exhibit conditional independence consistency if the distribution of temporally aggregated data entails a conditional independence set that is consistent with the d-separation set in the summary graph entailed by the original process.

It will be incorporated in the future version. Thank you for improving our manuscript. We welcome any further suggestions and feedback you may have!

We are very grateful for the constructive comments and the time devoted to our work.

[W1] VAR(1) is typically accepted as a linear model. It is not clear if f() covers another model. The terminology seems inconsistent to the notation.

In Section 3.1, our model definition is similar to the definition of a Nonlinear Vector Autoregressive (NVAR) model as described in [1]. However, in our paper, the function f can be either linear or nonlinear. Therefore, we refer to it as a 'general VAR(1)' in Section 3.1. To clarify, we have added some text in the revised version to emphasize that the VAR discussed in our paper encompasses both linear and nonlinear VAR models.

[W2] The number of variables seems to be s but not clearly defined.

Your understanding is correct. We have added additional text in the revised version to clearly define the number of variables as s.

[W3] What "k is large" means is not clear.

In our paper, we present the sufficient and necessary conditions for two types of consistency in three steps:

• Define the original model as a time-delay causal model (VAR) and define its temporal aggregated version.
• Approximate the aggregation of the time-lag causal model as the aggregation of the instantaneous causal model (at the end of section 3.1 for functional consistency as you mentioned, and section 4.1 for conditional independence consistency).
• Present the sufficient and necessary conditions of consistency for the aggregation of the instantaneous model, for any finite positive k, and any choice of g(k).

Among these steps, we only apply the large k condition in the approximation step. Yes, from a mathematical perspective, "k is infinity" would imply a perfect approximation. However, we write "k is large" for the following reasons:

1. According to our experiments, a large but not extremely large(infinity) k is enough to make sure the approximation reasonable. Please see Figure 9 in Appendix F. When k=30, the results of aggregation of time-delay model and the one of instantaneous model become very similar.

2. An infinite k introduces several mathematical challenges in terms of assumption or definition. For instance, the convergence of infinitely aggregated data heavily depends on functions f and g. If f is linear and g(k) = k, then the aggregated data loses randomness due to the law of large numbers. We require $g(k) \sim O(\sqrt{k})$ to maintain a converged variance. Additionally, we must assume data centralization to avoid infinite expectations. If f is nonlinear,(a) an appropriate g that fits f is necessary to maintain a well-defined variance; (b) different variables in the aggregated data may require different g functions; and (c) for some nonlinear f, the variance of aggregated data may never converge. In practical scenarios, it is unrealistic to expect that observed data will perfectly align with these requirements of suitable g and centralization. Furthermore, even if we overcome these assumption and definition issues, analyzing the central limit of non-independent and non-identically distributed variables remains a complex problem in probability theory.

In summary, an infinite k is unnecessary and impractical in real-world scenarios, and addressing convergence issues detracts from our work's fundamental question: "Is aggregation of causality recoverable?". Therefore, we use "large k" in the paper. In the revised version, we have included references to experimental results when mentioning "large k" for additional clarity.

[W4,Q1] g(k) is defined as "any normalization function like g(k)=1". Then, how do you guarantee that $(X_1 -X_{k+1})/g(k)$ vanishes?

We have corrected this in the revised version. Our intent is to convey that in Step 3, when we begin with the aggregation of the instantaneous model and consider any finite k, the choice of g(k) does not matter at all.

[W5] Definition 1 uses undefined terms such as "imply," "compatible," and "some degree of inconsistency." Their definition is not very clear.

We've made it clear in the revised version and rewrite the explaination for Definition 1. Modified Definition 1: Functional Consistency means that there exists a function $\hat{f}$ such that the temporally aggregated data $\overline{X}$ is described by $\overline{X} = \hat{f}(\overline{X}, e)$, where $e$ is a noise vector with independent components that are only related to the independent noises encountered in the process, denoted as $e_{2:k+1}$.

[1] Morioka, Hiroshi, Hermanni Hälvä, and Aapo Hyvarinen. "Independent innovation analysis for nonlinear vector autoregressive process." International Conference on Artificial Intelligence and Statistics. PMLR, 2021.

Thank you once again for the time you dedicated to reviewing our paper. Please see our response above. As you probably know, the discussion period will end soon, and we hope for your feedback and the opportunity to respond to it. Could you please kindly let us know whether your comments were properly addressed by our response and whether you have other comments?

Best wishes,

Authors of #7390

There seems to be a significant gap between the description and the intended outcome. For instance, I find the expression $\overline{X}=\hat{f}(\overline{X},e)$ problematic, as the noise component can take any value, implying an identity $\hat{f}(\overline{X},e) = \overline{X}$. The issue may stem from an inadequate definition of underlying stochastic processes, and it appears challenging to rectify. Regrettably, I feel compelled to recommend rejection for this version.