Data-Driven Selection of Instrumental Variables for Additive Nonlinear, Constant Effects Models

We appreciate your inspiring positive feedback and suggestions. Please see below for our responses to your specific comments.

W1. The article focuses on the constant effects model. However, this is not a weakness per se, nor do I believe it constitutes a reason for rejection.

A1: we would like to mention that linear models are common in the social sciences and ought to be more common in economics and elsewhere [Bollen (1989); Angrist & Evans (1996); Acemoglu et al., (2001); Spirtes et al., (2000)]. Furthermore, a series of articles have focused on studying the Additive Linear, Constant Effects (ALICE) model, including works by [Bowden et al. (2015); Kang et al. (2016); Silva & Shimizu (2017); Guo et al. (2018); Windmeijer et al. (2021)]. Unlike the ALICE model, our approach extends to a more general framework—Additive Nonlinear, Constant Effects (ANICE) models. In other words, our work explores a more challenging scenario, where $g(\cdot)$, $f(\cdot)$, and $\varphi_*(\cdot)$ may be non-linear functions.

S1. I suggest that the authors discuss the research approach for the additive nonlinear, non-constant effects model, as this would make the article more comprehensive.

A1: This is a great point! We have examined the applicability of the CAT condition in scenarios with non-constant causal effects and fully additive relationships among variables. Our results suggest that the CAT condition can still hold under these settings. Specifically, we consider two data-generation mechanisms illustrated in Figure 2(a) and 2(b) of the manuscript:

(a) Valid IV set {$Z_1, Z_2$}:

U = \varepsilon_U, \quad Z_1 = \varepsilon_{Z_1}, \quad Z_2 = \varepsilon_{Z_2}, \quad X = {Z_1}^2 + {Z_2}^2 + U + \varepsilon_X, \quad Y = X^2 + U^3 + \varepsilon_Y, $$where the noise terms $\varepsilon_U$, $\varepsilon_{Z_1}$, $\varepsilon_{Z_2}$, $\varepsilon_X$, $\varepsilon_Y$ are independent. Following the kernel-based or moments-based IV estimators, we have $\hat{f} _ 1(X) = \hat{f} _ 2(X) = f(X) = X^{2}$. Consequently,

\mathcal{A} _ {X \to Y \parallel Z_1} = U^3 + \varepsilon_Y, $$which is independent of $Z_2$. Likewise, for $Z_2$,

\mathcal{A}_{X \to Y \parallel Z_2} = U^3 + \varepsilon_Y, $$which is independent of $Z_1$. These imply that \{$X, Y|| \{ Z_1, Z_2 \}$ \} satisfies the CAT condition. (b) **Invalid IV set \{$Z_1, Z_2$\}:** Compared to (a), the generation mechanism for $Y$ changes to

Y = X^2 + Z_2 + U^3 + \varepsilon_Y.

According to the IV formula, $\hat{f} _ 1(X) = f(X) = X^{2}$. Hence,

\mathcal{A}_{X \to Y \parallel Z_1} = U^3 + Z_2 + \varepsilon_Y, $$which depends on $Z_2$. Therefore, {$X, Y|| \{ Z_1, Z_2 \}$ } violates the CAT condition.

We will incorporate the above discussion into the main text.

Q1: Is it possible to validate the algebraic equation condition (Assumption 1)? Additionally, the distinct causal effect biases (Assumption 2) seem to be verifiable.

A1: Firstly, because the probability density of noise terms cannot be fully determined from observational data, Assumption 1 cannot be directly validated. In practice, violating Assumption 1 imposes an extremely strict condition on an invalid IV set. Notably, one may not need to explicitly verify Assumption 1 or 2. Once {$X, Y|| \{ Z_1, Z_2 \}$ } violates the CAT condition, it indicates that {$Z_i, Z_j$} is an invalid IV set. In our work, we include Assumption 1 to show the necessity and sufficiency of the CAT condition.

Next, you are correct that Assumption 2 can be validated using observational data. While it holds for most invalid IV sets, it does not guarantee soundness, making it weaker than Assumption 1. As shown in Example 2 (line 278), this also highlights why Assumption 1 is necessary.

We appreciate your comments and suggestions, and we hope the following response addresses your concerns.

W1. The linear assumption from treatment X to effect Y is still strong...Under this assumptions, actually the core condition (CAT condition) is a direct consequence of existing conditions based on generalized independence noise (GIN) condition.

A1. Regarding the linear assumption, we would like to mention that linear models are common in the social sciences and ought to be more common in economics and elsewhere [Bollen (1989); Angrist & Evans (1996); Acemoglu et al., (2001); Spirtes et al., (2000)]. Furthermore, a series of articles have focused on studying the Additive Linear, Constant Effects (ALICE) model, including works by [Bowden et al. (2015); Kang et al. (2016); Silva & Shimizu (2017); Guo et al. (2018); Windmeijer et al. (2021)]. Unlike the ALICE model, our approach extends to a more general framework—Additive Nonlinear, Constant Effects (ANICE) models. In other words, our work explores a more challenging scenario, where $g(\cdot)$, $f(\cdot)$, and $\varphi_*(\cdot)$ may be non-linear functions.

Regarding the GIN condition and CAT condition, both the proposed CAT and GIN conditions use auxiliary variables to test independence among variables. However, their strategies differ: GIN’s basic approach is that, given a reference variable, it tests the independence between the auxiliary variable and that reference. In contrast, the CAT condition is similar to a “cross-test”: given a reference IV $Z_i$, it tests the independence between the auxiliary variable and another candidate IV $Z_j$. Moreover, the GIN condition is designed to identify the causal structure of latent variables within a linear non-Gaussian model, whereas the CAT condition specifically evaluates the validity of IV sets within the ANICE model.

W2. To testify the validity of one (truly valid) IV, it seems that at least one another truly valid IV is needed. Then when there is only one truly valid IV, can this algorithm still correctly identify it?

A2. You are correct: in order to test the validity of one IV, at least one other truly valid IV is needed. Notably, we do not need to know in advance whether that other IV is truly valid. If K=1, our method will fail, as the CAT condition relies on "cross test" to exclude invalid IV sets. In such cases, one can use single-IV methods, such as those proposed by Xie et al. (2022), Burauel (2023), and Guo et al. (2024).

Q1. ...Does that then yield weaker identifiability (e.g., the exclusion restriction condition being untestable)? Except for this, could the authors please discuss more on how these two works connect to each other, e.g., from the technical side?

A1: Yes, we would like to mention that, if at least two valid IVs exist in the system, our method offers a key advantage in identifying IVs violating the exclusion restriction assumption—a capability Guo’s method lacks. Generally, although both proposed conditions use auxiliary variables to test independence, Guo et al. (2024) focus on determining whether a single variable is a valid IV, whereas our approach validates an entire IV set. Roughly speaking, given a reference IV $Z_i$，Guo et al. (2024) test the independence between the auxiliary variable and $Z_i$ itself. By contrast, the CAT condition is more akin to a “cross-test”: given a reference IV $Z_i$, it tests the independence between the auxiliary variable and a different candidate IV $Z_j$. It is precisely the information provided by the “cross-test” that gives the CAT condition a broader capacity to test the exclusion restriction assumption.

Thank you for acknowledging the significance of our theoretical contributions and the novel of the algorithm. We hope the following response addresses your concerns.

W1. The Auxiliary Variable...using Eq.(5).

A1. Yes, the auxiliary variable can be viewed as a pseudo-residual. We would like to emphasize that identifying valid IVs is not always straightforward due to unmeasured confounders, often requiring substantial domain knowledge [Pearl, 2009; Imbens & Rubin, 2015]. This highlights the importance of a data-driven approach for testing IV validity. To the best of our knowledge, the independence property involving such an auxiliary variable and a reference IV has not been previously recognized as a criterion for assessing the validity of an IV set.

W2. the title of the paper

A2. Our paper’s title was based on prior work addressing IV set selection [Guo et al., 2018; Windmeijer et al., 2021; Silva & Shimizu, 2017]. To avoid potential misunderstanding, we will update the title to “Data-Driven Selection of Instrumental Variable Sets for Additive Nonlinear, Constant-Effects Models.” Thanks to you–hope it is clear.

W3. The ANICE model...which may limit the applicability...

A3. We would like to mention that linear models are common in the social sciences and ought to be more common in economics and elsewhere [Bollen (1989); Angrist & Evans (1996); Acemoglu et al., (2001); Spirtes et al., (2000)]. Furthermore, a series of articles have focused on studying the Additive Linear, Constant Effects (ALICE) model, including works by [Bowden et al. (2015); Kang et al. (2016); Silva & Shimizu (2017); Guo et al. (2018); Windmeijer et al. (2021)]. Unlike the ALICE model, our approach extends to a more general framework—Additive Nonlinear, Constant Effects (ANICE) models. In other words, our work explores a more challenging scenario, where $g(\cdot)$, $f(\cdot)$, and $\varphi_*(\cdot)$ may be non-linear functions.

W4. real-word datasets...suggesting the need for further validation and robustness checks.

A4. According to your suggestion, we conducted two additional real-world experiments:

1. Fulton Fish Market Data. This data study on the price elasticity of demand for fish. Our analysis focuses on the 111 samples and 10 key variables: the outcome logquantit ($logq$); the treatment logprice ($logp$), 3 candidate IVs ({$wave$, $wind$, $rainy$}), and covariates ({monday, tuesday, etc}). Cunningham, (2021) showed that both $wind$ and $wave$ can serve as valid IVs w.r.t. $logp \to logq$. Using the CAT method with K = 2, we found that {$wind, wave$} had the smallest distance correlation $dCor = 0.21$. Furthermore, distance correlation independence tests yielded a p-value of 0.98 for $\mathcal{A} _ {\widetilde{wind}}, \widetilde{wave}$ and a p-value of 0.95 for $\mathcal{A}_{\widetilde{wave}}, \widetilde{wind}$. These results suggest that we cannot reject {$wind, wave$} as a valid IV set w.r.t. $logp \to logq$, aligning with Cunningham’s findings.

Cunningham, Scott. Causal inference: The mixtape. Yale university press, 2021.

2. Education Wage Data. This dataset studies the effect of education on wages. Our analysis consisted of 663 individuals and 13 variables: the outcome logarithm of wage ($lwage$), the treatment years of education ($educ$), 8 candidate IVs, {father’s education ($feduc$), mother’s education ($meduc$), $urban$, $tenure$, $age$, $married$, $black$, $hours$}; and covariates, {$IQ$, $exper$, $expersq$}. Wooldridge et al., (2016) showed that both $feduc$ and $meduc$ can serve as valid IVs w.r.t. $educ$ and $lwage$. Using the CAT method with K = 2, we found that {$feduc, meduc$} yielded the smallest distance correlation (dCor=0.16). The distance correlation independence tests yielded a p-value of 0.03 for $\mathcal{A} _ {\widetilde{feduc}}, \widetilde{meduc}$ and a p-value of 0.12 for $\mathcal{A}_{\widetilde{meduc}}, \widetilde{feduc}$. These results imply that we cannot reject {$feduc, meduc$} as a valid IV set, consistent with Wooldridge et al., (2016).

Wooldridge, Jeffrey M. Introductory Econometrics: A Modern Approach 6rd ed. Cengage learning, 2016.

Q1. how can you verify...on real-world datasets? Is there a reliable method for this?

A1. Verifying whether a variable is a valid IV in real-world datasets is inherently challenging, as it cannot be directly tested. Typically, domain knowledge is used to determine whether a variable qualifies as an IV, but such information is often absent, making it difficult to select valid IVs and achieve unbiased causal estimates. Hence, a data-driven approach is needed. In general, one can rule out a variable's validity as an IV using necessary conditions proposed by existing studies (e.g., Pearl's instrumental variable inequality or our CAT condition). However, confirming that a variable is truly valid requires additional assumptions—such as Assumption 1 introduced in our paper.

Thank you for your helpful comments. Please find our responses below.

S1. time complexity analysis

A1: Let $n$ denote the sample size, $m=|\mathbf{Z}|$, and $p=|\mathbf{W}|$. The time complexity of our algorithm consists of three components:

1. Caculate the covariates' residual: $\mathcal{O}(n \cdot m\cdot p^2)$;
2. Find the valid IV set: $\mathcal{O}(n^2 \cdot \binom{m}{K} \cdot K^2)$;
3. Estimate the causal effect: (1) for the TSHT method, $\mathcal{O}(n\cdot (K+p)^2)$; (2) for the GMM method, $\mathcal{O}(n \cdot K^2)$.

Hence, the overall computational complexity is $\mathcal{O}(n^2 \cdot \binom{m}{K} \cdot K^2 + n\cdot m\cdot p^2 + n\cdot (K+p)^2)$.

S2. run-time scaling... some error analysis

A2: We first present run-time and error rate comparisons with baseline methods. Partial results (case 2, 1000 samples) are shown below:

where the error rate is the proportion of invalid IV set selections, and “–” indicates no output from the method.

The results above indicate that, although the runtime is longer—mainly due to distance correlation—our method achieves the lowest MSE and error rate.

Next, we present an experiment with four IVs, where $\{Z_1,Z_2\}$ are valid and $\{Z_3,Z_4\}$ violate Assumption 1. The results are as follows：

These results indicate our method also performs well in this case.

S3. For CAT Condition, provide a natural language explanation

A3: In general, the CAT condition describes the independence between candidate IVs and auxiliary variables, similar to a "cross-test". Specifically, given a reference IV $Z_i$, we test the independence between the auxiliary variable $\mathcal{A} _ {X \to Y || Z_i}$ and another candidate IV $Z_j$. Likewise, using $Z_j$ as the reference, we test the independence between $\mathcal{A}_{X \to Y||Z_j}$ and $Z_i$. If both $Z_i$ and $Z_j$ are valid, these conditions hold simultaneously, confirming the CAT condition.

S4. Figure 5 with a dashed/dotted line

A4: Following your suggestion, we updated Figure 5 with a dashed line to clearly show the true causal effect.

Q1. Under what settings...be prior knowledge?

A1: In Mendelian randomization studies [Burgess et al., (2017)], multiple candidate genes often serve as valid IVs. Thus, a small K (e.g., K=3) can be used. We would like to clarify that introducing K aims to avoid combinatorial search. If prior knowledge about K is unavailable, one may start with K=2.

Q2. ...have prior knowledge of which covariates W form a valid adjustment set...

A2: Here, we assume the covariate set is known by default. If such prior knowledge is absent, we can use other variables as covariates for $Z_i$, typically including the remaining IVs $\mathbf{Z}\setminus Z_i$ [Silva & Shimizu (2017)]. In practice, the choice of some covariates can often be straightforward. For example, factors such as age and sex commonly influence the effectiveness of drugs on disease recovery.

Q3. What if K is greater than the true number of IVs?

A3: You are right. Theoretically, when K exceeds the true number of valid IVs, the candidate IV set should fail to satisfy the CAT condition. Therefore, in the absence of prior knowledge, we recommend setting K=2. Hence, in practice, we suggest that users validate IVs incrementally (in ascending order) to ensure robustness and prevent the inclusion of unnecessary invalid IVs.

Q4. If multiple IVs are used for effect estimation, ... inappropriately large K? ...

A4: Yes, if K exceeds the true number of valid IVs, the estimated causal effect using $\mathcal{S}$ will be biased. We conducted a robustness experiment for cases where K is larger than the true number of valid IVs. As expected, we observed an increased MSE in causal effect estimation, compared to when K is set correctly. Thus, we recommend validating IVs incrementally (in ascending order) to ensure robustness and avoid introducing unnecessary invalid IVs.

Q5. If K=1, what benefits does your method provide that single-IV methods do not?

A5: We would like to clarify that our method applies only to cases where $K\ge 2$. If K=1, our method will fail, as the CAT condition relies on "cross test" to exclude invalid IV sets. However, if K>1, the most significant advantage of our method is its ability to identify IVs violating the exclusion restriction assumption, whereas single-IV methods cannot.