Data Fusion for Partial Identification of Causal Effects

Thank you for the thoughtful and constructive review. We’re glad to hear you enjoyed the paper and appreciate your openness to revisiting your score. Below, we address your main point of confusion and clarify the intended interpretation of our setup and assumptions. We also respond to your additional comments and outline how we will strengthen the final version of the paper.

Problem Setup and Population Relationships

Thank you for raising this important clarification. Our setup involves just two source populations: the RCT cohort (S=1) and the observational cohort (S=0). Our target estimand is defined over the combined population (the mixture of these two), weighted by their respective covariate distributions. However, Our framework is mathematically flexible to accommodate an alternative estimand that chooses either observational (S=0), RCT (S=1) or a separate third population (S=2) as target population. The adaptation requires modifying the marginal covariate distribution in our estimand from $P(\mathbf{X})$ to $P(\mathbf{X}|S=s)$ and adjusting the breakdown frontier calculations accordingly. This maintains all the sensitivity analysis benefits while targeting the population highlighted in your comment. In the revised manuscript, we explicitly present both estimand options (combined populations and specific target population), discuss their appropriate use cases, and demonstrate how our sensitivity analysis framework applies to each. This will better highlight the flexibility of our approach while addressing the valid concern you've raised about target population selection.

Using Project STAR as a concrete example: both cohorts were drawn from Tennessee school districts—the RCT population consisting of students randomized to different class sizes, and the observational population consisting of students assigned non-randomly. Our target population are all students, not just within one cohort.

With this clarification, we can explain the statement in lines 44-46: The ATE over this combined population is point-identified if either:

1. The observational data satisfies no unmeasured confounding (NUC), or
2. The experimental and observational samples satisfy study exchangeability

This is because we can decompose the CATE as: $\mathbb{E}[Y(1) - Y(0) \mid \mathbf{X} = \mathbf{x}] = \sum_{s \in \{0,1\}} \left[ \mathbb{E}[Y(1) \mid \mathbf{X} = \mathbf{x}, S = s] - \mathbb{E}[Y(0) \mid \mathbf{X} = \mathbf{x}, S = s] \right] P(S = s \mid \mathbf{X} = \mathbf{x})$

The only non-identifiable components are the potential outcomes for $S=0$. Either NUC or transportability suffices to identify these terms, which is why we develop sensitivity analysis for scenarios where neither assumption fully holds.

CATE vs. ATE Estimation Differences

You are right that precise estimation of bounds for CATE is harder compared to ATE. Regarding lines 242-243 and CATE estimation: The same efficient influence function (EIF) derivation approach applies to both ATE and CATE, but with key differences in implementation. For CATE estimation (for a prespecified $\mathbf{x}$):

1. We must estimate conditional expectation functions at the covariate value $\mathbf{x}$ of interest rather than averaging across the entire population
2. This introduces additional estimation challenges, particularly for high-dimensional $\mathbf{X}$. In particular, it reduces the resultant effective sample size and just inflates the confidence intervals.

While our theoretical framework extends to CATE estimation, practical implementation would rely on stronger smoothness assumptions for tighter uncertainty quantification.

Selecting $\alpha$ for the Boltzmann Operator

You've raised an important practical question. We discuss this briefly in Appendix E, but note that it is not referenced clearly in the main text. In our implementation, we found $\alpha$=10 provides a good balance between approximation quality and numerical stability across datasets of moderate size (n = 1000 to 5000). For larger datasets (n>10,000), higher values ($\alpha$=25+) can be used without numerical instability concerns.

Practically, we recommend:

• Starting with $\alpha$=10 for most applications
• Testing sensitivity to $\alpha$ by comparing results with $\alpha$=1 and $\alpha$=50
• For very large datasets, increasing to $\alpha$=50+
• For small datasets (n<500), using $\alpha$=0.1-1 to prioritize numerical stability

For most practical purposes, we found that our performance is not overly sensitive to moderate changes in $\alpha$, but extremely large values and/or small sample sizes can introduce numerical instability due to the steep gradients in the smooth approximation. In the revision, we will bring this guidance to the main text or add a more explicit reference to Appendix E to ensure it’s easy to find.

Simulation Study Extensions

We appreciate your suggestion to expand our simulations. While we focused on demonstrating bound behavior across DGPs, we agree that examining additional statistical properties would strengthen the paper. In the revision, we will add:

1. Sample size efficiency analysis showing estimator performance at n=250, 500, 1000, 5000, and 10000.
2. Robustness checks of the asymptotic properties when model assumptions are moderately violated
3. Sensitivity analysis for the $\alpha$ parameter in the Boltzmann estimator, specifically looking at behavior across different sample sizes.

These additional analyses will provide stronger empirical validation of our theoretical results and offer more practical guidance for implementation.

Fantastic point and you are absolutely spot on. To clarify more precisely: In our setup, due to the internal validity of the RCT, $E[Y(a) \mid S=1, X]$ is always identifiable. However, the reason $E[Y(a)]$ is not point identifiable without additional assumptions is because $E[Y(a) \mid S=0, X]$ remains non-identifiable, in general

This relationship can be expressed mathematically through the law of total expectation: $E[Y(a)] = E[E[Y(a) \mid X]] = E[E[Y(a) \mid S=0, X] P(S=0 \mid X) + E[Y(a) \mid S=1, X] P(S=1 \mid X)]$

Since the second term inside the expectation is identifiable but the first term isn't (without additional assumptions), this explains the partial identification challenge. This formulation demonstrates why obtaining partial identification bounds for $E[Y(a)]$ and $E[Y(a) \mid S=0]$ are mathematically synonymous.

As promised, we will update our paper to explicitly incorporate $E[Y(a) \mid S=0]$ as an alternate estimand, with appropriate adaptations to our framework.

Regarding study design: Our work addresses what is often called a "nested" study design. This is precisely the case in our STAR example, where students from the same Tennessee school districts were either part of the randomized trial or the observational component of the study. We will, for sure, clarify this in the paper.

Thank you so much for helping us improve the paper.

Thank you for your detailed review and critical questions. We hope the following notes help clarify some misunderstandings as well as elaborate on feedback we will incorporate into our final version.

Regarding sensitivity parameter selection by domain experts

Thank you for this important question. We designed our framework with interpretability in mind, particularly for domain experts who may not be statisticians. In Section 6.2 with the Project STAR dataset, we demonstrate how our parameters translate to meaningful quantities: when $\gamma = 0.05$, this means a confounder would need to shift student scores by more than 30 points on average (on a scale where scores range from 486-745) to invalidate our findings. Such concrete interpretations make the framework accessible to education researchers.

The breakdown frontier approach specifically addresses the challenge of parameter selection by visualizing robustness across a range of values rather than requiring precise point estimates. In practice, domain knowledge informs reasonable parameter ranges—for instance, when observational and RCT participants come from the same hospital system, limiting $\gamma$ to 0-0.05 may be reasonable, while cross-system comparisons might warrant $\gamma$ in 0-0.25 to account for greater heterogeneity.

Choice of sensitivity model and comparison to alternative approaches

While we build on established sensitivity analysis principles, our approach offers several key innovations for the data fusion context:

1. First, we developed a dual-parameter framework that simultaneously captures both unmeasured confounding ($\rho$) and study exchangeability violations ($\gamma$) in a unified model. This enables analysis of how these distinct sources of bias interact—revealing that they can either compound or counteract each other, an insight not captured by applying single-dataset sensitivity approaches separately to each data source.

2. Second, regarding our choice not to use MSM: while MSM effectively parameterizes propensity score sensitivity for a single observational dataset, this approach becomes problematic in multi-dataset settings where propensity scores have fundamentally different interpretations (true randomization probabilities in RCTs versus estimated probabilities in observational data). Our direct parameterization of outcome deviations provides a more consistent interpretation across the fused datasets.

3. Third, our parameters are deliberately designed as scale-invariant, relative magnitude measures. This allows meaningful interpretation regardless of the outcome scale—a critical feature when combining datasets that may have different measurement properties. For instance, $\gamma = 0.1$ consistently means "up to 10% relative deviation in outcomes" whether we're measuring blood pressure, test scores, or income. Our approach thus represents a principled extension of sensitivity analysis to the data fusion setting rather than a direct application of existing single-dataset methods.

Choice of Target estimand

Thank you for this important observation. Our target estimand (ATE over the combined populations) was deliberately chosen because it addresses the common data fusion objective of leveraging complementary strengths of both datasets to estimate treatment effects for a broader population. This aligns with the paradigm described by Parikh et al (2023), where RCT findings are extended to wider populations rather than transported to a specific target.

We fully agree that in many applications, the observational population (S=0) may be the primary population of interest. Our framework is mathematically flexible to accommodate this alternative estimand. The adaptation requires modifying the marginal covariate distribution in our estimand from $P(\mathbf{X})$ to $P(\mathbf{X}|S=0)$ and adjusting the breakdown frontier calculations accordingly. This maintains all the sensitivity analysis benefits while targeting the population highlighted in your comment. In the revised manuscript, we explicitly present both estimand options (combined population and observational-only population), discuss their appropriate use cases, and demonstrate how our sensitivity analysis framework applies to each. This will better highlight the flexibility of our approach while addressing the valid concern you've raised about target population selection.

Missing Consistency assumptions

Thank you for the suggestion. We note that the standard SUTVA and consistency assumptions are included in the first paragraph of Section 2. Following your recommendation, we have updated our manuscript and added them to our list of assumptions for completeness.

Literature review expansion

Thank you for these helpful resources. We agree that these are relevant to our approach and have added them to our literature review in the updated manuscript. In particular, Schweisthal et al. (2024) is similar to our work in that it proposes a partial ID framework for combining multiple datasets, albeit it focuses on combining only observational datasets and considers a different subset of assumption violations. We will clarify this distinction and note the complementarity between these approaches in the final version. Van der Laan et al. (2025) and Wang et al. (2024) are good examples of approaches for facilitating valid inference in our setting, and we will be sure to reference them in our final version. Although, they do not appear to focus on partial identification or sensitivity analysis. We have included an expanded literature review on data fusion methods for causal inference in our appendix, and thank the reviewer for providing these sources as additional relevant literature to include.

References:

• Parikh, H., Morucci, M., Orlandi, V., Roy, S., Rudin, C., & Volfovsky, A. (2023). A double machine learning approach to combining experimental and observational data. arXiv preprint arXiv:2307.01449.

Dear Reviewer,

We acknowledge and build upon the vast existing literature on sensitivity analysis while making several distinct contributions:

1. Novel data fusion approach: We uniquely leverage both experimental and observational designs to yield tighter partial identification bounds than would be possible with either data source alone. Our framework explicitly addresses the interaction between different bias types in data fusion contexts.

2. Accessible sensitivity parameterization: Our representation makes sensitivity analysis interpretable to domain experts without specialized statistical training. By parameterizing deviations as relative percentages from standard causal assumptions and combining this with breakdown frontier visualizations, we enable practitioners to intuitively assess the robustness of their conclusions.

3. Scale-invariant parameterization: While approaches like Bonvini et al. (2022) use sensitivity parameters on an absolute scale, our parameters are defined in relative, scale-invariant terms. This critical difference allows for meaningful comparisons across different outcome measures and studies, facilitating meta-analysis and cross-study interpretation.

4. Methodological integration: We bring together existing ideas from sensitivity analysis, data fusion, and efficient influence functions in a novel way that yields practical solutions for real-world problems. This integration bridges previously separate methodological approaches to address increasingly common analytical challenges.

In the revised manuscript, we will include a more extensive literature review that thoroughly contextualizes our work within existing approaches, including important frameworks like those developed by Bonvini and colleagues. This will better highlight both our intellectual foundations and our novel contributions to the field.communication and arguments across contexts.

We appreciate your review and would like to address your concerns about the work's significance and the sensitivity model.

Regarding significance and relevance

We respectfully disagree that our setting is highly specialized. Data fusion causal inference methods are increasingly crucial across healthcare, economics, and policy research, where combining experimental and observational data is necessary but fraught with challenges. For instance, consider the case of drug repurposing Wegovy or Ozempic for weight loss. Any time one has an experiment to examine a treatment that already exists, we will probably have both. Our approach addresses a significant methodological gap: while sensitivity analysis has a rich history in causal inference and data fusion methods are growing rapidly, robust sensitivity frameworks specifically for data fusion settings have been notably absent. Our contribution thus serves an important and expanding research area rather than a niche application.

Regarding the sensitivity model

The critique about "unassessable sensitivity models" applies to virtually all sensitivity analysis methods, as their fundamental purpose is to explore the implications of untestable assumptions. The strength of our approach lies in its transparent parameterization:

1. We use interpretable parameters ($\rho$ and $\gamma$) that quantify relative deviations in intuitive terms (e.g., $\rho = 0.2$ means confounding shifts outcomes by at most 20%)
2. Our parameters are scale-invariant, facilitating meaningful cross-study comparisons
3. We take a conservative worst-case bounds approach that avoids optimistic modeling choices
4. Our visualization framework enables practitioners to reason clearly about robustness thresholds

Rather than claiming to solve the inherent untestability of assumptions, we provide a principled framework for researchers to assess how strong violations would need to be to invalidate their conclusions. We believe this aligns with the core purpose of sensitivity analysis by enabling researchers to reason transparently about the robustness of their findings.

Thank you for the extremely detailed and thoughtful review. We appreciate the time you took to read the paper closely and provide such substantive feedback.

Sensitivity Model & Incompatibility:

We chose a ratio-based sensitivity model in part for its scale-invariance, which allows it to generalize across different outcome scales and covariate profiles. While a difference-in-means model (as you suggest) could also be used, we see this as a modeling choice rather than a structural limitation, and our framework could accommodate such a formulation with mild adjustments. We agree that briefly justifying this design decision in the main text would strengthen the paper.

Regarding our parametrization of study exchangeability violations via $\gamma$, we appreciate your proposed alternative of bounding deviations only from the unidentified component of the observational distribution. However, we chose to define $\gamma$ via a contrast that is marginalized over $T$ because it more directly reflects violations of study exchangeability as differences in potential outcomes across study populations.

To illustrate, suppose that for some covariate-treatment pair $(\mathbf{x}, t)$, there is a nonzero difference in expected potential outcomes between experimental and observational units who received treatment t: $\left| \mathbb{E}[Y(t) \mid \mathbf{X}=\mathbf{x}, S=1, T=t] - \mathbb{E}[Y(t) \mid \mathbf{X}=\mathbf{x}, S=0, T=t] \right| > 0.$ Now, under your alternative parametrization, $\gamma = 0$ would correspond to equality in the unobserved counterfactual group (T = 1−t). But this would not imply overall study exchangeability. The marginal difference, $\left| \mathbb{E}[Y(t) \mid \mathbf{X}=\mathbf{x}, S=1] - \mathbb{E}[Y(t) \mid \mathbf{X}=\mathbf{x}, S=0] \right|$, would remain nonzero (as long as treatment propensities are nonzero). Thus, this alternative $\gamma$ may appear to label settings as “no study exchangeability violation” even when outcome distributions differ between the samples.

We agree that your suggestion is principled and could support a valid alternative model. We chose our $\gamma$ specification because it provides a transparent interpretation of study exchangeability failure that integrates naturally with the interpretation of the rest of our sensitivity framework.

Target estimand: This is a good question, and a point we plan to clarify further in our final version. Our decision to target the CATE over the combined population was deliberate. This estimand reflects the common goal in data fusion problems to combine complementary information from both the RCT and observational datasets to estimate effects across a broader population. This perspective aligns with the “generalizability” framework outlined by Parikh et al. (2023), where the objective is to extend findings from a randomized study to a larger or more representative cohort, rather than transporting them to an external population.

That said, we fully agree that in many settings the observational population ($S=0$) may represent the primary target of interest. Our methodology is flexible with respect to the target estimand, and can be adapted to this case. Specifically, this involves adjusting the marginal covariate distribution in the estimand from $P(\mathbf{X})$ to $P(\mathbf{X}|S=0)$, with corresponding modifications to the breakdown frontier calculation. All core components of the sensitivity analysis remain applicable. In the revised manuscript, we will discuss both estimand options (combined population and observational-only), clarify their respective use cases, and explain how our framework supports either goal. We believe this addition will better highlight the flexibility of our approach and address the important point you’ve raised.

Estimation Considerations:

We really appreciate the depth of this feedback and the opportunity to address these estimation considerations. We address each below:

• (1) Related literature. Thank you for the helpful references. We agree that these works are highly relevant, and we will cite them and expand our discussion of related approaches to handling non-smooth functionals. In particular, we will better position our contribution in relation to both smooth approximation–based strategies and alternative approaches that avoid smoothing through structural or margin assumptions. We chose a smooth approximation framework (via Boltzmann operators) to preserve pathwise differentiability and enable influence function–based inference in a principled and generalizable way. One of these references, Levis et al. (2023), adopts a similar approximation strategy using the LSE function. We will incorporate a discussion of our approach and highlight the tradeoffs between it, and similar smooth approximation approaches, compared to other approaches that do not employ smooth approximations.
• (2) Approximation error. We agree that quantifying finite-$\alpha$ approximation error is an important theoretical question. Our current work focuses on establishing the validity and practical utility of the smooth bounds using Boltzmann approximations, and Lemma 2 guarantees their convergence to the sharp bounds as $\alpha \to \infty$. In practice, we found moderate values of $\alpha$ (i.e. \alpha=10) offered a favorable tradeoff between smoothness (for stable estimation) and tightness (for informative bounds), and we demonstrate their empirical behavior in Section 6. We will also include additional simulation results that explore the sensitivity of our estimator to different $\alpha$ values across varying sample sizes. We recognize that a more formal characterization of the bias–variance tradeoff introduced by finite $\alpha > 0$ would strengthen the theoretical guarantees. Prior work (like those you cited) offers promising techniques in this direction, and we will cite these in the final version and highlight this as a compelling direction for future work. However, we view this refinement as complementary to the current contribution, which introduces a general sensitivity framework with valid and efficiently estimable bounds. We believe our approach adds meaningful insight even without a full approximation error analysis, and hope it lays the foundation for future extensions that can incorporate these refinements.
• (3) Theorem 2 and Nuisance parameter rates. We thank the reviewer for suggesting that the statement of Theorem 2 should include the regularity conditions and the convergence rate conditions for the nuisance parameter estimators. In the revised version, we will add the necessary rate conditions on $\hat{g}$, $\hat{e}$, and $\hat{\mu}$ into the theorem statement itself. These regularity conditions are analogous to the ones described in Appendix B of Robins, Rotnitzky and Zhao (1994), and Section 3 of Rudolph et al (2025). In the proof, we will also explicitly define the empirical process term, $(\mathbb{P}_n - P)(\hat{\phi} - \phi)$, and the decomposition of the second-order remainder term, $R_n$. We will further clarify the conditions under which the empirical process term is $o_p(n^{-1/2})$, and that the second-order remainder is $o_p(n^{-1/2})$ provided the nuisance estimators converge at sufficient rates (i.e., standard conditions such as $|\hat{\mu} - \mu| \cdot |\hat{g} - g| = o_p(n^{-1/2})$, etc.). While this is expected to follow under standard regularity conditions for EIF-derived estimators of smooth approximations, we agree that a formal derivation of the second-order term is necessary to justify these conditions rigorously, and we will include this in the revised version.

Sharpness & Bounded Outcomes:

You are correct that Lemma 1 establishes the validity of our bounds, but not sharpness. This was an error on our part. We will revise the text to reflect this more precisely. We appreciate the suggestion to incorporate outcome bounds directly into the sensitivity model, by truncating $v(\mathbf{x}, t, \gamma)$ and $w(\mathbf{x}, t, \rho)$ with known support $[b, B]$, to yield sharper bounds under the model assumptions. That said, we chose not to incorporate additional min/max operations because it would add further non-smooth structure to an already challenging estimation problem. While we acknowledge this could be remedied using a similar smoothing approach to the one we already use (or one of the other existing approaches you reference), our current framework already applies Boltzmann smoothing between the functions $v(\mathbf{x}, t, \gamma)$ and $w(\mathbf{x}, t, \rho)$ that are at the heart of our partial identification method. Introducing further smooth approximations for support truncation would require additional derivations and tuning, which we viewed as beyond the scope of this initial contribution. We agree that this is a promising direction, and we will add a discussion in the revised paper clarifying that sharper bounds may be attainable using known outcome support, but that doing so would involve additional modeling and inference challenges we chose not to pursue here. We hope future work can build on our framework to incorporate this work.

References:

• Parikh, H., Morucci, M., Orlandi, V., Roy, S., Rudin, C., & Volfovsky, A. (2023). A double machine learning approach to combining experimental and observational data. arXiv preprint arXiv:2307.01449.
• Robins, J. M., Rotnitzky, A., & Zhao, L. P. (1994). Estimation of regression coefficients when some regressors are not always observed. Journal of the American statistical Association, 89(427), 846-866.
• Rudolph, K. E., Williams, N. T., Stuart, E. A., & Diaz, I. (2025). Improving efficiency in transporting average treatment effects. Biometrika, asaf027.

Thank you so much for your inputs and suggestions. It will surely make our paper stronger and better.