Thank you a lot for your positive feedback and your comments! We are happy to see that our claims, methods, theory, and experiments are well received and do not raise any additional concerns.

Here, we would kindly like to elaborate on why our paper is not an incremental contribution of the work of Schweisthal et al. but instead a standalone work with multiple novelties in a different setting.

(i) Applicability to arbitrary IVs and target bound width minimization: We show that the existing bounds for discrete instruments from Lemma 1 (which Schweisthal et al. used and which are also just an adaptation of the Manski bounds) can be applied to other instrument types (continuous, high-dimensional) by using arbitrary partitioning functions, enabling to transfer and generalize the bounds to new unconsidered settings such as Mendelian randomization (MR) or indirect experiments with complex nudges. This is a novel theoretical finding. While this may seem straightforward at first sight, to the best of our knowledge, we are not aware of any prior work considering that connection, i.e., even our naive baseline leveraging $k$-means clustering has not been considered before.

This is orthogonal to the work of Schweisthal et al. who derived model-agnostic learners in the discrete IV setting, while we focus on complex and continuous IVs. Further, this finding allows us to develop the new objective of directly targeting bound width minimization during representation learning to learn optimal partitions (Eq. (8)). Based on this, we make two major theoretical contributions regarding optimized training:

(ii) Stability by avoiding alternating learning: A straightforward implementation minimizing the bounds following Eq. (8) would require alternating learning. The reason is that, after every update step of $\phi(z)$, the quantities $\mu_\phi^a(x, \ell)$ and $\pi_\phi(x, \ell)$ are not valid for the updated $\phi$ anymore and would need to be retrained to ensure valid bounds. This is computationally highly expensive and results in unstable training and convergence problems. However, our method circumvents these issues: by using our novel Theorem 1, we show that, while training $\phi(z)$, the quantities $\mu_\phi^a(x, \ell)$ and $\pi_\phi(x, \ell)$ can be directly calculated. (see also our subsection “Implications of Theorem 1” on page 4).

For that, we can simply evaluate the nuisance functions, which only need to be trained once in the first stage. Therefore, we avoid any need for alternating learning, resulting in more efficient and stable training. Here, also note that Theorem 1 and its proof in Appendix A1 target effective estimation of our target quantities, and thus is orthogonal to the works about discrete instruments which aim for the derivation of bounds.

(iii) Improved finite sample robustness: Even using our stable training procedure from above, optimizing for Eq. (4) only yields valid bounds in expectation on the population level. However, if the discrete representation learning results in highly imbalanced marginal probabilities during training (i.e., $\mathbb{P}(\phi(Z)=\ell)$ is small for some $\ell$), this can result in high estimation variance of the nuisance estimates and thus unreliable bound estimates. We show this more formally in our Theorem 2 where we provide theoretical guarantees for the asymptotic behavior.

In contrast, we avoid these problems: by using our theoretically motivated custom loss from Eq. 19 with the respective regularization from Eq. 17, we enforce lower estimation variance during training and thus more reliable bound estimates.

In sum, we only leverage the formulation of the closed-form bounds of Schweisthal et al. from the discrete IV setting ( - which is also not their main contribution but only extending Manski bounds to the CATE - ) as a simple starting point for our method. Thus our major contributions are independent of the contributions of Schweisthal et al., which are model-agnostic meta-learners in a different setting (discrete vs. complex continuous IVs).

Action: We further improved the comparison to previous work in our paper to clarify the novelty of our method.

Thank you for your positive and very actionable review! We took your comments at heart and improved our paper as follows.

Response to Claims and Evidence

• Stability of our method compared to adversarial approaches: This is a very interesting point! As widely investigated in the literature, adversarial (typically min/max) optimization suffers from instability due to joint optimization of competing objectives, which often leads to slow convergence, gradient issues, and sensitivity to hyperparameters. In contrast, our two-step approach decouples nuisance function estimation and partitioning to provide a more controlled, stable learning process. Even in simple empirical cases, when we estimated the bounds in an adversarial manner by retraining $\mu_\phi$ and $\pi_\phi$ after updating $\phi$ (instead of using our Eq. 3 and 4), we could not even achieve convergence. This highlights the superior stability of our method.

Action: For the final version, we will report results with more exhaustive tuning of the adversarial baseline to provide a fair comparison.

Response to Methods and Evaluation Criteria

• Real-world data: For benchmarking, we use synthetic DGPs that are closely tailored to the real-world settings in Mendelian randomization (MR), such as polygenic risk scores and SNPs, and provide results for different levels of complexities. We added new experiments with real-world data from a chemotherapy study containing genetic variants. This allows us to the effect of exposure (smoking) on cancer progression (outcome). We provide the data description, results, and short interpretation here: https://anonymous.4open.science/r/IVRep4PartId-714C/rebuttal/rebuttal_experiments.pdf. In sum, we observe that our method provides expected results while showing similar Width and clearly more robust estimation compared to the naive baseline, which confirms the effectiveness of our method.

Action: We will include the real-world experiments in our paper.

Response to Weaknesses

• Regarding the points “improved stability against adversarial methods” and “real-world experiments”, we kindly refer to our response from above.
• Limited benefits in simpler scenarios: We agree that the benefits in terms of improved tightness are limited in the simpler setting. However, even though our method is designed for more complex settings, we still achieve similar performance in tightness (Width), while also providing more robust estimation (MSD) here. This shows that our method is suitable in both simpler and more complex settings.

Comments suggestions

• Loss formulation for $L_b$ in Eq. (16): While it might seem counterintuitive, our formulation in Eq. (16) is correct. By our Theorem 1, we ensure that in population, the upper bound is always greater than the lower bound, and thus, the loss cannot become negative. In finite samples, by having large estimation errors in our bound estimates, this could change in theory. However, by using our regularization loss in Eq. (17), we directly avoid such unreliable estimations. Also, when running our experiments, we never observed a negative loss, undermining its applicability.
• Intuition for $L_{aux}$: Here, as a motivation for cross-entropy, we followed the heuristic that if the activations before the discretizations can be separated more easily, they should be more diverse. We also did some runs with the empirical Wasserstein distance loss to directly enforce more distributional distance in Z given the partitions. However, the cross-entropy loss performed stable. Further, the auxiliary loss only reduces the estimation variance, but does not affect the average width or coverage (see also our experiments in Appendix K), thus turning out to be useful but not crucial for our method.

Response to questions

• For loss mechanics and stability, we kindly refer to our answers above
• Interpretation of Partitions: In principle, the clustering is only an internal mechanism to optimize our learning objective. However, under certain assumptions (see also Appendix F.1), maximizing for similar propensity scores within clusters and high heterogeneity between the clusters will lead to the closest bounds. Thus, one could implicitly identify genetic variations with similar influence on the exposure within one cluster.
• Auto-Tuning: Indeed a major benefit of our method compared to the baseline is the _robustness regarding $k$, since hyperparameter selection is hard in applied causal inference due to a lack of access to the ground truth CATE. However, we nevertheless provide practical guidelines in Appendix F.2 for the optimal selection of $k$. Therein, we propose two approaches: (1) an expert-informed approach and (2) a data-driven approach, which can be used seamlessly in practice.

Thank you a lot for your positive review and your helpful feedback! We took your comments at heart and improved our paper as follows.

Response to Claims and Evidence

Wording of contribution: Thank you for this remark! We use the term “effective” to reference the good performance of our method when optimizing for our different objectives: (1) learning tight bounds with low average width, (2) providing valid bounds with full coverage, and (3) demonstrating robust estimation without requiring alternating learning or retraining of nuisance functions while providing stable performance with low MSD between different values of $k$. However, we agree that this terminology is a bit loose.

Action: We will spell out more clearly which property of our method we refer to, and we will thereby spell out our contributions more clearly to avoid confusion.

Response to Methods and Evaluation Criteria

MSE* and Width* : We agree that the MSE* and Width* would be also interesting metrics for Datasets 1 and 2. However, to calculate these metrics, we need to approximate the oracle bounds. Since we use continuous IVs for datasets 1 and 2 to show diverse settings, ground truth bounds cannot be estimated by closed-form solutions. For the high-dimensional Dataset 3 (and Dataset 4 in the additional experiments in Appendix E), we model the IVs to be dependent on a lower-dimensional discrete space such that we can approximate the oracle bounds. However, we agree that our paper would benefit from explaining this more thoroughly.

Action: We will include an additional paragraph where we discuss our considered metrics in more detail.

Response to Weaknesses and Other Comments Or Suggestions

• Improved Presentation: Thank you for your careful checks! Action: We will use the additional page of the camera-ready version to improve our presentation significantly including FIgures 3 and 4.
• Figure 3: As you correctly pointed out and as presented in Appendix H, our current Figure 3 represents one training step. Action: To improve clarity, we will add our loss function to the final Figure and distinguish between (i) the training loop to update the parameters, and (ii) the final forward pass to estimate the final bounds.
• Motivation behind metrics: As mentioned above, we will provide an additional discussion behind our choice of metrics. Here, we shortly want to summarize the most important aspects: The Coverage(# CATE within estimated bounds / n), and Coverage* (# ground truth bounds within estimated bounds / n) should always be 1 and as high as possible to allow for reliable decision-making. The Width (sum of upper minus lower bound / n) and Width* (“Width for runs with Coverage* of at least 95%) express certainty about the CATE, and should be, in principle low. However, note that through estimation variance, low values of Width can become falsely overconfident (tighter than the oracle bounds), which is why we introduce the Width* for Datasets 3 and 4. The MSD shows the variation for different values of $k$. Low values indicate robust behavior. This metric is especially important in real-world applications where we do not have access to the oracle CATE or oracle bounds, and thus cannot calculate the Coverage, Coverage*, or Width*. This makes it hard to select a $k$ which guarantees validity but also a certain tightness of the bounds. In Appendix F, we provide an extended discussion about the role of $k$ and practical guidelines for selection.

Response to Questions

• Results Dataset 3: On average, we show improvements of about 9 % for MSE* and 2% for Width*. For the latter, even though we filter for runs with coverage above 95%, there still can be up to 5% observations with overconfident bound estimations with low width for the naive baseline, biasing the gap to appear smaller. Further, the standard deviations are inflated by summarizing the runs over different $k$. Within $k$, these effects appear stronger, as shown in Figure 5. Here, we can also see that for low values of $k$, coverage is optimal for both methods while only for the rarer evaluated high values of $k$ does the coverage start to decline rapidly for the naive baseline.
• MSD score and role of $k$: For the MSD score and the reporting of different $k$, we kindly refer to our answer from above. In Appendix F, we provide an extended discussion about $k$ and provide practical guidelines for selection that are suitable depending on the specific problem setting: (1) an expert-informed approach and (2) a data-driven approach. In our experiments, we provide the results for the respective values of $k$ in Table 3 and Figure 5, such that we can check the performance under different selection strategies.

Action: To address all points, we will add an extended discussion of the metrics and results to our paper.

Thank you for your review and the opportunity to clarify multiple points of our paper! We took your comments at heart and improved our paper as follows.

Response to concerns around claims and evidence, theoretical claims, and relation to broader scientific literature

• Novelty of our paper and implications of Theorem 1: We would like to emphasize that, while the extension of existing closed-form solutions to complex instruments is novel (i.e., even the naive baseline has not been considered yet), this is only a small part of our contribution. Importantly, instead of only deriving valid bounds, we focus on a method for improved estimation. Based on this, we introduce the new objective of directly minimizing bound width during representation learning (Eq. 8).

Importantly, in Theorem 1, we show that this objective can be optimized without alternating learning: we can directly evaluate the necessary quantities from pre-trained nuisance functions by leveraging Eq. 3 and Eq. 4. (otherwise, we would need to retrain $\mu_\phi$ and $\pi_\phi$ after every update step of $\phi$ in an alternating way). Thus, with our formulation, we avoid instability and high computational cost. (see also “Implications of Theorem 1” in our paper). Lastly, to ensure robustness in finite samples, we regularize for balanced partition probabilities (Eq. 14), reducing estimation variance as shown in our Theorem 2.

• Clarifications on our objective in Eq. (1), and claims about “tightness” and “validity”: The validity of our bounds in population follows straightforwardly from Theorem 1. In finite samples, as for every method, full validity (i.e., guaranteed 100% coverage) can never be proven. However, by using our regularization loss, we mitigate the risk of overconfident estimation, which is unlike existing methods that do not account for estimation uncertainty.

In addition, we refer to tightness by minimizing the expected bound width as much as possible given some representation $\phi$. Importantly, this usually does not lead to sharp bounds (i.e., the “tightest possible ones” for all $x$ which would only minimize the population tightness in Eq. 12), since this would not allow us to reduce the estimation variance. This is also our motivation for our objective in Eq. (1): We aim to reduce the average bound width to get robust estimates of “informative” bounds for the largest part of the population. Here, “informative” refers to “being useful for the underlying decision-making problem”. E.g., for classical CATE estimation, informative means that the lower bound is greater or the upper bound is below some decision threshold (often 0).

Action: While we are consistent in our terminology with previous work on partial identification, we will add a discussion about the different terms to improve the clarity of our work.

Response to experimental designs and analyses

• Adaptation of existing methods: As these methods often include additional assumptions over the continuous interventional distribution in the form of additional constraints, the adaptation is not straightforward. However, in Appendix E, we provide results for additional adapted baselines, such as extensions of methods that were designed for point identification.
• Significance of results: As standard in machine learning, we avoid making inferences about statistical significance since this would require careful adjustment for dependencies and multiple testing, but instead report mean and standard deviation over multiple seeds.
• Role of $k$ in Table 3: Dataset 1 is defined in a simple way such that a naive partitioning with $k=3$ can already closely approximate optimal bounds. Interestingly, with k=4 the partitioning gets worse, yielding only a coverage of 0.49 for the naive baseline, while still yielding 1.00 for our method. This shows the robustness of our method, and even may indicate that the narrow bound width of 0.83 for the baseline for k=3 may be due to falsely overconfident bound estimates.
• Evaluation and selection of $k$: We average over the k-values considered in our sensitivity analysis as reported in Table 3 and Figure 5. The intuition here is that, in real-world applications, one does not have access to the ground-truth CATE or oracle bounds, and thus coverage cannot be checked to select the best k. When only considering width as the criterion, this would lead to always choosing the tightest bounds, which are likely to not yield full coverage, especially for the baseline. In Appendix F, we provide an extended discussion over the role of $k$ and give guidelines on how to select k in practice.

Response to other comments or suggestions

• Thanks for your careful remarks! We will adjust our paper accordingly. Regarding the “Implications of Theorem 1”, we kindly refer to our answers above.

Response to Questions

• Thank you for your questions! We addressed all of them in our answers above.

Thank you for your detailed feedback and the overall positive evaluation of our manuscript! We will take all your comments to heart and improve our manuscript accordingly. Below, we provide answers to all your questions

Response to Methods and Evaluation Criteria

• Benchmark datasets: We agree that providing insights on real-world data is an interesting extension for our paper. However, it is standard in causal inference literature to rely on synthetic data-generating processes for benchmarking, since the CATE (and for our setup, also optimal bounds on the CATE) can never be observed in real-world data. Thus, we provide benchmarking using synthetic DGPs that are closely tailored to the real-world settings in Mendelian randomization (MR), such as polygenic risk scores and SNPs, and provide results for different levels of complexities. Further, by using our DGP for datasets 3 (and 4 in the Appendix), we can approximate oracle bounds, allowing us to check for validity also with respect to optimal bounds and not only compared to the CATE, which is an extension over previous work in similar settings.

Action: To further strengthen our paper, we provide additional insights on real world data in MR (https://anonymous.4open.science/r/IVRep4PartId-714C/rebuttal/rebuttal_experiments.pdf). Overall, our findings are consistent with our synthetic setup and indicate robust performance of our method

Response to Experimental Designs or Analyses

• Additional baselines and experimental setups: Thank you! Importantly, we focus on the highly relevant setting with complex instruments and binary treatments as it has been neglected in prior work. On the other hand, this implies, that our method is not tailored or applicable to setups considered by existing papers (i.e., our method cannot be naively adopted to continuous treatments; and, for simple discrete IVs, no tailored representation learning is necessary since the closed form bounds can be estimated directly). Thus, we focus on our considered datasets. However, we agree that additional baselines besides the naive one are interesting. Thus, in Appendix E, we provide results for additional adapted baselines to show that our tailored method is clearly superior and necessary.

Response to Strengths and Weaknesses

• Contribution: We agree that the reformulation of $\mu$ and $\pi$ is one of our main contributions as this allows us to avoid alternating learning. However, we would kindly highlight our major other contributions: (i) Novel setting: We are the first to directly focus on partial identification with complex instruments such as in MR, (ii) Our learning algorithm: We do not only avoid alternating learning and leverage discrete representation learning but also encourage reduced estimation variance by our tailored loss function as shown in Theorem 2.

• Dataset 1, Table 3 performance: As the DGP for Dataset 1 is simple, a naive clustering with $k=3$ could already lead to close-to-optimal bounds. However, importantly, for Datasets 1 and 2, lower width does not necessarily indicate better performance. By using only finite data for estimation, the bounds could also be falsely overconfident, even if they lead to full coverage of the true CATE, as the naive baseline does not try to reduce estimation variance. Since, in these DGPs, we use continuous IVs, we cannot approximate the oracle bounds (as for Datasets 3 and 4). Instead, the main finding here is that our method leads to robust estimates for different $k$, while the naive baseline highly depends on the selection of $k$.

• How to set $k$ in practice: As mentioned, a major benefit of our method compared to the baseline is the robustness regarding $k$. However, since hyperparameter selection is hard in applied causal inference due to a lack of access to the ground truth CATE, we provide practical guidelines in Appendix F2. Therein, we propose two approaches: (1) an expert-informed approach and (2) a data-driven approach, which can be used seamlessly in practice.

Response to other Comments

Thank you for your careful reading! All of your points are very helpful and we will include them in the camera-ready version of our paper.

Response to Questions

• Thank you for your questions! For the first two questions, we kindly refer to our responses above.
• Invalid IVs: As in typical MR and IV settings, we need to ensure that the exclusion and independence assumptions hold. However, unlike usual methods, we do not require the relevance assumption: we can – in principle – also use SNPs that are not associated with the exposure /treatment. Since these irrelevant IVs do not lead to closer bounds, our method will just not use their information. Our datasets 3 and 4 indeed contain 75% of irrelevant IVs, which demonstrates the strong performance of our method in this relevant scenario.