Active Treatment Effect Estimation via Limited Samples

Thanks for your advice! Here is our response point by point:

Q1: The proof of Lemma 4.4 borrows notation from Ghadiri et al. without defining it or referencing that paper, making it hard to follow without previously reading the other paper. In particular, $2 y=t+Z \bigodot \mu$. Also, it does not appear that $\mu$ is formally defined in the main body of the text, but is presumably consistent with the definition from Ghadiri et al.

Thanks for the careful review and wonderful comment! Here $\mu$ indicates that the vector composes of $Y_i(0)+Y_i(1)$. and $\bigodot$ denots the Hadamard product. We agree that, even though we had previously stated that these local notations were adopted from Ghadiri et al., it is important to make this explicit again in the main text or appendix. We have revised the manuscript accordingly with our utmost care and sincerity, which will be presented with full rigor and clarity in the final camera-ready version.

Q2: Add network experiment

Thanks for your advice. First, we have conducted large-scale experiments across a range of values for both $d$ and $n$, as shown in RW kg3L. Second, we have fully implemented your suggestion and carried out additional experiments under settings with network interference. Specifically, we followed both the simulation setup and the real-world dataset configuration used in the work by Lu et al. [1]. We replicate their basic settings wholly. We refer reader for the experimental results in the link https://anonymous.4open.science/r/Causal-Active-Sample-B0C5/README.md.

Q3: Expeiremnt focuses on d.

Refer to RW kg3L.

Q4: Notation issue Thanks for your comment! We will fix this notational issue by choosing another symbol to denote the row of the vector or matrix to avoid potential incomprehension.

Q5: The proposed ATE estimator in Addanki et al. instead recursively partitions the data using the GSW algorithm from Harshaw et al., so presumably the sample complexity would be different.

Thanks for your comment. The sample complexity for Addanki et al is $O(dlog(d)+d/\epsilon)$ for the ITE estimation; for ATE estimation, in their excellent paper, the complexity could be controlled by "ATE estimation. For ATE estimation we give a randomized algorithm that selects at most $s$ individuals for treatment/control assignment and obtains an error of $\widetilde{O}\left(\sigma / \sqrt{s}+\left(\left\|\boldsymbol{\beta}^1\right\|+\left\|\boldsymbol{\beta}^0\right\|\right) / s\right)$," We will revise it in the final version. Thanks for your wonderful comment!

---

[1] Adjusting auxiliary variables under approximate neighborhood interference, X Lu, Y Wang, Z Zhang, arXiv preprint arXiv:2411.19789.

Thanks for your review and comments! Here are the responses to all of your questions.

Q1: It would be helpful to see results as a function of covariate dimensionality.

Thanks for your suggestion! We provide the following supplementary experiments on the performance differences as d varies with $n=1000$ samples. The advantage of our RWAS algorithm over traditional CRA/GSW methods is especially apparent when the sample dimension d is large, as a result of the optimal sample complexity in active sampling process.

Q2: Further description on the core novelties of this method like the adaptation of IRD.

Thanks for your concern. Please see the response to RW PucS.

Q3: Revision on the word like outstanding

Thank you for this helpful note. In response, we have carefully revised the wording in the beginning of Section 4 to remove exaggerated adjectives such as "outstanding" and "superior". We now use more neutral, objective language and anchor our claims directly to specific empirical results and theoretical guarantees. This revision aims to improve clarity and expression throughout the manuscript. We sincerely appreciate your feedback on this matter.

Q4: How to emphasize the role of partitioning & subsampling

Thanks for your concern! The overall target of partitioning and subsampling is to find efficient causal effect estimators with limited cost on sample collection and computation, but focusing on different components.

• Subsampling finds the most representative samples in terms of covariate distribution by selecting and up-weighting ``influential'' directions in the data.

• Partitioning seeks for well-balanced assignments and provide unbiased effect estimators with controllable expected error.

In Algorithm 1, partitioning and subsampling are deployed in two disjoint subset of the entire samples, one of which applies GSW with regression adjustment, and the other applies IRD. Through weighting and averaging, we obtain an unbiased estimate with the total sample size being O(d).

Besides, the key idea is to improve the MSE through well-balanced assignments and representative samples. As stated in line 65, two types of techniques are applied to achieve these goals: partitioning and subsampling. The IRD algorithm selects the representative samples to ensure a controllable estimation error. At the same time, the GSW design is applied to achieve balanced assignments by partitioning the samples into treatment and control groups. This motivates the following thought, serving as another line of motivation:

• We can use O(1) samples for the GSW regression-adjusted estimator.
• However, learning the regression coefficient requires O(d) samples.
• Therefore, we divide the data into two disjoint sets, one of which applies GSW with regression adjustment, and the other applies IRD, using O(d) samples to learn the regression coefficient from the first set.
• These two sets each construct an estimator, and after weighting and averaging, we obtain an unbiased estimate, with the total sample size being O(d). Note that using either set alone as an estimator may be biased because we actively select individuals corresponding to X with good representational properties for the estimation. This group is not independent and is randomly selected from the entire sample.

Q5: Does this method easily extend to multiple treatments?

Thanks for your question. It is not complicated to propose a solution for multiple-treatment extension. For example, the estimator on the treatment effects between two treatment levels $Z=z_1$ and $Z=z_2$ can be constructed through our method restricting treatment allocation on $\{z_1,z_2\}$ as long as the positivity assumption $P(Z_i=z)>0$ holds for targeted levels. On the other hand, it is possible to find methods to compare different treatment pairs through one sampling process, or combine multiple processes to further increase efficiency, which requires further research. It might be highly nono-trivial due to the proof related to the martingale in the GSW design.

Thank you for your thoughtful review and valuable questions! We address your questions point-to-point in the following.

Q1: However, the authors should provide more explanation or context about the proposed method.

1. Insightful motivation.

Motivating example. Suppose $X$ is $n \times 2$, and we want a subset to approximate $X^\top X$ for regression. Random sampling may cluster points along a single direction, leading to rank deficiency or poor conditioning. In contrast, Algorithm 2 adapts sampling probabilities to promote diversity: it increases the chance of selecting points in underrepresented directions by updating $B_i$. This ensures that new samples help span missing dimensions. After a few rounds, the selected subset captures both primary axes and preserves the geometry.

To avoid over-representing outliers, constraints on $\alpha_i$ limit how much influence any point can have. As a result, the spectrum of the reweighted matrix $A^\top A$ remains close to that of the full dataset (identity matrix after normalisation). We avoid redundancy while preserving rare but critical directions—achieving near-optimal regression performance with far fewer, well-chosen samples.

2. Detailed interpretation of main algorithm

Another line of motivation, refer to RW RneH for the Q4.

3. Additional comparision

• The technical novelty (i): First, we refine the previous result in Chen & Price (2019) to adapt to a finite-sample setting. It resorts to the refinement of the parameter in the $\epsilon$-reweighting strategy, compared to that of Chen & Price (2019), Definition 2.1. Intuitively, consider the OLS estimator regressing $Y$ on $X$ for any arbitrary weight $w_i$. We have $E[w_i (\hat{Y}_i^{ols} - Y_i)] = 0$, so the expectation of the noise term vanishes. However, in finite samples, such sum may usually deviate from zero, necessitating further refinement of the weighting parameters.

• The technical novelty (ii): It brings significant challenges since, compared with the setting in Chen & Price (2019), each individual's outcome is not fixed as $Y_i$, instead, the outcome is randomly chosen as $Y_i(0), Y_i(1)$, jumping between the real world and the counterfactual world.

Q2: technical novelties compares to Chen & Price (2019)?

Kindly see Q1-Additional comparison

Q3: Explain Algorithm 1

Kindly see Q1-Detailed interpretation of main algorithm.

Q4: The lines 149-157's writing, and the meaning of Condition (ii).

The second condition (ii) is intended to impose an upper bound on the total weighting factors $\alpha_i$. Specifically, this condition ensures that the cumulative weights are controlled, preventing any single sample or a small group of samples from excessively dominating the overall estimator.

To clarify the mathematical intuition, we provide the following rewritten version of the paragraph. Briefly, it includes (i) Spectral approximation condition, (ii) Bounded cumulative weight condition and (iii) Single-point influence condition. Due to space limitations, we kindly refer you to the anonymous link: https://anonymous.4open.science/r/Causal-Active-Sample-B0C5/README.md.

Q5: The parameter analysis of the lower bound

Overall, whether setting it as $|\hat{\tau}-\tau|\leq \epsilon$ or $0.1$ are both correct (as long as $\epsilon < 0.1$); We sincerely appreciate your suggestion—explicitly including the $\epsilon$ parameter in the estimation error bound improves clarity and helps readers better understand the overall process and how the Shannon-Hartley upper bound applies to the estimation error.

Q6: Notation meaning

“∗” is a “null option”; also refer to the dear reviewers to Algorithm 4 in the appendix. the notation $\mathcal{N}_b^{\pi}(i)$ is deferred in the above anonymous link.

---

We are eager to hear your feedback. We’d deeply appreciate it if you could let us know whether your concerns have been addressed.

Acknowledgement and General Response to RW kg3L, RW PucS , RW RneH, RW fVjN

Thrilled to receive such a positive reception from dear reviewers! Also, sincerely thank all reviewers for their insightful suggestions! In this general response, we carefully synthesized the reviewers’ suggestions to facilitate the AC’s assessment and, more importantly, use them as a blueprint to guarantee the quality of the camera-ready version. We have addressed these points:

1. Detailed and intuitive explanation of motivation, methods and algorithms (RW PucS, RW RneH). In our rebuttal and our revised version, we provide (i) insightful motivation and intuition, (ii) additional comparison with previous literature and (iii) detailed interpretation of the main algorithm, especially for audiences who are unfamiliar with causality or active sampling.

2. Experiments (RW kg3L, RW fVjN). We provide additional performance comparisons upon the dimension $d$, the size $n$, the network interference, etc., to check in which circumstance the method is more advantageous. These empirical results back up our theories and intuitions.

3. Mention some misleading statements and notations (RW kg3L, RW PucS, RW fVjN). Thanks for the meticulousness. Our improved version has made a few isolated definitions more transparent and intuitive.

If have any further questions, please don’t hesitate to contact us directly. We will respond as promptly as possible and do our utmost to address every concern thoroughly. Let’s enjoy this fruitful discussion together.

---

For RWkg3L: Thanks and please kindly find below our concise and clear rebuttal addressing your concerns.

Truly appreciate your strong endorsement! Here is our response:

Q1: the synthetic experimental design appears to be sound. In the experiments, there are several settings where CRA / GSW actually outperformed the proposed estimator. Can you provide more intuition or justifications as to why the proposed estimators tend to perform better or worse?

Thanks for your advice! More intuitions are as follows:

1. The advantage of our RWAS algorithm over traditional CRA/GSW methods is especially larger when the sample dimension $d$ is large. This is because we have proven, through bounding arguments, that the sample complexity of RWAS is relatively optimal during the active sampling process that achieves a (1+\epsilon) relative fitting error, which is O(d/\epsilon). As a result, when the sample dimension dd is relatively large, our error is minorer than that of other methods.

2. The data structure (i.e., whether there exists a subset with efficient global representational capability) plays an important role in comparing the performance of these algorithms. Compared to the GSW algorithm, our key differences are that

   • We adopt the IRD active sampling strategy (yielding a subset whose $X^\top X$ distribution closely mirrors the overall distribution), thus obtaining regression coefficients from $X$ to the individual treatment effect (ITE = Y(1)-Y(0)); and
   • We apply regression adjustment to GSW itself (leveraging the regression coefficients from the former step). Intuitively, suppose the covariate distribution of this subset closely approximates the overall distribution. In that case, (i) our regression-adjusted GSW will be more efficient, and (ii) our IRD will produce a smaller fitting error for estimating the overall ATE.

3. Motivated by your inspiration, we provide the following supplementary experiments on the performance differences as d varies, which aligns with the above intuition.

The results indicate that our RWAS methods dominate the previous methods, especially when $d$ is large, which is relevant in practice. The code is available at https://anonymous.4open.science/r/Causal-Active-Sample-B0C5/README.md.

Q2: Minor typo.

Sincerely, thanks for your careful review! We revise this topo from "Set-ting" to "Setting".