Efficient Causal Decision Making with One-sided Feedback

Thanks very much for the insightful comment. Yes, $Y(0)$ is essentially treated as 0 as this is the natural way to define $Y(0)$ in the one-sided feedback problem such as bank loan. However, the decision making task with one-sided feedback is not just to predict the sign of $Y(1)$.

First, in the decision evaluation task, we aim to estimate the precise quantity of the value function $V_1(\pi)$. To this end, we have developed two different estimation strategies for binary and continuous outcomes.

Second, in the decision learning task, we demonstrate that the problem can be reformulated as a weighted classification problem (Proposition 5.3). Here, the label is $L={\mathbb{I}}\\{\widehat{\psi}(X,Y,A)>0\\}$ is based on whether an estimated value $\widehat{\psi}(X,Y,A)$, which approximates $\mathbb{E}\\{Y(1)\mid X\\}$, is positive. Additionally, we also need the estimated weights $W = |\widehat{\psi}(X,Y,A)|$, which requires estimating the quantity of $\mathbb{E}\\{Y(1)\mid X\\}$. This task is particularly challenging, as we can only observe $Y(1)$ for individuals who received action $A=1$, and without the no unmeasured confounders assumption, we do not have that $\mathbb{E}\\{Y(1)\mid X,A=1\\}=\mathbb{E}\\{Y(1)\mid X,A=0\\}$. To address this, we establish identification results and develop estimation strategies using shadow variables.

We sincerely appreciate your insightful and constructive comments. Below, we have provided our detailed, point-by-point responses.

Extension to non-binary action settings: Thanks for the insightful comment. Yes, the work can be naturally generalizes to multi-action settings. Consider the practical example of advertising: an advertiser observes user engagement (such as clicks) only when an ad is shown to the user. When no ad is displayed, the advertiser receives no information about whether the user might have clicked or engaged with the ads.

Suppose there are $K$ ads in total. Each ad $k\in\\{1, \dots, K\\}$ can be considered as an action, while the "no show" scenario can be treated as action $A=0$. In this case, the performance of ads (e.g., click-through rates) is only observed when an action is taken from $\\{1, \dots, K\\}$. No feedback is received when no ad is displayed ($A=0$). We can define the potential outcome for action $k$ as $Y(k)$, the decision rule as $\pi(x,k)=P(A=k\mid X=x)$ and the corresponding value function as $V(\pi)=\mathbb{E}\\{\sum_{k=1}^KY(k)\pi(X,k)\\}$. The identification and estimation strategies can be similarly established using shadow variables.

Optimality of the semiparametric efficiency bound $\Gamma(\pi)$: The semiparametric efficiency bound is defined as the supremum of the Cramer-Rao lower bounds for all parametric submodels. This bound describes how difficult it is to estimate the target estimand, namely the value function in our context, under the general semiparametric framework. A value estimator that achieves the semiparametric efficiency bound $\Gamma(\pi)$ is the most efficient estimator (with minimum asymptotic variance) within the class of semiparametric models.

Wider range of numerical experiments: In the real data application, we observe that the means of the proposed method’s values are higher than those of IPW-eff, while the variation is much smaller. The reason this improvement is less pronounced compared to synthetic scenarios is that the scale of the binary outcome is much smaller (the outcome is defined as 1 if the loan is repaid and -1 if the applicant defaults).

For the synthetic scenarios, we have conducted additional experiments with more complex outcome functions and treatment assignment mechanisms. Specifically, in Case 1, the potential outcome is generated by $Y(1) = 6X_1 - 3X_1^2 -3X_2 + 6X_3^2 + 3X_1X_2+3X_1X_3+1.5X_1X_2 +\epsilon$, where $\epsilon$ is generated from a normal distribution with mean 0 and standard deviation 0.5. The action $A$ is generated from $A\sim \text{Bernoulli}\\{\varphi(X,Y(1))\\}$, and $\text{logit}\\{ \varphi(X,Y(1))\\} =1/[1+\exp\{0.5-X_1-X_1X_2-0.1Y(1)\}]$. In Case 2, the potential outcome $Y(1)$ follows a Bernoulli distribution with probability of success $1/\{1+\exp(2X_1+X_2+X_3+X_1X_2+X_1X_3+2X_2X_3)\}$. The action $A$ is generated from $A\sim \text{Bernoulli} \\{\varphi(X,Y(1))\\}$, and $\text{logit}\\{ \varphi(X,Y(1))\\} = 1/[1+\exp\{-0.5-X_1+0.5X_1X_2-0.5Y(1)\}]$. The other settings are the same. We present decision evaluation results in Table 1 and Table 2.

Table 1: Simulation results for case 1: (a) $0.0\pi_d+1.0\pi_u$, (b) $0.3\pi_d+0.7\pi_u$, (c) $0.6\pi_d+0.4\pi_u$.

Thanks very much for the insightful comment.

Yes, opportunity cost is exactly reflected in our new value function. The value function $V_1(\pi)= \mathbb{E}\\{Y(1)\pi(X)\\}$ is defined to capture the outcomes of decisions under one-sided feedback, reflecting missed potential in cases where an action is not taken. Specifically, we define the potential outcome $Y(1)$ as the outcome if an individual were approved. If $Y(1)$ is positive, it indicates a profit opportunity, for instance, in a banking context where approving a loan could lead to financial gains. When the decision is to approve ($\pi(X)=1$), the profit is $Y(1)\pi(X)=Y(1)\times 1 = Y(1)$. However, if the loan is rejected ($\pi(X)=0$), the profit is $Y(1)\times 0 =0$. Thus, the opportunity loss is measured by $Y(1)-0$ under the rejection decision, which is exactly reflected in the value function because the value function becomes smaller with such a rejection decision. Therefore, the value function $V_1(\pi)= \mathbb{E}\\{Y(1)\pi(X)\\}$ directly incorporates this concept of opportunity cost, quantifying the outcome of both the action taken and the loss incurred from not taking it.

Beyond the banking example, this framework is relevant to many decision-making scenarios with one-sided feedback, such as hiring, admission and advertising. In the hiring/admission example, one-sided feedback arises because a decision-maker only observes candidates' actual performance if they decide to hire/admit them. If a candidate is rejected, the decision-maker receives no feedback $Y(1)$ about how that individual might have performed in the role. This one-sided feedback reflects opportunity cost in hiring/admission decisions, as the decision-maker misses good performance outcomes if strong performers are declined. In advertising, an advertiser only observes user engagement (such as clicks) if the ad is shown to the user. When the ad is not displayed, the advertiser receives no information $Y(1)$ about whether the user might have clicked on it or engaged with it. Thus, the outcome—whether the ad would have been effective—is only observed if the action (showing the ad) is taken. This one-sided feedback reflects opportunity cost as well: if the ad is not shown, the advertiser misses a potential click or engagement that could have led to a conversion or sale.

The inspiration for this value function stems from a need to reflect real-world decision-making processes with one-sided feedback. This is not merely a technical convenience; rather, it is based on a fundamental need to address opportunity costs explicitly. Hence, this value function is useful and interpretable in many practical applications where decisions must account for potential outcomes that are only observable under certain conditions.

We sincerely appreciate your insightful and constructive comments. Below, we have provided our detailed, point-by-point responses.

Questions:

1. The authors state that assumption 4.2 guarantees the uniqueness of $\phi(u,y)$. Can the authors explain how this assumption guarantees this? Also, is it a sufficient condition for uniqueness, or could there be necessary or more relaxed conditions?

Re: The observed distribution $w(x)=P(A=1\mid X=x)$ has the following relationship with $\varphi(u,y)$: $w^{-1}(x) = \int \frac{f(y|x,1)}{\varphi(u,y)}dy.$ Suppose there exist two different quantities $\varphi(u,y)$ and $\tilde{\varphi}(u,y)$ that satisfy the above equation. Then we have

$$\int\frac{f(y|x,1)}{\varphi(u,y)}dy=\int\frac{f(y|x,1)}{\tilde{\varphi}(u,y)}dy.$$

Let $h(u,y) = \frac{1}{\varphi(u,y)}- \frac{1}{\tilde{\varphi}(u,y)}$. Substituting this into the equation, we obtain:

$$\int h(u,y)f(y|x,1)dy=0.$$

By Assumption 4.2, it follows directly that $h(u,y)=\frac{1}{\varphi(u,y)}- \frac{1}{\tilde{\varphi}(u,y)}=0$ almost surely, and thus $\varphi(u,y)$ is unique. Please refer to Appendix A.1 for more detailed proof. This is a necessary condition for nonparametric identification of $\phi(u,y)$. However, when $\phi(u,y)$ belongs to a parametric/semiparametric model class, the completeness condition can be weakened. For example, Shao and Wang (2016) considered a semiparametric model for $\phi(u,y)$ and establish identification with relaxed assumptions. We will include detailed discussion in the revised version.

Reference: Shao, J. and Wang, L., 2016. Semiparametric inverse propensity weighting for nonignorable missing data. Biometrika, 103(1), pp.175-187.

2. Regarding assumption 4.2, it is worthwhile to discuss in the paper how restrictive this assumption is. Perhaps the authors could mention some cases where it does not hold.

Re: Thanks for the constructive comment. We will include a more detailed discussion by incorporating our responses to the weaknesses and Question 1 in the revised version. We believe this will offer a clearer and more straightforward interpretation of this assumption.

3. How do high-dimensional categorical covariates affect the estimation for the conditional expectations as required by assumption 5.1?

Re: Please refer to our response to the weaknesses.

4. The authors mentioned that the maximizer for the efficient estimator denoted by $\hat{\pi}$ is a natural estimator for the true optimal decision rule $\pi^*$, which is a maximizer of the true value function. Although in experiments, the authors show percentages of making correct decisions (PCD), can they provide any discussion on how far $\hat{\pi}$ can be from the true optimal rule $\pi^*$ in general?

Re: Thanks for the insightful question. Generally, the convergence rate of $\hat{\pi}$ to $\pi^*$ depends on the complexity of the decision rule class $\Pi$. For smooth parametric decision rules, $\hat{\pi}$ converges to $\pi^*$ at a rate of $n^{-1/2}$ (Wu and Wang, 2021). In contrast, for non-smooth parametric decision rules—such as $\mathbb{I}\\{\beta^T\theta(X)>0\\}$, where $\theta(X)$ represents a specific transformation of the covariates—the convergence rate is $n^{-1/3}$ (Wang et al. 2018). For nonparametric decision rules, the rate of convergence depends on the complexity of the decision rule class. We will include a detailed discussion of these points in the revised version.

Reference:

Wu, Y. and Wang, L., 2021. Resampling-based confidence intervals for model-free robust inference on optimal treatment regimes. Biometrics, 77(2), pp.465-476.

Wang, L., Zhou, Y., Song, R. and Sherwood, B., 2018. Quantile-optimal treatment regimes. Journal of the American Statistical Association, 113(523), pp.1243-1254.

We sincerely appreciate your insightful and constructive comments. Below, we have provided our detailed, point-by-point responses.

Weaknesses:

Shadow variable assumption: Shadow variables (SVs) can be identified through two main approaches. First, we can leverage expert knowledge in specific domains. For instance, in fairness-oriented hiring or admissions scenarios, sensitive attributes such as age and gender, which are unrelated to action assignments, can serve as SVs. Similarly, in medical treatment contexts, when the intervention is conveniently deliverable, factors like a patient’s access to transportation or distance to a healthcare center, which are unrelated to treatment assignment, can be considered SVs. Second, when we do not have prior expert knowledge, we can still automatically generate representations from observed covariates that serve the role of SVs, as demonstrated in Li et al. (2024). Therefore, SVs can be either selected or generated in practical settings, reducing the reliance on restrictive prior assumptions.

Conditional completeness assumption (Assumption 4.2): Assumption 4.2 is necessary to guarantee the nonparametric identification of $\phi(u, y)$. However, it is not restrictive in many practical scenarios. As discussed in the paper, when $Y(1)$ is binary, we only require $Z$ to be either continuous or a categorical variable with more than two levels. For cases where $Y(1)$ is continuous, the assumption holds if $f(y \mid x, 1)$ belongs to certain common distributions, such as exponential families and location-scale families. Hu and Shiu (2018) provides some other sufficient conditions for this assumption in different applications. We will include more detailed discussion of this assumption in the revised version.

Reference: Hu, Y. and Shiu, J.L., 2018. Nonparametric identification using instrumental variables: sufficient conditions for completeness. Econometric Theory, 34(3), pp.659-693.

Consistency of regression with high-dimensional discrete covariates: Regression with high-dimensional discrete covariates is indeed challenging, as converting these variables into numerical forms (e.g., one-hot encoding) can significantly increase the feature space. This often results in a sparse design matrix, particularly when most data points belong to a small subset of levels, complicating computation and optimization.

As discussed in the paper, our methodology supports the use of machine learning and deep learning models to fit the regression, allowing for various strategies to address these challenges:

Embeddings: Use learned representations from neural networks to represent categorical variables in a dense, lower-dimensional space. Level Clustering: Cluster levels based on similarity (e.g., frequency or behavior) to reduce cardinality. Tree-Based Models: Employ models like random forests or gradient boosting, which can handle categorical variables natively without explicit encoding. Regularization: Apply techniques such as Lasso, Ridge, or group lasso to prevent overfitting. Bayesian Hierarchical Models: Pool information across levels, improving estimation for rare categories. We will include this discussion in the revised version of the paper to address these considerations in greater detail.