Counterfactual Learning under Rank Preservation

This paper proposes a new identification condition, under which the individual counterfactual can be identified. The condition requires the rank of counterfactuals to be in accordance with the observed outcomes. The authors then provide an algorithm for learning the counterfactuals via empirical risk minimization. The proposed algorithm is evaluated on synthetic and real data.

Authors study counterfactual inference problem and propose a novel identification/estimation strategy, demonstrating its advantages through existing methods.

The paper proposes a new method for counterfactual estimation. The method needs slightly less stringent assumptions than existing approaches. Based on a novel convex loss function, the paper proposes a kernel-based counterfactual estimator and shows its unbiasedness.

The paper introduces a rank preservation assumption to identify counterfactual outcomes. Building on this foundation, it proposes a loss function that yields an unbiased estimator, and further develops a kernel-based estimator for empirical estimation.

The paper introduces a new family of assumptions for the identifiability of the joint distribution of potential outcomes. It is a relaxation of the assumption in some recent papers. Being familiar with monotonicity in other contexts (such as $Y_x \geq Y_{x'}$ for $x \geq x'$), I found it counter-intuitive, as it relies on monotonicity on error terms, instead of more common notions of fixing $U$ while being monotonic on the causes. Basically the point is that from one single observable outcome we can infer all potential outcomes (as in the additive error model where the error is shared by all potential outcomes). I found it intriguing, but currently I don't think the material is presented clearly enough.

One major source of confusion is that Definitions 4.1 and 4.5 are claims about statistics (i.e., functions of samples, not distributions). Equation 4.2 is about a distribution. I would revise the logic of these steps with care.

Line 147 is redundant: per line 23, $Y = f_Y(X, Z, U_Y)$ (I'm not sure where ``$U_X$'' comes from). There is no need to introduce $U_x$ and $U_{x'}$. The SCM is $Y_x = f_Y(x, Z, U_Y)$. I guess that this notation is inspired by consistency equations like $Y = XY_1 + (1 - X)Y_0$ interpreted as $Y = XU_1 + (1 - X)U_0$. But in a typical SCM framing this would just be represented as the two-dimensional error term $U_Y = (U_0, U_1)$ (I understand that a common misunderstanding is the belief that error terms should be scalars - in general they can be infinite-dimensional). This non-orthodox way of using the SCM notation was already a source of confusion to me... Assumption 3.1 feels redundant given a standard SCM notation. I guess it could be interpreted as $Y$ being a non-trivial function of all of the $U_Y$ variables for all values $x$ of $X$, but then it would be unclear what being monotonic with respect to a vector. I'd phrase it instead as "the error term can be summarized by a one-dimensional quantity" (e.g., in the additive error model we assume no interaction between the latent causes and the observed causes $Y = f_1(X, Z) + f_2(U_y)$, and hence we can just define $U' = f_2(U_y)$. The same trick is used by Balke and Pearl in their original work in instrumental variables, where they use $R$ to distinguish them from $U$.)

Given that, the point is that quantiles of the conditional distribution of $Y$ can then be mapped back-and-forth between potential outcomes. This is certainly more general than the additive error model, but it doesn't mean it's more satisfying. I can make sense of additivity being believable if additivity is OKish with respect to the observed variables (generalized additive models are used in applied sciences for several reasons, one of them being not particularly bad in some domains). It can also be falsified by checking for homoscedasticity of residuals. The generality of rank preservation may actually be less appealing, since it's a vacuous claim about possible structure. Any continuous conditional distribution can be parameterized as a monotone function of a $U(0, 1)$ random variable for instance. Moving from Assumption 3.2 to 4.2 requires appreciation for even more fine-grained differences among unfalsifiable models.

It is true that even though $U_Y$ is potentially infinite-dimensional, we can fold real infinite-dimensional spaces into the real line $\mathbb R$ because they have the same cardinality, but in general we would lose the smoothness $f_Y$. That's why we can have outcomes of discrete treatments modeled with a single "error term", the structural equations not being smooth anymore. This makes the interpretation of the discrete case also harder to understand: it is lacking an interpretation of when it is sensible, and a clear contextualization of when it fails. Without these, appealing to being more general than previous papers is not sufficient. My take is that a practitioner might as well feel free to also ignore those papers. There wasn't much in the discussion or paper on trying to engage with these questions. The mainstream cross-world counterfactual constraint assumptions (e.g. $Y_x \geq Y_{x'}$ for $x \geq x'$; or additive errors) are usually argued for by proposing possible mechanisms (e.g., treatments don't harm individuals, interactions among groups of variables is weak/non-existent). The cited Xie et al. paper provides some of the insights, but e.g. for additive error models $Y = f_1(X, Z) + f_2(U)$ there is no need to start from considering something special about the functional shape of $f_2(\cdot)$, it's the notion of (lack of) interaction that is doing the heavy lifting. So even there the picture is somewhat incomplete. In this regard, it is less clear which mechanistic insight the current version of the paper is adding on top of it.

Minor:

• $Y_x = f_Y(x, z, U_x)$ is not a clear piece of notation. It seems to suggest both $x$ and $z$ being fixed. Should it be instead $Y_x = f_Y(x, Z, U_x)$ or $Y_{xz} = f_Y(x, z, U_Y)$?

Reject