Invariant Deep Uplift Modeling for Incentive Assignment in Online Marketing via Probability of Necessity and Sufficiency

Dear Reviewer UCNa,

Thank you for taking the time to review our work, we sincerely appreciate your insightful comments, which have helped improve our paper. Below, we provide point-by-point responses to each concern raised.

1. Mathematical Formulations: We appreciate the valuable feedback and have incorporated a comprehensive notation table in the appendix to improve clarity. In our final revision, we will further enhance these explanations to ensure better understanding of our methodology.

2. Computational Complexity Analysis: We analyze the computational complexity of each module in our framework:

Invariant Property Learning Module

• Objective loss and negative log-likelihood: $O(n)$ (linear complexity)
• KL divergence: $O(n \times d)$, where $d$ is the feature dimensionality

Feature Selection Module

• Forward propagation (linear transformation + batch normalization): $O(n \times h)$, where $h$ is hidden layer dimension
• Mask generation (Gumbel-Softmax): $O(k \times m)$ for $k$ iterations on $m$ mask elements
• Total complexity: $O(n \times h + k \times m)$

Discrepancy Balancing Module

• IPM with MMD distance: $O(n^2)$

The overall time complexity of IDUM is $O(n^2) + O(n \times h) + O(k \times m)$.

Empirical Runtime Comparison

We compare the computational efficiency of different methods on two datasets:

If you have any further questions to discuss, we are willing to reply as soon as possible.

Dear Reviewer 4KLk,

Thank you for taking the time to review our work, we sincerely appreciate your insightful comments, which have helped improve our paper. Below, we provide point-by-point responses to each concern raised.

Cons

1. Open-source code: We have included our code in the Supplementary Material for review. Upon acceptance of the paper, we will open-source all code and datasets to ensure reproducibility, and will provide the GitHub link in the final version.

2. Mathematical formulation: Additionally, we have added a notation table in the appendix for clarity, and will further expand the explanations in our final revision to enhance understanding.

If you have any further questions to discuss, we are willing to reply as soon as possible.

Dear Reviewer xxdc,

Thank you for taking the time to review our work, we sincerely appreciate your insightful comments, which have helped improve our paper. Below, we provide point-by-point responses to each concern raised.

Methods And Evaluation Criteria:

We will follow your advice to include the the detailed definitions of AUUC and Qini coefficients and revise the presentation in our final version.

Weaknesses:

1. Three components are derived from previous research

Our main contribution is to solve the OOD problem in uplift modeling by introducing the PNS risk, which has been not fully investigated by previous works.

In our work, we build upon existing methods to construct our model architecture. However, due to differences in application scenarios, additional components are required to address the problem effectively. For OOD uplift modeling, we account for the in-distribution shift between treatment and control groups by employing a discrepancy regularizer—a common solution that is not our key contribution. To address the PNS risk, we introduce a novel intervention and prediction procedure for balanced embedding learning, which distinguishes our approach from prior work. However, we observed that the computational cost of PNS-based feature optimization was prohibitively high. To mitigate this, we adopted a Gumbel-Softmax-based feature selection mechanism.

The theoretical foundations of our work are primarily based on Pearl's causal framework, with additional insights adapted from Yang et al. (2024) to enhance rigor. However, our approach is not a direct application of their theorems, but rather a tailored extension for uplift modeling. First, our focus differs fundamentally from Yang et al. (2024): while they address image classification, we tackle uplift estimation with dual prediction heads. These domains differ critically in objectives and assumptions—for instance, uplift modeling requires counterfactual reasoning absent in standard classification. This divergence necessitates theoretical adaptations. Second, we explicitly model the interaction between prediction heads and balanced embeddings within our theoretical framework, which is uniquely designed for our problem setting. Finally, all prior works have been properly cited to maintain academic norms and contextualize our contributions.

2. Online experiments.

Following your suggestions, we conducted an online experiment to evaluate our proposed IDUM against the baseline CFRNet. Due to the limited time of rebuttal and the cost of algorithm deployment, we only keep it for three days.

Experimental Setup:

In-Distribution (ID) Test: We selected two user groups with similar distributions to the training data.

Out-of-Distribution (OOD) Test: We used two user groups with distinct distributions to assess generalization.

Evaluation Protocol:

Due to the constraints of online inference, we could not obtain the true responses for users unaffected by the deployed algorithm. Instead, we compared the watch time improvement across experimental groups as the key metric.

• Watch time evaluation | Method | ID | OOD | |---------|-------|--------| | CFRNet | 0 | 0 | | IDUM | 0.012%| 0.028% |

• Cost evaluation | Method | ID | OOD | |---------|-------|--------| | CFRNet | 0 | 0 | | IDUM | -1.21%| -1.75% |

We report the percentage of watch time improvement relative to CFRNet's performance (baseline, set at 0). And also, we evaluate the cost of two methods, for that we assign less incentives (-1.21% and -1.75%) to the user group and get competitive performance (0.012% and 0.028%).

The results demonstrate that our proposed IDUM achieves further gains in the OOD testing environment.

Questions

1. clarify the settings of the Lazada dataset

The Lazada dataset is a public dataset for uplift modeling, which can be found at: https://drive.google.com/file/d/19iSXsbRXJWvuSFHdcLb0Vi9JCP9Fu41s/view

In the dataset description, there are 86 fields in total, of which 83 are features(f0-f82). We use the column of is_treat as treatment $t$, and the column label as the response $y$. Usually, for uplift modeling, $t$ represents whether the user is assigned the discount or coupon, $y$ represents whether the user is converted.

2. The variance

We apologize for the error in reporting the variance. As you pointed out, the variance was incorrectly magnified by a factor of 10. We will correct this in the final version of our paper. Additionally, we plan to release the code, dataset, and training logs upon obtaining permission from our collaborating company, and the variance can be identified in the training logs. Thank you for your careful attention to this issue.

If you have any further questions to discuss, we are willing to reply as soon as possible.

Dear Reviewer UDxr,

Thank you for taking the time to review our work, we sincerely appreciate your insightful comments, which have helped improve our paper. Below, we provide point-by-point responses to each concern raised.

Weaknesses:

1. Additional experiments:

Due to the inherent constraints of our model's design, we are unable to generalize our approach to domains with significantly different characteristics within the rebuttal period. However, in response to your feedback, we have conducted additional experiments on the Criteo dataset (ad click-through rate prediction) to validate our method's ability.

Training Data: 80% (in-distribution)

ID Test Data: 10% (in-distribution)

OOD Test Data: 10% (unbiased sampling)

This partition allows us to evaluate both in-distribution performance and out-of-distribution generalization.

2. Gumble softmax features:

Both the Lazada dataset (with over 80 features) and the Production dataset (with over 100 features) exhibit high dimensionality. To address the computational challenges posed by such large feature sets, we employ Gumbel-Softmax-based feature selection, which effectively reduces computational costs while preserving predictive performance.

While the Lazada dataset lacks feature interpretability, our analysis of the Production dataset reveals that certain features—such as Phone_type, Region, Silcon, and Age, etc., are consistently retained across selections. This suggests their importance in modeling uplift across diverse scenarios.

If you have any further questions to discuss, we are willing to reply as soon as possible.