From Deterministic to Probabilistic World: Balancing Enhanced Doubly Robust Learning for Debiased Recommendation

I think this paper can be further improved by addressing the following concerns.

• Empirical insights of advantages of probabilistic models over deterministic ones. Since this paper focuses on extending the deterministic setting to the probabilistic ones, a natural question arises: How do variants of PMF perform in comparison to their MF counterparts, especially when coupled with DR estimators? Thus, I wonder how MF with SOTA DR estimators perform, compared with PMF+BEDR.
• In the experiments, the base model is PMF. However, in practical scenarios, neural network-based MF models like NCF and graph neural network-based MF models such as LightGCN are widely adopted. Hence, it is crucial to incorporate empirical insights into the paper by considering alternative backbones like NCF, LightGCN, etc. This exploration would enrich the experimental findings and align the study more closely with real-world applications.
• Some typos can be fixed by further proof-reading. For example:
  • What is $\mathcal{O}$ in Lemma 1? Is it $\mathbf{O}$? If not, it is not defined.
  • lemma 1 -> Lemma 1, theorem 1 -> Theorem 1, etc.
  • Consider bolding Remark 1-3.

1. Motivation

• Why the randomness of learned propensities and imputed errors should be considered?
• I personally cannot understand why there is a correlation between the error of propensity scores and the error of imputed errors.
• How did you implement the random propensity scores and imputed error? A detailed explanation of this process is missing.
• Actually, the proposed empirical covariance term does not have any variance of propensity scores and imputed errors. It even can be computed with deterministic models.

2. Proof of Theorem 1

• I cannot understand some parts of the proof of Theorem 1.
• In the first line, why did you take the expectation of $e_{u,i}$? it is a deterministic value.
• In the second line, why did you use the double expectation formula? Over which variable is each expectation calculated? I think just one expectation is enough.
• This proof is over-complicated. You can just go to the final line from the second line. It is a basic mathematics E[AB] = E[A]E[B]+Cov[A,B].

3. Definition 1

• Cov[A,B] = E[AB] - E[A]E[B].
• However, in Definition 1, the term E[A]E[B] is missing.
• The missing $E[\frac{\hat{p}-o}{\hat{p}}] E[e-\hat{e}]$ becomes zero only when either propensity model or imputed errors are correct.
• However, in practice, these are inaccurate. Even if they are accurate, the covariance term is zero.

4. The balancing correction term

• $\tilde{e} = \hat{e} + \epsilon(o-\hat{p})$.
• In this term, the imputed error is corrected by the error of the propensity model. What is the intuitive concept of this term?
• The derivatives on the $\mathcal{L}_e^{Bal}$ wrt $\epsilon$ should have $\epsilon$ in itself since $\mathcal{L}_e^{Bal}$ has the square of $\epsilon$. I cannot follow this derivative.