Transfer Learning for Nonparametric Contextual Dynamic Pricing

We thank the reviewer for the detailed and helpful comments. The additional simulation results are given in this anonymized link. https://docs.google.com/document/d/e/2PACX-1vRBfGzJo3ETCltTWfOi_0p4RjLbsUJo6g9z0J-Ckm2m6fL0fahJWSrrptiFwOGCyxhtHNuyHsQP0tOh/pub

Q1 Our theoretical guarantees hold for any $d$. Like most nonparametric works, however, our approach does encounter curse of dimensionality in high-dimensional settings, which can be seen from its regret upper bound. To mitigate this, one can impose additional structures on the reward function $f(x,p)$, where $x=(x^{(1)}, ..., x^{(d)})\in \mathbb R^d$ is the context. For example, one can adopt a sparse additive model $f(x,p)=\sum_{i=1}^d g_i(x^{(i)}, p)$, with sparsity $s \ll d$ (i.e. most $g_i(\cdot,\cdot)$ are zero functions). In this setting, the effective dimension is $s$ rather than $d$, albeit at the potential cost of model misspecification. Following your comment, we will provide a detailed discussion on this aspect.

Q2 First, we note that the context $X$ can encode useful information such as product specifications and consumer features, thus allowing the revenue function $f(x,p)$ to remain roughly stable across different products conditioned on a rich $X=x$. For example, in our real data experiments in Section 5.2, loans differ in their terms (e.g. duration). The source and target data differ in market conditions (i.e. different US regions). Despite these discrepancies, our method still improves performance, showing its robustness. Second, we further test robustness of our method under posterior drift (i.e. when revenue functions are different between source and target). Following the setup of Configuration 1 in Scenario 1, we modify the true source revenue function as $f(x, p) = \theta_0' + x^{\top} \theta' p + \tilde{\theta}' p^2,$ where $\theta_0' \sim \mathcal{N}(\theta_0, \sigma^2)$, $\theta' \sim \mathcal{N}(\theta, \sigma^2 I_d)$, and $\tilde{\theta}' \sim \mathcal{N}(\tilde{\theta}, \sigma^2)$, with $\theta_0, \theta, \tilde{\theta}$ being the target revenue function parameters. We set $\sigma = r \cdot \min(|\theta_0|, \|\theta\|_\infty, |\tilde{\theta}|)$ and control the drift severity by varying $r$. Results (Figure 1 in the link above) show our method performs well and is robust under moderate posterior drift.

Q3 One key hyperparameter is the smallest exploration radius $\tilde{r}$. While the regret bound remains valid for any choice of $\tilde{r}$ as discussed in Remark 4.2, the optimality of our TLDP algorithm relies on the specific $\tilde{r}$ given in equation (13). That choice depends on two parameters often unknown in practice: the transfer exponent $\gamma$ and the exploration coefficient $\kappa$. In practice, $\kappa$ can be estimated from the source data, while $\gamma$ can be estimated by measuring the empirical overlap between the source and target covariate distributions using a small portion of the target data (e.g. $\sqrt{n_Q}$ samples). In our synthetic experiments, we used the true values of $\kappa$ and $\gamma$ to compute $\tilde{r}$. To assess robustness, we have conducted additional simulations under Configuration 1 in Scenario 1, using a list of mis-specified values of $(\kappa,\gamma)$, while keeping the true values fixed at $\kappa = 0.6$ and $\gamma = 1$. Results (Figure 2 in the link above) indicate that our method remains robust under moderate misspecification of these parameters.

Q4 As discussed in our response to your previous question, one limitation would be that to achieve optimal regret, our approach requires the knowledge of the unknown quantities $(\kappa, \gamma)$. It would be interesting to rigorously extend our method such that it is adaptive to the knowledge of $(\kappa, \gamma)$ and further establish theoretical guarantees for such an algorithm. In the revision, we will discuss more on this aspect of our limitation.

Q5 If the demand (thus revenue) function depends explicitly on time (e.g. seasonality, holiday, weekly effects), we can include time as an additional dimension in the context space. Our partition-based procedure then proceeds similarly and the theoretical guarantees still hold. However, if the revenue function $f(x,p)$ changes over time in a rather arbitrary fashion, we then need to rely on additional procedures such as change-point detection or moving-window estimation to counter the non-stationarity. We conjecture the regret upper bound will inevitably inflate in this case. As for inventory consideration, we conjecture one needs to reformulate the revenue maximization problem into a constrained optimization due to inventory limit. Based on deterministic approximation, Wang et al 2025 (on dynamic pricing with covariates) proposes a dual-based method for dynamic pricing under parametric models. We believe similar ideas can be used to help solve price $p_t$ in our Algorithm 1 under inventory constraint. We will comment on these aspects in the revision.

We thank the reviewer for the detailed and helpful comments. The additional simulation results are given in this anonymized link. https://docs.google.com/document/d/e/2PACX-1vRBfGzJo3ETCltTWfOi_0p4RjLbsUJo6g9z0J-Ckm2m6fL0fahJWSrrptiFwOGCyxhtHNuyHsQP0tOh/pub

W1) The classic $0$ or $p$ revenue model is a special case of our model. Formally, revenue $Y$ is $p\cdot D(x,p)$, where $D(x,p)$ is the random demand given context $x$ and price $p$ with $\mathbb E(D(x,p))=d(x,p)$. This leads to $\mathbb E(Y)=f(x, p) = p d(x, p)$. Thus we can directly model $f(x,p)$. When $D(x,p)$ is Bernoulli, it gives the 0 or $p$ revenue. However, demand $D(x,p)$ can be continuous (e.g. when sold by weight/volume like rice/gas). Also, we clarify in simulation scenario 2, we use [0,1]-truncated Gaussian centred at $f(x, p)$.

W2)&3) Covariate shift is a widely used scheme in transfer learning (Suk&Kpotufe2021, Cai et al 2024). The context $X$ can encode product specifications and consumer features, thus allowing revenue function to remain roughly stable across products or markets when conditioned on a rich $X$. In Section 5.2, loans differ in their terms (e.g. duration). The source and target data differ in market conditions (i.e. different US regions). Despite the discrepancies, our method still improves performance, showing its robustness.

We further test robustness under posterior drift. Following the setup of Configuration 1 in Scenario 1, we modify the true source revenue function as $f(x, p) = \theta_0' + x^{\top} \theta' p + \tilde{\theta}' p^2,$ where $\theta_0' \sim \mathcal{N}(\theta_0, \sigma^2)$, $\theta' \sim \mathcal{N}(\theta, \sigma^2 I_d)$, and $\tilde{\theta}' \sim \mathcal{N}(\tilde{\theta}, \sigma^2)$, with $\theta_0, \theta, \tilde{\theta}$ being the target revenue parameters. We set $\sigma = r \cdot \min(|\theta_0|, \|\theta\|_\infty, |\tilde{\theta}|)$ and control the drift severity by varying $r$. Results (Figure 1 in the link above) show our method performs well and is robust under moderate posterior drift.

Q1) The independence assumption (i.e. source data generated by a fixed behavior policy) is a standard assumption in bandit and RL literature, which simplifies theoretical analysis by enabling concentration inequalities for independent data. Our analysis can be extended to allow temporally dependent observations, such as those satisfying mixing conditions. Note that one must impose some structural conditions on the adaptivity of source data, as without them, data with arbitrary dependence can be uninformative. It would be interesting to allow source data to be collected via an adaptive policy. Doing so introduces additional complexities in terms of dependence structures and requires careful handling in the analysis to ensure valid results. We leave this for future research.

Q2) The constant $C_r$ is used to define the smallest exploration radius $\tilde{r}$, which intuitively controls a bias-variance trade-off: $\tilde{r}$ small enough to yield fine local estimates v.s. large enough to avoid excessive partitioning and exploration. We do not have a single optimal value for $C_r$ in closed form as it depends on unknown quantities (e.g. $c_Q$ and $c_{\gamma}$). However, any constant $C_r$ in a range that satisfies the bounding inequalities leads to optimal regret (in order $O(\cdot)$). This is inline with bandit and RL literature, where one often settles for an order-wise optimal choice rather than a numerically unique constant.

Q3) You are correct that $Q_X$ is a probability distribution, so Assumption 2.2 implies $c_Q\leq 1$. The constant 8 in $C_r^4c_{\gamma}c_Q\geq 8$ is merely sufficient for technical simplicity, and by no means necessary and sharp. While we conjecture that it may be technically possible to sharpen $8$ to some smaller constant to cover the choice of our experimental design, the magnitude of these constants does not affect the order of regret. In fact, our experiment shows our method is robust when the sufficient condition $C_r^4c_{\gamma}c_Q\geq 8$ is violated.

Q4) Misspecification of $\gamma$ and $\kappa$ affects the optimal choice of $\tilde{r}$, and consequently, the regret bound, as discussed in Remark 4.2. In practice, $\kappa$ can be estimated from source data, while $\gamma$ can be estimated by measuring the empirical overlap between the source and target covariate distributions using a small portion of target data (e.g., $\sqrt{n_{Q}}$ samples). In our experiment, we use true $\kappa$ and $\gamma$ to compute $\tilde{r}$. To assess robustness, we conduct additional simulations under Configuration 1 in Scenario 1, using a list of mis-specified values of $(\kappa, \gamma)$, while fixing true values at $\kappa = 0.6$ and $\gamma = 1$. Results (Figure 2 in the link above) indicate our method remains robust under moderate misspecification of these parameters.

We hope our responses help resolve your concerns and positively influence your evaluation.

We thank the reviewer for the detailed and insightful comments.

Comparing our work with Cai et al (2024). Cai et al (2024) study transfer learning for nonparametric contextual MAB under covariate shift. In their setting, actions (i.e. arms) are discrete and for simplicity are treated as a constant (i.e. the number of arms $K \asymp 1$). The simplification $K \asymp 1$ results in their regret bound losing explicit control of $K$, making their approach inapplicable to infinite or uncountable action spaces. In contrast, dynamic pricing presents a unique challenge due to its continuous action space (i.e. prices), requiring more intricate methodologies than that in Cai et al (2024). Our approach effectively addresses this by adaptively partitioning the joint covariate-price space. This adaptation ensures optimal regret scaling while accounting for both the covariates (of dimension $d$) and price (of dimension $1$). We will include the discussion in our revision.

Nonparametric assumption. By not assuming a specific functional form for the demand curve, a nonparametric approach allows for more adaptability in capturing complex relationships between price and demand, especially when the true underlying demand structure is unknown or difficult to specify a priori. Therefore, compared with parametric model, the nonparametric model offers greater flexibility, which motivates our work.

Partially linear models in Bu et al (2022). In terms of the partially linear model as you mentioned in Bu et al (2022), it covers two different demand models, $d(x,p)=bp+g(x)$ and $d(x,p)=f(p)+a^\top x$. For the first one, we believe that similar techniques developed in this paper, namely the adaptive exploration/partition of the covariate space, could be particularly useful for handling the nonparametric part $g(x)$ of the demand function and its transfer learning. On the other hand, due to the linearity in price, we conjecture a local linear demand model should be fitted within each bin to leverage the structure of the problem and achieve the optimal regret (instead of a local constant revenue function as in our paper). Since there is no interaction between $p$ and $x$ and there is linearity structure in $p$, we believe sharper rates than the ones in our paper can be achieved.

For the second one, we conjecture a different upper bound strategy other than our adaptive exploration/partition should be used to leverage the linearity in $a^\top x$ and the smoothness in the one-dimensional function $f(p)$. In addition, for the linear part, under the covariate shift framework, we believe that techniques developed in existing literature, such as those in Lei et al (2021) and He et al (2024), can be applied for its transfer learning.

A thorough investigation of transfer learning under the partially linear demand models is indeed an interesting avenue for future research. Following your suggestion, we will discuss these aspects in the revision.

References

[1] Cai, Changxiao, T. Tony Cai, and Hongzhe Li. "Transfer learning for contextual multi-armed bandits." The Annals of Statistics 52.1 (2024): 207-232.

[2] Bu, Jinzhi, David Simchi-Levi, and Chonghuan Wang. "Context-based dynamic pricing with partially linear demand model." Advances in Neural Information Processing Systems 35 (2022): 23780-23791.

[3] Lei, Qi, Wei Hu, and Jason Lee. Near-optimal linear regression under distribution shift. In International Conference on Machine Learning, pages 6164–6174. PMLR, 2021.

[4] He, Zelin, Ying Sun, and Runze Li. Transfusion: Covariate-shift robust transfer learning for high-dimensional regression. In International Conference on Artificial Intelligence and Statistics, pages 703–711. PMLR, 2024

Thank you for your appreciation, especially in acknowledging our presentation and our novelty.