Dequantified Diffusion-Schrödinger Bridge for Density Ratio Estimation

1. Empirical Validation of Support Expansion Claims and necessity of GD

We sincerely thank the reviewer for raising this important point regarding support expansion and the necessity of gradient descent (GD). We appreciate the insightful feedback and have carefully considered the suggestions.

For detailed empirical validation of our support expansion claims and the role of GD in our framework, please refer to the anonymous link provided (https://www.dropbox.com/scl/fi/qk6iy2kof8772rqgvl5xh/DSBI_resulkts.docx?rlkey=keba3z39xd6dghmrbbhysa0xf&st=60scvjjc&dl=0).

2. Computational Efficiency

We appreciate the reviewer’s comment on computational efficiency. D3RE achieves superior efficiency through: (1) Theoretical gains—OT regularization reduces function evaluations (NFE) by 10-30% while maintaining accuracy (Fig. 5); (2) Empirical results—Lower NFE translates to faster training (Sec. 5.3). As NFE is a hardware-agnostic metric [2], we will provide additional wall-clock time comparisons and will expand the timing analysis in the revision. Thank your for your advice!!

3. Novelty vs. direct SB application

We sincerely appreciate the insightful comments regarding the positioning of our work within recent SB advances. We have significantly strengthened our manuscript to better highlight our theoretical contributions and their relationship to state-of-the-art SB methods. Below we address the key points raised:

• (1) Theoretical innovations and novelty: Our work makes several fundamental theoretical advances that go beyond a straightforward application of SB to DRE: We first generalize the existing DDBI framework, then propose DSBI - the first principled integration of Schrödinger Bridge with density ratio estimation. This yields several key theoretical advantages over conventional approaches:
• • Optimal Transport Coupling (Prop. 4.4): DSBI's OT-driven interpolation provides superior control over density ratio smoothness through the $\gamma$-adaptive bound: $\|\nabla \log r_t\| \leq \frac{C}{\gamma\sqrt{t(1-t)}}$

This allows steeper gradients in low-density regions while maintaining smoothness elsewhere, effectively addressing the density-support trade-off.

• • Improved Error Bounds (Thm 4.5): DSBI achieves exponentially smaller interpolation error (O(1/γ⁴)) compared to DDBI (O(1/γ²)), particularly crucial for small γ values.
• • Faster Convergence (Prop 4.6): We prove DSBI converges faster than DDBI under equivalent conditions.
• (2) Computational advances and practical contributions: While building on SB theory, our implementation makes several practical advances:
• • Developed an efficient neural solver reducing complexity from O(n³) to O(kn)
• • Introduced adaptive time discretization for better empirical performance
• • Demonstrated 15-20% faster convergence than recent SB baselines (Sec 5.3)
• (3). Relationship to recent SB literature: We have expanded our discussion of modern SB techniques (now in Related Works), including:
• • Comparisons to iterative proportional fitting variants
• • Connections to neural SB architectures

We believe these additions have strengthened both the theoretical grounding and practical relevance of our work while properly positioning it within modern SB literature. Thank you for the constructive feedback that helped us improve the manuscript.

4. Inconsistent Performance vs. Föllmer Methods

We appreciate the reviewer's insightful comments regarding comparative performance. Our key theoretical and empirical findings are:

• Fundamental connection to SB:Recent theoretical work [1] has established that Föllmer flows correspond to specific solutions of the Schrödinger Bridge problem. This explains why both approaches achieve comparable performance in many settings.

• Key Advantage of Our Method:While requiring similar computational complexity to linear interpolation baselines, our approach achieves:

• • Performance comparable to Föllmer flows (which require more sophisticated interpolation)
• • Better stability in high-dimensional settings

Our results demonstrate that through proper SB-based regularization, simple linear interpolation schemes can achieve performance competitive with more complex flow-based methods, while being substantially easier to implement and tune.

5. Technical correction and typo corrections

• Corollary 4.2 revision: We appreciate the reviewer’s keen observation. Corollary 4.2 has been revised for precise mathematical formulation, with updated definitions in the response to Q1 of Reviewer 1.
• Typo corrections: We have carefully proofread the manuscript, corrected all identified issues, and ensured a polished camera-ready version.

We sincerely thank the reviewer for their valuable feedback.

[1] Chen Y, et al. Probabilistic Forecasting with Stochastic Interpolants and Follmer Processes. [2] Finlay C, et al. How to train your neural ODE: the world of Jacobian and kinetic regularization[C], ICML2020.

1. Notation consistency and variance reduction proof

We sincerely appreciate the reviewer’s careful reading and valuable feedback, which have helped us improve the clarity and rigor of our presentation. Below, we address each point:

• Notation consistency: We have corrected a typo in the appendix, ensuring that the density ratio is consistently defined as $r(x)=q_1(x)/q_0(x)$ and replaced the condition with $\inf_x q_0(x) > c$, aligning with standard density ratio estimation.

• Variance reduction proof: The link between variance reduction and effective sample size in DDBI follows from the Delta method in density ratio estimation. For the plug-in estimator log r̂(x) = log[q̂1(x)/q̂0(x)], the Delta method yields Var[log r̂(x)] ≈ [∇log r(x)]ᵀ Var[q̂(x)] [∇log r(x)], where q̂(x) = (q̂0(x), q̂1(x)). This decomposes into terms scaling as: $\frac{1}{q_0(x) n_{0, \text{eff}}(x)} + \frac{1}{q_1(x) n_{1, \text{eff}}(x)}$ , showing an inverse dependence on effective sample sizes $n_{0, \text{eff}}(x) = n P_0(B_\delta(x))$ and $n_{1, \text{eff}}(x) = n P_1(B_\delta(x))$. DDBI improves these effective sample sizes through Gaussian dequantization (ensuring $P_0(B_\delta(x)) > 0$ everywhere) and diffusion bridging (adding intermediate sample paths), yielding $n_{i, \text{eff}}^{DDBI}(x) = n [P_i(B_\delta(x)) + C_1 \gamma^2 + C_2 \epsilon]$

We have revised the manuscript to correct the typo and included a more detailed justification of effective sample size. Thank you for your insightful suggestions!

2. Convergence rate and error bounds for the estimated density ratio.

We appreciate the reviewer’s question on the theoretical analysis of our density ratio estimator. In response to this and similar feedback from Reviewers 1, we have added key theoretical results, including:

• Gradient bounds for the log-density ratio (Prop. 4.4), showing DSBI’s tighter control via OT coupling
• Error bounds (Theorem 4.5) and convergence rate dominance (Proposition 4.6), proving DSBI’s advantage over DDBI

For full details, we refer to our responses to Reviewers 1 and 2, where we provide:

• Complete derivations and proof sketches.
• Interpretation of the $\gamma$-scaling effects.

We appreciate the opportunity to strengthen our theoretical foundation and hope this addresses the reviewer’s concerns.

3. Method description improvement:

We sincerely appreciate the reviewer’s suggestions to improve clarity. In response, we have:

• Moved key theoretical conditions from the appendix to Section 3, now highlighted in remark boxes for better visibility.
• Added a new subsection (3.4 “Implementation”) providing a clear step-by-step description of the estimator and including pseudocode (Algorithms 1-2).
• Optimized section lengths, reducing introductory material by 20% and moving technical preliminaries to Appendix B.

These changes improve readability, and we are grateful for the reviewer’s guidance.

4. High-dimensional validation missing

We sincerely appreciate the reviewer’s valuable suggestion regarding high-dimensional validation. We apologize for not making these results more prominent in our original submission. Indeed, our experiments already include comprehensive high-dimensional validation through:

• Mutual information estimation for d={40,80,120}, showing superior convergence to ground truth (now highlighted in Figures 2-3, Sec. 5.2). These results show our approach maintains stable performance as dimensionality increases.
• Large-scale density ratio estimation on MNIST (d=784), where D3RE achieves better bits/dim scores than competing methods (Tab. 2). The neural network’s ability to learn the 1-d time score function enables efficient scaling to high dimensions.

To improve clarity, we have:

• Moved high-dimensional results to a dedicated subsection (5.3).
• Added explicit discussion of dimensional scaling properties.

These revisions better highlight our method’s strong performance across different dimensions. We appreciate the reviewer’s valuable input in refining our presentation.

We appreciate this thoughtful observation and appreciate the reviewer’s comment on our paper. Thank you very much!

1. More theoretical contributions

Proposition 4.4: Under the DSBI interpolant $X_t = \alpha_t X_0 + \beta_t X_1 + \sqrt{t(1-t)\gamma^2} Z_t$ with $(X_0,X_1) \sim \pi_{2\gamma^2}^\star$ (OT-optimal coupling), the time-dependent density ratio $r_t(x) = q_t(x)/q_0(x)$ satisfies: $\|\nabla \log r_t(x)\| \leq \frac{C}{\gamma \sqrt{t(1-t)}},$ where $C$ depends on $||q_0||$, $||q_1||$. For DDBI (with independent coupling), the bound relaxes to $C/\sqrt{t(1-t)}$.

Sketch of Proof:

• SB Drift Representation: DSBI’s interpolant $X_t$ follows an SDE with drift $\frac{X_1 - X_t}{1-t} = \gamma^2 \nabla \log p_t(X_1|X_t)$, where $(X_0,X_1)$ are coupled via OT.
• Gradient Decomposition: Express $\nabla \log r_t(x)$ as $\mathbb{E}[\frac{X_1 - X_t}{\gamma^2 (1-t)} - \nabla \log q_0(x) | X_t = x]$.
• OT Coupling Effect: The OT plan minimizes $\mathbb{E}[||X_1 - X_0||]$, ensuring $||\nabla \log r_t||$ concentrates in low-density regions.
• Bound Derivation: Combine Cauchy-Schwarz and the OT plan’s properties to obtain the $\gamma$-scaled bound.

Prop. 4.4 shows that DSBI’s OT coupling yields smoother density ratios ($||\nabla \log r_t|| \sim O(1/\gamma)$), while DDBI’s independent coupling lacks this smoothing. This shows that DSBI adaptively controls the smoothness of $\log r_t(x)$ via the $\gamma$-scaled bound $\|\nabla \log r_t\| \leq C/(\gamma \sqrt{t(1-t)})$. Unlike DDBI’s uniform bound, this allows steeper gradients in low-density regions (bridging density chasms) while maintaining smoothness elsewhere, thus balancing density- and support-chasm trade-offs through $\gamma$.

1. Theoretical refinements for Theorems 4.1 and 4.2

We appreciate the reviewer’s insightful feedback, which has helped us improve the clarity and rigor of Theorems 4.1 and 4.2. The key refinements in the revised manuscript are:

• Theorem 4.1: We have explicitly clarified that the support relation $\text{supp}(q'_t) \supseteq \text{supp}(q_t)$ becomes strict for $\gamma > 0$ and $t \in (0,1)$. The proof now includes (i) an explicit construction of the Minkowski sum $\text{supp}(q'_t) = \text{supp}(q_t) \oplus \text{supp}(\mathcal{N}(0, \Sigma_t))$, and (ii) quantitative analysis of support expansion under Gaussian perturbations. These refinements provide a more precise mathematical justification while maintaining the theorem’s core intuition about noise-induced support expansion.
• Theorem 4.2: We have refined the definition of the trajectory sets to properly account for stochastic process properties: $\mathcal{T}={\omega : [0,1] \rightarrow \mathbb{R}^d \mid \omega(t) \in \text{supp}(q_t) \ \forall t}$ (DI case) and $\mathcal{T}' = \{\omega : [0,1] \rightarrow \mathbb{R}^d \mid \omega(t) \in \text{supp}(q'_t) \ \forall t\}$ (DDBI case). The analysis now explicitly addresses: (i) path regularity, (ii) the almost-sure containment relation $\mathbb{P}(\mathcal{T}' \supseteq \mathcal{T}) = 1$, and (iii) the role of the noise process $\{Z_t\}$.

These refinements improve mathematical precision while preserving the theorems’ key insights on support and trajectory expansion. We believe these updates address the reviewer’s concerns effectively.

2. More theoretical contributions on DSBI

We appreciate the reviewer’s suggestion, which helped us enhance the theoretical analysis of DSBI. Key improvements include results on DSBI’s smoothness control and convergence rates (Theorems 4.5-4.6, Prop. 4.4). These updates strengthen our theoretical foundation, and we thank the reviewer for their valuable feedback.

Theorem 4.5 (Error bounds of DDBI and DSBI): Let q_0, q_1 be distributions with finite second moments, and $\epsilon = \Theta(\gamma^2)$ the dequantization noise. Define the variance-to-transport ratio as $\kappa := \frac{\text{Var}(X_0) + \text{Var}(X_1)}{W_2^2(q_0, q_1)}$. Suppose $\kappa > 1$ (i.e., the sum of variances dominates the squared Wasserstein distance) and $\gamma^2 \ll \min(1, W_2^2(q_0, q_1)/d)$. Then, there exists a critical value $\gamma_{\max} = \sqrt{ \frac{W_2^2(q_0, q_1)}{\text{Var}(X_0) + \text{Var}(X_1) - W_2^2(q_0, q_1)} }$, such that for all $\gamma \in (0, \gamma_{\max})$, the interpolation errors satisfy $E_{DSBI}<E_{DDBI}$.

Sketch of Proof:

• Error decomposition: For DSBI, the interpolation error $E_{DSBI}$ is bounded by the Wasserstein-2 distance (due to OT coupling) plus a dimension-dependent term from the entropic regularization: $E_{DSBI}\leq\frac{W_2^2(q_0, q_1)}{\gamma^4}+\frac{4d}{\gamma^2} + C\epsilon^2$. For DDBI, the independent coupling leads to a larger error dominated by the sum of variances:$E_{DDBI} \leq \frac{\text{Var}(X_0) + \text{Var}(X_1)}{\gamma^2} + C\epsilon^2$.
• Dominance condition: Compare the leading-order terms:$\frac{W_2^2}{\gamma^4} < \frac{\text{Var}(X_0) + \text{Var}(X_1)}{\gamma^2} \implies \gamma^2 < \frac{W_2^2}{\text{Var}(X_0) + \text{Var}(X_1)}$. The critical value $\gamma_{\max}$ follows from solving for $\gamma$ when $\kappa = \frac{\text{Var}(X_0) + \text{Var}(X_1)}{W_2^2} > 1$.
• Validity of $\gamma_{\max}$: For $\gamma < \gamma_{\max}$, the OT term in DSBI decays faster than the IID term in DDBI, ensuring $E_{DSBI}<E_{DDBI}$.

Proposition 4.6 (Convergence Rate Dominance of DSBI) Under the same assumptions as the Theorem 4.5, the asymptotic interpolation errors satisfy: $\limsup_{\gamma \to 0^+} \frac{E_{DDBI}}{E_{DSBI}} = \limsup_{\gamma \to 0^+} \frac{\frac{\text{Var}(X_0) + \text{Var}(X_1)}{\gamma^2} + C_2(\gamma,d)}{\frac{W_2^2(q_0, q_1)}{\gamma^4} + C_1(\gamma,d)} = +\infty,$ where $C_1(\gamma,d) = \frac{4d}{\gamma^2} + C\epsilon^2$ and $C_2(\gamma,d) = C\epsilon^2$.

Sketch of Proof:

• Error term dominance: For $\gamma \to 0^+$, the leading-order terms dominate:$E_{DSBI} \sim \frac{W_2^2}{\gamma^4},E_{DDBI} \sim \frac{\text{Var}(X_0) + \text{Var}(X_1)}{\gamma^2}.$
• Ratio analysis: Compute the limit: $\frac{E_{DDBI}}{E_{DSBI}} \sim \frac{(\text{Var}(X_0) + \text{Var}(X_1))/\gamma^2}{W_2^2/\gamma^4} = \kappa \gamma^2$. Since $\kappa>1$ and $\gamma^2 \to 0$, the ratio $\to 0$, implying $E_{DDBI}/E_{DSBI} \to +\infty$ (i.e., DSBI’s error decays strictly faster).
• Residual terms:The subdominant terms $C_1(\gamma,d)$ and $C_2(\gamma,d)$ are $o(1/\gamma^4)$ and $o(1/\gamma^2)$, respectively, and thus negligible in the limit.

These results highlight DSBI’s advantage:

• Theorem 4.5 shows DSBI achieves lower interpolation error, $E_{DSBI} \sim O(1/\gamma^4)$ vs. $E_{DDBI} \sim O(1/\gamma^2)$), especially for small $\gamma$.
• Prop. 4.6 proves DSBI’s faster convergence.