Global Well-posedness and Convergence Analysis of Score-based Generative Models via Sharp Lipschitz Estimates

In response to weakness part:

1. Our main Hessian bound result for smooth initial data, Theorem 3.5, is based on Assumption 2.3, which is a typical (though not strictly necessary) technical condition in the case of smooth initial data to ensure the existence of solutions up to time $t = 1$. Theorem 3.1 and the subsequent Corollary 3.2, similar to results in previous works as noted in Remark 3.3, are included for reader's convenience and serve as preparation for another key result in this paper, Theorem 3.4, which demonstrates the sharpness of the threshold time given in Theorem 3.1 and Corollary 3.2. That said, we agree with the referee that the phrase 'minimal general assumption' could be controversial. Therefore, we have removed the term 'minimal'.

2. Indeed well-posedness and convergence of numerical scheme would normally follow from the Lipschitz bound. However, there is a rewarding bonus from our work here. The convergence estimates for smooth initial data in Chen et al. (2023) rely on two key elements: (1) a short-time pointwise uniform Lipschitz bound, and (2) an average bound on the Hessian for longer times. In contrast, our approach leverages a pointwise Lipschitz bound that is local in space but global in time, and so is applicable to the more general case, including potential Wasserstein bound which is sketched in Section C.10.

In response to questions:

1.-3. Thanks for pointing out the typo. We have corrected it and conduct a deeper proofreading to fix the notations.

4. First we would point out that our Assumption 2.3 is just to assume the tail of the distribution is upper bounded by some Gaussian. For smooth initial data, Assumption 2.3 represents a typical (though not strictly necessary) condition to ensure the existence of solutions of Fokker-Planck equation of the forward process. We added a line before Assumption 2.3 to explain this.

5. Thanks for pointing out the notation problem. In the updated submission, we use $n$ to denote the total number of dimension.

6. We have rewritten Theorem 4.3 and provided definition (or related equation number) of the relating coefficients.

7. We agree that under the triangle inequality (Proposition A.3, or the one in Chen et al. (2023)) approach for KL bound, all complexity bounds with the same error level, will require $T=\mathcal{O}(\log n)$. So we have included a new Remark 4.4 after Theorem 4.3 to state the assumed regime of the near linear $\mathcal{O}(n\log n^2)$ complexity bound.

8. Indeed, we are working on the Lipschitz bound of the score function based on probability flow ODEs. New analytical tools, similar to Proposition A.3 or Theorem 2.1 of Chen er al. 2023, need to be developed for convergence and complexity bound. Should the reviewer point out other tools to this end, we are happy to cite them in our manuscript. We will complete this study in a future research. In the updated manuscript, we give a sketch of proof on Wasserstein distance bound in Section C.10 of the Appendix section based on the point-wise estimates obtained here.

Thanks for the careful reading, we have fixed the typo mentioned in the review and finished rounds of proofreading to fix the notations. Below we list our response to the questions.

1. This is a very reasonable question. First we would point out that our Assumption 2.3 is just to assume the tail of the distribution is upper bounded by some Gaussian, in particular, a distribution with bounded support follows such a assumption. For smooth initial data, Assumption 2.3 represents a typical (though not strictly necessary) condition to ensure the existence of solutions. We added a line before Assumption 2.3 to explain this.

In addition, non-smooth initial data are more common in real-world applications. However, the $O({1/t})$ blow up in (1) of inequalities (13) (Theorem 3.9 in the previous version) indicates that the denoising process can only be traced back to a certain $t > 0$, which essentially corresponds to a Gaussian-like initial data.

2. Thank you for pointing this out. Indeed, the assumption that "the density is smoothly defined on $D_0$" is not necessary. We only require that the initial distribution $p_0$ is a probability density supported on $D_0$, as can be seen from the proof. This assumption has been removed. Additionally, following the suggestion of another referee, we have chosen to state the estimates in (13) directly in the revised version rather than formulating them as a theorem.

1. For Theorem 3.1, we further clarify its connection to prior works (in Remark 3.3). Based on the referee’s suggestion, we add a sentence immediately before the statement of Theorem 3.1 to demonstrate that the associated time threshold is sharp (Theorem 3.4), which is new.

Regarding Theorem 3.9 in the previous version, we have followed the referee's advice by presenting it, in the revised version, as a standard result (16) rather than formulating it as a theorem. As discussed in our paper, the key focus is on whether these standard estimates can be further improved. We also refined the statements following (16) to emphasize two key points: (1) the $1/t$ gradient bound is generally sharp and cannot be improved, and (2) the Hessian bound can be improved from $1/t^2$ to $1/t$ for ``most" $x$. For the reader's convenience, we kept the proof of (16), as it is straightforward and some steps, such as (32) in the previous version, are used for the proof of Theorem 3.10 (Theorem 3.9 in the updated version).

Regarding Theorem 4.1, we noted in Remark 4.2 in the previous version that it is a special case of more general results (e.g., Chapter IV of Ikeda $\&$ Watanabe (2014)). Since uniform Lipschitz continuity is often assumed as a standard condition for long time existence of solutions for convenience, we believe it would be beneficial for readers, especially non-experts, to explicitly state this special case as a theorem and provide its proof, where uniform Lipschitz continuity may not hold. Thanks to the referee's suggestion, we have added several sentences just before the statement of Theorem 4.1 and in Remark 4.2 to emphasize that this is a special case of known results.

2. We apologize for the inconvenience during your reading, and have revised to list the assumptions in Section 2.2. Regarding whether those technical assumptions are reasonable, for smooth initial data, Assumption 2.3 represents a typical (though not strictly necessary) condition to ensure the existence of solutions. Also, if we assume $D^2g$ is bounded, $g$ grows at most quadratically and $Dg$ grows at most linearly. Hence it is reasonable to expect that $\alpha_1$ behaves similar to $|x_0|^2\sim O(n)$ and $\nabla g(x_0)$ behaves similar to $|x_0|\sim O(\sqrt{n})$. From the computational point of new, the main issue is to track dependence on the dimension $n$ that could be potentially very large.

3. The bound on line 1320 of the previous version is due to (14) of Theorem 3.5 in the revised version ( the same as (11) of Theorem 3.5 in the previous version) and the fact that $\max(a,b)\leq a+b$ after absorbing all other constants depending on $C_0$, $\alpha_2$ and $L$ but independent of $n$. We added this explanation in the revised version.

4. As mentioned in the paper, if the Hessian bound improvement ${1/t^2}\to {1/t}$ can be established for every $x$, then an improved convergence rate of the Wasserstein bound follows immediately from Theorem 3 of Bortoli (2022). Our result shows that this Hessian bound improvement typically holds except on a set of measure zero of $x$. Accordingly, the theoretical assumption of $O(1/t)$ Hessian bound is meaningful in most situations when addressing the convergence rate issue. We share the referee's view that it would be ideal if we could quantify rigorously how this small exceptional set (usually inevitable for non-convex data set) could impact the convergence rate. However, this is mathematically very challenging due to the complex topological structure of the exceptional set, which we plan to investigate in the future. A sentence is added in the revised version to clarify this point.

In response to Weaknesses

1. To the authors' best knowledge, we shall include two results in the updated Remark 4.6: 1) 'Nearly -Linear Convergence Bounds for Diffusion Models via Stochastic Localization' (2024) that provides a $d$-linear complexity bound in the early stopping setup; 2) a concurrent 'KL Convergence Guarantees for Score Diffusion Models under Minimal Data Assumptions' (2024) that provides a linear in dimension bound without early stopping.

2. We agree with the referee that the material in Subsection 2.2 is well-known to experts. Nevertheless, we believe it is more convenient for readers, particularly non-experts, to include it in the main text. A sentence will be added at the beginning of subsection 2.2 to clarify this point.

3. We shall provide a comparison to results in Chen et all. (2023) as Remark 4.6 in the revised version. We agree that we have a complexity bound slightly higher that Theorem 2.5 in Chen et all. (2023) under similar condition of initial data, although both of them are polynomial in dimension. On the other hand, since our new Hessian bound is homogeneous in time, our theorems apply to uniform discretization in time, which does not require any prior knowledge of the target distribution and hence is more realistic. In particular, we provide a sketch of proof about how to use those pointwise Hessian bound to obtain convergence rate under Wasserstein metric in whole space.

4. Thank you for the suggestion. We shall insert a paragraph at the beginning of the appendix summarizing the structure of the manuscript.