Convergence Of Consistency Model With Multistep Sampling Under General Data Assumptions

1. Thank you very much for your suggestions. We have submitted a revision to resolve some of your comments.
2. $x_{t_i}$ vs $x_i$. Consistency model has both (a) a sequence of time steps for training, which goes from $0$ to $T$; (b) a sequence of time steps for inference, which decreases in the multi-step sampling process. For clarity, we directly use time $t$ as the subscript of $x$;
3. Could you please clarify the concerns on the scope of our work? We start with a general family of forward processes defined in Section 2 and present a sequence of theorems under this general setting. For illustration purposes, we instantiate our main result (the first theorem) with some VP and VE SDEs in the case studies and derive more interpretable upper bounds based on our main theorem. In fact, to the best of our knowledge, our results are much more general than previous works (Lyu et al. 2023, Li et al. 2024b, Dou et al. 2024) because they all focus only on VP SDEs but our results capture both VP and VE SDEs.
4. Please refer to Section G in the new version for empirical results.

I have read the author's answer, and I will increase my score as part of my concerns have been addressed.

However, I still find the practical contribution of the paper + the presentation (overall structure, ..) as detailed previously not clear. I understood your contributions (that you explicit again in your previous comment) but, for me, the impact of your paper seems limited as it is described now in the paper.

For 'actionnable' input, I already gave explanations in my review. I think that a lot of notations + the structure is confusing and sometimes shadows the key messages. I think the paper would benefit from a refactoring, focused on concise, sufficient and clear notations. Some reviewers seem to share my thoughts about the presentation.

For the new empirical study, I totally share eesS reviewer's opinion. I t should be included in the main text and better explained and introduced.

1. Our bounded support assumption is satisfied by the distribution of natural images because all images have finite pixels with ranges in $[0,1]$. The light tail and smoothness assumptions are mainly for theoretical purposes, both are common in theoretical works of diffusion models (Chen et al. 2023a, Chen et al. 2023b, Lee et al. 2023). Importantly, our results scale peacefully in those problem parameters.
2. Please refer to Section G in the new version for empirical results.

1. The RHS of eq (7) would be small for a non-trivial set of problems. RHS is controlled by: the consistency error $\epsilon_{cm}$ and $\frac{\alpha_{t_1}^2}{\sigma_{t_1}^2}$. $\epsilon_{cm}$ is small for reasonable consistency functions. For now, let's assume $\epsilon_{cm}=0$ and focus on the other term. $\frac{\alpha_{t_1}^2}{\sigma_{t_1}^2}$ characterizes the convergence of the forward process to the Gaussian distribution, which goes to $0$ quickly for reasonable forward SDE. Please refer to our first technical Lemma in Section F for this connection. Indeed, the success of diffusion models relies on fast convergence: we add sufficient noise so that the forward marginal distribution is close to some Gaussian; we then approximate the marginal distribution with Gaussian and run the reverse process to generate samples. In OU-process (Case study 1), $\frac{\alpha_{t_1}^2}{\sigma_{t_1}^2} \approx \exp(-2t_1)$ so $W_2 = O(R^{1.5}\exp(-t_1/2))$, which goes to $0$ exponentially fast as $t_1\to\infty$; in VE process (Case study 2), $\frac{\alpha_{t_1}^2}{\sigma_{t_1}^2} \approx t_1^{-2}$ so $W_2 = O(R^{1.5}T^{-0.5})$. This means the term $\frac{\alpha_{t_1}^2}{\sigma_{t_1}^2}$ is ignorable when we choose a proper forward SDE with a reasonably large $t_1$.
2. In Case study 1, we show that two-step sampling should improve the sampling quality, but additional sampling steps will have diminishing improvement based on our upper bound. This is consistent with the finding in (Luo et al. 2023). The optimal choice of $N$ may vary for different forward processes.
3. Please refer to Section G in the new version for simulation results.

Thank you for providing insightful feedback. The reviewer-author discussion period is ending soon. Could you please let us know if we have addressed all of your concerns? If there are no remaining issues, we would greatly appreciate it if you could consider adjusting your score accordingly.

1. Our main theorem requires two assumptions: (a) bounded support of data distribution: This assumption is satisfied by natural images because every image has finite pixels with value bounded in $[0,1]$; (b) the approximate self-consistency (Assumption 1): This assumption aligns with the loss function in the training process. The dependency on distribution $P_{\tau_i}$, the marginal distribution of the forward process, is fine since we can directly sample from it by adding noise to the image dataset. It's also common for theoretical works on diffusion models to make similar assumptions on the score functions;
2. Technical breakthroughs: (a) avoid Lipschitz condition on consistency function: one difficulty in translating the self-consistency property to sampling quality is how to account for the error of starting with Gaussian noise rather than the truth marginal distribution. Previous works (Lyu et al. 2023, Li et al. 2024b, Dou et al. 2024) assume Lipschitz condition on the ground truth consistency function. Lipschitz continuity can be used to analyze this error directly, but the unknown Lipschitz coefficient makes the result hard to interpret. We avoid making such assumption by using data-processing inequality; (b) use the chain rule of KL to analyze multi-step sampling: we need KL divergence on the starting distribution to apply data-processing inequality to avoid the Lipschitz assumption on the ground truth consistency function. However, W2 distance is a more natural quality metric for the generated data. It is challenging to translate between KL and W2 in the iterative process of multi-step sampling. To solve this issue, we apply the chain rule of KL to study how the mismatch (in terms of KL) in starting distribution changes during multi-step sampling and only translate KL to W2 in the last step; (c) general forward processes: previous works are limited to variance preserving forward processes, while we analyze a general family of forward processes that captures both variance preserving and variance exploding forward processes.
3. Please refer to Section G in the new version regarding empirical evaluation.
4. Please let us know if there are additional concerns about our assumptions or other aspects.

Thank you for providing insightful feedback. The reviewer-author discussion period is ending soon. Could you please let us know if we have addressed all of your concerns? If there are no remaining issues, we would greatly appreciate it if you could consider adjusting your score accordingly.