Flow Matching for One-Step Sampling

Thank you for the review. Here are our responses to your comments.

(1) there are no comparisons to other flow matching approaches or generative models (diffusion, consistency models) provided (2) The evaluations are only done on a simplistic 2D dataset and the color transfer task, where I have difficulty assessing how good the result really is

Thank you for the comment. Our approach offers a significant advantage by generating results in a single step, without relying on the computationally expensive and complex processes of ODE or SDE solvers, nor does it involve distillation techniques. This sets it apart from existing methods that depend on these solutions.

(3) No evaluation and comparison of the computational efficiency of the generative model is offered (which was the main motivation of the approach). Its unclear to me how the single-step sampling here actually works? We still have the solve the ODE, don't we?

We have to solve ODE only in the training process to get couplings, in the sampling process we do not need to solve ODE. In the sampling step, we simply feed a random point that was obtained by sampling from the original Gaussian distribution $\rho_0$ to the input of the neural network and get a new image as output.

(4) Theory is also quite hand-wavy. For example, it is unclear to me how importance sampling is used to estimate the integral of Eq (1). Which distributions are replaced here ? I.e., what is the sampling distribution?

Thank you for your comment. We aimed to avoid overwhelming the reader with an overly theoretical tone, focusing on providing only the essential concepts to maintain clarity. Many of the details regarding importance sampling were already thoroughly discussed in the Ryzhakov et al. paper, where the exact formula is provided. However, we will revise the text to avoid any potential misunderstandings. Specifically, the formula (1) for the exact velocity involves integrals where the integrand is multiplied by an unknown density, $\rho_1$, the explicit form of which is not known but can be sampled from. This scenario is a typical application of importance sampling. It is important to note that since we must evaluate the integral in the denominator, this evaluation may be biased, leading to self-normalized importance sampling (SIS). To mitigate this issue, rejection sampling can be employed as an alternative to SIS, as outlined in the referenced article. We include this clarification in the revised version of the text.

(5) It is unclear to me how tolerance and sigma interact and why we need sigma. From the presented theory, sigma would not be needed, would it? But without sigma, only very poor prototypes are learned, so this seems to be unsatisfactory to me

The non-zero sigma makes the conditional map invertible. This is extremely important in our setup, as we solve the inverse ODE. If we immediately set sigma to zero, then even exact prototypes will not fill the entire $\rho_0$ distribution, since real data lie on a manifold whose dimensionality is less than the dimensionality of the space. In this case, our model will not know how to behave at points where learning is fundamentally impossible. Thus, we have to artificially add sigma and noise with the same variance so that the prototypes we get lie in the whole space. Note, however, that the smaller the sigma, the more accurate, theoretically, our solution becomes (the prototypes become closer to the exact prototypes). But with small sigma we have to take a smaller parameter ``tol'' to solve ODEs, which in turn affects the speed of the algorithm. Thus, here we have a tradeoff between accuracy and speed.

Let us answer your questions:

comparisons to other flow matching and consistency models (As they have the same goal) is needed. Add more experiments for more complex target distributions.Add more detail for the color tansfer task and compare to other methods (also qualitatitively)

We would greatly appreciate any guidance on appropriate metrics for numerical comparison, particularly for the color transfer task, as, to the best of our knowledge, no existing method sufficiently measures the quality of color transfer.

We appreciate your review. Let's discuss your specific comments.

More elaboration on the method of Ryzhakov would be helpful.

Thank you for the comment. We provide a concise overview of the main ideas of the Ryzhakov method in the introduction to maintain the paper's readability. A more detailed mathematical exposition is presented in Section 3.

Unclear about motivation and comparison to related works.

Our motivation is that one step allows one to much faster generate images, without numerically solving ODE or PDE. We write this in the introduction section. We are eager to solve the problem of long sampling process to be able to generate huge amount of new data much faster. We expanded the “Introduction” section to focus more on the problem statement and limitation of our method.

Lack of numerical results.

We would appreciate any suggestions from the Reviewer regarding suitable quality metrics for 2D cases and image generation. However, we believe that the ability to perform one-step sampling without relying on heuristic techniques such as distillation is a significant advantage of our method, as demonstrated by the promising MNIST results included in the updated PDF.

Let us answer your questions:

Where exactly is the convexity in the problem?

Since one-step generation in our case is analogous to straight trajectories in classical Flow Matching when solving ODEs, we expect that these trajectories would not overlap for all possible model inputs. This could potentially imply the monotonicity of our model, suggesting that it might be the gradient of a convex function. However, as we have not delved deeper into this aspect, we have removed the mention of convexity from the text. Thank you for your insightful comment.

How does this method compare to related work?

While direct quantitative comparisons are challenging due to the absence of established metrics for 2D data and MNIST datasets, we believe our method's ability to perform one-step generation without an ODE solver is a significant advantage. We would be grateful to receive any feedback regarding suitable metrics for evaluating these specific data.

We value your feedback. We will now address your comments in detail.

The experiments described in the paper are extremely limited. It is shown that 8 Gaussians can be generated using the proposed sampling method...

Direct quantitative comparisons are difficult to make because there are no standard metrics for evaluating 2D data or MNIST dataset. Nevertheless, we argue that a key strength of our approach lies in its ability to generate results in a single step, without the need for an ODE or SDE solver. This sets our method apart from many others, which depends on such solvers. Moreover, our method does not rely on distillation techniques. We would appreciate any suggestions for appropriate metrics to assess performance on these particular datasets. We have added MNIST experiments, rewritten the color transfer experiments and included in the new PDF file.

It is rather unclear what a prototype is, whether it is the point in $\rho_0$ one would converge to if there are no errors in ODE solver, or whether a prototype can be any point in $\rho_1$ that you happen to converge to...

We thank the reviewer for the comment. We would be grateful for more specific feedback from the reviewer regarding any aspects of the prototype generation process that may be unclear or require further elaboration.

We have added a footnote to the Introduction to further clarify this point. Additionally, we have expanded our answer about prototypes in the subsequent discussion below. We'll make a deeper investigation into the properties of prototypes in a future research.

When prototypes are found in $\rho_0$, noise is first added to the position in $\rho_1$, but the motivation for this is hardly satisfactory...

The non-zero sigma makes the conditional map invertible. This is extremely important in our setup, as we solve the inverse ODE. If we immediately set sigma to zero, then even exact prototypes will not fill the entire $\rho_0$ distribution, since real data lie on a manifold whose dimensionality is less than the dimensionality of the space. In this case, our model will not know how to behave at points where learning is fundamentally impossible. Thus, we have to artificially add sigma and noise with the same variance so that the prototypes we get lie in the whole space. Note, however, that the smaller the sigma, the more accurate, theoretically, our solution becomes (the prototypes become closer to the exact prototypes). But with small sigma we have to take a smaller parameter ``tol'' to solve ODEs, which in turn affects the speed of the algorithm. Thus, here we have a tradeoff between accuracy and speed.

The language of the paper is unfortunately rather problematic with numerous missing articles (the, a), improper prepositions, and awkward sentences that are hard to interpret. However, the problems are not too severe for the material to be understood and should be relatively easy to correct in a final version of the paper.

Thank you for the comment, we carefully read the text and corrected found errors, rewrote some parts for better understanding.

The two algorithms are very similar to each other. The only difference seems to be that if you have a discrete label, a separate mapping is learned for each value of the label. It would have been sufficient to just keep Algorithm 2.

To maintain clarity and simplicity, we have presented the first algorithm in its core form. It directly highlights the fundamental aspects of our method and can be readily applied in practical settings.

The notation for buffer B varies in different parts of the text.

We thank the reviewer for pointing out the notational error. We have corrected this in the revised paper.

Let us answer your questions:

What is a prototype? How would a prototype be defined? For each point in $\rho_1$ are there many potential prototypes?

Thank you for the question. Mathematically, we define prototype as $X_0(x_1)\in\mathcal{R}^d$, which in practice is just a point that we get after solving the ODE. Yes, each point may have many prototypes and the prototype of the given point depends on the tolerance of the ODE solver. We use the phrase ``exact prototype'' when we mean the point in $\rho_0$ we get to if there are no errors in ODE solver and if we know $\rho_1$ exactly.

Does the $\sigma$ in (1) have anything to do with the $\sigma$ in (3)? Isn’t the $\sigma$ in (1) necessary to make the mapping invertible, while the $\sigma$ in (3) is related to the tolerance?

If we use invertible map, then under conditional map the initial Gaussian $\rho_0$ will move to the Gaussian distribution located near the point $x_1$ with variance $\sigma$. To model the inverse situation, we add a noise with the same variance to the points from the target distribution. Thus, $\sigma$ in (1) and the variance of $\epsilon_l$ in (3) are the same.

Why is no noise added in (4), just like in (3)?

Thank you for your comment, it was a typo, we modified the text.

Thank you for your review. We will now respond to your comments.

Language: there are many spelling and grammatical errors, and the sentences are sometimes incomprehensible. Especially in the introduction, the main ideas are hard to comprehend, especially if one does not know the previous work in detail. The experiment in Section 4.2 should be explained in more detail or a reference needs to be added if it is a common method for image colorization. Line 114: there is no "Introduction and Related Work" section, these are two separate sections in the paper.

We thank the reviewer for the comments. We have carefully revised the paper to address the identified issues, including correcting grammatical errors and rewriting line 114 for improved clarity. We have also tried to provide a comprehensive overview of the necessary background information in the preliminaries and introduction sections. To maintain readability, we have avoided overly detailed mathematical explanations and paper overviews, as these are already covered in the cited works. We would be grateful for specific feedback on any particular areas that require further elaboration.

We have improved the Experiments section, particularly regarding the color transfer task. To our knowledge, there are currently no established metrics or common approaches for evaluating color transfer performance. We believe that our method's ability to perform one-step sampling without relying on distillation, a heuristic technique, is a significant advantage. This is further supported by the promising MNIST results included in the updated PDF.

Let us answer your questions:

Sec 3.3: what does the >>tol<< parameter have to do with the generation? Why and how does it affect the shape of the base distribution?

We thank the reviewer for the question. The tolerance parameter (tol) in our work controls the accuracy of the numerical ODE solver employed to find pairs of points from different distributions. In essence, the ODE solutions serve as prototypes for target points. Figure 1 visualizes the target points and their corresponding prototypes obtained through the ODE solver, rather than the generated points themselves. Poor ODE solver parameters (tol, $\sigma$) lead to inaccurate prototypes, which in turn result in poor training data and ultimately poor generation quality. In the inference step, i.e., image generation, tol is not used since we are not solving ODEs.

Page 3, footnote 2: "up to a constant": Is it a constant factor or a constant term?

Up to a constant factor, i.e., not normalized. We added the clarification to the text.