Interaction-Force Transport Gradient Flows

We thank the reviewer for writing a long and critical review. We must point out that the review is filled with misunderstandings, which we do our best to clarify below. We would appreciate it if the reviewer could please consider our clarification.

 This theory is full of remarkable, non-trivial, theoretical results. Transmitting all of this mathematical beauty into practical algorithms is praiseworthy.

Thank you. But such a positive assessment does not match your score.

(E) The practical validation is rather weak... ... visual performance of the method is somewhat disappointing... the method leaves a considerable number of points far from the support of target distribution... vanilla MMD flow, are better

We are deeply confused by this assessment. The only explanation we can think of is that the reviewer completely misunderstood the point of our experiment and intuition of unbalanced transport. We have clearly stated twice in the manuscript (Fig.3 & 4 captions), that color intensity indicates points’ weights and mention that weights vanish. The "points far from the support" the reviewer was referring to are likely (since we are confused by the review) the points of zero mass, hence the reviewer's comment is likely due to a major misunderstanding. Nonetheless, we will improve the presentation further to avoid misunderstandings. We will explicitly say: The hollow/white circles indicate the particles that have already vanished.

Please note: as the paper indicates, the performance of the IFT is overwhelmingly good. Hence the review comment does not make sense.

only a couple of 2D experiments

We have now included results in higher dimensions in the attached PDF.

strange that the paper develops theory of spherical IFT gradflows.. does not require this theory (Theorem 3.6) because spherical MMD (IFT) coincides with conventional MMD (IFT) flow

The reasoning seems flawed here. Our results such as Thm 3.6, precisely establish this special-case equivalence of SIFT and allow an easy implementation. Without our theory, there is no literature on the PDE of a mass-preserving flow in the MMD/IFT, then no implementation is possible.

The reviewer claimed our theory is "not needed" after she/he read our theory. Without this result, one cannot implement the mass-preserving flow. Furthermore, rigorously speaking, it is also incorrect to say two gradient flows are equivalent. Hence, the logic of the claim that our result is ”not needed” is not sound

[7] (not cited!)

We have now cited a subsequent/expanded work by the same group [Y. Mroueh and M. Rigotti] in the revised preprint. This paper covers and overlaps with the older paper you mentioned, we believe the coverage is now enough.

Furthermore, the paper you mentioned did not contain the PDE theory and principled gradient structure we uncovered. Hence, although related, it should not be used as an argument to undermine our contribution.

my skepticism extends to MMD gradflow in the joined Wasserstein-MMD geometry

Unfortunately, we failed to find sound reasoning for this "extension" of skepticism. We never claimed MMD-Mmd-Gf (which is not our focus) is superior. IFT is what we advocate and the Wasserstein contribution made the difference. The review simply cited an old paper related to but didn't theoretically study MMD-MMD-GF, let alone IFT, and claimed this "extends" their "skepticism" to IFT. We do not see any sound reasoning here.

Therefore, if we fairly solve MMD minimizing movement step, the resulting μℓ+1 will mix μℓ+12 and target π, i.e., the teleporting of mass will occur.

Sorry, we did not get what is the question or concern here. The statement is straightforward from our PDE/ODE characterization in Thm 3.6.

(a) What are the properties of inverse operator 𝐾1. In particular, why is it linear

Do we understand it correctly, that the reviewer is asking why is the inverse of a linear operator, linear? Please see also the first paragraph in Sec 2.2 for properties of the integral operator and references that can provide more basics of kernel methods and the integral operator. Or we didn't understand your question correctly?

(c) In the appendix, proofs. Why does the equality hold:

First, there is a typo that the inner products should be in unweighted L2 (thank you for helping us fix this), i.e.,

This correct relation follows from the textbook definition of the RKHS (and its norm). See standard text such as Cucker, F., & Zhou, D. X. (2007). Learning theory: an approximation theory viewpoint (Vol. 24).

substituted with a single step of project GD. What is the reason? How do such approximations affect theoretical and practical properties of the proposed method?  

In optimization, it is often desirable and significantly faster to perform inexact iterations to be more computationally efficient. That is the reason of our implementation. As our paper does not focus on the theory of the exact-inexact relation, hence it does not affect the existing analysis of the paper. In practice, we have observed such changes do not affect the performance significantly but improve the speed. Since eq (18) is a convex program, inexact iterations are preferred.

difficult to read...A lot of specific mathematical terms were used

First, we will consider your suggestions.

Our paper is a mathematically rigorous treatment, hence some mathematical terms are necessary. Plus, we have already opted for a minimal set of common terminologies for ML researchers on the topic. It also seems the other reviewers did not experience the same difficulty. Nonetheless, we will try to make the paper more accessible to non-experts (than it already is).

Dear reviewer,

Thank you for your feedback on our manuscript. We have carefully considered your comments and suggestions and have made the revisions. We have also included new experiments and done our best to answer your questions. The rebuttal took a tremendous amount of effort and we want to make sure it has been read.

As the discussion period will be closed soon, we kindly ask for your feedback on the rebuttal. Have we addressed your concerns? Is there anything else we can improve?

Thank you again for your time and effort.

Authors

Thank you for taking the time to carefully read and review the paper. We appreciate that you have noticed the many subtle features we put into the manuscript. Thank you for the kind summary in the "Strengths" section. Below, we respond to a few concerns and suggestions.

the roughness of the standard deviation bands in Figure 2 for the proposed method?

Thank you for the careful read and for noticing the details.

First, we believe the comment refers to the std band in Fig.2, which occurs around and below 10^{-3} magnitude. We suspect that could be due to that we reported in the y-axis in log scale. Then, it is expected the bands get rougher the lower they get. We observe that the same happens to the "MMD flow+noise" when its performance improves eventually (over more iterations); so it does not pertain to IFT.

this limitation is already mentioned briefly in the discussion, I feel that a bit more could be said about scalability of the presented algorithm. This does not necessarily have to be included in the main body of the paper, but rather in the appendix where the algorithm is presented.

Indeed, we agree that scalability is important and deserves a more detailed discussion, at least in the technical details in the appendix.

First, we note that our current implementation only looks at the batch case. Here, IFT performs very well as in, e.g., Fig.2. As we have larger data sets in higher dimensions, the vanilla MMD might be limiting.

Additionally, to speed up the implementation and hence scalability, we have reported that we perform a single step of projected gradient descent instead of computing a full solution to the optimization problem (18). This is effective as (18) is a convex program.

We note that we have not yet explored stochastic algorithms; we have focused on full-batch so far. Hence stochastic algorithms and parallel implementation could be future directions to improve scalability.

Since we are also interested in applications to generative models, there are possibilities such as deep-net features inside the kernels, as used in MMD-GAN works.

more examples/comparisons to the work of Arbel et al. [2019] in an appendix.

Thank you for the suggestion -- we agree. We have now added new experiments, whose results we report in the PDF:

• experiments in higher dimensions, vs Arbel et al. 2019;
• new experiments that uses the Wasserstein-Fiher-Rao flow of the MMD energy. To the best of our knowledge, this is the first implementation of the MMD energy in this flow;
• many miscellaneous improvements in terms of scalability and implementation, as suggested by the review. We will provide the details in the appendix of the revised manuscript as you suggested.

After considering all of the reviews and the authors' rebuttal, I am inclined to keep my rating as is. I appreciate the additional experiments provided in the PDF file as part of the rebuttal.

We thank the reviewer for the fair assessment and constructive suggestions. We are glad that the exposition of the paper is easy to follow -- thank you for this feedback. We have incorporated some of your suggestions and left more challenging ones for future work.

experiments run are relatively simple and low-dimensional, and it is not clear how practical

Your assessment is correct -- indeed our experiments are only a proof-of-concept. We have now included results in higher dimensions in the attached PDF. We also plan to investigate more realistic applications in the future.

numerical example, have you tried any heuristic approaches,

We appreciate your constructive suggestions.

First, we stated at the end of Sec.5 (L317-319) that there are simple improvements that can be made to improve the algorithm, and we deliberately left them out to keep the paper focused; almost playing handicapped. The IFT still outperforms the pure Wasserstein flow.

Second, thank you for the valuable suggestions. We believe they can definitely help with practical performance. As we reported in the paper, we actually let IFT run with a handicap (i.e. every IFT iteration counts as two iterations). We did so to show that IFT already has an advantage without further heuristic tricks. We have focused on implementing the WFR flow during the rebuttal to provide another new algorithm for comparison. We will further investigate your suggestions in the near future and comment in our Sec.5 if any of the tricks help.

comparison of behavior for KL

We agree that more experiments can strengthen the paper. A direct comparison between energy MMD vs KL is difficult, as we explain below. However, we did run the newly implemented WFR flow and reported them in the attached PDF.

First, We must note that KL minimization is NOT the topic of this paper: our focus is a type of (unbalanced-transport) gradient flows and optimization with applications to the MMD-minimization tasks. IFT is tailored to address the gap in the literature on "MMD flow". We also do not claim that MMD inference is superior to KL inference, while some other papers might do so. Furthermore, we look forward to applying our theory to generative models using MMD, e.g., [Galashov et al].

Nonetheless, we appreciate the reviewer raising this. We can think of two interpretations of the question:

1. KL energy with the proposed IFT flow: Recall that the MMD step is a discretization of the differential equation

One could view the update step as a step in the "kernel-mean-embedding space": let $e(\mu):=\operatorname{\mathcal{K}} \mu$ be the kernel mean embedding of $\mu$, then the update rule is

However, as there is no guarantee that the velocity $\log \frac{\mathrm{d}\mu^{\ell}}{\mathrm{d}\pi}-...$ to be in the RKHS. Hence, this step is theoretically interesting, but it's unclear how the infinite-dimensional update can be implemented in a principled way.

2. KL energy with the WFR flow: this case belongs to the "KL-inference" where one only has access to the score (or potentials) of the target e.g. $\nabla \log \pi$, instead of the "MMD-inference" (our paper, and Arbel et al., etc.) where we have access to the target $\pi$ via its samples but not the score $\nabla \log \pi$. Therefore, how to perform a "fair comparison" is unclear to us at the moment (e.g. how many samples from $\pi$ ? Noisy evaluation of the score?).

In summary, we agree that the reviewer's proposal is interesting and can see room for future work. At this moment, we can only leave comparing MMD vs KL as future work as they apply in different tasks and settings (score-based vs sample-based).

We have now implemented the WFR flow of MMD. It works only slightly worse than IFT but does not come with guarantees. See our overall rebuttal summary and the PDF.

Reference

Galashov A, de Bortoli V, Gretton A. Deep MMD Gradient Flow without adversarial training. arXiv; 2024

Dear reviewer,

Thank you for your feedback on our manuscript. We have carefully considered your comments and suggestions and have made the revisions. We have also included new experiments and done our best to answer your questions.

As the discussion period will be closed soon, we wish to ask for your feedback on the rebuttal kindly. Have we addressed your concerns? Is there anything else we can improve?

Thank you again for your time and effort!

Authors

Thank you for the constructive suggestions and the critical review. We have incorporated most of your suggestions, e.g., new experiments. We did find the one comparison you suggested with KL inference difficult, for which we explained the reason.

including experiments in dimensions larger than 𝑑=2

Done. The results (attached PDF) are consistent with the other experiments.

performing comparisons... (Yan et al., 2023, Lu et al., 2019)

We agree that more experiments can strengthen the paper. We have run new experiments and reported them in the attached PDF, such as WFR flows used by Yan et al. and Lu et al.

First, We must note that KL minimization is NOT the topic of this paper: our focus is a type of (unbalanced-transport) gradient flows and optimization with applications to the MMD-minimization tasks. IFT is tailored to address the gap in the literature on "MMD flow". We also do not claim that MMD inference is superior to KL inference, while some other papers might do so. Furthermore, we look forward to applying our theory to generative models using MMD, e.g., [Galashov et al].

Nonetheless, we appreciate the reviewer raising this. We can think of two interpretations of the question:

1. KL energy with the proposed IFT flow: Recall that the MMD step is a discretization of the differential equation

One could view the update step as a step in the "kernel-mean-embedding space": let $e(\mu):=\operatorname{\mathcal{K}} \mu$ be the kernel mean embedding

However, as there is no guarantee that the velocity $\log \frac{\mathrm{d}\mu^{\ell}}{\mathrm{d}\pi}-...$ to be in the RKHS. Hence, this step is theoretically interesting, but it's unclear how the infinite-dimensional update can be implemented in a principled way.

2. KL energy with the WFR flow [Yan & Lu et al.]: Their methods are adaptations of the original Wasserstein-Fisher-Rao flow. Their cases belong to the "KL-inference" where one only has access to the score (or potentials) of the target e.g. $\nabla \log \pi$, instead of the "MMD-inference" (our paper, and Arbel et al., etc.) where we have access to the target $\pi$ via its samples but not the score $\nabla \log \pi$. Therefore, how to perform a "fair comparison" is unclear to us at the moment (e.g. how many samples from $\pi$ ? Noisy evaluation of the score?).

In summary, we agree that the reviewer's proposal is interesting and can see room for future work. At this moment, we can only leave comparing MMD vs KL as future work as they apply in different tasks and settings (score-based vs sample-based).

We have now implemented the WFR flow of MMD. It works only slightly worse than IFT but does not come with guarantees. See our overall rebuttal summary and the PDF.

Proposition 3.8 immediately follows from two well-known facts... for the KL case, it is not a surprising property ...

We have already clearly stated that it is standard (L229-230). Neither did we claim it is surprising. It is also immediately evident that KL is not the focus of this paper, as can be seen from the shortness of Sec.3.3. Hence, we believe our current presentation is not likely to cause confusion. We emphasize that the discussion of KL is for completeness and to show that IFT enjoys the best of both worlds.

Bonus: Prop. 3.8 is not as trivial as it seems -- we just fixed an error in the current revision: the LSI only holds along the mass-preserving spherical flows (SIFT) over $\cal P$, but the flow over $\cal M ^+$. This technicality does not change ML applications, especially with MMD. But anyway, thanks for helping us notice this.

For the MMD case, the results are quite novel... other works... (Arbel et al., 2019). Thus, I am wondering, are the provided proofs of exponential convergence rates actually important...?

Thank you for the positive assessment. We will add a small discussion in the revision as per your comment. A sketch: (Arbel et al., 2019) does not contain (global) convergence analysis. For example, their Prop.2 states energy is non-increasing. However, this is not equivalent to convergence and is easily satisfied by many flows. The mathematical limitation to their "MMD flow" is that the MMD is in general not guaranteed to be convex along the Wasserstein geodesics. [Arbel et al.] also contains a heuristic noise injection procedure, albeit without good analysis characterization. In contrast, we believe establishing the first global (exponential) convergence in our paper is, needless to say, important. Not to mention our theorems/proof are clean and do not contain un-verifiable assumptions. Therefore, in our case, we have both practical performance and theoretical guarantees

Does your approach have some practical use cases?

This paper is mainly methodological and analytical, but we also value practical application. MMD flow applications in the literature mainly include, e.g.,

• image process, as done by [Hertrich et al.] and that research group at TU Berlin
• generative models (MMD GAN)
• MMD two-sample test

We also see some new directions of new deep generative models that are based on gradient flows of the MMD e.g. in [Galashov et al.]

Strengths: The paper provides the proof of...

We emphasize that the proofs are a part of but not the whole of our contributions, which also include the discovery of the IFT gradient structure, the implementable algorithms, PDE theory of the dissipation mechanism of the MMD and inf-convolution with Wasserstein (e.g. Prop 3.2, Thm 3.6). The work also advances the state-of-the-art understanding of "MMD gradient flow", which already has sizable literature.

Reference

Galashov A, de Bortoli V, Gretton A. Deep MMD Gradient Flow without adversarial training. arXiv; 2024

Dear reviewer,

Thank you for your feedback on our manuscript. We have carefully considered your comments and suggestions and have made the revisions. Per your suggestions, we have also included new experiments and done our best to answer your questions.

As the discussion period will be closed soon, we wish to ask for your feedback on the rebuttal kindly. Have we addressed your concerns? Is there anything else we can improve?

Thank you again for your time and effort.

Authors