Piecewise deterministic generative models

Thank you for your thorough review and the time you have taken to provide valuable feedback on our paper. Below are our responses to your comments.

• Regarding the first two points mentioned in the Weaknesses

  We agree with the remarks and we will add clearer discussions on the limitations of our models in the next version of the paper. As the reviewer observed, the computational cost of the ratio matching loss in the case of ZZP grows linearly in d, which indeed makes the algorithm costly for higher dimensional datasets. As pointed out by the reviewer, subsampling over dimensions as suggested in lines 238-239 increases the variance of the unbiased estimator of the loss function. On the other hand, notice that the cost of the loss function associated with BPS and RHMC does not increase with the dimension, since the velocity vectors lie in a continuous space and the whole vector is updated at the same time, instead of flipping each component separately. Nevertheless, it is challenging to tune conditional normalising flows when the dimension increases, hence this constitutes the main limitation for BPS and RHMC.

• Moreover, given that many alternative choices exist other than the proposed method, in-depth discussions on the proposed method's merits would be appreciated for evaluating its importance.

  We agree with the reviewer that it is important to motivate our novel generative approach and we will add a paragraph to the introduction to improve this aspect. In general, we expect PDMPs to be successful at modelling data distributions that are supported on restricted domains, since it is very simple to adapt their deterministic dynamics to incorporate boundary terms (see e.g. [1]), especially if compared to diffusion processes. Similarly, data distributions that are a mixture of continuous density with point masses can be modelled adapting the ideas in [2], while extensions to data on Riemannian manifolds can be obtained leveraging flows which do not leave the manifold (see e.g. [3] for a PDMP on a sphere).

  We also believe that further empirical studies will be beneficial to highlight the specific advantages of PDMPs over other methods. In the revised paper we plan to include comparisons with other generative models on the synthetic datasets we considered.

• In addition, I find that the presentation should be improved. For example, many statements contain multiple prepositional phrases, which are difficult to parse and understand.

  Concerning this remark by the reviewer, we interpret that it refers to sentences such as those in lines 136-138, or 316-318. We will rewrite these and similar sentences to make the presentation more streamlined and easy to read. In addition, we will modify the paper extensively in order to improve the general presentation, taking into consideration the feedback that we received by all reviewers. For example, we will improve the exposition of Section 2 by introducing intuitive explanations and visual aids such as trace plots of the forward processes. We will also consider starting Section 2 with a description of RHMC and its time reversal, using this simple process to illustrate the main ideas to the reader in a more gentle way before discussing PDMPs in full generality. We will work on improving other sections, for instance including better discussions of our assumptions and additional material on the implementation of our algorithms, emphasising the main differences with diffusion models.

[1] Bierkens, J., Bouchard-Côté, A., Doucet, A., Duncan, A. B., Fearnhead, P., Lienart, T., Roberts, G., & Vollmer, S. J. (2018). Piecewise deterministic Markov processes for scalable Monte Carlo on restricted domains. Statistics & Probability Letters, 136, 148-154.

[2] Bierkens, J., Grazzi, S., van der Meulen, F., & Schauer, M. (2023). Sticky PDMP samplers for sparse and local inference problems. Statistics and Computing, 33, 8.

[3] Yang, J., Łatuszyński, K., & Roberts, G. O. (2024). Stereographic Markov Chain Monte Carlo. arXiv. https://arxiv.org/abs/2205.12112

Thank you for your thorough review and for taking the time to read and appreciate the full extent of our paper. We value your detailed feedback and have addressed your concerns below.

• Weaknesses (originality, significance)

  • Our paper focuses on establishing theoretical foundations and validating our approach with toy datasets. We highlight that our main contribution in this paper was proposing this novel methodology together with the theory for this class of methods. We believe this was a significant contribution, as deemed by the reviewer.
  • In the numerical experiments, PDMPs show promising results, with faster convergence to the data distribution in terms of computational time (see the pdf attached to this rebuttal, Figure 1). We agree that demonstrating their performance on more complex, real-world problems would showcase practical utility more effectively, and this is an important future direction we wish to work on.
  • We expect PDMPs to be successful at modelling data distributions supported on restricted domains, as it is very simple to adapt their deterministic dynamics to incorporate boundary terms (see e.g. [1]), especially compared to diffusion processes. Similarly, data distributions that are a mixture of continuous density with point masses can be modelled adapting the ideas in [2], while extensions to data on Riemannian manifolds can be obtained leveraging flows which do not leave the manifold (see e.g. [3] for a PDMP on the sphere).
  • As the reviewer suggests, in the next revision we will work on comparisons with other generative methods (e.g., variational autoencoders and normalising flows) on the datasets we consider.

• Weaknesses (quality) and Questions: Under H2, the authors state that the condition...

  We understand the remarks asking for more clarity regarding the assumptions that we require, and we agree that it is important to discuss whether these hold in practice. We will add these clarifications to the revised paper. We stress that H2 is the only assumption that depends on the data distribution. Let us comment on them.

  • H1 is about the chosen stationary distribution of the forward PDMP and is satisfied when this distribution is the multivariate standard normal, but also in the “variance exploding” variant, that is when $\psi(x) = 0$ for all $x$.
  • H2 is only required to prove Theorem 1 and is discussed in Appendix D.1. H2 holds when the stationary distribution of the PDMP is Gaussian and the data distribution has tails at least as light as those of a Gaussian in the case of ZZP and BPS, or in the case of HMC when it admits finite moments up to order 2. H2 is then satisfied in all our toy examples and in general when the data distribution is supported on a bounded set, as e.g. when the dataset is composed of images. We will give more information on this aspect in Appendix D.1 of the revised paper. We remark that in the case of ZZP and BPS much weaker conditions on tails of the data distribution can certainly be obtained by developing theory which is tailored for the context of generative modelling.
  • H3 is verified for the three PDMPs we consider, as mentioned in Appendix A.2. We will add comments on it in the appendix.
  • H4 is very technical, but is about the dynamics of the forward process rather than about the data distribution. It was verified in previous work for a specific version of BPS and can be verified for ZZP and RHMC after some technical arguments. Since this is out of the scope of our submission, we leave this verification for future works dedicated to this technical question.

• Clarity We thank the reviewer for having found our paper theoretically solid. We agree that more intuitive and visual explanations could make the content more accessible. In order to improve the clarity of our paper, we will make the following changes.

  • Include a sample trajectory for the forward process of each sampler; you can find them in Figure 2 of the PDF attached to this rebuttal.
  • Add a dedicated ‘Training’ section in the appendix, which will be self-contained and include a step-by-step guide for each of the three PDMP considered. This will clarify the overall training procedures and make them easy to implement.
  • Another option we are considering is to start Section 2 with a description of RHMC and its time reversal, using this simple process to illustrate the main ideas and intuition in a more gentle and visual way. We could then outline the main ideas of the training procedures for the characteristics of RHMC, leaving the most technical details to a new appendix we plan to add. If the reviewer thinks this is a good idea, we would gladly make this addition to the revised version of our paper.

• Could the authors include computational time with their simulations? As we previously mentioned, we include in the attached pdf a plot (Figure 1) displaying performance for each generative method on the Rose dataset, as a function of computational time. In addition, we plan to make the same type of comparisons on the other datasets that we consider, and add the resulting plots to the paper.

• A small query, in Proposition 2... We consider that the uniform distribution on $\{ \pm 1 \}^d$ is also a distribution on $\mathbb{R}^d$, hence the proposition applies. Based on your comment we will add a clarifying remark.

[1] Bierkens, J., Bouchard-Côté, A., Doucet, A., Duncan, A. B., Fearnhead, P., Lienart, T., Roberts, G., & Vollmer, S. J. (2018). Piecewise deterministic Markov processes for scalable Monte Carlo on restricted domains. Statistics & Probability Letters, 136, 148-154.

[2] Bierkens, J., Grazzi, S., van der Meulen, F., & Schauer, M. (2023). Sticky PDMP samplers for sparse and local inference problems. Statistics and Computing, 33, 8.

[3] Yang, J., Łatuszyński, K., & Roberts, G. O. (2024). Stereographic Markov Chain Monte Carlo. arXiv:2205.12112

We thank the reviewer for their time invested in our paper and their feedback. We believe we addressed all the raised issues below.

• There is a big jump in the style of writing between Section 1 and 2...

  • We agree the description of a time inhomogeneous PDMP is technical. In order to alleviate this difference in style between the two sections, in the next version of the paper we will add at the beginning of Section 2 the trace plots that can be found in the pdf attached to the rebuttal (Figure 2). These plots show the dynamics of the three PDMPs we discuss in the paper and hence help the reader develop a clear idea of the noising processes and their time reversal.
  • In addition, one option that we are considering is to first present our methodology for the specific case of RHMC, discussing the forward process and its time reversal, before introducing PDMPs in full generality. RHMC is perhaps the simplest of the three processes and the characteristics of its time-reversal are relatively straightforward to present and can be intuitively interpreted. We could then briefly outline the main ideas of the training procedures to estimate these characteristics, leaving the most technical details to a new appendix. If the reviewer thinks this is a good idea, we would gladly make this addition to the revised version of our paper.
  • Finally, in the supplement, we will add a section dedicated to the simulation of the forward PDMPs, with special emphasis to the case in which the limiting distribution is a multivariate standard normal. We think these additions will give the reader a more gentle, visual introduction to PDMPs and their time-reversals.

• You provide convincing examples on simple synthetic datasets. Is you family of methods able to generate data also in more challenging real-world examples?

  We are happy the reviewer found the experiments on synthetic datasets convincing. The numerical experiments, albeit on simple datasets, indeed suggest that it is worth conducting more extensive studies and comparisons on real-world datasets. For the moment, this paper laid the theoretical foundations to this new class of methods. Obtaining further results on more complex datasets is an important future direction that we wish to work on.

• When you compare the results of the various methods you show results for "steps". Is this a fair way of comparing the methods? ...

  We agree with the reviewer, and have added to the pdf attached to the rebuttal a plot (see Figure 1) which reports the computational time on the x-axis and the performance on the y-axis, for the “rose” dataset. This plot clarifies the computational cost of our algorithms as compared to DDPM, and we indeed observe clear improvements. We plan to make the same type of comparisons on the other datasets that we consider and add the resulting plots to the paper.

We thank the reviewer for their time invested in our paper and the relevant questions. Below, we believe we address all the raised concerns.

• To me, the descriptions of approximating the process characteristics with normalizing flows are unclear. ...

  We understand that we did not provide sufficient details on our methodology to obtain approximations of the characteristics of BPS and RHMC via normalising flows. We will clarify our implementation further both in the main text and in the supplement of the updated version of the paper, including pseudo-algorithms and step by step guides to reproduce the experiments.

  • Let us give a brief overview of the methodology here. Our procedure to approximate the characteristics of the backward BPS and RHMC is built on minimising an empirical counterpart of the (maximum likelihood based) loss displayed between lines 252 and 253. Such empirical loss is obtained with the standard approach of using Monte Carlo estimates for the inner expectation as well as the integral with respect to the probability distribution on the time variable. Concerning the time variable, we draw a time $\tau$ independently from $\omega$ and use it as the time horizon for the forward PDMP, obtaining $(X_\tau,V_\tau)$.

  • Then, the normalising flows framework is used to model the conditional density that appears in the loss. Essentially, we define a normalising flow that takes as input a $d$-dimensional standard normal random vector and outputs another $d$-dimensional random vector. The architecture of the normalising flow is described in Appendix E.1. Then, the normalising flow must be conditioned on the position and time variable since it models a conditional density. We achieve this by embedding the position and time via MLPs (with size given in Appendix E.1) and then injecting the outputs in the NF architecture at different points. The output of the NF is then a random vector which is approximately distributed according to the conditional distribution of v given x and t. The obtained conditional NF defines an invertible deterministic mapping $v \mapsto T_{(x,t)}(v)$ and hence gives an explicit expression for the modelled conditional density by the change of variables formula. Therefore, this can be used to compute the empirical loss.

• Are there particular data characteristics that these processes would be most suitable for? It seems that images with sharp boundary transitions are better modeled by the authors proposed approach. Would you say that is the case?

  This is a very important question, which will be best answered through extensive experimentation. As the reviewer noticed, our numerical simulations indeed suggest that sharp boundary transitions are modelled well by our algorithms compared to DDPM. We also expect PDMPs to be successful at modelling data distributions that are supported on restricted domains, since it is very simple to adapt their deterministic dynamics to incorporate boundary terms (see e.g. [1]), especially if compared to diffusion processes. Similarly, data distributions that are a mixture of continuous density with point masses can be modelled adapting the ideas in [2], while extensions to data on Riemannian manifolds can be obtained leveraging flows which do not leave the manifold (see e.g. [3] for a PDMP on a sphere).

• Do these methods have issues like being only able to generate from one mode if the data distribution is multimodal?

  While the non-reversibility of PDMPs can lead to improved convergence, it is not yet clear whether this is beneficial in the context of multimodal distributions. Nonetheless, it is worth mentioning our 2D experiment with the Gaussian grid, which was designed to test the robustness of the different generative methods with respect to mode-collapse. The Gaussian grid at hand is an unbalanced multimodal dataset (see line 686), and we can see that the PDMP samplers all have better performance than DDPM with respect to the MMD metric. One can visually check on Figure 2 that the data coverage of RHMC compares favourably to DDPM. This suggests that PDMPs can generate from multimodal distributions.

• Why does RHMC do so well for such a small number of steps?

  This is an interesting and perhaps surprising behaviour. Our intuitive explanation is that velocity refreshments can guide the position vector of the process towards the data distribution quite efficiently assuming the last step is performed close enough to the time horizon and assuming the conditional distribution of the velocity vector given the position vector is learned accurately enough. This intuition is supported by the fact that this phenomenon is not observed for the Zig-Zag process when the refreshment rate is set to 0, in which case the model requires a larger number of reverse steps to give good performance. We will add a comment about this in the paper.

[1] Bierkens, J., Bouchard-Côté, A., Doucet, A., Duncan, A. B., Fearnhead, P., Lienart, T., Roberts, G., & Vollmer, S. J. (2018). Piecewise deterministic Markov processes for scalable Monte Carlo on restricted domains. Statistics & Probability Letters, 136, 148-154.

[2] Bierkens, J., Grazzi, S., van der Meulen, F., & Schauer, M. (2023). Sticky PDMP samplers for sparse and local inference problems. Statistics and Computing, 33, 8.

[3] Yang, J., Łatuszyński, K., & Roberts, G. O. (2024). Stereographic Markov Chain Monte Carlo. arXiv. https://arxiv.org/abs/2205.12112

We thank the reviewer for their valuable review. We hope the following responses will answer their main remarks.

• We will include a paragraph in the introduction motivating the PDMP approach, based both on theoretical and empirical aspects. We expect PDMPs to be successful at modelling data distributions that are supported on restricted domains, since it is very simple to adapt their deterministic dynamics to incorporate boundary terms (see e.g. [1]), especially when compared to diffusions. Similarly, data distributions that are a mixture of a continuous density with point masses can be modelled adapting the ideas in [2], while extensions to data on Riemannian manifolds can be obtained taking advantage of flows which do not leave the manifold.

• We acknowledge that the current focus of our paper has been on establishing the theoretical foundations and validating our approach with toy datasets. We underline that developing the theoretical foundations for this class of methods is non-trivial, which is our main contribution in this paper. The experiments on 2D data show faster convergence to the data distribution in terms of the number of reverse steps (Table 2 in the paper), which also translates in better computational time (see the PDF attached to the rebuttal, Figure 1). As the reviewer suggested, we will work on empirical studies to highlight the specific advantages of PDMPs over other methods, focusing in particular on sample efficiency. Regarding the mixing rates of PDMPs, improved convergence over traditional, reversible methods has been observed first in the physics literature [4], while quantitative estimates on the rate of convergence to the stationary distribution were obtained using the hypocoercivity approach in [5], hence these can be compared to existing results for diffusions for a given limiting distribution. Moreover, the recent work [6] shows that PDMPs can at most achieve a square root speed-up compared to the overdamped Langevin diffusion in terms of relaxation time, and that this can be achieved in some cases.

• Contrary to what was erroneously written in the manuscript, we compared our model to the improved DDPM framework of [7], which already improves over the standard technique. While it is clear that this does not represent the current state of the art, we also found it to be a reasonable comparison, since our method is novel and, as the reviewer pointed out, there are many engineering tricks improving the performance of diffusion based models.

• We agree with the reviewer that it is beneficial to give clearer explanations on the training phase of our models. We will provide a dedicated ‘Training’ section in the appendix of the revised paper, that will include algorithm blocks for each sampler and will highlight how our training procedures differ from well-established methods. We will also give more details on other aspects as for instance the simulation of the forward process.

• On the novelties compared to [8] The work of [8] characterises the time reversals of a wide class of Markov processes with jumps. However, the conditions of [8] are abstract and stated in a “language” that differs from the standard in the context of PDMPs. Our contribution is twofold: (i) in Proposition 1 we give simple assumptions on the characteristics of the PDMP under which the process admits a time reversal, for which we give the backward characteristics; (ii) in Proposition 2 we consider the two abstract types of transition kernels that are used in the PDMP literature, namely deterministic mappings which change the velocity vector only and velocity refreshments, and give explicit formulas for the corresponding backward rates and kernels. Moreover, in Propositions 4,5, and 6 in the appendix we give clean statements for the time reversals of ZZP, BPS, RHMC, where the only assumption we require is a technical condition on the domain of the generator. Therefore, while the abstract machinery for time reversals was developed in [8], in our paper we specialise it to PDMPs pruning the statement to the cleanest form possible.

• We will provide a clearer explanation on the different terms in Equation (9) in the next version of the paper. The first term corresponds to the error caused by initialising the backward process at the limiting distribution instead of at the law of the forward process at time $T_f$. The second term is a consequence of using approximate backward rates defined with the estimated density ratios.

• On lines 238-239 If we draw uniformly at random a subset of the components $i=1,\dots,d$, then the expectation of the empirical loss coincides with the theoretical loss $\ell_I$ given in Proposition 3. Hence the estimator is unbiased, although its variance will increase.

• In the classical framework of PDMPs, a PDMP is indeed constructed by specifying the rates and transition kernels for each type of jump.

• The papers we cite at the end of the description of each PDMP on page 3 do not give quantitative upper bounds on the rate of convergence to the limiting distribution, hence they cannot be used to compare these PDMPs to e.g., the OU process. As mentioned above, there is both theoretical and empirical evidence that PDMPs can give faster convergence to a given probability distribution.

[1] Bierkens, et al. (2018). Piecewise deterministic Markov processes for scalable Monte Carlo on restricted domains.

[2] Bierkens et al. (2023). Sticky PDMP samplers for sparse and local inference problems.

[4] Michel et al. (2015). Event-chain Monte Carlo for classical continuous spin models.

[5] Andrieu et al. (2021). Hypocoercivity of piecewise deterministic Markov process-Monte Carlo.

[6] Eberle et al. (2024). Non-reversible lifts of reversible diffusion processes and relaxation times.

[7] Nichol et al. (2021). Improved Denoising Diffusion Probabilistic Models.

[8] Conforti et al. (2022). Time reversal of Markov processes with jumps under a finite entropy condition.