Monte Carlo guided Denoising Diffusion models for Bayesian linear inverse problems.

Dear reviewer, We thank you for the time and effort taken to review our manuscript. Regarding your comments:

• We think that the point 3 is an empirical contribution in itself since we show that the current methods used for solving linear inverse problems with diffusion do not actually correctly sample the posterior in the most straightforward cases.
• While we agree that our method is computationally more expensive than existing works such as DDRM and DPS, we believe that this is the price to pay to obtain more faithful approximations of the posterior. Indeed, as we show in the Gaussian mixture experiment, DPS and DDRM do not actually sample approximately from the posterior and cannot be made more precise using more computational resources. On the other hand, our method has an extra parameter (the number of particles $N$) which controls the accuracy of the approximation. We firmly believe that there is an intrinsic tradeoff between accuracy and computational burden when dealing with the posteriors of generative models. That being said, in moderate dimensions, drawing $N$ parallel samples from DPS or DDRM has the same computational cost as running one particle filter with $N$ particles, which returns $N$ samples that approximate the posterior (the resampling cost is negligible in front of the cost of running the diffusion). In much larger dimensions, taking $N$ particles in MCGDiff will result in $N$ collapsed particles, which is not qualitatively the same thing as running DPS or DDRM with $N$ samples. However, we advocate in our experiments that in such cases, one should use a small number of particles and instead run $M$ parallel MCGDiff with $N/M$ particles, to obtain a particle approximation of the posterior with $M$ different and independent particles. In such cases, the computational cost of our algorithm is larger than that of DPS and DDRM (in the sense that with the same computational cost you can draw $N$ i.i.d. samples from DPS and DDRM). While it is true that the particles from a single particle filter lack diversity in global level features, still, we believe that the quality of our proposal allows us to draw samples in the support of the posterior, and not outside, as it is the case of the other methods. Hence running parallel particle filters allows us to have samples that are globally diverse.
• For our imaging experiments, we believe that no method can quantitatively assert the performances of all three algorithms, as we do not have samples from the posterior to compare to. This is precisely the reason why we focus on tractable experiments in the main paper. In the appendix though, we have compared MCGDiff and other algorithms in some image tasks by at first generating 100 samples from the posterior and then showing the 2 samples from those that are the furthest apart. We can see that MCGDiff seems to obtain more coherent samples, even though, as we said before, this is purely a qualitative evaluation and is of course subject to discussion.

Thanks for the response, that clarifies the computational question I posted, although I am still not fully convinced by the claim that 'extra parameter (the number of particles N) which controls the accuracy of the approximation', increasing the number of particles will certainly increase computational cost but not necessarily improving the accuracy, my concern is still on how the intrinsic degeneracy issue in particle filter will affect the whole algorithm. I would suggest adding 'limitations of the method' to your final version. Although I still have minor concern as described above, the novelty and the soundness of this work is good, I will keep my weak accept score.

Dear reviewer,

We would like to start by thanking you for the time and effort you put into reviewing our paper. We are glad that you have appreciated our work!

Indeed, FID is not suitable for Bayesian reconstruction problems and as far as we are concerned, there are no other consistent methods that permit a reliable assessment of such reconstructions. Note that in the particular case where the posterior is concentrated, it is possible to “confidently” compare different reconstruction algorithms, through the LPIPS metric using the groundtruth image. However, we have been particularly interested in ill-posed problems, where this type of comparisons is not relevant. One line of exploration could be to run MCGDiff with a very large number of particles and use the resulting samples as a groundtruth with some metric over distributions, for example the sliced Wasserstein distance.

Dear reviewer, We thank you for the time and effort taken to review our manuscript and the positive feedback. We thank you for the points that you raised concerning the bibliography and the speed issues and will be integrating it into the main text in the future. We will also add a precise pseudo code that reflects exactly the source code, in the appendix.

• We would like to point out that we refrain from comparing to the ground truth. Indeed, our main focus on the numerical part is indeed the mixture of gaussians and mixture of funnel cases. The main reason is that in those cases it is possible to have access to samples from the real posterior and we show that our algorithm outperforms the competition in all settings and manages to match well the posterior.

• Indeed in figure 5 and 6 we see that unlike the other existing methods, MCGDiff finds all the modes but in some cases the tails of each mode are thinner. A straightforward answer to your question, relying on the theorems presented in the main paper, would be to increase the number of particles being used. There are also other alternatives, as for example using the unadjusted Langevin algorithm initialized at the particles resulting from MCGDiff and targeting the posterior of the model. This requires having the score at time 0, to which we may not have access as the data distribution may not have a density in practice. However, we can replace it with the score at some timestep epsilon, close to 0. We have tried this method and in some cases it leads to better spreading around the mode and smaller sliced Wasserstein distance.

• The current work is an effort to handle bayesian linear inverse problems when using a pre-trained denoising diffusion generative model as a prior, without any need of (further) training. In this sense, the measurement $y$ can be a noisy linear observation of the state. But if we understand your question correctly, it concerns the training data for the underlying generative model, which we do not have control over as we use off the shelf trained diffusion models. However, if one has some prior information on the noise present in the dataset, perhaps there might a path forward. This is a really interesting yet difficult question that we believe is outside the scope of the current paper.

Dear reviewer, We thank you for the time and effort taken to review our manuscript. We are glad that you have appreciated our work and thank you for the positive feedback!

The code that is currently available in the anonymous git will be available once the submission process is over. Indeed, we also believe that MCGDiff can have a broader impact beyond image-related tasks. We are currently using it on healthy electrocardiogram (ECG) to develop interpretable outlier detection tools for electrophysiological anomalies, with great success.