Tuning Sequential Monte Carlo Samplers via Greedy Incremental Divergence Minimization

Thank you for your review.

The datasets used seem quite standard for evaluating SMC methods. One thing I am not sure of is whether just reporting normalising constant is sufficient to show the superiority of this method, it would have been nice to see something like ESS or some assessment of posterior quality like moment estimates.

Thank you for the suggestion! Theoretically speaking, the accuracy of the normalizing constant and expectations taken over the particles are closely related. That is, the variance of the normalizing constant estimate is the cumulation of the variance of the normalizing constant ratios $Z_t/Z_{t-1}$, which corresponds to setting the integrand of expectation taken over the particles to be the indicator function as $\varphi = \mathbb{1}$. Therefore, if the normalizing constant estimate is accurate, the particle expectations will also be reasonably accurate. Indeed, we ran some experiments on two benchmarks: seeds and radon, where we computed the Wasserstein-2 distance of $N = 1024$ particles from SMC-LMC against ground truth samples from $10^5$ samples from the no-u-turn sampler. The Wasserstein-2 distance is closely related to the accuracy of the first and second moments. Also, it bounds the Wasserstein-1 distance, an integral probability metric, which quantifies the worst-case absolute error for 1-Lipschitz integrands. Therefore, it is a good metric for quantifying the error of particle expectations.

The results are shown in the following link:

https://imgur.com/a/80l62vS

(red line: Wasserstein-2 distance between the samples from vanilla SMC with the corresponding fixed stepsize and NUTS; blue line: Wasserstein-2 distance between the samples from our adaptive SMC sampler and NUTS.)

We can see that our adaptive SMC samplers achieve Wasserstein-2 distances close to the best fixed step size. However, it does not outperform fixed stepsizes as much as for estimating normalizing constants. This suggests that our adaptation scheme could be tailored to expectations taken over the particles for better performance.

Some interpretation as to why, eg, the method under-performed on the rats dataset would be welcome.

Thank you for pointing this out. With full honesty, it is unclear why this is the case. Upon close inspection, the optimization algorithm seems to correctly find the minimizer of the objective. Therefore, two explanations are possible: Either the sampler is underperforming for the given budget such that the estimate of the incremental KL is inaccurate, or the greediness of the scheme results in a suboptimal solution.

The experiments seem sound if a bit limited (as I previously mentioned) from what is relayed in the paper. I am curious why 32 replications were used to get the confidence intervals in the adaptive tuning vs fixed experiments but only 5 in the end-to-end optimisation experiments.

We agree with the reviewer that more than 5 evaluations for the end-to-end results would have been better. Therefore, since the submission, we re-ran the experiments using 32 replications for evaluation, which is now the same number as adaptive SMC, and included a more challenging problem (Pines). We observe that on Pines, our method does not outperform end-to-end optimization, which is unsurprising: end-to-end optimization should outperform our greedy scheme on some problems where the gradient noise is negligible. For the grid search experiments, we ran more problems from PosteriorDB, which now totals 21 benchmark problems. Please refer to the response reviewer 3HR9 above!

The idea of using the KL divergence to tune parameters of the proposal distribution in SMC also appears in (Shixiang et. al. Neural Adaptive Sequential Monte Carlo).

Thank you for pointing this out! We will add this to the list of works performing end-to-end optimization.

Is there any indication that this method under-explored the posterior or suffered more from particle degeneracy than the other compared methods?

Generally, our method should perform as well as a well-tuned vanilla SMC sampler, which may or may not fully explore the posterior depending on the problem and other configurations. In terms of particle degeneracy, our method should be the least susceptible since we are essentially maximizing the incremental weights. In a sense, our scheme is actively preventing degeneracy. The side effect would be that the biased normalizing constant estimates obtained during adaptive runs will be overestimated.

Thank you for your review.

Do the authors want to minimize incremental divergences in order to target the path measure of the annealed importance sampling procedure, or fine-tune an annealed importance sampling procedure in order to later perform SMC on the target density?

We kindly request more details on this question as our tuning procedure does not involve nor is specialized to annealed importance sampling (AIS).

Perhaps the question arose from the fact that the Feynman-Kac model formalism does not explicitly involve resampling, which might have led to the impression that optimizing the target path of the Feynman-Kac model is tuning an AIS procedure, not an SMC procedure. If this is the case, we would like to clarify that while our procedure can absolutely be applied to AIS (after all, SMC and AIS are a few resampling steps away from each other), our focus is purely on SMC samplers. That is, both AIS and SMC are algorithms for simulating the same Feynman-Kac model (it’s only the procedure for simulation that differs.) To be clear, we would like to bring attention to Algorithm 1, which shows the adaptive SMC implementation with our tuning scheme embedded in it. All we do is run Algorithm 1.

If this does not fully address the Reviewer’s question, we would be very happy to further clarify any point of ambiguity.

Thank you for your review.

One potential caveat is that some theoretical guarantees rely on assumptions (e.g., unimodality of the tuning objective) that may limit generalizability,

We agree that those theoretical assumptions are restrictive. As such, we are happy to mention that we were able to relax those assumptions during the review period. Now, even without unimodality, we are able to guarantee that our algorithm finds a point $\epsilon$-close to a local minimum with similar computational cost. Additionally assuming unimodality strengthens the guarantee so that the returned point is $\epsilon$-close to the global optimum.

The only major change we had to introduce is that we now use a different variant of the golden section search algorithm. In particular, we use the version described in “Numerical Recipes” (Section 10.1 in [1]). We found that this variant is able to guarantee finding a local minimum as long as the initialization satisfies a certain condition implying that the initial search interval contains a local minimum.

How sensitive are the experimental results to changes in key parameters, and does this sensitivity mirror the challenges encountered in practical scenarios?

Thank you for the great question. Overall, we didn’t spend much effort tuning the parameters, and all of our experiments were run with a single fixed set of parameters for each type of kernel. In more detail though, the only parameters that primarily affect the results are the regularization strength $\tau$ and the optimization accuracy budget $\epsilon$. (The rest of the parameters only change the amount of computation spent obtaining similar solutions.) $\epsilon$, for instance, does not affect the result much as long as it is small enough. On the other hand, the effect of the regularization strength $\tau$ seems to depend on the kernel but not so much on the problem. For instance, the performance of LMC isn’t very sensitive to $\tau$, and we thus use a small amount. On the other hand, KLMC seems to require stronger regularization to obtain good performance. While we suspect this has something with the persistence of the momentum, it is not entirely clear why. Overall, we do not expect the parameters to require much tuning, except when swapping the MCMC kernel.

Can the proposed method scale efficiently with the increased dimensionality and sample sizes often found in real-world problems?

We extended our experiments with higher dimensional problems since the initial submission. (Please refer to the response to 3HR9 for the new results.) In principle, dimensionality shouldn’t be a problem for our method. As shown in the experiment, our method should perform better or as well as the best-tuned vanilla SMC sampler. Therefore, as long as vanilla SMC can scale, our method should be able to scale as well. In terms of scaling with respect to the sample size $N$, as long as the subsampling size $B$ is smaller than $N$, the added overhead of our method should be negligible. More concretely, if at most $C$ objective evaluations are spent during each adaptation step, the total cost of running SMC with our adaptation is $O(B C + N)$. Thus, if $BC = o(N)$, our adaptive SMC scheme is just as scalable as vanilla SMC.

1. Press, William H. Numerical recipes 3rd edition: The art of scientific computing. Cambridge University Press, 2007.

Thank you for your review.

The experiment set up is too simple on a collection of toy benchmark datasets. It is not clear how it is applicable to the application domains of SMC, such as steering large language models and conditional generation from diffusion models.

We agree that more realistic experiments are always a good thing. As such, we added new benchmarks from both PosteriorDB and Inference Gym, including a high-dimensional benchmark problem (Pines) from Inference Gym, which is 1600 dimensional and multiple problems from PosteriorDB, including TIMSS and Three Men, both of which have more than 500 dimensions. Here is the full list of problems with their corresponding dimensionality:

The new experimental results can be found in the anonymized links below.

Comparison against end-to-end optimization:

https://imgur.com/pdk3Bza

(red dot: normalizing constant estimate obtained by our adaptive SMC sampler; blue line: normalizing constant estimate obtained by end-to-end optimization; dotted line: ground truth)

Here, we can see the comparison against end-to-end optimization results with the new Pines benchmark problem. Furthermore, unlike the experiments included in the original submission, we use the official code provided by Geffner and Domke [1], which we find to be better tuned. In particular, on Pines, we observe that end-to-end optimization performs slightly better than our adaptive SMC procedure, while on other problems, we obtain more accurate results.

Comparison against grid search for SMC with Langevin Monte Carlo kernels:

https://imgur.com/9JjJEiK

Comparison against grid search for SMC with kinetic Langevin Monte Carlo kernels:

https://imgur.com/mUeWbuz

(red line: the normalizing constant obtained by our adaptive SMC sampler; purple line: the normalizing constant obtained by vanilla SMC with a fixed stepsize; dotted line: ground truth estimate.)

Overall, for the comparisons against grid search, we observe a similar trend where our tuning procedure finds better or comparable results to the best-fixed step size. The only exception appears to be Rat, as observed in the original submission.

We would also like to point out that PosteriorDB is a state-of-the-art benchmark (with the corresponding paper published this year at AISTATS’25 [2]) that contains problems that have actually been used for practical applications or closely resemble such problems. As such, we believe our experiments adequately represent problems to which SMC is expected to be applied in practice. While evaluating our method on applications such as LLM steering and conditional generation from diffusion models would definitely be interesting, they will probably need their own investigation with more specialized solutions.

1. Geffner, Tomas, and Justin Domke. "Langevin diffusion variational inference." International Conference on Artificial Intelligence and Statistics. PMLR, 2023.
2. Magnusson, Måns, et al. "posteriordb: Testing, benchmarking and developing Bayesian inference algorithms." AISTATS’25, to be presented.