Active Sequential Posterior Estimation for Sample-Efficient Simulation-Based Inference

Thank you for your time and effort providing detailed feedback on our work. Please find our responses to your questions and comments below.

Note: Where applicable, we prefix sections with W-<x>, Q-<y>, or L-<z> to reference itemized comments in Weaknesses, Questions, and Limitations, respectively, numbered by the order in which they were mentioned.

[W-1] This is true, although SBI/Bayesian methods in general are sensitive to a choice of prior, and this is not a limitation exclusive to our method. Further, the active learning mechanism does not change the limiting behavior of the method insofar as its relation to APT [1], as discussed in Section 3.4 (i.e., we still have $q_\phi(\theta |x)\rightarrow p(\theta|x)$ as $N \rightarrow \infty$).

[1]: D. S. Greenberg, M. Nonnenmacher, and J. H. Macke. Automatic posterior transformation for likelihood-free inference, 2019.

[W-2] The acquisition function proposed for selecting simulation samples is motivated across Sections 3.1, 3.2, and 3.3, with the principal motivation being to improve on traditional SBI methods by exploring the simulation parameter space in a more principled manner over the course of the multi-round inference procedure. In particular, we want to do so in a way that is maximally informative to our model of the posterior, which can yield compounding benefits in posterior accuracy under $x_o$ that are particularly important in settings with low simulation budgets and/or high simulation costs.

[W-3] Please see our global rebuttal reply for details on additional empirical evaluation, including discussion around new simulation benchmarks and more representative metrics.

[W-4] Regarding syntax issues and small figure text Thank you for pointing these out, they will be corrected in our final manuscript.

[Q-2] Many of these parameters are intrinsic to the SUMO simulator or the traffic network. For example, exogenous parameters include the route choice set and link attributes (e.g., free flow speeds, number of lanes). Endogenous variables include link and path traffic statistics (travel times, flows, speeds), and departure times.

[Q-3] Our proposed approach falls into the class of simulation-based inference methods, and is indeed intended to be generally applicable in settings where statistical inference under an arbitrary mechanistic model is needed. There are many scientific domains that define problems of this nature (e.g., inferring parameters of biological processes, calibrating physics models to real-world observations, etc). In the transportation space, SBI methods like the one proposed can be used not only for inferring network demand (OD matrices), but also for many other dynamics in urban settings, such as human mobility and public transportation. Urban designers and city planners can reason about effects of various proposed changes using simulation models, as well as leverage inverse models learned with SBI to better characterize parameters required to achieve desired outcomes.

[Q-4] Evaluating the acquisition function as-is requires some means of inducing a distributional estimate over the parameters of the chosen model. BNNs and MC-dropout are flexible ways to achieve this in practice, but without this form our acquisition function cannot be used as intended. We are not otherwise aware of a clear way to sidestep this requirement without considering an entirely different acquisition function (and no such formulation exists in the SBI/BO literature, to the best of our knowledge).

[Q-5] Equation 3 captures average differences in the assigned likelihoods between the posterior model ensemble ($p(\theta | x_o, D)$, see also the marginalization above line 157) and any particular model instantiation under the weight posterior. It could perhaps be likened to a notion of "distributional variance" around $\theta$ under the model weight posterior $p(\phi |D)$, but it is distinctly different from the variance of $\theta$ under the posterior $p(\theta | x_o)$.

[Q-6] This is a great question, and very similar to the one that motivated this work. It is worth first noting that sequential SBI methods in general are focused on learning a particularly accurate picture of $p(\theta | x_o)$, i.e., the posterior under known observational data of interest. These methods explicitly condition on $x_o$ to produce each round’s proposal distribution, effectively “refocusing” the next round’s samples to reflect the model’s current understanding of $\theta$ that explain $x_o$. As a result, the posterior at a given round does not strictly correspond to regions previously explored in the parameter space (and so “exploitation” may be a bit misleading); rather, it captures the parameters expected to explain $x_o$ under the simulation model, given the simulation samples that have since been observed.

In any case, there are a few challenges when thinking about using the alternative distribution you mention in this case:

1. How can we ensure convergence to the true posterior if sampling from a modified distribution?
2. (similarly) How can we ensure the resulting exploration is consistent with the prior?

Our approach effectively implements your proposition by allowing the underlying model to "explore" regions of the parameter space it doesn't currently "understand" well, while remaining consistent with prior specifications and converging to the true posterior in the limit. Put another way, our method effectively produces the mentioned alternative "exploration distribution," but through a combination of components (the posterior model and a selection mechanism) that can be tractably mapped onto properties required for consistent Bayesian inference.

Thank you for your time and effort providing detailed feedback on our work. Please find our responses to your questions and comments below.

Note: Where applicable, we prefix sections with W-<x>, Q-<y>, or L-<z> to reference itemized comments in Weaknesses, Questions, and Limitations, respectively, numbered by the order in which they were mentioned.

[W-1] The motivation for Equation 4 is primarily discussed in Section 3.3 (lines 172-175), which describes how the $\theta^*$ that maximizes this notion of distributional uncertainty is not necessarily a likely parameter under the observational data $x_o$. Multiplying by the proposal prior, which captures the most up-to-date estimate of $p(\theta | x_o)$ at each round of the algorithm, allows us to re-weight this uncertainty by the approximate likelihoods of each parameter under $x_o$ and prioritize more likely $\theta$ during acquisition. This multiplication is primarily motivated by the desire to more closely align the effective likelihoods of parameters $\theta$ with the true proposal prior, which is ultimately present and corrected for in the NDE loss (see line 178).

[W-2, L-1] Regarding computational cost of acquisition: Section 3.4 describes the mechanism by which Equation 4 can be optimized over a fixed set of proposal samples during each round. While we do indeed need to evaluate the NDE for several candidate parameters and many network realizations, this acquisition evaluation can be performed very efficiently as a batched inference step under the NDE model.

Further, the additional computational overhead can be compared against SNPE (i.e., no acquisition function) for our primary traffic task, as the reported wallclock times (found in subfigures (b) for each of the calibration plots) provide the raw runtimes for each method when obtaining the 128 simulation samples for each setting. Empirically, this allows us to compare the total time spent by our algorithm (including the cost of optimizing the acquisition function), and we observe a negligible difference compared to SNPE over our explored horizons.

[W-3, Q-1] You raise a valid point: RMSNE is indeed a non-standard metric for evaluating SBI methods in the literature, and is employed in this work almost entirely due to its prevalence in the OD calibration community. In our traffic calibration setting, there are few why it’s used:

1. We are ultimately interested in producing good point estimates of the true OD matrices,
2. RMSNE is a standard in the OD calibration space and allows us to compare with other baseline methods like SPSA and PC-SPSA, and
3. While we may benefit downstream from having a good distributional estimate under $x_o$ (as SBI methods generally attempt to provide), for our task, we are mostly interested in how approximating the posterior can improve our method's intermediate exploration of the parameter space in service of producing good point estimates.

Nevertheless, we agree a systematic exploration of the final posterior accuracy under metrics better suited to capture distributional differences (not just point estimates) would be warranted in general. Please see our global rebuttal reply for more details here.

[W-4] Please see our global rebuttal reply for details on additional empirical evaluation, including discussion around new simulation benchmarks and more representative metrics.

Thank you for the favorable score revision and again for your valuable feedback! It is very much appreciated. A few additional comments to the points raised:

Regarding the formulation of the acquisition function

Thank you for clarifying. We agree, the family of functions

$\alpha(\tilde{\theta}, p(\phi, D)) = \tilde{p}(\tilde{\theta})(\mathbb{E}_{\phi^\prime\sim\phi |D}[\dots])^\lambda$

under parameter $\lambda$ constitutes valid choices for the acquisition function for any $\lambda$, facilitating different levels of emphasis on uncertainties at values of $\theta$ relative to their likelihoods under $\tilde{p}$. The choice to use $\lambda =1$ here is in part due to convenience as we did not set out to explore this parametric family explicitly, but it intuitively captures a desirable balance in the relationship between uncertainties and likelihoods of $\theta$.

In particular, under level sets $\alpha(\cdot, p(\phi, D)) = z$ (where $p(\phi, D)$ is held constant), as likelihoods $\tilde{p}(\tilde{\theta})$ decrease by a factor of $n$, the average deviation between $p(\theta | x, D)$ and $p(\theta | x, \phi)$ need only increase by a factor of $\sqrt{n}$, i.e., changes in uncertainty are sub-linear in the likelihood ratio. With the introduction of $\lambda$, we’d see this factor generalize to $n^{1/(2\lambda)}$, and may need to take additional measures to balance the resulting sensitivity between the terms. We find that $\lambda =1$ is a natural choice that reasonably captures the desire to explore potentially unlikely parameters with high uncertainties without ignoring them (e.g., $\lambda \rightarrow 0$) or relying too heavily on them (e.g., $\lambda \rightarrow \infty$). We nevertheless find it intriguing to explore the impacts of $\lambda$ on the effectiveness of the acquisition function in practice, and will aim to incorporate this alongside our additional ablation tests in the final manuscript.

Regarding the additive form, we find this slightly less intuitive, and potentially lacking some of the above-mentioned qualities. In particular, it only shifts the uncertainties for choices of $\theta$ rather than scaling them by their likelihood, meaning the relationship between terms is no longer dependent on a multiplicative factor. That is, under level sets of $\alpha$, absolute differences in likelihood $\tilde{p}(\tilde{\theta})$ need to be made up for by proportional absolute differences in uncertainty around $\tilde{\theta}$, which is slightly counter-intuitive (e.g., for any two $\theta_1,\theta_2$, the needed change in uncertainty is no longer determined by their likelihood ratio under $\tilde{p}$). Additionally, for $\lambda$ close to 1, $\theta$ with low likelihood may be too readily selected provided $\tilde{p}(\theta)$ is dominated by high uncertainty found outside of high-likelihood regions under $p(\theta | x_o)$.

We welcome further discussion if there are any outstanding questions or concerns. Thank you again!

Thank you for your time and effort providing detailed feedback on our work.

Note: We prefix sections with W-<x>, Q-<y>, or L-<z> to reference itemized comments in Weaknesses, Questions, and Limitations, respectively.

[W-1] The discussion around EIG was principally intended to help set the stage for desirable qualities of an acquisition function, despite the fact it is itself not directly possible to optimize in LFI settings due to its reliance on the likelihood $p(x|\theta,D)$. This drawback motivates an alternative formulation that captures these qualities, which begins with our introduction of a notion of distributional uncertainty (up to a divergence measure $\mathcal{D}$) in Equation 2. This is further connected to Equation 3 via the discussion in Section 3.3, which makes concrete an approach inspired by the general form in Eq. 2 and that seen in [1].

Line 157 is an expansion of the marginalization involved over NDE model parameters $\phi$ in defining $p(\theta|x,D)$, and is included primarily to highlight the relationship between $p(\theta|x,D)$ and $p(\theta|x, \phi)$. More exposition will be added to our final manuscript to better contextualize this form.

[1]: K. Kandasamy, J. Schneider, and B. Poczos. Bayesian active learning for posterior estimation. (IJCAI ’15)

[W-3] CI-1 Please see our global rebuttal reply for details on additional empirical evaluation, including discussion around new simulation benchmarks and more representative metrics.

[W-4] Appendix C.1.1 provides the hyperparameters used by our algorithm across explored experimental settings, as well as our NDE model’s structure, dropout rate, etc. We agree that a more principled analysis of hyperparameter sensitivity of heuristics for setting them in practice would be valuable. While performing such a study is prohibitive on our primary task due to computational constraints, the effects of the algorithm hyperparameters (e.g., $R$, $N$, $B$) and NDE hyperparameters (e.g., network structure, dropout rate, etc) will be explored on the smaller scale SBI benchmarks discussed above and reported in our final manuscript.

[W-5] These are valuable resources and indeed provide useful context for leveraging implicit models in Bayesian optimization contexts. While we believe mentioning these works in our literature review is warranted, there are a few reasons why they might otherwise have been seen as indirectly relevant:

• They primarily leverage GP surrogate models rather than models seen in more recent SBI literature (e.g., flow-based generative models) and those used in our work.
• Our efforts are focused on formulations that only require a direct posterior approximation, rather than assuming surrogate likelihoods or likelihood ratios are also available.
• They primarily focus on experimental design contexts, which embrace a slightly different set of assumptions, formulations, and intended applications.

[Q-1] The adjustment to Eq. 3 is motivated in Section 3.3 (lines 172-175), which describes how the $\theta^*$ that maximizes this notion of distributional uncertainty is not necessarily a likely parameter under the observational data $x_o$. Multiplying by the proposal prior, which captures the most up-to-date estimate of $p(\theta | x_o)$ at each round of the algorithm, allows us to re-weight this uncertainty by the approximate likelihoods of each parameter under $x_o$ and prioritize more likely $\theta$ during acquisition. See also [Q-3] below.

[Q-2] “Cost” here is referring to the time or compute resources spent evaluating the simulator under the mentioned parameter. We agree this should be more clearly stated.

[Q-3] Eq. 3 captures a means of quantifying uncertainty over the likelihood values assigned to $\theta$ under the NDE and its various realizations under $\phi\sim p(\phi|D)$. This does not capture the desire for $\theta$ to also be likely under the posterior $p(\theta|x_o)$, however. Our approach, along with many sequential SBI methods, is primarily concerned with learning an accurate view of the $p(\tilde{\theta}|x_o)$ under observational data $x_o$. Lines 172-173 are simply connecting this larger goal to the implications of optimizing Eq. 3, suggesting $\theta^*$ may be of low value if it’s unlikely under the observational data $x_o$. This motivates the adjustment to Eq. 3 that appears in Eq. 4 (addressed in [Q-1]).

[Q-4] Lines 191-192 convey that the optimum of $\alpha$ (call it $\alpha^*$) is captured in a sample $X_n$ of size $n$ (drawn over $\alpha$'s domain) in the limit $n \rightarrow \infty$. This could be expanded to include the relevant implications, namely,

$p(\alpha^* \in X_n) \rightarrow 1, n \rightarrow \infty$

or

$\text{argmax}_{\theta\in X_n} \alpha(\theta, p(\phi | D)) \rightarrow \alpha^*, n \rightarrow \infty$

This does not suggest that there exists a finite sample of size $n$ within which $\alpha^*$ is present, and instead only captures the limiting behavior for increasingly large sample sizes. In practice, we leverage this consistency with $\alpha^*$ given this limiting behavior.

[Q-5] The primary difference between ASNPE and SNPE in terms of computational overhead is the optimization of the acquisition function in Eq. 4. As is mentioned in Section 3.4, however, the acquisition function is computed over a fixed size parameter batch and NDE realizations $\phi\sim p(\phi|D)$ (with specific hyperparameters reported in Appendix C.1.1). This can be performed efficiently as a batched inference step under the NDE model, and typically yields a negligible difference in terms of raw runtimes between the two methods. For the traffic case study we explore in the paper, the raw runtimes of both methods (found in subfigures (b) for each of the calibration plots) allow us to compare the total time spent by our algorithm (including acquisition evaluation), and we observe a negligible difference compared to SNPE over our explored horizons.

Thank you for your time and effort providing detailed feedback on our work. Please find our responses to your questions and comments below.

Note: Where applicable, we prefix sections with W-<x>, Q-<y>, or L-<z> to reference itemized comments in Weaknesses, Questions, and Limitations, respectively, numbered by the order in which they were mentioned.

[W-1] There is indeed some overlap in notation, and the differences between these forms are mostly context-dependent:

• $q_{\phi}(\theta | x)$: the approximate posterior NDE model, with parameters $\phi$
• $p(\theta | x, \phi)$: a concrete posterior density produced by a particular setting of NDE weights $\phi$
• $\tilde{q}_{x, \phi}(\theta)$: the approximate proposal prior
• $q_{\phi, x}(\theta)$: same as $q_{\phi}(\theta | x)$, but with syntax adjusted to mirror the key result being used from [1] at line 178.

In particular, the form used on line 178 was constructed to resemble the setup of a key result we leverage from [1]. However, we agree that this may ultimately have made the connection to the notation elsewhere in our methodology unclear, and will revise this in our final manuscript (by both introducing each term if using a different form, and simplifying where applicable).

[1] D. S. Greenberg, M. Nonnenmacher, and J. H. Macke. Automatic posterior transformation for likelihood-free inference, 2019.

[W-2] Please see our global rebuttal reply for details on additional empirical evaluation, including discussion around new simulation benchmarks and more representative metrics.

[W-3] Regarding paper [1]: We distinguish between the following two problems in OD estimation:

1. General OD estimation problems (also known as travel demand estimation problems), where the outputs are stand-alone OD matrices that can be used for a variety of planning and operational network analysis, and
2. Model calibration problems, where the goal is to calibrate (or estimate) the inputs (such as the demand inputs specified as OD matrices) of a specific traffic simulation model and the output is a calibrated traffic simulator that can itself be used for analysis.

Paper [1] addresses Problem (i), while our work addresses Problem (ii). We would like to stress that this distinction is an extremely important one. In particular, the main challenges of Problem (ii) are due to using an intricate (e.g., stochastic, non-differentiable, high compute cost) traffic simulator. This calls for likelihood-free methods, sample-efficient methods, and methods robust to simulator stochasticity. In contrast to this, Problem (i) can be formulated as a differentiable problem, which can be tackled with a likelihood-based approach, has little-to-no compute cost challenges, and no need for sample efficiency. This distinction is also discussed in [A].

[A]: C. Osorio (2019) High-dimensional offline origin-destination (OD) demand calibration for stochastic traffic simulators of large-scale road networks. Transportation Research Part B: Methodological, Volume 124, Pages 18-43

Regarding paper [2]: This work addresses the similar problem of performing likelihood-free inference under expensive simulators/low budgets. However, it focuses almost entirely on formulations suited for time-series data, and aims principally to jointly learn both summary statistics and a classifier for performing density ratio estimation. These are no doubt important settings in this space, but this particular work ultimately differs in several significant ways form our paper (e.g., we use a single observational data point, work exclusively with direct posterior approximations, require a means of inducing uncertainty over NDE parameters, etc).

[W-4] Thank you for pointing these out, lines 268 and 294 are indeed missing words and will be corrected in our final manuscript. We also noticed the missing line numbers in several places (appears to be an odd formatting bug) and will ensure these are fixed.

Thank you for your consideration regarding the score, as well as the additional feedback. It is very much appreciated!

[W-1] We agree: the use of $q_{\phi, x}(\theta)$ (bullet #4) is inconsistent and should be replaced with $q_{\phi}(\theta | x)$ (bullet #1). We will make this change in our final version.

Regarding $q_{\phi}(\theta | x)$, or $q_\phi$, we generally use this to indicate that $q$ is a family of models parameterized by $\phi$. $p(\theta|x, \phi)$ is then used in contexts when $\phi$ is concrete (e.g., in an expectation like that of Eq. 2, or in the marginalization above line 157), and parallels the generic form we see with the marginal posterior $p(\theta | x, D)$. However, as you point out, this distinction isn’t particularly important (or at least the use of two separate forms isn't necessary), and we agree simply using $q_\phi(\theta |x)$ directly is better in these contexts given it's clear $\phi$ is concrete (i.e., it doesn't clash with the broader notion that $q_\phi$ can refer holistically to the model family). We will make these substitutions in our final version.

[W-3] We also think this discussion is valuable, and will be sure to include it in the literature review of our final manuscript.

In summary, W-1, W-2 (addressed in global rebuttal), and W-3 will each be incorporated in our final version. Thank you again for your consideration, and we welcome any further questions or discussion.

Given you indicated a willingness to revise your score, we wanted to kindly ask if you would look over our reply, which includes our plan to incorporate your suggestions (W-1, W-2, and W-3). If these changes meet your expectations and warrant the aforementioned score revision, doing so prior to today’s deadline would be greatly appreciated. Thank you again for your helpful feedback during this process!

Thank you for your time and effort providing detailed feedback on our work. Please find our responses to your questions and comments below.

Note: Where applicable, we prefix sections with W-<x>, Q-<y>, or L-<z> to reference itemized comments in Weaknesses, Questions, and Limitations, respectively, numbered by the order in which they were mentioned.

[W-1] Please see our global rebuttal reply for details on additional empirical evaluation.

[W-2] We agree this transition could be smoother. Section 3.4, however, sets up several practical considerations for how ASNPE can be implemented (e.g., efficiently approximating Eq. 4, training neural network-based NDEs with MC-Dropout, etc), and specific model details and hyperparameter settings are relegated to Appendix C.1.1 (primarily due to space constraints). In our final manuscript, we will incorporate a discussion on concrete implementation choices prior to the experimental setup, and direct readers to the Appendix for additional details.

[W-3] The intended scope of Section 4.2 was to introduce methods specifically used in OD calibration literature (i.e., just SPSA and PC-SPSA), while we relied on prior context for the likelihood-free inference methods (i.e., SNPE and MC-ABC). However, as you point out, MC-ABC is not properly introduced, and the exact version of SNPE remains unclear (although it is indeed APT). In our final manuscript, we’ll broaden the scope of Section 4.2 to include all reported methods and properly introduce them.

[W-4] This is a good point, and something we will incorporate alongside the smaller scale experiments run on benchmarks mentioned in [W-1]. As a quick theoretical discussion, however, the following reflects our observations on the large-scale setting we explore in the paper. Keeping the total number of simulation samples constant,

• The number of rounds $R$ dictates how many times we update the proposal distribution over the course of the simulation horizon. Increasing this value can enable quicker feedback to the NDE, requiring fewer simulation samples before re-training the model. When the prior is well-calibrated and simulation samples are representative of the observational data, this can have a positive compounding effect that boosts the rate of convergence to the desired posterior. However, for larger R the resulting batch sizes are smaller and the NDE receives noisier updates, which can have the opposite effect and hurt early performance when the prior is poor.
• The number of selected samples B per round is directly determined by R when the total number of simulations is held constant, and thus the above effects apply here.
• The number of proposal samples N per round governs the size of the parameter candidate pool over which the acquisition function is evaluated. Increasing this value allows us to consider more potentially relevant candidates under $\tilde{p}(\theta)$, and can thus increase the quality of the resulting B-sized batch. Given the acquisition function can be evaluated over this pool very efficiently (i.e., as a batched inference step through the NDE model), one can practically scale this up arbitrarily to increase the sample coverage over the proposal support (but with decreasing marginal utility).

[Q-1] Thank you for pointing this out. It should instead say R=4 (selection size B=32 is correct), and will be corrected in our final version.

[Q-2, L-1] Generally, prior I captures relatively high bias/low noise around the true OD parameter, while prior II is comparatively low bias/high noise. Our opinion as to why gaps between SNPE and ASNPE relies on a few observations:

1. Empirically, all methods under prior II benefit from its lower bias (seen very clearly in Figure 2) without suffering too greatly from the higher noise, indicating a particular sensitivity to the prior bias for this congestion setting and time of day.
2. ASNPE can identify useful simulation parameters more quickly than SNPE due its explicit acquisition mechanism, aligned with uncertainty in the NDE model.

Given the sensitivity around the prior in this setting, ASNPE can have an edge over SNPE by making early strides to correct for the bias through exploration, and this can have a significant compounding effect for multi-round inference on such short simulation horizons. Put another way, ASNPE attempts to counter uncertainty in the NDE model by employing a more principled exploration of the parameter space than SNPE, and the effects of doing so are magnified under the higher bias under prior I (and less prominent under the less biased prior II).

Thank you for your time and effort providing detailed feedback on our work. Please find our responses to your questions and comments below.

Note: Where applicable, we prefix sections with W-<x>, Q-<y>, or L-<z> to reference itemized comments in Weaknesses, Questions, and Limitations, respectively, numbered by the order in which they were mentioned.

[W-1] Please see our global rebuttal reply for details on additional empirical evaluation.

Regarding vanishing benefit: While we haven’t been able to experiment with large sample sizes (e.g., of the order 10^4-10^5) in our high-dimensional traffic setting due to computational constraints in our academic environment, we expect ASNPE’s advantage over SNPE to diminish as we collect increasingly large amounts of simulation samples. This is primarily due to the fact ASNPE and SNPE converge to the true posterior in the limit under samples from the simulation model, and both take similar steps to improve the accuracy of the posterior approximation under the observational data of interest. Precisely at what simulation budget we might expect the difference between methods to become negligible, however, is difficult to anticipate, and would require a more principled empirical study.

To this end, we will explore this question alongside the experimental results on the common SBI benchmarks mentioned above in our final manuscript. Given the smaller scale simulation models, we can better characterize the relationship between the dimensionality of the parameter space and the behavior of ASNPE and SNPE over longer simulation horizons.

[Q-1, Q-2, L-1] You raise a valid concern: the cost required for evaluating the acquisition function may not always be worthwhile when the simulation model is very cheap to evaluate. In these cases, more data may be more valuable than a principled exploration of the parameter space, and the time ASNPE spends on active learning may be better spent collecting additional simulation samples.

Precisely characterizing the kinds of problems where we expect the cost for acquisition to be worthwhile is difficult, but in general we posit our method will provide the most benefit in complex settings (e.g., stochastic, non-trivial, large parameter/observation spaces) where simulation samples are limited (due either to slow simulation models, limited computational resources, or both). Our primary traffic case study is a problem very much characterized by these qualities, which is at least in part why we elected to focus most of our efforts on this single problem. In these cases, ASNPE may exhibit large benefits over traditional methods given its more principled exploration of the parameter space, and with sequential inference, early improvements can compound heavily over time (i.e., informative samples produce better proposal distributions, from which the next round’s samples are drawn, and so on).

Regarding ASNPE’s relative computational overhead, note that acquisition evaluation can be performed very efficiently as a batched inference step under the NDE model, and typically yields a negligible difference in terms of raw runtimes between the two methods. For our primary traffic task, the reported wallclock times (found in subfigures (b) for each of the calibration plots) provide the raw runtimes for each method when obtaining the 128 simulation samples for each setting. Empirically, this allows us to compare the total time spent by our algorithm (including acquisition evaluation), and we observe a negligible difference compared to SNPE over our explored horizons.

[Q-3] The convergence of $q_\phi$ to the true posterior is discussed in Section 3.4. In particular, we leverage a result from APT [1] (also known as SNPE-C), which states that training our NDE model parameters via maximum likelihood $\min_\phi \tilde{\mathcal{L}}$, where

$\tilde{\mathcal{L}}(\phi) = -\sum_{i=1}^N \log \tilde{q}_{x,\phi}(\theta_i)$

and

$\tilde{q}_{x,\phi}(\theta) = q_{\phi}(\theta)\frac{\tilde{p}(\theta)}{p(\theta)}\frac{1}{Z(x,\phi)}$ (line 178)

implies $q_\phi(\theta|x) \rightarrow p(\theta|x)$ as $N \rightarrow \infty$ (along with the proposal posterior $\tilde{q}_{x,\phi}(\theta)\rightarrow \tilde{p}(\theta|x)$). This setup allows us to incorporate the true proposal prior $\tilde{p}(\theta)$ in the training loss without additional explicit corrective terms. Further, this limiting behavior holds even when optimizing $\tilde{\mathcal{L}}$ over samples drawn from the transformed distribution under the acquisition filter, as it shares its support with the proposal prior (and the same can be said for any distribution with this quality, in the limit as $N \rightarrow \infty$).

[1] D. S. Greenberg, M. Nonnenmacher, and J. H. Macke. Automatic posterior transformation for likelihood-free inference, 2019.