Inductive Domain Transfer In Misspecified Simulation-Based Inference

Thank you for the constructive and detailed review, and for raising important points to clarify in our work. Below, we address them point by point.

W1+Q1. OT with $\rho=0$

Thank you for this interesting analysis. First, we would like to clarify that the inductiveness of our approach is not inherently due to our choice of $\rho=0$, but rather from how we optimize $g_{\phi}$ (Stage 1) and $q_{\xi}$ (Stage 2), allowing inference without solving an OT problem at test time. Our learning scheme can naturally be extended to $\rho > 0$ by computing an optimal $P_{ij}$ in Stage 1 (Line 7) using RoPE’s OT solver, and integrating the computation of $\alpha_{ij}$, and thus the OT plan, by batch for the training of Stage 2. This process remains inductive. We will clarify this in L129–140.

Second, we believe that one commonly used term for the matching problem in Equation 1 is the one used in L88: "semi-balanced entropic OT problem" (though “semi-relaxed” or “semi-unbalanced” are also used), following recent works like [30], [A, B] with $\rho = 0$ and [C] with $\rho > 0$. This corresponds to a specific instance of unbalanced OT, with $\infty$ as hyperparameter on the first marginal and $\rho$ on the second. After the introduction, we simplified terminology to "OT" or "OT coupling" for conciseness. If the reviewer finds it important to be more specific throughout the text, we welcome precise suggestions.

Finally, we agree with the reviewer’s point on behavior with $\rho = 0$. [D] offers good intuitions on the semi-balanced OT loss without entropic regularization. They show that minimizing the support and masses of the target distribution for a fixed input one (i.e., finding a minimal Wasserstein estimator) generalizes k-means depending on the inner cost. Entropic regularization on $\mathbf{P}$ naturally yields a soft k-means. These problems still differ from ours, as the target support is $\{ h_{w^\star}(x_s^j) \}$, and we optimize $g_{\phi}$ over inputs $\{x_r^i\}$. Therefore, embeddings from simulations act as prototypes (or clusters) over which we fit real sample embeddings. Setting $\rho > 0$ would encourage uniform use of these prototypes for fitting real data (whose validity is questionable in mini-batch settings). However, as mentioned, $\rho = 0$ scales better than $\rho > 0$ thanks to the closed-form; the OT solution does not depend on the sampled batch and provides empirically strong performance. We will clarify this analysis and the referenced arguments in the revised manuscript.

[A] Clark, R. A., Needham, T., & Weighill, T. (2025, April). Generalized dimension reduction using semi-relaxed Gromov-Wasserstein distance. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 39, No. 15, pp. 16082–16090).

[B] Van Assel & al. (2025). Distributional Reduction: Unifying Dimensionality Reduction and Clustering with Gromov-Wasserstein. Transactions on Machine Learning Research.

[C] Van Assel, H., & Balestriero, R. (2024). A graph matching approach to balanced data sub-sampling for self-supervised learning. In NeurIPS 2024 Workshop: Self-Supervised Learning—Theory and Practice.

[D] Canas, G., & Rosasco, L. (2012). Learning probability measures with respect to optimal transport metrics. Advances in Neural Information Processing Systems, 25.

W2. Finetuning objective: clarifying joint OT+supervised objective

The idea was that, unlike in RoPE where the finetuning is done first, then $g$ is fixed and the OT coupling is applied to the embeddings obtained by $g$, in our framework we propose to train $g$ jointly on both the supervised objective (using the calibration set) and the OT objective (using both the calibration set and a large unsupervised set of real observations). The motivation is that, since the calibration set is small and we want to avoid overfitting by the encoder $g$, the OT objective act as a regularizer—especially given the large number of unpaired real observations it is trained on. We will make the quoted sentence clearer in the revised version.

W3. + Q2. Complexity and hyperparameter tuning:

• We would like to clarify that the NPE training stage, followed by the finetuning of $g$, is shared between RoPE and our method. The only additional step we introduce is the amortization stage, which is specifically designed to remove the need for simulation access at test time, a requirement that remains in RoPE, even in its "inductive" variant (RoPE-single sample).

• Hyperparameter selection: In this regard, our approach does not differ significantly from RoPE. In their work, the authors suggest that practitioners, if possible, hold out a validation set of labeled real observations from the calibration set to tune $\gamma$ (and $\rho$), or otherwise rely on default values informed by a sensitivity analysis. Similarly, we provide a sensitivity analysis with respect to $\gamma$ (please refer to W3 in the reply to reviewer TtjD ) and $\lambda$ on the light tunnel benchmark (w.r.t LPP).

The $\lambda$ parameter was analyzed on the light tunnel benchmark with synthetic noise added at a 10% level. We observe that as long as $\lambda > 1$, the results remain relatively stable.

• Calibration set: When we refer to the calibration set, we mean the small set of labeled real observations $(x_r, \theta)$. In both our framework and the RoPE baselines, there is only one calibration set, which is used to train the supervised objective. We kindly ask the reviewer to point us to the source of this confusion so that we can clarify it appropriately in the revised version.

Questions:

Q1. $\rho=0$ discussion, posterior computation, self-calibration.

• For the discussion about the choice of $\rho=0$ please refer to W1.
• The posterior approximation in Stage 1 for $\rho = 0$ remains unchanged and is expressed as a mixture of simulated posteriors, weighted by the optimal transport plan coefficients: $\tilde{p}(\theta \mid x_r^i) := \sum_{j=1}^{N_{\text{OT}}} \alpha_{ij} \, q_{\psi^\star}(\theta \mid h_{\omega^\star}(x_s^j))$, where $\alpha_{ij} := N_u P_{ij}^\star$. Setting $\rho = 0$ does not alter the assumptions underlying RoPE’s equation (Eq. 4): $p(\theta \mid \mathbf{x}_r) = \int p(\theta \mid \mathbf{x}_s) \pi^{\star}(\mathbf{x}_s \mid \mathbf{x}_r) d\mathbf{x}_s$, but only changes how the distribution $\pi^\star$ is approximated.
• Indeed, we cannot guarantee the self-calibration property as in RoPE (balanced RoPE without the relaxation of the column-marginal constraint). Since in our case we don't constrain the column-marginal at all, we cannot guarantee that $\sum_{i=1}^{N_u} P_{ij}^\star = \frac{1}{N_{OT}}$, so the derivation of the self-calibration property of balanced RoPE (RoPE appendix, p. 20) does not hold. However, in practice RoPE also relaxes this constraint, so self-calibration is not guaranteed as well.

Q2. How should users tune $\lambda$ ?

Please refer to W3.

Q3. OT+supervised terms purpose

The calibration set $D_{calib}$ is a small supervised set of pairs $(\theta, x_r)$, from which we can simulate the corresponding $x_s = S(\theta)$ (with $S$ being the simulator). It is used to train the supervised objective, specifically, to finetune the encoder $g$ so that it embeds $x_r$ as close as possible to its corresponding $x_s$. The entropic OT term compensates for the uncertainty in the supervised matching, which arises due to the limited size of the supervised calibration set. It enables a soft matching between $g(x_r)$ and $h(x_s)$, and also allows us to train on a large unlabeled set of real observations $x_r$, denoted as $D_{OT}$. As a result, $g$ is optimized jointly on both the supervised and OT objectives. We observe that the pure OT baselines perform worse than both RoPE and our proposed method. Additionally, in the $\lambda$ sensitivity experiment shown above, we demonstrate that heavily down-weighting the supervised loss (e.g., $\lambda = 0.01$) leads to significantly degraded performance.

Q4. CS and SIR tasks

We chose not to analyze the CS or SIR tasks, as the RoPE paper indicated that the level of misspecification in these cases was limited and vanilla NPE or OT without a calibration set already performed well. Instead, we introduced a more complex use case: the real cardiac biomarker inference task from the arterial pressure wave benchmark. This represents a substantially different challenge compared to the RoPE cardiac biomarkers task, which was entirely synthetic.

Q5. results consistency

We did not have access to the RoPE code, so we implemented our own version. Naturally, this may lead to some variations in results. However, the main reason for the differences in baseline scores is the use of batch normalization. As noted in the implementation details (p. 22–25, appendix) of the RoPE paper, they do not use batch normalization in the NSE encoders, which results in a very low LPP for the vanilla NPE baseline in the highly misspecified benchmarks. To ensure a fair comparison, we added batch normalization to all baselines in our implementation (we will mention it in the revised manuscript).

Thank you for your clarifications regarding the inductiveness of the approach and hyperparameter selection. Apologies for misusing the term "calibration set", I am aware that there is only one. The other set I was referring to was $D_{unpaired}$. The paper is sufficiently clear on this point.

Thanks for the references regarding the semi-balanced OT with $\rho=0$. This addresses my concerns regarding terminology. Whether $\rho=0$ is considered to be semi-balanced OT seemingly depends on the setting (another example is this paper, where again $\rho>0$ [1]). I still think the method the authors end up with can be framed without any OT (kernel weighted posterior mixture), and the paper would benefit from at least discussing this, in my view, simpler framing.

[1] Janati, H., Muzellec, B., Peyré, G., & Cuturi, M. (2020). Entropic optimal transport between unbalanced gaussian measures has a closed form. Advances in neural information processing systems, 33, 10468-10479.

Regarding the experiments:

First, I'm not convinced by the author's reasoning to not include SIR and CS (may it be in the appendix). For a paper that is so closely aligned with or based on another paper (RoPE), testing on the full benchmark suite proposed by said paper would be valuable.

Specifically, the argument that these tasks are not "misspecified enough" is questionable. Wouldn't it be good to see that the proposed method still works well even if the level of misspecification is not large? Do users always know the level of misspecification beforehand? A method tackling misspecification should produce reasonable results even if the simulator is not, or only slightly, misspecified, no?

Second, it is unfortunate that RoPE does indeed not seem to provide code, and this is certainly not the author's fault. I agree that some variations in results may be expected when reimplementing a method.

However, for some settings/experiments the differences are quite large? And why would the use of batch normalization in encoders have such a large effect on the LPP? I really don't understand this. Could the authors explain? Are there other differences in implementation to the ones listed in the RoPE appendix?

I appreciate the additional experiments and clarifications. I don't have any more questions at this point, and I will raise my score to a 4.

Thank you for the constructive and encouraging review, and for highlighting both the strengths of our approach and valuable directions for comparison and clarification. We address the reviewer’s comments and suggestions below.

W1. comparison with non-calibration set baselines

This is indeed a very important point. In the ICML paper of the baseline RoPE work, the authors report the performance of two baselines without a calibration set: NPE-RS (Huang et al.) and NNPE (Ward et al., "Robust Neural Posterior Estimation and Statistical Model Criticism"). They show that while these methods perform better than vanilla NPE, they still fall short compared to baselines that incorporate a calibration set — even when the calibration set is small. We will add this to the discussion of the baseline experiment results (Section 4.3.1).

W2. As a minor exposition point, I recommend to state clearly the dimensions involved in each problem; that is parameter dimensionality (I think it's 2-4-1-4 or similar) and the dimensionality of the summary statistics. Relatedly, it'd be useful to know to which dimension (of parameters and data) is the method expected to scale.

Thank you for the comment. We will clarify this point in the revised manuscript. In addition, we will include a discussion on scalability with respect to dimensionality. Indeed, we consider here only low-dimensional use cases, which are common in various real-world applications. However, the discussion regarding applicability to higher-dimensional settings is similar to that presented in the baseline RoPE work. The higher-dimensional parametric space identifiability problem could be solved by generating and training the NPE over more simulated samples.

The main limitations in higher dimensions are twofold:

1. The misspecification between simulated and real interactions across the different $\theta_k$ could become more significant with the increase in dimension, increasing the complexity of the matching of embeddings of real observations with simulated ones. This requires larger sizes of calibration sets.
2. The embedding dimension required to encode sufficient statistics for predicting the posterior is expected to grow with the parameter dimensionality, which may complicate the coupling problem. In Chen et al. [1], they propose and validate a heuristic where the dimension of the summary statistics embedding is set to $d = 2K$, where $K$ is the dimension of $\theta$. In our implementation, we use an overparameterized embedding dimension of 16 regardless.

[1] Chen Y, Zhang D, Gutmann M, Courville A, Zhu Z. Neural approximate sufficient statistics for implicit models. Proceedings of the International Conference on Learning Representations (ICLR), 2021.

However, we expect our framework to alleviate these limitations. Specifically, the joint training of the OT and supervised objectives could help in situations where the small calibration set is insufficient to project the representation of the real observations onto the simulated ones. This can be balanced with the unsupervised OT coupling, which can be trained on an abundance of unlabelled data.

Moreover, our choice of using $\rho=0$ benefits from a closed-form solution for the coupling, computed independently for each sample in $D_u$, involving $N_{OT}$ operations. Each operation consists of computing the Euclidean distance between a pair of embeddings in $\mathbb{R}^d$, which is achieved in $O(d)$. Since we operate in the mini-batch OT setting we have a complexity at each step of $B_t N_{OT} d$, where $B_t$ is the batch size. In comparison, RoPE which considers $\rho>0$ and cannot operate on mini-batches has a complexity of $O(log(N_u N_{OT})N_u N_{OT} d)$ where $N_u$ is the overall number of real observations. Hence, our approach would scale better than RoPE in these high-dimensional settings.

Thank you for the constructive review and for highlighting the practical benefits of our approach and key areas for improvement. We address the reviewer’s concerns point by point below.

Weaknesses:

W1. An intuitive explanation of optimal transport is not provided in the main text. This can make it somewhat hard to understand for the readers.

Thank you for the suggestion, we will clarify the motivation and operations involved in Equation 1 in the corresponding section of the revised manuscript.

W2. The method seems to be suspectible to noise in the calibration set, which can potentially limit its applicability in a wide range of real world problems where calibration sets are scarce and noisy.

It is true that we have identified some sensitivity to noise in the synthetic noise experiment. However, our approach worked better than the RoPE-single-sample baseline, and since RoPE-full-test is calculated and evaluated on the test set itself, it is perhaps able to couple even noisy samples when the noise model is relatively simple.

It should be emphasized that in the real noisy benchmark (the cardiac biomarkers), our method outperformed RoPE. For clarity, we will emphasize this point in the revised manuscript in the results discussion.

W3. The paper uses mini-batch OT with a fixed entropy regularization parameter gamma, but does not explore how sensitive results are to this value.

We performed a sensitivity analysis w.r.t the entropy regularisation param $\gamma$ on the light tunnel benchmark. The results below are for the lpp metric.

Like in RoPE, we observe for our method that when the calibration set is small, it is beneficial to select a larger value for the entropy regularization parameter, as the OT matching becomes more diffuse i.e closer to an uniform distribution. On the other hand, when the calibration set is large (e.g., 1000 samples), it is more likely that a good pairing exist between those and the real data, hence we can reduce $\gamma$ to obtain a sharper coupling.

W4. + Q2 The method is evaluated only on problems with up to four parameters. SBI applications often involve ten or more parameters. It is unclear how the OT coupling or the normalizing flow posterior will scale in such regimes, or whether performance degrades under the curse of dimensionality.

Indeed, we consider here only low-dimensional use cases, which are common in various real-world applications. However, the discussion regarding applicability to higher-dimensional settings is similar to that presented in the baseline RoPE work. The higher-dimensional parametric space identifiability problem could be solved by generating and training the NPE over more simulated samples.

The main limitations in higher dimensions are twofold:

1. The misspecification between simulated and real interactions across the different $\theta_k$ could become more significant with the increase in dimension, increasing the complexity of the matching of embeddings of real observations with simulated ones. This requires larger sizes of calibration sets.
2. The embedding dimension required to encode sufficient statistics for predicting the posterior is expected to grow with the parameter dimensionality, which may complicate the coupling problem. In Chen et al. [1], they propose and validate a heuristic where the dimension of the summary statistics embedding is set to $d = 2K$, where $K$ is the dimension of $\theta$. In our implementation, we use an overparameterized embedding dimension of 16 regardless.

[1] Chen Y, Zhang D, Gutmann M, Courville A, Zhu Z. Neural approximate sufficient statistics for implicit models. Proceedings of the International Conference on Learning Representations (ICLR), 2021.

However, we expect our framework to alleviate these limitations. Specifically, the joint training of the OT and supervised objectives could help in situations where the small calibration set is insufficient to project the representation of the real observations onto the simulated ones. This can be balanced with the unsupervised OT coupling, which can be trained on an abundance of unlabelled data.

Moreover, our choice of using $\rho=0$ benefits from a closed-form solution for the coupling, computed independently for each sample in $D_u$, involving $N_{OT}$ operations. Each operation consists of computing the Euclidean distance between a pair of embeddings in $\mathbb{R}^d$, which is achieved in $O(d)$. Since we operate in the mini-batch OT setting we have a complexity at each step of $B_t N_{OT} d$, where $B_t$ is the batch size. In comparison, RoPE which considers $\rho>0$ and cannot operate on mini-batches has a complexity of $O(log(N_u N_{OT})N_u N_{OT} d)$ where $N_u$ is the overall number of real observations. Hence, our approach would scale better than RoPE in these high-dimensional settings.

We notice that the points regarding dimensionality and scalability are repeatedly brought up by the reviewers, and we are planning to add a discussion of these aspects in the Conclusions and Limitations section.

Questions

Q1. To make sure I understand correctly: the supervised calibration loss is mainly useful when the calibration pairs are accurate, while the OT loss provides robustness when those pairs are noisy or mismatched. Is that correct?

Indeed, that is the case, but not only. Recall that the main setting we consider here is one where the calibration set is too small for a purely supervised domain transfer. To address this, we introduce an additional unsupervised OT loss. The OT loss provides a regularisation for matching real observations with simulated ones, even when the calibration set is small, which is the basic setting of the problem we aim to solve.

This effect is evident from the baseline experiments in Figure 2, where we observe that our method outperforms the finetune-only baselines at small calibration set sizes. However, as the size of the calibration set increases, the performance gap diminishes.

Q2. dimensionality and scalability

Please refer to W4

We thank the reviewer for the thoughtful analysis of our work and address next the main concerns:

weaknesses

Clarity:

W1. Section 2.2 (lines 83-97): Adding OT background.

We will clarify in the revised version the introductive parts on OT as the reviewer suggests, specifically in the Entropic OT coupling paragraph section 2.2. (essentially explaining what we mean by soft matching)

W2. Section 3.2: Clarifying final posterior estimation and amortisation

Perhaps the following explanation can make Section 3.2 more intuitive and clear. From Section 3.1 (referred to as "joint training only" in the ablation study on p.8), we obtain an estimated posterior density that is effectively a mixture of simulation-based posteriors, with mixture coefficients learned through the method in 3.1. What Section 3.2 introduces is a way to infer this density directly from the real observation $x_r$. The idea of using such a mixture as an approximation of the true posterior $p(\theta \mid x_r)$ is derived by prior work, RoPE (see Eq. (4)--(5), Section 3). The key insight from RoPE is that the OT plan approximates the joint distribution over $(x_r, x_s)$. By leveraging this approximate joint distribution, we can marginalize over $x_s$ (which becomes a discrete sum in our setting) to estimate the posterior [to be clarified in the paper Section 2.2, L97]

W3. Figure 1: Clarifying visualisation.

Thank you for these suggestions that will be taken into account to improve Figure 1 in the revised manuscript. We will clarify our workflow by illustrating more clearly the inputs/outputs, which parts are optimized and the objectives they are trained on. Additionally, we will include a visualization of the baseline vanilla NPE (and NSE), which serves as a preliminary stage for our framework on top of the figure.

W4. Dataset notation.

We thank the reviewer for the helpful suggestion, we will add brief sentences in the Training procedure to describe each considered dataset.

Quality:

W5. + Q4: dimensionality and scalability analysis:

Thank you for this interesting remark. Indeed, we consider here only low-dimensional use cases, which are common in various real-world applications. However, the discussion regarding applicability to higher-dimensional settings is similar to that presented in the baseline RoPE work. The higher-dimensional parametric space identifiability problem could be solved by generating and training the NPE over more simulated samples.

The main limitations in higher dimensions are twofold:

1. The misspecification between simulated and real interactions across the different $\theta_k$ could become more significant with the increase in dimension, increasing the complexity of the matching of embeddings of real observations with simulated ones. This requires larger sizes of calibration sets.
2. The embedding dimension required to encode sufficient statistics for predicting the posterior is expected to grow with the parameter dimensionality, which may complicate the coupling problem. In Chen et al. [1], they propose and validate a heuristic where the dimension of the summary statistics embedding is set to $d = 2K$, where $K$ is the dimension of $\theta$. In our implementation, we use an overparameterized embedding dimension of 16 regardless.

[1] Chen Y, Zhang D, Gutmann M, Courville A, Zhu Z. Neural approximate sufficient statistics for implicit models. Proceedings of the International Conference on Learning Representations (ICLR), 2021.

However, we expect our framework to alleviate these limitations. Specifically, the joint training of the OT and supervised objectives could help in situations where the small calibration set is insufficient to project the representation of the real observations onto the simulated ones. This can be balanced with the unsupervised OT coupling, which can be trained on an abundance of unlabelled data.

Moreover, our choice of using $\rho=0$ benefits from a closed-form solution for the coupling, computed independently for each sample in $D_u$, involving $N_{OT}$ operations. Each operation consists of computing the Euclidean distance between a pair of embeddings in $\mathbb{R}^d$, which is achieved in $O(d)$. Since we operate in the mini-batch OT setting we have a complexity at each step of $B_t N_{OT} d$, where $B_t$ is the batch size. In comparison, RoPE which considers $\rho>0$ and cannot operate on mini-batches has a complexity of $O(log(N_u N_{OT})N_u N_{OT} d)$ where $N_u$ is the overall number of real observations. Hence, our approach would scale better than RoPE in these high-dimensional settings.

We notice that the points regarding dimensionality and scalability are repeatedly brought up by the reviewers, and we are planning to add a discussion of these aspects in the Conclusions and Limitations section.

W6. + Q3 LPP metric details: LPP computations and number of samples

The number of test samples for the pendulum/causal-chamber benchmarks is mentioned in lines 226-227 (1000 for all, the same scale of test set size as in RoPE). Indeed for the pulse-waves benchmark it is not explicitly mentioned, but it's also 1000. Thank you for noticing, we will add it to the revised manuscript. As for the computation of LPP, there is no need to sample from the posterior as the normalising flows model models the density function of the posterior and we just derive from it the log density value of the ground truth parameter. For the calibration score, the acauc, we sample a 1000 samples from each posterior density.

W7. Hemodynamics calibration data: data collection & labelling process

Indeed, the process of aligning the CO measurements obtained via the thermodilution method to the corresponding ABP segment is important for understanding the root cause of the noisy labels. Due to space constraints, we did not elaborate on this in the paper, but we will include a paragraph about it in the supplementary material in the revised manuscript.

### Significance:

W8. Calibration data requirement: being more explicit about this requirement. We mention the use of a calibration set and the semi-supervised settings in lines 37-39 in the intro when mentioning the prior work of RoPE. However we can make it clearer that we follow the same settings in our work.

Questions

Q1. Section 3.2, amortisation clarification.

The intuition is correct. As explained in response to W2 RoPE has already showed that the real posterior can be approximated by the mixture of simulated posteriors obtained by the coupling. So indeed the $\alpha_{ij}$ correspond to the level of "match" between the parameters $\theta$ of a real observation and those of a simulated one. In 3.2 however, we amortise this mixture-based density using normalising flows.

Q2. For the real hemodynamics data, how is the calibration dataset obtained? How are real $\theta$ values measured in practice?

The heart rate is measured from ECG and the cardiac output by the invasive thermodilution. Then the HR and CO measurments are aligned with the corresponding pulse-wave segment using the patient's electronical record, which can induce noisy missaligned pairs of $x$ and $\theta$. We can elborate about that in the revised manuscript in the supp. material.

Q3. How many samples were used to compute the LPP metric? This is important to avoid bias as noted in Lueckmann et al. 2021.

Please refer to W6

Q4. How does the method scale to higher-dimensional parameter spaces? The current evaluation is limited to relatively low-dimensional problems. Please refer to W5

Q5. Have you explored approaches to mitigate the reduced posterior confidence (ACAUC < 0) observed with larger calibration sets?

We believe that it is essentially a matter of hyperparameter tuning on ($\gamma$, $\lambda$), that we did not empirically explored. We envision solutions for future works considering for instance adaptive strategies for $\lambda$ given $\gamma$. One of them could be to merge both losses into a unified transport problem with additional constraints on the transport plan where $\lambda$ would become similar to a lagrangian multiplier.