Diffusion Bridge Implicit Models

Thank you for your comments. Despite negative, your constructive suggestions are very helpful for improving our work. We provide our responses below.

Can the authors clarify how their work presents any novel insights compared to the original DDIM paper (or other works mentioned here [1,2]) since this is not obvious to me from the current explanation in the main text?

Thank you for giving us the opportunity to explain where our contribution lies. We are aware of your mentioned works, and we fully understand and agree with your opinion that the underlying idea has been well studied in works like generalized DDIM. For example, we frequently compare with DDIM and related works in Table 1, Appendix C.3/C.4. We believe we have given proper position of our work in the literature and made as many theoretical investigations as we can. We want to argue that, extending these ideas to certain field (diffusion bridge) exhibits differences from prior work, provides new insights, and makes valid contributions to the community.

• Theoretically, the generalized DDIM works are distinct from DDBM: (1) though the coefficients before $x_t$ and the network are quite general, even including non-isotropic ones, they don't have the $x_T$ term as DDBM (Equation (35)) (2) DDBM ends in a delta distribution and has singularity at time $T$, so the first sampling step is forced to be stochastic. However, the forward process considered in generalized DDIMs never ends in delta distribution and never encounters or handles this singularity issue. If DDBM cannot be subsumed by generalized diffusion, then DBIM cannot be subsumed by generalized DDIM.
• Technically, we believe that being simple is important and requires insights. For example, though gDDIM is applicable for general $F$ and $G$, the integral in its Equation 19 only admits closed-form solution in certain scenarios, and deriving these solutions takes effort. If you look at Equation (35) in our paper, the coefficients are much more complex than the standard diffusion model. Through the DDIM-inspired method, we derive the simplified ODE quite neatly. Another example is that, a key insight of DPM-Solver[2] compared to DEIS[1] is to use the change-of-variable $\lambda_t=\log(\alpha_t/\sigma_t)$ (half of the log-SNR). This greatly simplifies the process of developing high-order solvers, making DPM-solver simpler to implement and easier to follow by subsequent works. We also develop such an insight for DDBMs in our paper: $\lambda_t=\log\left(\frac{b_t}{c_t}\right)=\frac{1}{2}\left(SNR_t-SNR_T\right)$. This is quite different from DPM-Solver, and is novel and insightful to make high-order diffusion bridge solvers work.
• Practically, we have released the code at the anonymous site https://anonymous.4open.science/r/DBIM. We feel that though DDBM has aroused a lot of follow-up works, they all rely on the cumbersome codebase and formulations presented by DDBM. Besides, they are unaware of the possible simplifications and accelerations. Our work definitely provides a fresh perspective to the diffusion bridge community. We believe it can benefit other works based on DDBM, and can inspire future studies. We sincerely hope that our contribution to the community can be recognized. We also note that [3], focusing on fast sampling of a process that is more similar to standard diffusion than us, is well recognized by its reviewers.

[1] Fast sampling of diffusion models with exponential integrator [2] DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps [3] MRS: A Fast Sampler for Mean Reverting Diffusion based on ODE and SDE Solvers (ICLR2025 Submission, current score 8865)

I would recommend updating Table 3 to remove some baselines like DDRM and Pi-GDM and instead compare DBIM with the latest flow matching or interpolant models.

Thank you for your suggestion. We have revised the table and use gray color to annotate works that don't require paired training. As for flow matching or interpolant models, we find them mainly focused on image generation instead of image translation tasks, and we don't have their results on our settings (edge2handbags, DIODE, ImageNet inpainting), so we are unable to add them. We have discussed the relationship between DBIM and flow matching in Appendix C.3.

I want to thank the authors for taking the time to address my concerns and apologies for a delayed response. Here are my thoughts on the author response:

Firstly, the authors point out,

Theoretically, the generalized DDIM works are distinct from DDBM: (1) though the coefficients before 
 and the network are quite general, even including non-isotropic ones, they don't have the 
 term as DDBM (Equation (35)) (2) DDBM ends in a delta distribution and has singularity at time 
, so the first sampling step is forced to be stochastic. However, the forward process considered in generalized DDIMs never ends in delta distribution and never encounters or handles this singularity issue. If DDBM cannot be subsumed by generalized diffusion, then DBIM cannot be subsumed by generalized DDIM.

I acknowledge the fact that DDBM's have a different structure than gDDIM (of course they do since they are bridges between arbitrary diffusions as compared to DDIM which creates a transport map between Gaussian and data distributions). However, this was not my main concern. As pointed in my review,

The main idea of all these methods is to modulate the drift of the reverse diffusion/flow process to enable fast sampling during inference (See also [3] for a similar approach). Since DDBMs also possess a semi-linear drift, extending these generic frameworks to obtain the corresponding DDIM sampler for DDBMs is trivial

DDBM like other diffusion/flow models has a semi-linear drift and so it feels trivial to construct samplers like DBIM by existing theory in exponential integrators, conjugate integrators etc.

Secondly, the authors point that,

Technically, we believe that being simple is important and requires insights. For example, though gDDIM is applicable for general and, the integral in its Equation 19 only admits closed-form solution in certain scenarios, and deriving these solutions takes effort. If you look at Equation (35) in our paper, the coefficients are much more complex than the standard diffusion model

While I agree on the notion of simplicity and acknowledge the contribution in deriving DPM-Solver for bridges, in the worst case implementing gDDIM requires 1 dim numerical integration of coefficient which is trivial so I don't agree here with the latter claim.

Lastly, regarding empirical aspects, the authors state,

As for flow matching or interpolant models, we find them mainly focused on image generation instead of image translation tasks, and we don't have their results on our settings (edge2handbags, DIODE, ImageNet inpainting), so we are unable to add them

I still feel interpolants are a valid baseline since, in fact, they form a bridge between arbitrary distributions. A lot of recent work in this field has focused on improving interpolants. In light of these improvements, adding such a baseline is important since it informs us when we should use DDBM vs an interpolant.

In light of the theoretical arguments made by the authors, I would increase my rating from 3 to 5; however, I am still leaning towards a rejection since I feel the methodological contributions are marginal and the empirical baselines need to be improved for more clarity on the utility of this method.

Thank you for your positive comments. We provide our responses below.

To conclude, the methods SinSR & DBIM (proposed here) are not directly equivalent but one can be derived analogously to the other one with some effort.

Thank you for informing us of this work, and we have added it to the related work section. We think it is better to call SinSR a mean-reverting diffusion process, instead of a diffusion bridge process, as we find it similar to [1]. Besides, compared to us, SinSR did not connect it to ODE or develop high-order solvers.

[1] MRS: A Fast Sampler for Mean Reverting Diffusion based on ODE and SDE Solvers (ICLR2025 Submission, current score 8865)

Thank you for your positive comments. We provide our responses below.

the notation of $q_{tT}$ in Eq.(7) is confusing.

Thank you for pointing out. We have fixed it to $q_{T|t}$.

NIT - the literature section should be revised significantly, for example, Schrodinger in line 502: Schr"odinger, schrodinger in line 509, 600, 606, sde line 521, rosenbrock in line 543, Dpm-solver in line 576, ode in line 644 should have properly capitalized letters.

Thank you for your suggestion. We are unsure about whether we should manually change the titles in the references. We export them directly from Google Scholar. We are afraid that changing them may make the academic indexing systems unable to recognize them.

DDBM proposed the pred-x parameterization motivated by the EDM framework to enhance the predictions and derive a set of scaling functions for distribution translation. Is your work applicable to this extension? If not, what are the challenges?

Sure, we are already using the pretrained DDBM checkpoints for experiments on edge2handbags and DIODE, which apply complex scaling and skip connections to the network input and output. As detailed in Appendix F.1, as long as we have a "wrapper" that takes xt and outputs x0-prediction, we can substitute it into our algorithm. We have released the code at the anonymous site https://anonymous.4open.science/r/DBIM, and welcome to refer to the DDBMPreCond class in ddbm/karras_diffusion.py.

Any intuitions on why a non-Markovian property is preferred?

Intuitively, the non-Markovian process reduces the stochasticity in each step (in Equation 11, the variance of the Gaussian is smaller). Using a more deterministic process can reduce the variance of the sampling trajectory and enable faster convergence to the target.

some discussions on OT-accelerated diffusions, such as [1,2], may be helpful.

Thank you for your suggestions. We have added them to the diffusion Schrodinger bridge part in the related work section.

Once again, thank you for your constructive feedback and for considering our paper for acceptance. We have revised our paper accordingly.

Thank you for your positive comments. We appreciate that you carefully read the math and proofs. We provide our responses below.

I didn't find a reference to this section from the main text. Furthermore, I think it is important to somehow cover related works in the main text (maybe, with some references to the appendix)

Thank you for your suggestion. We have added a small related work section in the main text with references to the appendix.

No source code in the submission. It is important for reproducibility.

We agree. We have released the code at the anonymous site https://anonymous.4open.science/r/DBIM to ensure reproducibility.

Appendix C.1, lines 892-897. Why do you take gradient $\nabla_{x_{t_{n+1}}} \log q^{\rho} (x_{t_{n + 1}} \vert x_0, x_T)$, not $\nabla_{x_{t_{n+1}}} \log q^{\rho} (x_{t_{n + 1}} \vert x_0, x_{t_n}, x_T)$. Also, the eq. 50 is incorrect, while the result (lines 904 - 906) is correct. Please check the derivation.

Thanks for pointing out! We do miss $x_{t_n}$ in several places and have fixed them.

Line 256 - The statement that $x_T$ is cancelled out under the choice of $\rho_n$ seems to be correct, but a link to the proof should be given.

Thank you for your suggestion. We briefly prove it in the footnote.

Do I understand correctly, that different variants of variance parameter $\rho$ appear only at inference, i.e., training with eq. 13 is agnostic to $\rho$?

Yes, as $\rho$ only affects the relative loss weightings at different timesteps for DBIM. It does not affect the global minimum, which is minimum at all timesteps. Appendix C.2 also gives explanation.

You mention that eq. (8) (from DDBM) is equivalent to (16). But on par with (16) you have additional booting noise required by theory. How previous works (e.g., DDBM) manage to avoid the necessity of this noise?

DDBM divides each step into 2 substeps: (1) simulating SDE and (2) simulating ODE. Therefore, the first step is always simulating the SDE, which has no singularity. $\eta=1$ in DBIM can be seen as a better discretization of the SDE than the Euler step.

Lines 427 - 428: ‘straightforward mapping without the involvement of stocahsticity’ - but you have booting noise still, yes?

Yes, our sampling always has the booting noise. We are expressing that maybe directly mapping the condition is enough. But in practice, the first step needs stochasticity to avoid the singularity. Only stochasticity in intermediate timesteps can be omitted.

Once again, thank you for your constructive feedback and for considering our paper for acceptance. We have revised our paper accordingly.

Thank you for your positive comments. We provide our responses below.

The diversity of the proposed method should be evaluated quantitatively.

Thank you for your suggestion. On the ImageNet inpainting task, following CMDE and BBDM, we calculate the standard deviation of 5 generated sampled (numerical range 0~255) given each observation (condition), averaged over all pixels and 1000 conditions.

The diversity score keeps increasing with more NFE. Surprisingly, we find that the our $\eta=0$ case exhibits larger diversity score than the $\eta=1$ case. This demonstrates that the booting noise can introduce enough stochasticity to ensure diverse generation. Moreover, the $\eta=0$ case tends to generate sharper images. This may favor the diversity score which is measured by pixel-space variance.

Can you compare your method with I2SB for inpainting with freeform masks and evaluate I2SB on small NFEs?

Our inpainting model on ImageNet is trained by ourselves, not directly taken from I2SB. The reason is that, the checkpoints I2SB released are actually not the standard bridge as DDBM: for inpainting, they start from condition with randomly added noise, instead of clean condition; starting from the noisy condition, they set the intermediate noise level as 0, so no more noise is additionally added, and the model becomes an interpolant (or flow matching) model.

As the training takes around 2 weeks on 8xA100 GPUs, we are unable to additionally training the free-form inpainting model in the rebuttal period. Center inpainting is a more difficult task than freeform inpainting (as in I2SB paper, its FID is higher), so we believe experiments on center inpainting is convincing. We report the FIDs of I2SB (center inpaint) under small NFEs as follows:

We currently compute the coefficients $a_t,b_t,c_t$ in 64-bit to enhance precision, so the DBIM result is slightly better than the original paper. I2SB also performs reasonably under small NFE, as its posterior sampling is the same as DBIM($\eta=1$) under its noise schedule (Appendix C.3).

Can you discuss the limitations of your method or provide failure cases?

Sure. Please refer to the conclusion section in the revised paper.

Can you comment on how much slower high-order methods are than first-order methods regarding inference time? Does the quality improve further for bigger orders like 4? Can you additionally provide inference time for DBIM and other methods to show the inference time acceleration?

We report the average sampling time (in seconds) of 8 batches on a single A100, under the batch size of 16.

As the networks are of the same architecture and size, the inference time is approximately proportional to the NFE. Other costs, like computing the coefficients in the sampler, are negligble. Higher orders than 3 will not further improve the quality. With higher orders, the estimation error of the higher-order derivatives will also increase. As seen in Table 6, the improvement from 2nd-order to 3rd-order is already small and even worse. According to the experiences in diffusion model solvers, order 3 or even 2 is the best.

Discuss some missing related work

Thanks for pointing out! We have added proper citations and discussions in the related work section.

Typo: It looks like in Eq. 7, it should be $\bar w_t$ instead of $w_t$.

Thanks for careful reading! We have fixed it in the revised paper.

Once again, thank you for your constructive feedback and for considering our paper for acceptance. We have revised our paper accordingly. We have also released the code at the anonymous site https://anonymous.4open.science/r/DBIM to ensure reproducibility.