Implicit Bridge Consistency Distillation for One-Step Unpaired Image Translation

W1. In Eq. (6), two PF-ODEs starting from different domains are connected together at $\sigma_{max}$, how to guarantee the smoothness of score function (i.e., gradient) at this point (since one directly uses noisy to solver attached to domain B)? If not smooth, how will the error be like?

We thank the reviewer for the insightful question. While IBCD introduces a continuous but non-differentiable point at the center of the PF-ODE trajectory ($t=0$), we argue that Theorem 1 of the original CD paper (Appendix A.2) remains applicable.

First, we consider the Lipschitz condition for $f_\theta (x_t,t)$. Since the primary difference between IBCD and CD lies in the $t$ direction, we focus on this aspect. The output of $f_\theta$, which predicts the clean target domain image, remains constant along a given PF-ODE trajectory, regardless of $t$. Therefore, the Lipschitz condition is not affected by the non-differentiable point while the trajectory is continuous. Since the change in the $x_t$ direction is not different from CD, we can still use their Lipschitz assumption.

Second, we examine the local truncation error of the ODE solver. The non-differentiable point is precisely captured by our discretization scheme. The gradient used at this point is a combination of gradients from both sides of the trajectory, ensuring stable numerical integration. For example, consider an Euler solver. For the forward direction (domain A to B):

• The interval $i=[-1,0]$ uses the gradient at $-1$ (from domain A).
• The interval $i=[0,1]$ uses the gradient at $0$ (from domain B).

On the other hand, for the Backward direction (domain B to A):

• The interval $i=[0,1]$ uses the gradient at $1$ (from domain B).
• The interval $i=[-1,0]$ uses the gradient at $0$ (from domain A).

Consequently, due to the properties of the consistency function and our careful handling of the non-differentiable point, the error bound of the consistency function remains $O((\Delta t)^p)$ which is same as the original CD.

W2. In L261, the authors claim one of the challenge is to employ only local consistency. However, this should be called a global consistency following [1]. I can hardly understand why such strategy is a challenge, given that most distillation works use such a loss.

In response to the reviewer's comment, we would like to clarify that the consistency loss employed in IBCD is indeed a form of local consistency (vanilla CD), as defined in CTM. As extensively discussed in Section 5.2 and Appendix C.3 of CTM, this local consistency has key limitations, primarily its recursive nature.

We argue that this recursive nature of local consistency increases distillation errors in IBCD, as we will explain later. However, we have also revised our analysis of the sources of distillation errors in vanilla IBCD to adopt a more nuanced perspective:

1. Model Capacity Constraints: Unlike other methods, our student model is tasked with learning a bidirectional translation function using a single model, inherently limiting its capacity. Previous work [2] has demonstrated that bidirectional consistency models can underperform compared to vanilla CM.

2. Combination of Different ODEs: Unlike the Teacher model, where the two ODEs share a timestep and are differentiated solely by their initial conditions, the two ODEs in IBCD are entirely separate and interconnected. This can similarly impact model capacity and learning complexity.

3. Recursive Nature of Local Consistency Loss: As the local consistency loss is applied recursively, local fitting errors accumulate sequentially from the boundary condition to the trajectory's end. Consequently, the translation process, involving a longer path, incurs a larger accumulated distillation error compared to the generation process (updated Figure 8).

W3. The authors state that vanilla IBCD faces mean prediction phenomenon, but provides no convincing analysis on it. Original CD seems not to face such challenge.

Based on your review, we acknowledge the lack of theoretical support for the mean prediction phenomenon. While we initially introduced this phenomenon to illustrate an example of distillation error in vanilla IBCD, we have removed this content from the main text. Instead, we now focus on why distillation error is an issue in vanilla IBCD (W2 and section 3.2).

Our auxiliary losses were designed to address this fundamental distillation error and provide flexibility in balancing reality-faithfulness, not to specifically target the mean prediction phenomenon. Therefore, our core argument remains unaffected by this change.

W4. The ablation study is somewhat confusing. Why vanilla IBCD is only a point rather than a broken line like the others?

Per your suggestion, we've modified the vanilla IBCD model to also incorporate a trade-off curve format. The data point prior to the update signifies the initial state of the model before additional training with the auxiliary loss function.

W5. I cannot see the efficacy of DMCD and cycle loss in toy data.

All components contribute to the effectiveness of the toy experiment. When DMCD is applied alone, it reduces the number of samples translated to low-density regions, as evidenced by the decreased density between the spirals. However, this comes at the cost of reduced modal coverage, leading to thinner spiral arms. Introducing cycle loss helps to mitigate this issue by maintaining the reduction in low-density translations while expanding the modal coverage, resulting in wider spiral arms and lower density between spirals. The recovery in modal coverage with cycle loss can be attributed to the difficulty of mapping samples back to the source domain when they are translated to the same point (reduced modal coverage) in the target domain. Finally, adaptive DMCD further enhances the ability of DMCD to reduce low-density translations. In addition to mitigating distillation error, auxiliary losses provide a mechanism for flexible control over the trade-off between reality and faithfulness by allowing for the adjustment of weights like EGSDE (Figure 9).

W6. From Tab. 3 and Fig. 6, it seems that adaptive weighting may harm the performance, which is not consistent with conclusion in toy data.

We agree the potential negative impact of adaptive DMCD on the low reality-high faithfulness region. That said, the adaptive DMCD is usually a usful option since we can use adaptive DMCD when prioritizing FID and omitting it when SSIM is the primary concern.

While a definitive analysis is still needed, we hypothesize that the interaction between cycle loss and adaptive DMCD might be the root cause. Cycle loss, by transforming teacher's ODE trajectories trained with consistency loss, could potentially disrupt the semantic interpretation of consistency loss magnitudes, which are crucial for adaptive DMCD weightings. High cycle loss might lead to trajectories that deviate significantly from the teacher's, potentially undermining the assumption that consistency loss magnitude correlates with trajectory learning difficulty. This speculative explanation suggests a promising direction for future research: adaptively applying cycle loss to mitigate conflicts between auxiliary losses.

Q1. The authors propose to use one generator for two domains, which may be unreasonable or hard to achieve in practice. I think the whole pipeline is compatible with two independent pre-trained DMs.

As you pointed out, our pipeline can be compatible with two independent teacher models with a simple modification. In this case, it can be used as is even if the two domains are not trained in a single model, which expands the scope of our framework. Thanks for the great comment.

References:

[1] Kim, Dongjun, et al. "Consistency trajectory models: Learning probability flow ode trajectory of diffusion." ICRL (2024).

[2] Li, Liangchen, and Jiajun He. "Bidirectional Consistency Models." arXiv preprint (2024).

Dear reviewer etdT,

We've carefully considered your valuable feedback and have made the following revisions to our manuscript and responses to your questions:

1. Clarified the error induced by non-differentiable ODE trajectories.
2. Clarified the difficulty in vanilla IBCD and removed the term "mean prediction phenomenon."
3. Clarified the ablation studies.

As the discussion period deadline is approach fast, we would appreciate it if you could provide your feedback whether our revision and rebuttal have fully addressed your concerns.

We appreciate your time and consideration.

Best regards, Authors

Thank you for your kind words and for taking the time to review our work. We appreciate your understanding and the increase in your score.

As you suggested, we will include the results for Q1 in the revised manuscript. We also thank you for the insightful suggestion regarding diversity.

Thank you again for your valuable feedback.

W1. Missing comparison of the results for bidirectional translation.

We appreciate the reviewer's suggestion to include quantitative comparative experiments on bidirectional translation. In response, we have added a comprehensive evaluation in Appendix D.6, where we conduct opposite translation (Dog→Cat, Dog→Wild, Female→Male) and cycle translation (Cat→Dog→Cat, Wild→Dog→Wild, Male→Female→Male) tasks. While bidirectional models and public checkpoints are limited, our results in Table 6, Figures 12 and 13 demonstrate that our model's bidirectional performance is on par with its unidirectional capabilities.

(Partial) Table 6: Quantitative comparison of unpaired image-to-image translation tasks (opposite & cycle translation)

W2. Missing comparison of computation cost with the existing methods to show the efficiency of the proposed method.

Per your suggestion, we have conducted additional experiments comparing the actual inference time of our method with major open-source baselines. As shown in (new) Table 5 and Appendix D.4, our methodology demonstrates a significant advantage in inference computational complexity.

(Partial) Table 5: Quantitative comparison of model inference times.

W3. The results in Table 3 show the model which added DMCD loss, cycle loss and adaptive DMCD degrades the performance in terms of PSNR and SSIM compared to the method using IBCD only.

To clarify the comparison, we've updated Table 3 to benchmark against the lowest FID achievable by each individual component. Additionally, we've introduced a new metric, PSNR-Teacher, which uses the DDIB teacher's output as a PSNR ground truth. This metric allows us to assess the resolution of distillation error, a primary focus of our auxiliary loss approach.

Our results demonstrate that each added component consistently reduces FID beyond the lower bound achievable by vanilla IBCD, while effectively mitigating the inherent PSNR trade-off and minimizing distillation error. Notably, updated Figure 6 highlights that even under identical PSNR conditions, the application of auxiliary losses consistently yields lower (improved) FID scores or vice versa.

Table 3: Quantitative ablation study results in the Cat→Dog task under the lowest FID.

W4. The zero in the first row of Eq. (6) might be $\chi_A \cap \chi_B$.

Thanks for your careful reading. Typo fixed.

Dear reviewer CDyb,

We've carefully considered your valuable feedback and have made the following revisions to our manuscript and responses to your questions:

1. Added bidirectional translation evaluation results.
2. Added inference computational cost comparison results.
3. Clarified ablation studies.

As the discussion period deadline is approach fast, we would appreciate it if you could provide your feedback whether our revision and rebuttal have fully addressed your concerns.

We appreciate your time and consideration.

Best regards, Authors

W1: The method seems too incremental, appearing to be merely a combination of DDIB and consistency models.

In contrast to your misunderstanding, our work introduces a novel image translation model framework that uniquely satisfies four crucial properties: one-step, unpaired, bidirectional, and non-discriminator. This framework establishes a new state-of-the-art performance across various tasks and evaluation metrics while maintaining these desirable properties.

Extending consistency distillation to DDIB, particularly in a bidirectional manner, has never been tried before due to several technique challenges. To address this, we have developed several novel approaches such as adapting consistency distillation to diffusion bridge trajectories (timesteps, training scheme), designing appropriate boundary conditions for bidirectional training, and carefully parameterizing the model for bidirectional diffusion bridge (Appendix B). Furthermore, we introduced additional auxiliary losses, strategically designed to mitigate the distillation loss and simultaneously improve reality and fidelity. These losses were carefully integrated into the IBCD framework with framework-aware modification (DMCD, adaptive DMCD, bidirectional cycle loss).

Extensive ablation studies validate the significance of these advances in achieving our framework's superior performance across various tasks and evaluation metrics. Therefore, our work cannot be considered as incremental.

W2. In Table 3, the FID improvement from adding Cycle and DMCD is marginal.

As you correctly noted, the bottom three components of Table 3 do not directly compare the superiority of FID. Instead, they present a scenario where PSNR is considered in situations with similar FID values.

To clarify the comparison, we've updated Table 3 to benchmark against the lowest FID achievable by each individual component. Additionally, we've introduced a new metric, PSNR-Teacher, which uses the DDIB teacher's output as a PSNR ground truth. This metric allows us to assess the resolution of distillation error, a primary focus of our auxiliary loss approach.

Table 3 demonstrates that each added component consistently reduces FID beyond the lower bound achievable by vanilla IBCD, while effectively mitigating the inherent PSNR trade-off and minimizing distillation error. Notably, updated Figure 6 highlights that even under identical PSNR conditions, the application of auxiliary losses consistently yields lower FID scores or vice versa.

Table 3: Quantitative ablation study results in the Cat→Dog task under the lowest FID.

W3. Most baselines in Table 2 show the variance of FID, while the author didn't.

The criteria for indicating and not indicating the variance of the results are the same as those adopted in many previous studies. Models that indicate variance in quantitative results (ILVR, SDEdit, EGSDE, SDDM) utilize a probabilistic sampler (SDE), generating probabilistic samples for each translation. Consequently, their variance is explicitly indicated. In contrast, models that employ non-deterministic sampling during the translation process (ours, CycleDiffusion, GAN-based models, etc.), do not indicate variance as there is no inherent probabilistic mechanism in the translation process.

W4. With only a single step, the stochasticity is significantly reduced. The authors should include several other related metrics that highlight diversity, such as the Inception Score.

To address the reviewer's concern, we additionally conducted a quantitative diversity analysis using the density-coverage metric[1] in addition to the FID metric (updated Table 2). This metric, which separately evaluates quality and diversity, is more suitable for our tasks, as Inception Score's (Inception v3) classification granularity is insufficient to assess diversity within a single domain (dog or female).

As shown in the Table 2 (please refer to the updated text), our one-step non-deterministic mapping approach outperforms baselines in both quality (density) and diversity (coverage).

W5. More failure cases should be provided for better understanding of the method's limitations.

Per your suggestion, we have updated Figure 11 to include failure cases for each task, providing a more comprehensive visualization of our model's performance (14 → 35 cases).

References:

[1] Naeem, Muhammad Ferjad, et al. "Reliable fidelity and diversity metrics for generative models." ICML (2020).

Dear reviewer UgkH,

We've carefully considered your valuable feedback and have made the following revisions to our manuscript and responses to your questions:

1. Clarified our contributions and the determinism of the inference process.
2. Clarified the ablation studies.
3. Added diversity evaluation (density-coverage) results.
4. Added extra failure cases.

As the discussion period deadline is approach fast, we would appreciate it if you could provide your feedback whether our revision and rebuttal have fully addressed your concerns.

We appreciate your time and consideration.

Best regards, Authors

This is just a kind reminder as the deadline for the paper revision period is approaching. We are looking forward to hearing your feedback and will be happy to answer any further questions.

Dear Reviewer UgkH,

As the discussion deadline is getting closer, we would like to kindly remind the reviewer that we are waiting for your valuable feedback to our responses.

Please note that we've addressed all your feedback by clarifying our contributions, ablation studies, and adding diversity evaluation and failure cases.

So we would greatly appreciate it if you could review our revised manuscript and provide your feedback.

Thank you for your time and consideration.

Best, Authors