Efficient Rectified Flow for Image Fusion

Thank you for your valuable comments, especially for recognizing the technical merits and performance of our method. Below are our responses to your questions.

• W1: The method does not incorporate any explicit mechanisms to model modality-specific characteristics.

Thanks for your comment. We acknowledge that, compared to our method, dual-branch encoders or cross-modality attention modules offer a more intuitive advantage for image fusion tasks, as they explicitly integrate features from different modalities. However, we would like to emphasize that input images often suffer from quality issues such as noise and blur. Our method leverages the capability of generative models to enhance image quality during the fusion process—something traditional fusion modules are typically unable to achieve. Furthermore, as shown in Tables 2 and 4 of our paper, our method achieves superior performance in terms of the MI metric, indicating that it effectively preserves critical information from the source images even without relying on conventional fusion modules.

• W2: Indirect infrared input may bias the latent space and weaken infrared feature encoding.

Our method achieves leading performance in terms of the MI metric across multiple datasets. The MI metric typically reflects the richness of information from the input images contained in the fused image, which indicates that our method does not suffer from insufficient encoding of infrared-specific semantics.

• W3: The unclear definition and learning of $M$.

Here, $M$ can be either a linear or a nonlinear operation. It encompasses the learned, fixed, and task-specific components mentioned by the reviewer. In this paper, we adopt the Expectation-Maximization algorithm used in DDFM [1]. However, we do not intend to limit it to a single algorithm. For example, in CCF [2], multiple types of conditions are employed, such as Basic Conditions, Enhanced Conditions, and Task-specific Conditions.

• W4: The additive fusion in Equation (13) lacks analysis.

Our formulation defines the fusion velocity as an additive combination of the pretrained Rectified Flow $v_\theta(f_t)$ and a posterior correction term $\nabla_{f_t} \log p(i,v \mid f_t)$, which serves to guide the generative process toward better alignment with the input modalities. While this additive form may appear simplistic, it is inspired by techniques in score-based generative modeling, where posterior guidance is often introduced as a gradient correction to the original flow or score field. Empirically, we found this formulation to be stable during generation, as the pretrained flow already provides a strong data prior, and the posterior term acts as a relatively localized correction based on specific input instances.

Reference

[1] Zhao Z, Bai H, Zhu Y, et al. DDFM: denoising diffusion model for multi-modality image fusion[C]//Proceedings of the IEEE/CVF international conference on computer vision. 2023: 8082-8093.

[2] Cao B, Xu X, Zhu P, et al. Conditional controllable image fusion[J]. Advances in Neural Information Processing Systems, 2024, 37: 120311-120335.

Thank you for your valuable comments and for acknowledging the effectiveness of our method across various image fusion tasks. Below are our responses to your questions, and we hope they address your concerns.

• W1: The pretrained Rectified Flow model is not specifically designed for image fusion.

Thanks for your comment. We acknowledge that the pretrained Rectified Flow model was not specifically designed for image fusion tasks, and compared to traditional image fusion methods, our approach does not intuitively fuse features from the two source images. However, we would like to emphasize that our method leverages the powerful generative capability of Rectified Flow to produce high-fidelity fused images, while incorporating rich information from both images through posterior sampling to ensure the quality of the fusion results. Therefore, the design of our method does not compromise the final performance. Moreover, some approaches utilizing Rectified Flow have also been successfully applied to other low-level vision tasks [1,2,3], which further demonstrates the feasibility of our method.

• W2: Additional ablation experiments on Rectified Flow and VAE structure.

We have added ablation studies on VAE and Rectified Flow. For experiments related to inference speed, please refer to weakness 3 mentioned by reviewer uZpR. The quantitative results of the ablation studies are shown in the table. We conducted a comparison on the Roadscene dataset.

• Q1: Providing generalization tests under different modal combinations.

Please refer to weakness 2 mentioned by reviewer fHBh.

• Q2: Further ablation of the model architecture.

We have added the results of image fusion under different VAE architectures. We conducted experiments using VQ-VAE (with a downsampling factor of 4 and without attention) and trained the VAE structure using the same training procedure. The results are shown in the table below.

As shown in the table, although the VAE without attention offers a slight advantage in inference speed, its reconstruction capability is inferior to that of the VAE with attention. Therefore, we choose the attention-based VAE as our pretrained model.

• Q3: Other starting points to improve fusion consistency.

Regarding why we do not use noise as the starting point, you can refer to Question 1 raised by Reviewer uZpR. As for using dual modal images as the starting point, since Rectified Flow only accepts a single image as input, we perform a simple fusion of the two modality images before conducting experiments. The fusion can be formulated as:

$$I_{\text{fused}} = \alpha \cdot I_1 + (1 - \alpha) \cdot I_2.$$

Here, $\alpha = 0.5$, and our experimental results are shown in the table.

We were pleasantly surprised to find that using the fused image as the sampling starting point led to improved performance compared to using the original visible image, as reflected in the evaluation metrics. This demonstrates that our method still holds potential for performance gain when different images are used as starting points. In future versions of the paper, we plan to further explore the fusion performance when using images containing different proportions of modality information as starting points, in order to fully uncover the potential of our method in image fusion.

Reference

[1] Zhu Y, Zhao W, Li A, et al. Flowie: Efficient image enhancement via rectified flow[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024: 13-22.

[2] Li J, Cao J, Guo Y, et al. One diffusion step to real-world super-resolution via flow trajectory distillation[J]. arXiv preprint arXiv:2502.01993, 2025.

[3] You W, Zhang M, Zhang L, et al. Consistency Trajectory Matching for One-Step Generative Super-Resolution[J]. arXiv preprint arXiv:2503.20349, 2025.

Thank you for the valuable comments. We appreciate the reviewer’s recognition of the excellent fusion performance and generalization ability of our method. Below are our responses to the reviewer’s questions.

• W1: Validating the proposed technique on additional fusion methods.

Thanks for your comments. About weakness 1, you can refer to weakness 2 of Reviewer uZpR.

• W2: The lack of evaluations on additional tasks or datasets.

We have supplemented our method's performance on medical image fusion. Specifically, we conducted tests on the Harvard Medical Image Dataset, which includes MRI-CT, MRI-PET, and MRI-SPECT image pairs. The fusion results are shown in the table.

As shown in the tables, our method consistently outperforms the baseline DDFM on the MRI-CT fusion task, demonstrates a clear advantage on MRI-PET, and achieves comparable performance to DDFM on MRI-SPECT. Overall, our approach exhibits superior performance in medical image fusion compared to DDFM. Notably, our method requires no additional training and offers faster inference, further highlighting its great potential for medical image fusion applications.

• W3: Why RFfusion also achieves better quantitative performance than DDFM?

Our method RFfusion, employs a VAE for image reconstruction during the fusion process and is trained on a fusion dataset to better adapt to the fusion task, enabling high-quality image fusion. As a result, we achieve improvements in evaluation metrics compared to DDFM. In addition, the pre-trained generative model used in RFfusion provides better image generation performance than that of DDFM, which further contributes to the superior performance of our method in terms of quantitative metrics.

• Q1: Detailed information about the pretrained Rectified Flow.

We use checkpoint 12 from the Rectified Flow [2] as the pretrained model, which is trained on the LSUN Church dataset with a resolution of 256×256.

• Q2: Evaluating the proposed method on additional datasets and fusion tasks.

Please refer to weakness 2.

• Q3: The rationale behind the choice of additional loss functions.

These loss functions are incorporated into the VAE training process to enhance the reconstruction quality of the fused images. They evaluate whether the fused image effectively preserves the critical information from the input images from multiple perspectives, including structure, color, and saliency. By introducing diverse loss terms during training, the model not only focuses on the overall image quality but also better retains the essential information from multiple sources, thereby improving the comprehensive performance of the fusion results.

Reference

[1] Zhao Z, Bai H, Zhu Y, et al. DDFM: denoising diffusion model for multi-modality image fusion[C]//Proceedings of the IEEE/CVF international conference on computer vision. 2023: 8082-8093.

[2] Liu X, Gong C, Liu Q. Flow straight and fast: Learning to generate and transfer data with rectified flow[J]. arXiv preprint arXiv:2209.03003, 2022.

Thank you for the valuable suggestions. We appreciate the reviewer’s recognition of the performance improvements achieved by our method. In response to the questions raised by the reviewer, we have provided the following answers, and we hope they will address the concerns.

• W1: Quantitative fusion performance lacks a clear advantage over other methods.

Thanks for your comment. We acknowledge that, compared to other fusion methods, our approach does not demonstrate an absolute and comprehensive advantage across all evaluation metrics. However, in terms of the overall average performance, our method still maintains a leading position. More importantly, rather than merely surpassing existing methods in fusion quality, we aim to emphasize the significant improvement in fusion speed achieved by our approach. Compared to the baseline, our method brings substantial enhancements in both inference speed and various performance metrics. Therefore, we believe that even if more effective diffusion model fusion methods are proposed in the future, our approach can be readily adapted to them and will likely achieve even better results in both speed and effectiveness. This is the core contribution of our work.

• W2: Lacking validation on other diffusion-based methods.

We have supplemented our method by integrating it with CCF [1] as the baseline to verify the generalization capability of our approach. The experimental results are shown in the table.

As shown in the table, our method accelerates CCF from 200 steps sampling to single step sampling. Although the performance is somewhat reduced compared to the original CCF method, it still demonstrates the great potential of our method for integration with other diffusion model fusion methods.

• W3: Lacking ablation studies to isolate the individual contributions of VAE and Rectified Flow to inference speed improvement.

We have added ablation experiments on the inference speed improvements of VAE and Rectified Flow. The experimental results are as follows, and we will update the article with these results in a later version.

The table clearly shows that both VAE and Rectified Flow contribute to improved inference speed, which validates the argument presented in our method.

• Q1: Why use visible images as the sampling starting point?

Compared to the traditional DDPM algorithm, Rectified Flow focuses more on direct flow matching between image distributions and therefore no longer relies on Gaussian noise as a necessary component. By starting the sampling process from a visible image, the model can leverage the visible prior, enabling the generation of more informative and richer fused images.

• Q2: Only using the MSRS dataset in the second stage.

In the second stage of VAE training, we introduce the RFfusion sampling process. Since this process relies on inference operations, the training must be conducted with a relatively small batch size, and the overall training time increases significantly. The primary objective of stage two is to optimize the VAE decoder's ability to reconstruct fused images, rather than to focus on the generalization performance of the fusion method across multiple datasets. Therefore, we choose to train on a relatively small dataset, which significantly reduces the training time without compromising the final performance of our method.

-Q3: The paper should clarify the VAE training details.

In the first stage of training, we trained for a total of 20 epochs on the LLVIP [2] and MSRS [3] datasets. However, we observed that the model typically converged within 4 to 5 epochs, which took approximately 4 to 5 hours on a V100 GPU. In the second stage, we trained for 40 epochs on the MSRS dataset, with the entire process taking about one day. The pre-trained VAE we used was the VQ-VAE from LDM [4], with a downsampling factor of f = 4 and equipped with an attention mechanism.

Reference

[1] Cao B, Xu X, Zhu P, et al. Conditional controllable image fusion[J]. Advances in Neural Information Processing Systems, 2024, 37: 120311-120335.

[2] Jia X, Zhu C, Li M, et al. LLVIP: A visible-infrared paired dataset for low-light vision[C]//Proceedings of the IEEE/CVF international conference on computer vision. 2021: 3496-3504.

[3] Tang L, Yuan J, Ma J. Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion network[J]. Information Fusion, 2022, 82: 28

[4] Rombach R, Blattmann A, Lorenz D, et al. High-resolution image synthesis with latent diffusion models[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022: 10684-10