Deno-IF: Unsupervised Noisy Visible and Infrared Image Fusion Method

Q1: Training complicacy and computation complexity.

A1: We appreciate this insightful observation. To enhance methodological clarity, the final version will include a detailed algorithmic presentation in pseudocode format.

Besides, we want to clarify the computational efficiency in the testing phase. While the iterative optimization in the convolutional low-rank module increases computational overhead, the complexity is confined to the training phase. This module serves solely as an implicit knowledge distillation mechanism to guide the joint denoising and fusion network. During inference, it is completely bypassed, not incurring additional computation. The computational efficiency comparison results in Tab. 3 in the submitted manuscript also indicate that our method achieves optimal efficiency (including parameter numbers, FLOPs, and inference time) during the testing phase.

Q2: No-reference metric BRISQUE and no-reference results under different-level noise.

A2: BRISQUE measures image quality by quantifying deviations from natural scene statistics in spatial domain. It extracts spatial features from the empirical distribution of locally normalized luminance and products of locally normalized luminance under a spatial natural scene statistic model. The features capture losses in naturalness caused by distortions, generating a holistic measure of quality. The no-reference comparison results under multi-level noise are reported in the following table. Key observations are analyzed from two perspectives:

1. Robustness to increasing noise severity

As noise level escalates, comparison degradation-aware image fusion methods exhibit significant performance degradation, while our method maintains stable quality preservation. It demonstrates our method’s inherent robustness to multi-level noise interference.

2. Effectiveness per noise level

At each noise level, our method achieves best-in-class BRISQUE scores, outperforming comparison degradation-aware image fusion methods. The consistently low BRISQUE scores confirm our method's robustness in joint denoising-fusion, effectively maintaining naturalness under noise interference.

We will add the introduction and selection reasons of BRISQUE and other metrics and provide the above results in the final version.

Q3: Clarification on multi-level Gaussian noise.

A3: Thanks for the query about multi-level Gaussian noise targets. To validate the robustness, we perform the experiment where both the visible and infrared images suffer from gaussian of different-level Gaussian noise. We will provide qualitative results of multi-level noise in the final version, which demonstrates our method’s robustness to multi-level noise interference.

Q4: Parameter settings in the convolutional low-rank optimization module.

A4: These settings stem from principled trade-offs and computational efficiency:

1. Principled trade-off

These parameters critically balance three competing objectives, including data fidelity, regularization, and releasing terms. The parameter settings directly correlate with the characteristics of source images, particularly noise levels. Higher noise level demands stronger regularization to suppress noise while lower noise level requires weaker regularization to preserve details.

2. Computational efficiency imperative

While deep learning-based noise estimation networks exist, such approaches incur prohibitive computational overhead, especially when the training process is already time-consuming. The manually designed estimator can achieve optimal efficiency-accuracy trade-off.

Considering these two aspects, a compromise is made by using manual operations. We will add these reasons of manually designed operations for parameter settings in the final version.

Q5: Does increasing $k_1,k_2$ lead to performance improvements?

A5: Theoretically, increasing the size of convolutional kernel can contain more structural information for inference. However, excessively large convolutional kernels constrain the convolutional low-rank nature of more complex scenes, which may potentially comprise the preservation of complex information.

In terms of experimental verification, we increase $k_1,k_2$ and retrain the network. As empirically validated in the table below, appropriately increasing $k_1,k_2$ improves the performance. When $k_1,k_2$ may be excessively (due to time and equipment limitations, we increase $k_1,k_2$ to 36 for feasible experiments to observe trends), enforcing convolutional low-rank constraints over largely expansive regions risks damaging local details. It is reflected in a slightly decreasing performance trend.

Besides, computationally, increasing kernel sizes $\{k_1,k_2\}$ generates a convolutional matrix of dimensions $hw\times{k_1k_2}$, leading to prohibitive computational costs.

Thanks again for this question which provides more validation and optimized settings of $k_1,k_2$. We will add the experiments about different settings of $k_1,k_2$ in the final version.

Q6: Progressively increasing setting of $\kappa$.

A6: The dynamically increasing setting of $\kappa$ is motivated by the need to balance competing objectives: data fidelity (similarity losses between source and fused data) and regularization (convolutional low rankness of fused images). Through extensive experiment, we observe that: i) Large initial $\kappa$ forces the network to over-prioritize the low-rank constraint, causing fused image to be excessively simplified (overly low-rank). It suppresses critical structural details in source images, leading to serious information distortion. ii) Small initial $\kappa$ allows data fidelity to dominate early training, progressively incorporating richer information into fused image. However, a fixed small $\kappa$ later renders the regularization ineffective, failing to refine high-frequency details. The scheduled increasing strategy of $\kappa$ prioritizes data fidelity to preserve scene content in the early stage and gradually strengthen the low-rank prior to refine textures in the later stage.

Q7: Typographical issues to be corrected.

A7: Thanks for point out the typos. We will carefully correct them as “intra-modal recovery and inter-modal recovery and fusion” and “different-level Gaussian noise” in the final version and further improve the description.

Q1: Connection between two modules.

A1: We sincerely appreciate this for this constructive suggestion. We want to emphasize that the connection between these modules operates not through superficial explicit-level structural transfer of data or feature, but fundamentally via deep implicit physics-driven constraint propagation. The ultimate goal is to obtain a joint denoising and fusion module via an unsupervised approach, yet struggles to infer high-quality structural cues to guide network optimization. To address it, the convolutional low-rank optimization module leverages convolutional low-rank priors to derive clean components. These components serve as i) physics-driven prior injector to regularize fused images, and ii) implicit teachers that enable knowledge distillation of denoising priors directly into the final joint denoising and fusion network. The design enables simultaneous noise suppression and information fusion while eliminating the need for low-rank module reuse during testing, thereby enhancing real-time inference efficiency. We will emphasize the deep implicit connection in the final version.

Q2: Differences with prior low-rank methods in other domains.

A2: While low-rank methods exist in image processing (e.g., image smoothing, image restoration, image denoising), the proposed method differs from existing works in both task formulation and low-rank theory:

1. Task formulation

The proposed method differs in input complexity and objective shift. For input complexity, we handle multi-modal data rather than single-modal inputs. For objective, we focus on joint denoising and fusion rather than a single image processing task.

2. Low-rank theory

The proposed theory differs in energy function formulation and optimization strategy. For energy function, the regularization term is based on convolutional low-rank prior by the convolution nuclear norm. Compared with standard low-rank theory, it can avoid excessive smoothing and structural damage for better local structure preservation, noise robustness, and physical interpretability. For optimization strategy, previous researches directly implement the objective function as the loss of network. Differently, we design a two-phase optimization strategy. In the first phase, we rewrite the energy function and iteratively address the subproblems with analytical solutions. In the second phase, we design a deep network to distill the physics-driven priors and realize denoising and fusion jointly.

We will highlight the contributions of the proposed method over existing low-rank methods in other domains in the final version.

Q3: Replace Fig. 1 with more noticeable noise.

A3: We appreciate this insightful recommendation. We will replace the qualitative examples in Fig.1 with the comparison examples where both the visible and infrared images suffer from more clearly noticeable noise in the final version. Compared with the combination of SOTA image denoising and fusion methods, the proposed method can restore and represent essential high-quality structural information from source images with greater clarity in the fusion result.

Q4: Annotate the block number in equations.

A4: Thanks for this suggestion. In both intra-modal recovery and inter-modal recovery and fusion parts, there are many blocks. To avoid the confusion between the block numbers of these two parts, we discuss them separately. In Eq. (10) for intra-modal recovery, the output of the current block (e.g., $\hat{**f**}_v$) is only related to the input of the current block (e.g., $**f**_v$). It does not involve complex update process as Eqs. (6a)-(6d). For the sake of formula simplicity, the block numbers will not be given. In Eq. (11) for inter-modal recovery and fusion, the input of each block includes visible, infrared and even earlier fused features. However, the directions of intra-modal recovery and inter-modal recovery and fusion are different. The earlier blocks of inter-modal recovery and fusion need to fuse the features of the later blocks of intra-modal recovery. This is clearly represented in Fig. 2, so the block number will not be labeled to avoid confusion.

Q5: Reason of selecting CTNet for comparison.

A5: The rationale of selecting CTNet for comparison is two-fold: generalization validation between unsupervised and supervised approaches, and architecturally relevant comparison.

1. Validating unsupervised generalization against supervised performance

CTNet represents a SOTA supervised denoising approach, requiring paired clean and noisy images for training. Our method is an unsupervised denoising and fusion framework, leveraging the convolutional low-rank prior without needing clean data. Including CTNet for comparison serves a critical purpose: testing the generalization and denoising effectiveness of our unsupervised approach. Comparing against a top-performing supervised denoising method, our unsupervised method achieves competitive or even superior denoising performance. It validates the robustness and applicability of our approach, especially when paired training data is unavailable or difficult to obtain.

2. Architecturally relevant comparison

CTNet and our method are fundamentally based on Transformer. By comparing with CTNet, it provides a controlled comparison point, specifically highlighting improvements of divergent frameworks and objectives. CTNet utilizes Transformer primarily to mine structural information and explore multi-level interactions for extra information, all within a single modality for denoising. In contrast, our method uses Transformer for both multi-modal image denoising and fusion. It simultaneously addresses intra-modal recovery and inter-modal recovery and fusion within a unified framework. Comparing against CTNet, a typical representative of Transformer-based denoisers, allows to directly highlight the effectiveness and novelty of our multi-modal denoising- and fusion-oriented Transformer design.

We will provide a brief introduction to CTNet and the reason for leveraging it for comparison in the final version.

Q6: Typo and description.

A6: Thanks for pointing out the issues related to typo and detailed description. The parameter of SwinFusion+CTNet is 52.38M. The circle size in Fig.1 represents the correct parameter number. MAP represents classic Maximum A Posteriori inference to integrate physics-driven priors with data-driven fitting for robust decomposition. We will carefully correct the typos (extra “.” and parameter) and description in the final version and further correct the presentation of figures and tables.

Q1: Explain the necessity and motivation of fusing noisy infrared and visible images.

A1: We appreciate this constructive suggestion. We want to declare that the clean publicly available datasets do not reflect real-world acquisition. Methods should account for inherent noise from affordable hardware.

1. Dataset quality does not reflect real-world feasibility

Existing datasets are collected for the target of high image quality, thus relying on cost-prohibitive imaging systems to ensure high signal-to-noise ratios. For example, the platform of LLVIP is DS-2TD8166BJZFY-75H2F/V2, a binocular camera platform priced over $19,000. It not only has high-quality signal acquisition hardware devices, but also embeds dedicated and professional processing pipelines to enhance raw inputs, including adaptive AGC for dynamic range compression, DDE for nonlinear enhancement, and 3D DNR for denoising.

However, in practice, most deployed multi-modal systems use affordable devices that may cost only several hundred dollars. The imaging systems may introduce critical distortions, such as degraded signal acquisition (low-cost microbolometers and CMOS suffer from more obvious noise), and ineffective embedded processing pipelines. These factors will result in a large amount of noise in the collected signal, which is not as high-quality as the dataset.

2. Artificial bias

Existing dataset is dedicated to providing benchmarks for evaluating fusion performance. As declared, the construction of dataset involves rigorous manual screening. Some initially captured samples with low quality maybe discarded. It creates a distributional gap: real-world images contain various noise absent in curated data.

We will add the discussion on necessity and motivation of fusing noisy images in the final version.

Q2: Feasibility and disadvantages of replacing the convolutional low-rank optimization module with an unsupervised denoising model.

A2: We sincerely appreciate this insightful question, which crucially enhances the methodological rigor and expositional clarity of our work. The convolutional low-rank optimization module pre-inferences clean data as i) physics-driven prior injector to regularize fused images, and ii) implicit teachers that enable knowledge distillation of denoising priors. It seems like a pre-processing module but in fact, it is a core enabler of physics-aware and training-free supervision.

It is possible to replace it with an unsupervised denoising model. However, the substitution may introduce critical disadvantages:

1. Theoretical grounding and interpretability

Convolutional low-rank optimization is theoretically grounded in mathematical foundations and explicit priors, including matrix decomposition and low-rank representation theory. The objective function, constraints, and convergence of iterations are interpretable and theoretically guaranteed. Unsupervised denoising models rely on heuristic architectures with less rigorous justification for denoising mechanisms. Although certain priors can be embedded in network architecture, the success relies more on empirical design, preventing theoretical explanation and formal verification.

2. Training-data dependency

Convolutional low-rank optimization is training-free, operating directly on individual noisy inputs without the need of prior dataset. It is critical for unique or rare images lacking training samples. In contrast, while unsupervised denoising models avoid paired supervision, they still demand large training data, remaining fundamentally data-driven. The performance depends on training data distribution.

3. Overfitting risks and generalization

As convolutional low-rank optimization is training-free to per image, it does not rely on learning dataset-specific patterns, exhibiting robustness to out-of-distribution cases and eliminating overfitting risks. Unsupervised denoising models employ fixed networks trained on finite data distributions. When encountering noise types or content underrepresented in training, the learned mappings may fail to generalize, leading to degradation.

We will add the discussion in the final version.

Q3: Compare with more SOTA methods published in 2024 and 2025.

A3: Thanks for your helpful suggestion. To further strength the comparative analysis, besides comparing with Text-IF and DRMF published in 2024, we expand the competitors with three state-of-the-art methods published in 2025, including FusionBooster [r1], PromptFusion[r2] and OmniFuse [r3]. FusionBooster and PromptFusion are combined with CTNet for pre-denoising while OmniFuse is degradation-aware image fusion methods.

[r1] FusionBooster: A Unified Image Fusion Boosting Paradigm. International Journal of Computer Vision, 2025.

[r2] PromptFusion: Harmonized Semantic Prompt Learning for Infrared and Visible Image Fusion. IEEE/CAA Journal of Automatica Sinica, 2025.

[r3] OmniFuse: Composite Degradation-Robust Image Fusion with Language-Driven Semantics. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025.

As qualitative comparison is not available, we report quantitative results under four conditions (different datasets with different types of noise) below. Our method achieves optimal performance on most metrics. Less optimal performances of CT.+FusionBooster and CT.+PromptFusion stem from the focus on addressing visual degradation caused by distortions of critical features and details, prioritizing texture preservation. However, CTNet's limited denoising capability results in residual noise erroneously amplified in fusion results. The comparatively lower results of OmniFuse stem from the unified model for multiple types of degradations and supervised paradigm.

The computational complexity and efficiency of these SOTA methods and the proposed Deno-IF are reported below. Our method achieves lower computational complexity and higher inference efficiency.

We will add the results with [r1]-[r3] in the final version to improve the previous comparison.

Q4: Reason of selecting the five evaluation metrics.

A4: Thanks this question. Traditional fusion metrics can be categorized into similarity-based metrics (evaluating similarity between fused and clean source images) and statistics-based metrics (measuring fused image characteristics, such as SD, SF, MG, EN, etc.). However, when fusing noisy images, statistics-based metrics may produce misleading results as noise residuals can artificially inflate values (e.g., higher EN/SD/SF caused by noise rather than true useful information). We avoid this issue from two aspects.

1. Similarity-based metrics

We prioritize similarity-based metrics that explicitly measure information fidelity over statistical properties to ensure reliable and robust evaluation. The selected metrics, i.e., PSNR, SSIM, CC and FSIM, provide complementary assessments of: i) absolute intensity preservation, ii) structural and texture similarity, iii) global feature distribution consistency, and iv) frequency-domain characteristic retention. The orthogonal combination of these multi-dimensional metrics offers a comprehensive evaluation of information fidelity.

2. No-reference quality-based metrics

To assess image quality, we employ BRISQUE instead of conventional statistics-based metrics. By analyzing natural scene statistics through normalized pixel intensity distributions, BRISQUE detects unnatural distortions via mean-subtracted contrast-normalized coefficients and generalized Gaussian modeling. Its trained regression model penalizes noise-induced deviations from natural image statistics, yielding scores consistent with human perception and reliable for noisy scenarios.

Q5: Correct “inter-modal recovery” and “intra-modal recovery and fusion” in Fig. 2.

A5: Thanks for careful reading and catching the terminology inconsistency. The correct terms should be "intra-modal recovery" and "inter-modal recovery and fusion". The terms “inter-” and “intra-” are mistakenly swapped while “modal” is correct as it refers to distinct imaging modalities rather than “model”. We will carefully correct all instances in the final version.

Thank you for the rebuttal, which addressed some of my concerns.

However, I am still not convinced by the authors regarding the motivation.

It is well-known that thermal cameras are expensive. Thermal cameras at hundreds of dollars are basically not very useful in practise. Do you mean that, by using your method, people can use devices of several hundreds of dollars to achieve the performance of devices of thousands of dollars? To be honest, I doubt this point.

Additionally, thermal camera is usually applied in applications where pedestrian safety is a critical issue. In these applications, such as autonomous driving, this is not the case - "most deployed multi-modal systems use affordable devices that may cost only several hundred dollars". In these applications, people tend to use high-quality camera to capture better images to ensure safety. According to my data capturing experience, I don't think fusion with noisy images is necessary in these cases when high-quality thermal cameras are used in these important applications. Therefore, I still think this paper creates a "fake" issue and then propose a solution to this "fake" issue. I think this is the main problem of this paper. I am not sure this method is useful in practise, such as in autonomous driving.

Furthermore, would it be possible to provide fusion results in terms of other fusion metrics other than the five utilized in the paper? This will show more comprehensive comparison.

Q1: Implementation and interplay between the convolutional low-rank optimization and joint fusion network.

A1: We sincerely appreciate this constructive suggestion. We want to emphasize that the connection between these modules operates fundamentally via deep implicit physics-driven constraint propagation. The ultimate goal is to obtain a joint denoising and fusion module via an unsupervised approach, yet struggles to infer high-quality structural cues to guide network optimization.

To address it, the convolutional low-rank optimization module leverages convolutional low-rank priors to derive clean components. These components serve as i) physics-driven prior injector to regularize fused images, and ii) implicit teachers that enable knowledge distillation of denoising priors directly into the final joint denoising and fusion network. The design enables simultaneous noise suppression and information fusion while eliminating the need for low-rank module reuse during testing, thereby enhancing real-time inference efficiency.

We will emphasize the deep implicit connection and include a detailed algorithmic presentation in pseudocode format in the final version to enhance methodological clarity.

Q2: Optimize the training strategy or provide an analysis of computational training cost.

A2: Thanks for your insightful suggestions. We first perform an analysis of computational cost. As the training latency is dominated by the first stage (convolutional low-rank optimization module), we quantify the computational bottleneck within this module. In this module, computational complexity scales primarily with i) convolution kernel size ($k_1,k_2$) and ii) number of iterations ($T$) for solving the low-rank subproblem. The training costs of different settings are reported below. We observe that increasing $k_1,k_2$ incurs significant increase in training time while the number of iteration scales linearly with computational cost. Nevertheless, the entire training process remains practical, completing within several days.

When $k_1,k_2=12$, we change the number of iterations (T) and quantify the training cost.

A possible solution to optimize the training strategy is to adaptively adjust $k_1$ and $k_2$ based on the scene content, and dynamically update $T$ according to the convergence behavior. We will report the training cost in the final version. In future work, we plan to further develop adaptive strategies for determining convolutional kernels and iterative updates to improve training efficiency.

Q3: Annotation of operator “c” in Figure 2.

A3: Thanks for this question. The operator “c” represents feature concatenation. We will add the explicit annotation in Figure in the final version.

Q4: How noise suppression is handled in the fusion stage.

A4: We appreciate this opportunity to clarify our fusion methodology. We want to clarify that the fusion process is not a simple overlay of source images, but rather a sophisticated extraction, recovery, fusion, and reconstruction process implemented through the joint denoising and fusion module.

From the aspect of network architecture, noise is suppressed through Transformer blocks in both intra-modal recovery and inter-modal recovery and fusion.

From the aspect of network optimization, clean components derived with convolutional low-rank priors serve as i) physics-driven prior injector to regularize fused images, and ii) implicit teachers that enable knowledge distillation of denoising priors into the fusion network. It enables simultaneous noise suppression and information fusion. In the final version, we will emphasize the implementation of denoising functionality in the network more prominently to prevent potential misunderstanding.

Q5: Revise the phrase “inter-modal recovery and intra-modal recovery and fusion”.

A5: Thanks for your careful reading and reminder. The terms “inter-” and “intra-” are mistakenly swapped in the original manuscript. The correct terms should be "intra-modal recovery" and "inter-modal recovery and fusion". We will carefully revise the wording through the manuscript and ensure the precise use of terminology in the final version.