DeblurDiff: Real-Word Image Deblurring with Generative Diffusion Models

1. Lacks the quantitative evaluation of ablation study.

As real blurred images lack ground-truth references, we employ no-reference metrics for quantitative evaluation. Consequently, Table 2 in the main paper already reports the ablation results with two standard no-reference indices—NIQE and MANIQA. To further strengthen the quantitative evidence, we now provide an extended set of reference-based metrics on the GoPro dataset in the table below.

When the EAC module is removed (w/o EAC), the fidelity of DeblurDiff decreases, demonstrating that EAC effectively preserves information from the input image and contributes to improved restoration accuracy. When the features from the pre-trained diffusion model (SD) are not used in LKPN (w/o SD for LKPN), the model lacks the benefit of the iterative refinement guided by the denoised prior, making it difficult for LKPN to provide precise structural guidance. This highlights the importance of the interaction between the degradation estimator and the diffusion model.

2. In the Figure 13 of the supp, I think "Latent Kernel Prediction Network" should change a name.

Thank you for this suggestion. We agree that the term “Latent Kernel Prediction Network” in the Figure 13 of the supp is inappropriate. This network should be named “Latent Image Restoration Network”. We will revise the label accordingly.

3. If all methods are retrained on the proposed dataset, then it does not make sense to me that the proposed method performs worse on PSNR and SSIM, since there is a L2 loss during the training of this method.

As shown in Table 1 on the GoPro dataset, the PSNR of non-diffusion methods (e.g., FFTformer) is generally higher than that of diffusion-based approaches, with FFTformer outperforming all diffusion-based methods in PSNR. This is because non-diffusion models, such as CNNs or transformers, are typically optimized directly for pixel-level accuracy. In contrast, the training objective of diffusion models focuses more on generating high-quality, perceptually realistic images rather than pixel-level precise matching. Although L2 loss is included during training, the core optimization goal and the stochastic nature of the generative process may result in relatively lower performance on pixel-level metrics like PSNR and SSIM compared to methods specifically designed for these metrics. This is an inherent difference in the design objectives between generative and discriminative models.

4. Did you just retrain the DiffBIR or all compared methods. If you retrained all compared methods, then why did the proposed method performs worse on synthetic data. The proposed method also used L2 supervision between ground-truth and output, same as other method.

We provide a comprehensive comparison of all retrained methods in Table 1 of the supplemental material

The lower PSNR/SSIM scores for our diffusion-based method, despite using L2 supervision, stem from fundamental differences:

1. Optimization Focus: CNNs or Transformers (FFTformer) directly minimize the pixel-wise L2 error between the predicted output image and the ground truth, which inherently optimizes for high PSNR and SSIM. In contrast, our diffusion model is trained to predict the noise added to a latent representation at each denoising step. The L2 loss is applied between the predicted noise and the actual noise, not directly on the final reconstructed image. This indirect optimization prioritizes learning the data distribution and generating perceptually realistic details, often at the expense of perfect pixel-level accuracy. As a result, while our method produces visually superior results, its PSNR and SSIM metrics may be lower than those of CNNs that are explicitly trained to minimize pixel-level reconstruction error.

2. Model Prior: Diffusion models use a powerful generative prior, which can produce more realistic textures but may sacrifice slight pixel fidelity compared to CNNs or Transformers optimized for these metrics. When compared to the pre-trained diffusion-based approaches (ControlNet, DiffBIR, PASD), our method achieves higher scores in both reference-based (PSNR, SSIM) and no-reference metrics. Specifically, on the GoPro, DVD, and RealBlur datasets, our method consistently outperforms ControlNet, DiffBIR, and PASD in both PSNR and SSIM (0.44dB higher in PSNR on the GoPro dataset), demonstrating better restoration fidelity. This indicates that our method is more effective at preserving the information from the input image.

5. Some related method estimate the adaptive kernels on the original image, have you tried this idea instead of on the latent space? And advantage of estimate the kernel in the latent space?

Since diffusion models operate in the latent space to significantly reduce computational cost, we design our approach to be compatible with this paradigm. Therefore, we perform kernel estimation directly in the latent space instead of on the original high-resolution image. Estimating the kernel on the original image would not only contradict this efficiency principle but also lead to dramatically higher memory and processing demands, as the resolution is 8 times larger. Additionally, it would require continuous and expensive VAE encoding/decoding during both training and inference.

Estimating the kernel in the latent space is advantageous because it significantly reduces computational overhead while maintaining acceptable accuracy. The lower resolution makes the process more efficient and avoids repeated VAE operations.

We will cite and discuss the distinctions from these papers in the revised manuscript.

6. Long inference time makes it inapplicable for real time scenarios

This is indeed a limitation of our method. As discussed in L295–L297 of the main paper, we plan to optimize the multi-step diffusion into a single-step approach to accelerate inference.

7. The information loss of VAE is inevitable, which may lead to a slight blur on the final result.

Traditional VAEs often compress an image into a single latent vector, which can lead to severe information loss and blurry reconstructions due to the reliance on pixel-wise L2 loss. In contrast, the VAEs used in pre-trained diffusion models (Stable Diffusion and FLUX) are specifically retrained and designed differently. They do not compress the image into a single vector, but rather into a spatial latent feature map of size h/8 x w/8 (where h and w are the original image dimensions), which preserves much more spatial information.

Furthermore, their training process incorporates advanced loss functions, such as perceptual loss and adversarial loss, in addition to reconstruction loss. This design prioritizes perceptual quality and detail preservation, resulting in significantly clearer and more realistic reconstructions compared to traditional VAEs. While some information loss is still inherent, the VAEs in modern diffusion models are optimized to minimize this issue.

We acknowledge this limitation and believe it's an area where future research on diffusion models can focus.

1. The improvement in reference-based metrics (PSNR, SSIM) over prior methods is modest, despite substantial complexity.

Compared to the transformer-based FFTformer, our PSNR is lower, which is partly due to the inherent trade-off in diffusion models between perceptual quality and pixel-level accuracy. Our method prioritizes generating visually realistic and detailed results, especially on real-world images, over maximizing PSNR/SSIM.

When compared to the pre-trained diffusion-based approaches (ControlNet, DiffBIR, PASD), our method achieves higher scores in both reference-based (PSNR, SSIM) and no-reference metrics. Specifically, on the GoPro, DVD, and RealBlur datasets, our method consistently outperforms ControlNet, DiffBIR, and PASD in both PSNR and SSIM (0.44dB higher in PSNR on the GoPro dataset), demonstrating better restoration fidelity. This indicates that our method is more effective at preserving the information from the input image.

Furthermore, on all four benchmarks—GoPro, DVD, RealBlur, and RWBI—our method achieves the best performance in the no-reference metrics MANIQA and CLIP-IQA, indicating better perceptual quality. Our method prioritizes generating visually realistic and detailed results, especially on real-world images, over maximizing PSNR/SSIM, which aligns with the strengths of the diffusion framework.

2. Limited discussion on computational overhead of iterative kernel refinement during inference.

We analyzed the computational overhead, including runtime and GPU memory, in L86–L92 and Table 4 of the Supplemental Material. As shown in the following table, we provide a comprehensive comparison of other methods in terms of parameters, inference time, and FLOPs.

Compared to FFTformer, our method is slower in inference time, as FFTformer is not a diffusion-based method and thus inherently faster. ReShift, which does not use a pre-trained diffusion model, also has a runtime advantage. However, both methods achieve inferior visual quality compared to the approaches that does not use a pre-trained diffusion model.

Among all methods that leverage pre-trained diffusion models (ControlNet, DiffBIR, PASD, and ours), our method achieves a favorable trade-off between speed and performance. With an inference time of 46 seconds, our method is slightly slower than ControlNet (40 seconds) but 16.4% faster than DiffBIR (55 seconds). In terms of GPU memory, our method uses 7.8 GB, which is slightly higher than ControlNet (7.6 GB) but lower than DiffBIR (8.0 GB). Additionally, our method has fewer parameters and lower FLOPs than PASD.

This demonstrates that our iterative kernel refinement does not introduce significant computational overhead. More importantly, among all pre-trained diffusion-based methods, our approach achieves both higher fidelity (as shown by superior PSNR and SSIM in Table 1) and better perceptual quality. This indicates that our method delivers state-of-the-art performance without incurring excessive computational cost.

1. The compared methods are outdated

Thank you for your suggestion. We would like to clarify that the version of DiffBIR we compare against is not outdated. While the initial arXiv submission was in August 2023, the method has been significantly updated and was accepted to ECCV 2024. We compare with this latest and most competitive version of DiffBIR.

Similarly, PASD was also published at ECCV 2024, and for ReShift, we use the updated results from its TPAMI 2024 publication. Therefore, our comparisons are based on the most recent and up-to-date versions of these methods, ensuring a fair and current evaluation.

Importantly, the final published versions of both DiffBIR and PASD (in ECCV 2024) contain significant updates and improvements compared to their initial arXiv submissions, making them strong and contemporary baselines for comparison.

Furthermore, we have evaluated the "Efficient Visual State Space Model for Image Deblurring (EVSSM)", which was published at CVPR 2025, on the benchmarks. The results are shown in the following table. EVSSM$^*$ indicates that we retrain EVSSM on our training set.

Our method also achieves better performance on no-reference metrics compared to the latest image deblurring method from CVPR 2025 on real-world images, demonstrating better visual quality.

2. The proposed method is not the best on some metrics.

Our goal is to develop a diffusion-based framework that leverages the powerful generative prior of pre-trained models to produce high-fidelity and perceptually realistic results for image restoration. While pixel-level metrics like PSNR are important, our primary focus is on achieving a superior balance between fidelity and perceptual quality.

It is important to note that Hi-Diff is fundamentally a regression-based method built on a transformer architecture, which only incorporates features from diffusion models as guidance, rather than using a diffusion process to generate the final image. In contrast, our method is a fully diffusion-based framework.

Among all methods that leverage pre-trained diffusion models (ControlNet, DiffBIR, PASD, and DeblurDiff), our method achieves the highest PSNR, SSIM, and LPIPS scores on synthetic datasets (GoPro, DVD, RealBlur), as shown in Table 1 of the main paper, demonstrating better fidelity. This indicates that our method is more effective at preserving the information from the input image.

For no-reference metrics, Table 1 of the main paper shows that our method achieves the best MANIQA and CLIP-IQA scores on all four benchmarks: GoPro, DVD, RealBlur, and RWBI. On the RealImages dataset, our method also achieves the highest NIQE and MANIQA.

We acknowledge that on the Real Images dataset, our MUSIQ and CLIP-IQA scores are slightly lower than those of PASD. However, our overall state-of-the-art performance across the majority of metrics and datasets demonstrates the effectiveness and advancement of our approach.

3. Please provide a comparison of all methods in Table 1 in terms of the number of parameters and inference time.

We have provided runtime information for some methods in Table 4 of the Supplemental Material.

We provide a more comprehensive comparison of other methods, including the number of parameters and inference time, in the following table.

As discussed in L86–L92 of the Supplemental Material, FFTformer is not a diffusion-based method, which makes it faster but results in inferior visual quality compared to diffusion-based approaches. ReShift does not use a pre-trained diffusion model, giving it some advantages in runtime over methods that do; however, its visual performance is worse than diffusion-based methods such as DiffBIR, PASD, and DeblurDiff. Among all methods that leverage pre-trained diffusion models, our approach achieves both higher fidelity (as shown by better PSNR and SSIM in Table 1) and better perceptual quality. Moreover, our method has fewer parameters and lower FLOPs than PASD, and runs faster than both DiffBIR and PASD.

4. The NIQE and MANIQA metrics used in Table 2 are no-reference quality assessment indicators, and they exhibit large fluctuations. Furthermore, they cannot accurately evaluate visual quality. Metrics like PSNR, SSIM, and LPIPS would be more appropriate here.

Since Table 2 in the main paper evaluates results on real-world images where ground truth (GT) is unavailable, metrics like PSNR and SSIM cannot be computed. To provide a comprehensive comparison, we also report PSNR, SSIM, and LPIPS on synthetic datasets (GoPro) where GT is available, as shown in the following table.

When the EAC module is removed (w/o EAC), the fidelity of DeblurDiff decreases, demonstrating that EAC effectively preserves information from the input image and contributes to improved restoration accuracy. When the features from the pre-trained diffusion model (SD) are not used in LKPN (w/o SD for LKPN), the model lacks the benefit of the iterative refinement guided by the denoised prior, making it difficult for LKPN to provide precise structural guidance. This highlights the importance of the interaction between the degradation estimator and the diffusion model.

5. The novelty of the paper is not very strong. Degradation estimation in Latent Diffusion Models (LDM) has already been explored by many image restoration methods

Methods like DiffIR operate directly in the pixel space, while DeeDSR, S3Diff, and PASD, though performing estimation in the latent space, directly estimate degradation from the corrupted input. This makes their results highly dependent on the accuracy of the initial degradation estimation.

In contrast, our method introduces two key innovations:

1. Iterative Refinement with a Clean Prior: Methods like PASD, DeeDSR, and S3Diff perform a single, static degradation estimation at the beginning of the process, typically based solely on the corrupted latent input. This makes their entire restoration pipeline critically dependent on the accuracy of this initial, one-off estimate. If the initial estimation is poor, the error propagates irreversibly through the diffusion process, leading to suboptimal results. In contrast, our method introduces a dynamic, iterative refinement loop. We do not estimate the degradation once, but re-estimate it at every diffusion step. Crucially, our estimation leverages not just the corrupted input, but also the progressively denoised and cleaner latent features generated by the diffusion model itself. This denoised output serves as a powerful "clean prior," providing rich, high-quality structural information that guides a more accurate degradation estimate for the next step. This closed-loop feedback mechanism allows the degradation estimate to evolve and improve over time, correcting early mistakes and leading to significantly more robust and accurate results.

2. Deblur Kernel-Guided Reconstruction: Existing methods employ the estimated degradation as a generic conditioning signal (e.g., via concatenation as an auxiliary input). In contrast, our approach leverages the predicted deblurring kernel in a more precise and effective manner by integrating it through an Element-wise Adaptive Convolution (EAC) module, which performs feature-adaptive modulation in latent space. This allows the model to apply spatially adaptive deblurring operations. Such a design ensures tight coupling between the restoration process and the estimated degradation, thereby enhancing reconstruction fidelity and mitigating artifact generation.

These design choices result in a more accurate estimation and higher-quality restoration, representing a meaningful advancement over prior approaches. We will cite and discuss the distinctions from these papers in the revised manuscript.

Dear Reviewer,

Have you had a chance to review our new response? Does it address your concerns? We have already responded to your questions and compared the latest methods (EVSSM in CVPR 2025) in our initial rebuttal. We have also provided explanations regarding the performance of our method. If there are still any questions, we hope you can let us know.

Best,

Authors

1. The novelty of this paper

While it is true that some underlying techniques, such as kernel prediction and adaptive convolution, have been explored in prior research, our contribution lies in the novel integration and application of these concepts within a latent diffusion framework for image restoration, leading to a distinct and effective solution.

Specifically:

1. Diffusion-based Formulation: The "Spatio-Temporal Filter Adaptive Network" (ICCV 2019) directly estimates kernels from the blurred frame and its adjacent frames in a video sequence, leveraging temporal information. In contrast, our method is specifically designed for the latent space of pre-trained diffusion models. It does not rely on temporal context but instead estimates the degradation kernel by jointly leveraging the input latent of the blurred image and the progressively refined, denoised latent features generated by the pre-trained diffusion model during the iterative process. This integration of the diffusion model's strong generative prior fundamentally differentiates our approach from non-diffusion based methods.

2. Spatially-Variant Degradation Modeling: Unlike "Parallel Diffusion Models" (CVPR 2023), which estimates a single, global blur kernel, our method focuses on estimating a spatially-variant degradation. This is crucial for handling real-world blurs that vary across the image. Our LKPN network predicts location-specific kernels, enabling more accurate and localized restoration.

3. Iterative Refinement: We introduce an iterative mechanism where the denoised output from the diffusion model at each step provides a clean prior that actively guides the refinement of the spatially-variant degradation estimate. Non-diffusion methods (as mentioned above) perform "static, one-time" degradation estimation and restoration, whereas our method enables "dynamic, iterative" estimation and restoration. It fully leverages the diffusion model's strength in generating high-quality images. This is further validated by our ablation study in Table 2: the "w/o SD for LKPN" variant, which removes the iterative refinement by not using the diffusion model's features, results in significantly worse performance, confirming the critical role of our iterative design.

We agree that the related work section can be strengthened, and we will include the mentioned papers in the revised manuscript to provide a more comprehensive context.

2. Why the proposed iterative refinement process should converge

The process is designed to leverage the inherent property of diffusion models: the restoration quality improves progressively over the denoising steps. As the diffusion model generates increasingly cleaner latent features with each step, these high-quality features serve as a strong prior for our LKPN network. This enables LKPN to estimate more accurate deblurring guidance at each subsequent step.

As detailed in Section 5 of the Supp, we show that our LKPN, combined with the EAC module, generates increasingly accurate deblurring guidance over the diffusion steps, which in turn facilitates better image restoration. This dynamic, iterative interaction is a key advantage of our approach. Figure 10(c) shows that the MANIQA scores of the intermediate results progressively improve from left to right, with values of 0.43, 0.47, 0.55, and 0.58, respectively, clearly demonstrating the quality enhancement throughout the iterative process.

3. Why DeblurDiff is superior to other potential approaches

As discussed in L40–L50 and L121-L129 of the main paper, methods like DiffBIR rely heavily on the initial pre-deblurred result. When this initial estimate is poor (Figure 1(c)), it can lead to significant structural errors in the final output (Figure 1(g)).

In contrast, our method features an iterative refinement process, where the LKPN, guided by the denoised output of the diffusion model, progressively generates more accurate degradation estimates, leading to a clearer conditioning input and a feedback loop that produces a more distinct and artifact-free image. (Figure 1(h)).

4. Visualization of the filtered blurred images

In Figure 2 of the main paper, we show the intermediate results of the reconstructed latent clean images at different diffusion steps. These results are obtained by applying our estimated deblurring kernels in the latent space and then decoding them, effectively serving as visualizations of the filtered blurred images. Furthermore, Figure 10 in the Supp provides additional ablation studies that include these intermediate reconstructions.

We will provide a more detailed discussion and clarification on these visual results in the revised paper.

5. Comparison of Params, FLOPs, and runtime.

We have provided runtime information for some methods in Table 4 of the Supp. We provide comprehensive comparisons of model parameters and FLOPs in the following table.

As discussed in L86–L92 of the Supp, FFTformer is not a diffusion-based method, which makes it faster but results in inferior visual quality. ReShift does not use a pre-trained diffusion model, giving it some advantages in runtime over methods that do; however, its visual performance is worse than diffusion-based methods such as DiffBIR, PASD, and DeblurDiff. Among all methods that leverage pre-trained diffusion models, our approach achieves both higher fidelity (as shown by better PSNR and SSIM in Table 1) and better perceptual quality. Moreover, our method has fewer parameters and lower FLOPs than PASD, and runs faster than both DiffBIR and PASD.

6. The qualitative comparisons lack baseline methods retrained on the new dataset

The qualitative results (e.g., in Figures 1-2 and Figures 4-6) are consistent with Table 1, where DiffBIR and PASD were retrained on our dataset for fair comparison. In the revised manuscript, we will add an asterisk (*) to DiffBIR and PASD in the figure labels to clarify this.

7. Overfitting of the model on training data

In fact, as stated in line 217 of our paper, our training set includes only RSBlur and MCBlur, while the test sets consist of five distinct datasets: GoPro, DVD, RealBlur, RWBI, and Real Images, ensuring a strict train/test split and avoiding data leakage. Furthermore, both RWBI and Real Images are real-world blur benchmarks without ground truth, making overfitting to GT impossible. Notably, on the RWBI dataset, our method achieves the highest scores in MUSIQ, MANIQA, and CLIP-IQA among all compared methods. This demonstrates the strong generalization capability of our model rather than overfitting.

8. More details and experiments on the LKPN architecture.

In the revised manuscript, we will provide a comprehensive description of the LKPN architecture. Specifically, LKPN is a U-Net with 4 stages (often referred to as 4 levels), resulting in a total of 8 encoder and decoder blocks (2 blocks per stage). Each encoder block consists of a ResBlock followed by a self-attention layer, and each decoder block consists of a ResBlock followed by a nearest-neighbor upsampling layer. After each encoder block, a downsampling layer is applied to reduce the spatial resolution before feeding the features to the next stage.

The input channel is 8, and the number of hidden channels is 128 throughout the network. The total number of parameters is 144M.

Due to time constraints, we were only able to conduct ablation studies on different parameter scales of the LKPN network. The results are shown in the table below, where a larger LKPN with more parameters brings moderate performance improvement, indicating a positive correlation between model capacity and restoration quality.

We commit to releasing the full code and model configuration files to ensure complete reproducibility.

9. Detailed implementation of the proposed method

We are actively preparing the code, model weights, and corresponding test results. We commit to releasing the code to ensure reproducibility and facilitate future research.

10. Replacing convolution operations with attention

Our goal is to estimate a spatially variant kernel to guide the restoration of sharp images in the latent space, effectively preserving the information from the blurry input. We have explored replacing this dynamic filtering operation with a cross-attention mechanism. However, as shown in the table below, this alternative design resulted in inferior performance. Therefore, we find that the dynamic filtering approach is more effective for this task. This remains an interesting direction for future exploration.

11. Adding a pre-deblurred image as extra input

We have experimented with this approach by retraining our model to accept additional latents extracted from a pre-deblurred image (generated by a pre-trained DRM) as input, concatenated with the original conditioning features. The experimental results are shown in the table below.

Adding the DRM input leads to improved performance in NIQE, MUSIQ, and CLIP-IQA, while MANIQA slightly decreases. This indicates that incorporating the pre-deblurred image as an additional conditioning signal can enhance the model's perceptual quality to some extent.

12. Whether or not the FFTformer used in DiffBIR is retrained on the proposed dataset.

Yes, the FFTformer used in DiffBIR was retrained on our proposed dataset. We will clarify this point in the revised manuscript.