DGSolver: Diffusion Generalist Solver with Universal Posterior Sampling for Image Restoration

W1&Q1: Experssion and selection of noise schedule

Following the linear noise schedule, the formulations of $\bar{\alpha}_t,\bar{\beta}_t,\bar{\delta}_t$ are:

$\beta_t = \beta_{min} + (\beta_{max} - \beta_{min})\frac{t}{T},\beta_{min} = 0.0001,\beta_{max} = 0.02$,

$\delta_t = \delta_{min} + (\delta_{max} - \delta_{min})\frac{t}{T},\delta_{min} = 1\times 10^{-6},\delta_{max}=0.002$

$\bar{\beta}_t = \sum_0^t \beta_i$, $\bar{\alpha}_t = 1 - \sqrt{\prod_0^t (1 - \beta_i)}$, $\bar{\delta}_t = \sum_0^t \delta_i$

Notably, the total timesteps $T=1000, t\in\mathcal{Z}, 0\le t < T$. Since $\beta_t$ and $\delta_t$ are non-negative within the range of [0,1], three variables $\bar{\beta}_t,\bar{\alpha}_t,\bar{\delta}_t$ exhibit strict monotonic increase.

W2: Typos

We sincerely appreciate your meticulous review, and we will correct the misspelled $I_{res}^{\theta}$ in revision. We promise to revise other typos for readability.

Q2: Complexity of naive and queue-based sampling strategies

We utilize neural function evaluations (NFEs) to evaluate the computational complexity. Let $k$ denote the solver order and $n$ be sampling steps. For a $k$-order solver, naive sampling from time $s$ to $t$ requires interpolating $k-1$ intermediate points within the interval, resulting in $k$ NFEs per sampling step. Consequently, the overall computational complexity for $n$ steps is $O(nk)$. In contrast, the queue-based sampling strategy precomputes the values at (k−1) intermediate time points and caches them for reuse in subsequent steps. This reduces the total complexity to $O(k−1+n)$, offering a clear computational advantage for multi-step sampling. For quantitative results, please refer to Tab. r1 in Reviewer (9bRU-W1) and Tab. r3 in Reviewer (1zc3-W2&Q3).

Q3: Insights about controllable restoration.

Our core idea is to map different degradations into a shared representation space, and then leverage high-order solvers to enhance reconstruction accuracy, the queue-based sampling to improve efficiency, and the UPS mechanism to provide effective guidance for restoration. Hence, our model handles compound degradation through a unified framework, but the restoration order is indeed implicitly learned during training rather than explicitly controlled. Below we clarify some insights and potential extensions: (i) Add task-specific prompts as controllable factors to condition the solver. For example, we can mine degradation priors $c$ either from the image itself or from foundation models, and train model $I_{res}^{\theta}(I_t,I_{in},t,c)$ using paired data with restoration order. With learnable parameters, image restoration becomes controllable. (ii) Project the gradient guidance $\nabla \log p(I_{in}|I_t)$ onto orthogonal subspaces for degradation-specific restoration. For instance, we assign orthogonal bases $v_i, i=1,..,k$ for $k$ degradation types during training, and utilize the UPS term to project gradients $\sum_i^k \theta_i P_{v_i}\nabla \log p(I_{in}|I_t)$, where $\theta_i$ is a switch for activating the $i$-th degradation subspace and $P_{v_i}$ is a projection operator. This facilitates controllable and interpretable restoration. We sincerely appreciate your insightful comments, and we will conduct further research in our future work.

W1: Ablation and comparison with DiffUIR

We sincerely appreciate this constructive suggestion. We perform a detailed efficiency comparison with baseline DiffUIR, as presented in Tab. r7. Both methods maintain identical model parameters (7.73M), confirming our enhancements require no additional network capacity. Our method outperforms the baseline in terms of performance, but the overall time consumption increases. For a fair comparison, we apply our solver to DiffUIR for inference process, and the results show a significant improvement in restoration quality, as presented in Tab. r2 in Reviewer (9bRU-W3). This verifies our solvers can be integrated into related methods to enhance restoration accuracy without retraining. For detailed efficiency comparison, please refer to Tab. r1 in Reviewer (9bRU-W1).

Table r7: Ablation and comparison with DiffUIR

W2: Comparisons with 2025 SOTAs

For a fair comparison, we reimplemented and retrained several representative universal image restoration models, namely AwRaCLe[1], MaIR[2], and DeepSNNet[3], on our collected dataset using their released code and official settings. The quantitative results are presented in Tab. r8. Evidently, our method outperforms others. Thanks for your constructive comments, and we will include these results in revision.

Table r8: Quantitative comparisons with latest methods

[1]Rajagopalan S, Patel V M. AWRaCLe: All-weather image restoration using visual in-context learning[C]. AAAI, 2025.

[2]Li B, Zhao H, Wang W, et al. Mair: A locality-and continuity-preserving mamba for image restoration[C]. CVPR, 2025.

[3]Deng X, Zhang C, Jiang L, et al. DeepSN-Net: Deep semi-smooth Newton driven network for blind image restoration[J]. TPAMI, 2025.

W3: Counter-intuitive phenomenon between performance and sampling steps

Thanks for your insightful observation. We agree that the non-monotonic performance trend in Tab. 4 appears unintuitive at first glance. This counter-intuitive phenomenon may stem from two main factors. (1) Our approach to image restoration is motivated by the principle of commonality, aiming to handle diverse degradation types within a unified framework. In Appendix E.2, we discuss the model's behavior under compound degradation scenarios. It can be observed that across multiple sampling steps, the model tends to first remove the primary degradation before addressing secondary ones. Consequently, in cases where samples from the deraining or deblurring datasets also suffer from additional degradations (e.g., low-light conditions), the restored output may diverge from the available reference that typically reflects only the removal of the primary degradation, leading to reduced evaluation metrics despite better perceptual quality. (2) The stochasticity in both the forward and reverse processes, combined with network estimated errors, can cause slight influence on performance within an acceptable range.

W4: Datasets setting

Our task setting is closely aligned with that of DiffUIR. However, since each method adopts different dataset configurations for image restoration, we attempt to collect and standardize all the datasets used by these methods for fairness, enabling direct and meaningful comparisons. Besides, we strictly follow their open-source code and retrain all of them except autodir.

W5: SDEs motivation

The motivation behind adopting forward SDEs is that SDEs offer a broader perspective for analyzing the transition of probability distributions in a continuous manner. As discussed in Appendices A and B, both the perturbed process and deterministic implicit sampling can be viewed as first-order instances of the reverse-time SDE and ODE, respectively. Therefore, defining the diffusion process from the perspective of probability distribution in Eq. (4) is fundamentally equivalent to describing it through continuous SDEs in Eq. (5). Their performance comparisons can be found in Tab. 2, where the results for $k=1$ without UPS correspond to deterministic implicit sampling, while our high-order solvers demonstrate improved performance. Besides, SDEs formulation offers higher scalability, enabling the integration of our high-order solver and UPS, which are tightly coupled with the dynamics of the SDEs.

W6,7,8: Figure and table errors

Thank you for your careful observations regarding inconsistencies in notation, figures, and formatting. We will correct the inconsistency in the description of Figure 2, as well as the misaligned highlighted regions in Figure 5. In addition, Tables 2, 3, and 4 will be revised to include clearly marked highlights for improved readability. We will thoroughly check and address all similar issues in the revision for clarity and accuracy.

Q1: Generalization on other methods

Thank you for the thoughtful question. Within the domain of image restoration, UPS can indeed be generalized to related frameworks such as DiffUIR. We apply UPS to DiffUIR, which yields performance gains, as shown in Tab. r2 in Reviewer (9bRU-W3). Broadly speaking, UPS depends on residual information from given distribution to provide gradient guidance for diffusion reverse process. It can be seamlessly extended to other tasks wherein both prior and target data distributions are accessible, such as image translation. Therefore, we believe UPS exhibits strong scalability and generalizability. We leave a more comprehensive exploration of UPS in other vision tasks as promising future work.

Q2: Ablating sampling strategies on performance

In Appendix B.2, we theoretically show that cumulative error is positively correlated with the sampling interval $\Delta t$ . For a k-order solver, naive sampling from time $s$ to $t$ requires interpolating $k−1$ intermediate points, whereas queue-based sampling leverages precomputed points cached in a queue. As a result, the effective sampling interval in the queue-based strategy is larger than that of naive sampling, which may lead to higher theoretical error. To validate this, we conduct experiments, and the results are shown in Tab. r9. Performance of queue-based sampling is slightly lower than naive sampling. However, queue-based approach significantly reduces neural function evaluations (NFEs), achieving 2–3× higher efficiency than naive sampling (please refer to Tab. r1 in Reviewer (9bRU-W1) ).

Table r9: Performance for differnent $k$ with different strategies  

Q3: Solver order and inference efficiency

We use neural function evaluations (NFEs) to measure inference efficiency. Let $k$ denote the solver order and $n$ be sampling steps. For a $k$-order solver, naive sampling from time $s$ to $t$ requires interpolating $k-1$ intermediate points within the interval, resulting in $k$ NFEs per step. Consequently, the overall computational complexity for $n$ steps is $O(nk)$. In contrast, the queue-based sampling strategy precomputes the values at $k−1$ time points and caches them for reuse in subsequent steps, offering high efficiency by reducing total complexity to $O(k−1+n)$. Therefore, Both naive solvers and queue-based solvers have the same time complexity $O(k)$ only when $n=1$. When $n > 1$, the naive solvers are increasingly less efficient than queue-based solvers because $nk \ge k-1+n$. Under the queue-based sampling strategy, the difference in NFEs among different solvers equals the difference in their orders. As a result, their efficiency gap scales linearly. For quantitative results, please refer to Tab. r1 in Reviewer (9bRU-W1) and Tab. r3 in Reviewer (1zc3-W2&Q3).

I would like to thank the authors for their comprehensive rebuttal. I will retain my original recommendation.

W1&Q1: Challenges and opportunities arise from image restoration ODEs solvers

To the best of our knowledge, in the field of image generation, high-order solvers are primarily employed to accelerate the sampling process from random noise to image samples, with the only requirement being that the generated samples conform to the target distribution. There is no strict constraint enforcing a one-to-one correspondence between samples from the prior and those from the data distribution. Consequently, various solvers can be employed to approximate the data distribution without explicit constraints, and the potential of high-order terms remains largely underexplored due to the weak coupling between the prior and data. In contrast, image restoration involves a strong dependency between the prior (i.e., the degraded image) and the target data (i.e., the high-quality image). Therefore, we argue that the key challenge and opportunity for designing solvers in image restoration lies in effectively exploiting the relationship between these two distributions and embedding this relationship throughout the diffusion process. To this end, we propose DGSolver. Our key insights are: (1) We fully incorporate the residuals derived from both the prior and data distributions into the high-order solver. (2) The potential of residuals in designed solvers is further exploited through a Universal Posterior Sampling (UPS), which reinforces the connection between the prior and data distributions, thereby enhancing restoration performance, inference stability, and sample fidelity.

W2&Q3: Disscusion about queue-based sampling

High-order solvers can reduce the sampling steps, but they do not necessarily reduce the number of neural function evaluations (NFEs) in few-step sampling scenarios. NFEs directly measure the number of neural network calls with respect to total runtime. For a $k$-order solver, naive sampling from time $s$ to $t$ requires interpolating $k-1$ intermediate points within the interval, resulting in $k$ NFEs per step. Consequently, the overall computational complexity for $n$ steps is $O(nk)$. In contrast, the queue-based sampling strategy precomputes the values at $k−1$ time points and caches them for reuse in subsequent steps, offering high efficiency by reducing total complexity to $O(k−1+n)$. Therefore, only when $n=1$ do both naive solvers and queue-based solvers have the same time complexity $O(k)$. When $n > 1$, the naive solvers are increasingly less efficient than queue-based solvers for $nk \ge k-1+n$.

To verify that, we report runtime comparisons in Tab. r3 with a fixed image size of 512 and UPS deactivated. Apparently, queue-based strategy is more efficient than naive sampling. Additionally, as shown in Tab. r1 in Reviewer (9bRU-W1), enabling UPS further increases the computational cost, making the efficiency gap between the two strategies even more pronounced. Moreover, Tab. r9 in Reviewer (iqQ1-Q2) presents a performance comparison between the queue-based and naive strategies. Queue-based solvers possess a 2–3× speed advantage with comparable restoration quality to naive solvers.

In conclusion, queue-based accelerated sampling still remains effective when using fewer sampling steps  

Table r3: Time efficiency of differnent strategies

W3&Q4: Discussion about $\delta_T$

We appreciate your constructive suggestion. We respectfully clarify that performance comparisons under different values of $\overline{\delta}_T$ are presented in Tab. 3 (Page 8). As observed, the choice of $\overline{\delta}_T$ significantly influences restoration performance across different tasks, with higher values generally being more suitable. Specifically, the model achieves the best performance on deraining, low-light enhancement, and desnowing tasks with $\overline{\delta}_T = 1.0$; for dehazing, the optimal value is $\overline{\delta}_T = 0.9$; and for deblurring, $\overline{\delta}_T = 0.75$ yields the best results. Nevertheless, considering overall performance across tasks, $\overline{\delta}_T = 1.0$ emerges as the most favorable setting within our framework.

W4: Real world quantitative evaluation

To systematically evaluate the zero-shot capability of each method on real-world datasets (Appendix E, Tab. A1), we adopt PSNR for paired data and NIQE [1] for all data to assess perceptual quality, as reported in Tab. r4. Our method consistently achieves superior PSNR across various tasks. However, the non-reference metric (NIQE) exhibits patterns that diverge from the objective PSNR trend, where our performance is at a moderate level. This observation is consistent with prior studies [2,3], which highlight the inherent instability of NIQE. NIQE is highly sensitive to content variations and often fails to align with human perception, particularly under complex degradations. In conclusion, our method still remains highly competitive, as evidenced by both quantitative metrics and visual results (Appendix E, Fig. A6).

Table r4: Evaluation on Real-world image restoration tasks

[1] Mittal A, Soundararajan R, Bovik A C. Making a “completely blind” image quality analyzer[J]. IEEE Signal processing letters, 2012.

[2] Liu Y, Zhao H, Gu J, et al. Evaluating the generalization ability of super-resolution networks[J]. TPAMI, 2023.

[3] Zhang L, Zhang L, Bovik A C. A feature-enriched completely blind image quality evaluator[J]. TIP, 2015.

Q2: Disscusion about solver order

The performance and computational cost of different solvers are summarized in Tab. r5. Notably, UPS increases computational cost by ~2× due to additional backward propagation[1]. In our configuration, the solver with $k=2$ and UPS yields the best performance and is thus used as the default setting in the paper. The case of $k=3$ demonstrates the solver’s scalability, although performance reaches a saturation point within our framework. Nevertheless, when generalized to related frameworks such as DiffUIR, both $k=3$ and UPS bring additional improvements, as shown in Tab. r2 in Reviewer (9bRU-W3). In terms of efficiency, due to the queued sampling strategy yielding neural function evaluations (NFEs) of $n+k−1$ (as proved in W2&Q3), the computational difference between solvers of different orders is determined solely by the order gap. Consequently, FLOPs increase approximately linearly with $k$. In conclusion, the solver complexity grows linearly with order, while performance varies depending on the target framework.

Table r5: FLOPs vs. PSNR for differnent $k$ with queue-based sampling strategy (Steps=8)

  [1] Hobbhahn M, Sevilla J. What’s the backward-forward flop ratio for neural networks?[J]. Published online at epochai. org, 2021.

Thank you for the authors' substantial efforts in the rebuttal. It has addressed my concerns, and I will be raising my score from 3 to 4.

W1: Steamlined explanation of our key components

We thank the reviewer for pointing out the issue with presentation clarity. We will add a more streamlined method description for readability in Sec. 3. Simplified contents can be concluded as: The overview of our DGSolver is illustrated in Fig. 2. In the forward process, diffusion generalist SDEs map degradations into a shared, degradation-agnostic latent space. In the reverse process, our DGSolver integrates diffusion generalist solvers and universal posterior sampling to jointly enhance inference accuracy and stability, with respect to red and blue trajectories in Fig. 2. The former component reduces cumulative error by solving a semi-linear ODE, thereby guiding the solution toward the ground truth; the latter component further optimizes the solutions through gradient guidance along the learned data manifold."

W2: Residual Formulation and Degradation

From a degradation modeling perspective, residual-based and kernel-based approaches emphasize additive degradation restoration and inverse problem solving, respectively, with each carrying inherent limitations. However, we respectfully clarify that our use of residual modeling is not tied to a strict pixel-wise alignment assumption, but instead rooted in a signal decomposition perspective. From this broader view, residual decomposition offers a more general and widely applicable framework, extensively adopted across various image restoration tasks, such as super-resolution, deblurring, and denoising [1,2,3]. In contrast, kernel estimation often suffers from ill-posedness and increased complexity, making the decomposition less stable and kernel estimation harder to solve. In our DGSolver, residual modeling is seamlessly embedded into the diffusion generalist SDEs, enabling the unified design of both high-order solvers and the UPS mechanism. As a result, UPS becomes a plug-and-play component for diffusion-based methods that adopt residual modeling, whereas kernel-based approaches still depend on explicit kernel priors or estimation. That said, we acknowledge the limited capacity of residual modeling to generalize across all inverse problems. For instance, as shown in Tab. 2, UPS yields more substantial improvements in deraining (i.e., 0.15 dB (k=2), 0.12 dB (k=3)) than in deblurring (i.e., 0.07 dB (k=2), 0.06dB (k=3)). We believe that kernel-based modeling may perform well in specific inverse problems, but its applicability to broader restoration scenarios remains somewhat constrained, especially in the compound restoration tasks.

In a broad sense, our residual-based modeling exhibits the potential to be applied beyond the realm of image restoration. It can be seamlessly extended to other tasks wherein both prior and target data distributions are accessible, such as image translation.

[1] Zhang Y, Li K, Li K, et al. Residual non-local attention networks for image restoration[J]. ICLR, 2019.

[2] Liang J, Cao J, Sun G, et al. Swinir: Image restoration using swin transformer[C]. CVPR, 2021.

[3] Cui Y, Ren W, Cao X, et al. Revitalizing convolutional network for image restoration[J]. TPAMI, 2024.

W3: Typos and figure errors

We appreciate your attention to detail regarding typos and figure inconsistencies. We will revise "computationally intractability" to “…computationally intractable” in line 94 and capitalize “when” to “When” in line 111. Besides, we will ensure that all zoomed-in regions are spatially aligned across input, ground truth, and restored images in Fig. 5. All these issues will be thoroughly checked and addressed in the revision to ensure clarity and accuracy.

Q1: Disccusion about UPS and DPS

We thank the reviewer for pointing this out. We sincerely apologize for overlooking the discussion and connection to DPS in Section 3.3. We fully acknowledge and appreciate that UPS is partially inspired by DPS, and in fact, we have dedicated Section 2.2 to introducing the theoretical foundations and related methods of DPS. We will explicitly include a discussion in Section 3.3. The simplified addition is as follows: “DPS[1] circumvents the intractability of posterior sampling in diffusion models via a novel approximation, which is generally applicable to noisy inverse problems. Inspired by this, we propose universal posterior sampling from the perspective of residual modeling.”

Both UPS and DPS aim to incorporate gradient guidance into the diffusion sampling process. By omitting the measurement operator, our residual modeling is generalizable across compound degradations. Beyond omission of an explicit measurement operator, we believe UPS can be a plug-and-play component for diffusion models in field of image restoration. Given that the prior distribution (e.g. degraded images) and data distribution (e.g. high-quality images) are both available, we believe future variants of diffusion model will increasingly integrate residual prior into their formulation. In this context, UPS can be seamlessly coupled into their reverse process. Moreover, UPS can be seamlessly extended to other tasks wherein both prior and target data distributions are accessible, such as image translation. In contrast, kernel-based models are often limited by their formulation. In future work, we plan to explore the adaptability of UPS in other tasks.

[1] Chung H, Kim J, Mccann M T, et al. Diffusion Posterior Sampling for General Noisy Inverse Problems[C]. ICLR, 2023.

Q2: Performance and sampling steps

Thanks for your insightful observation. We agree that the non-monotonic performance trend in Tab. 4 appears unintuitive at first glance. This counter-intuitive phenomenon may stem from two main factors. First, the stochasticity in both the forward and reverse processes, combined with network approximation errors, can cause slight performance fluctuations across neighboring sampling steps, where the variations are within an acceptable range. Second, since we project all degradations into a shared degradation-agnostic representation and train the model on mixed degradation types, the model implicitly learns to handle compound degradations, as discussed in Appendix E.2. Consequently, when the number of sampling steps increases, the model may begin to address secondary degradations present in the image. For example, some samples in the deraining or deblurring datasets, also exhibit low-light conditions. After removing the primary degradation, the model may perform low-light enhancements, where the available ground truth typically reflects only the removal of the primary degradation. This mismatch means that additional sampling steps do not necessarily bring the output closer to the reference, and may even result in a slight drop in evaluation metrics.

Q3: Sample variety

Thanks for raising the concerns about sample variety. In field of image restoration, the primary focus is on achieving high-fidelity and consistent restoration results, rather than promoting sample diversity. Our work is therefore motivated by fully leveraging diffusion models to recover a unique, high-quality target, as opposed to sampling from a distribution of plausible outputs. As a result, sample variety is expected to be low. However, we still consider that our sample variety may arise from two sources: (i) the randomness of initial states sampled from the Gaussian prior, and (ii) the model’s capacity to handle compound degradations. For (i), we conduct a thorough evaluation across 10 random seeds, reporting the average and variances of metrics in Tab. r6. The results indicate strong robustness, with highly consistent outcomes across seeds. Notably, the use of UPS further stabilizes the inference process and reduces variance. For (ii), we conduct generalization experiments on compound degradations in Appendix E.2. Since the initial states are degradation-agnostic representations, more sampling steps allow the model to progressively eliminate secondary degradations, resulting in diverse yet plausible restorations.

Table r6: Performance of our DGSolver with different seeds

W1: Efficiency comparisons among universal methods

We appreciate the reviewer’s concern regarding inference efficiency. For fairness, we collect and mix the datasets used by comparison methods, with image resolution ranging from 256 to 1024 pixels. Accordingly, we evaluate model efficiency under three representative resolution settings, as summarized in Tab. r1. Let $k$ denote the solver order and $n$ be sampling steps. Obviously, our method (n = 3) and baseline DiffUIR remain competitive in computational cost and efficiency. When increasing $n$, memory consumption remains stable while time cost grows proportionally. Activating UPS that requires gradients backpropagation introduces per-step computational overhead and additional memory usage. To alleviate these issues, we employ a queue-based sampling strategy that significantly improves efficiency by reducing the computational complexity. Specifically, we use the number of neural function evaluations (NFEs) to evaluate the complexity. For a $k$-order solver, naive sampling from time $s$ to $t$ requires interpolating $k-1$ intermediate points within the interval, resulting in $k$ NFEs per step. Consequently, the overall computational complexity for $n$ steps is $O(nk)$. In contrast, the queue-based sampling strategy precomputes the values at $k−1$ time points and caches them for reuse in subsequent steps, offering high efficiency by reducing total complexity to $O(k+n−1)$. Tab. r1 also demonstrates that the queue-based solver achieves approximately a 2× efficiency improvement over naive solvers when $k=2$, and around a 3× improvement when $k=3$.

Table r1: Efficiency comparisons among universal methods. '-' means out of memmory.

Q1&Q2: Definitions about $q_{0t}$ and norm $\|\cdot\|$

We consider $q_0$ as the data distribution of $I_0$. In Eq. (4), $q_{0t}:=q(I_t\vert I_0,I_{res},I_{in})$ denotes a conditional probability distribution of $I_t$ conditioned on $I_{0},I_{res},I_{in}$ for simplicity. In Eq. (21), $\|\cdot\|$ is $l_2$-norm. We appreciate your comments and promise to refine the notations to avoid ambiguity in the revision.

Q3: Isolated contribution of our Solver

We appreciate your insightful suggestion. Our solver is built upon the prediction of residuals and noise, and is thus applicable to any diffusion-based models that adopt similar formulations. To validate its generality, we adapt our solver to DiffUIR. As shown in Tab. r2, DiffUIR consistently benefits from our solver across various configurations, leading to notable performance improvements.

Table r2: Adapt our solvers to DiffUIR