Solving Inverse Problem With Unspecified Forward Operator Using Diffusion Models

Thank you for the thoughtful and detailed feedback. We reply point-by-point here, to begin the discussion.

W1, W2, W3, Q2: We appreciate your suggestion to compare UFODM with FastDiffusionEM. While UFODM and FastDiffusionEM share certain methodological elements, our approach is distinct in its focus on handling unspecified forward operators. In light of your feedback, we recognize the value in conducting a direct comparison with FastDiffusionEM and will consider incorporating this in our future work. In terms of the comparisons drawn with BlindDPS and GibbsDDRM, it is essential to underscore that our primary objective is centered around scenarios involving unspecified forward operators. While attaining state-of-the-art performance on specific tasks is undoubtedly important, it was not the principal focus of our study. Our approach is designed to broaden the scope and applicability of solving inverse problems in more generalized settings.

W4: Thank you for your insightful comment on the realism of our approach to unspecified H in complex problem settings. While we acknowledge that our current setting does not entirely capture the multifaceted nature of real-world scenarios, we believe it represents a meaningful step forward compared to traditional settings of inverse problems and blind inverse problems. Our approach, by reducing restrictions typically encountered in these conventional frameworks, allows for more flexible and comprehensive solutions. This adaptability is crucial in complex scenarios where standard methods might not be sufficient. We provide a qualitative analysis in Figure 3, showcasing our method's adeptness in managing mixtures of two distinct degradations. This demonstration serves as a testament to the versatility of our approach. Our aim with this setting is not to fully replicate the intricacies of real-world degradations but to provide a more versatile foundation upon which further research can build. We envision future extensions of our work that will progressively incorporate the complexity of real-world scenarios.

W5: We have employed a parallel sampling method within the diffusion process. Each forward function in this process corresponds to a single sample in a batch. This parallel sampling technique allows us to simultaneously estimate samples from all possible function forms represented in the set, thereby significantly enhancing the efficiency of the process. This method alleviates the issue of having to run the diffusion process separately for each candidate $\mathcal{H}$, ensuring a more practical and time-efficient approach. Regarding the criterion based on the distance from the measurement, we recognize that this metric has its limitations and there is indeed room for improvement. We are committed to refining this aspect of our methodology in future work to develop a more robust and accurate criterion for selecting $\mathcal{H}^*$.

Q1: 'UFODM' stands for 'Unspecified Forward Operator using Diffusion Models'. This name succinctly captures the essence of our approach, focusing on addressing inverse problems where the forward operator is unspecified, employing diffusion model techniques.

Q3: In our methodology for handling unspecified forward operators, the parameters of the forward operator, are indeed inferred as part of the inverse problem-solving process. Our approach is designed to simultaneously estimate these critical parameters alongside the primary task of image restoration. This integrated estimation ensures that we adaptively and effectively address the specific characteristics of the degradation in each case.

Q4: This parameter selection is informed by the initialization strategy we adopted from SDEdit [1]. Contrary to using random noise as the starting point, our approach involves initializing with the measurement $\mathbf{y}$ perturbed by noise corresponding to a specific time step. This method aligns with strategies like CCDF, focusing on a more structured and potentially more effective starting point for the algorithm.

Q5: In our algorithm, the N is set to 3, which helps in keeping the number of Function Evaluations (NFEs) both manageable and within practical limits. Additionally, the optimization process involves 100 steps for parameter adjustment. It is important to note that this optimization process does not necessitate further NFEs.

Reference:

[1] Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu and Stefano Ermon. "SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations". In International Conference on Learning Representations (ICLR) 2022.

Thank you for the thoughtful and detailed feedback. We reply point-by-point here, to begin the discussion.

W1: We recognize that our demonstrations, focusing primarily on unspecified known forward operator types, might not fully capture the complexities of real-world scenarios where the forward operator is completely unknown. This is a common challenge in the field, as even training-based, all-in-one restoration methods typically restore images with degradation types that are represented in their training set. In our experiments, we deliberately chose degradations that represent typical inverse problems, aiming to provide a realistic yet controlled experiment environment. Our proposed method introduces reduced restrictions in problem settings, which enables more flexible and comprehensive solutions in complex scenarios. This approach marks a significant step toward addressing real-world settings more effectively.

W2: Thank you for your comment regarding the computational cost of our proposed method. To address this, we have included a detailed Efficiency Analysis in the appendix of our paper.

W3&Q1: To clarify, the selection of a finite number of forward operator types in our experiments was primarily driven by the need for simplicity and clarity in demonstrating our method. This aligns with common practices in the field, where even training-based, all-in-one restoration methods typically focus on a limited range of degradation types represented in their training datasets. Beyond this, our framework goes beyond simply executing parallel reconstructions. We incorporate an innovative approximation algorithm designed to progressively enhance the accuracy of latent variable estimates. In addition, we utilize Maximum a Posteriori (MAP) estimation for precise tuning both the forward operator and its associated parameters. The combination of these strategies distinctly sets our framework apart from standard parallel reconstruction approaches, offering enhanced reconstruction quality.

Q2: Thank you for your question regarding the real-world applicability of our research. The inverse problems we have addressed in our experiments, such as those presented in the FFHQ, AFHQ, and ImageNet datasets, are representative of common challenges in the field of inverse problems. The diversity and complexity of the data in these datasets provide a robust platform for demonstrating the efficacy of our approach in various contexts. We understand the importance of illustrating the practical significance of our work and are continually seeking opportunities to apply our method to real-world examples that further demonstrate its utility and impact.

Thank you for the thoughtful and detailed feedback. We reply point-by-point here, to begin the discussion.

1. Paper Presentation: We understand the concern about the placement of the background section defining the forward operator. To enhance clarity and flow, we will restructure the paper to introduce key concepts earlier, ensuring that the background information is presented in a more logical sequence.
2. Assumptions and Limitations: We recognize the need for a more qualitative explanation of the assumptions underlying our approach, particularly regarding the parameter $\varphi$ and its known prior $p(\varphi)$. In the revised version of our paper, we will provide a detailed discussion of these assumptions, their implications, and their limitations, especially in the context of solving real-world inverse problems.
3. Open-Source Code: Regarding the availability of open-source code, we have included the code as part of the supplementary material accompanying our paper. To further facilitate accessibility and usage, we are committed to making the code publicly available upon the publication of the paper.

Thank you for the thoughtful and detailed feedback. We reply point-by-point here, to begin the discussion.

W1 & Q1: The variable $i$ represents the iteration round within each timestep. We recognize that this approach results in some repetition of equations within the algorithm box. However, this repetition was intentional to preserve the integrity and clarity of the algorithm's presentation. It ensures that each step is explicitly stated, avoiding any ambiguity about the process or the sequence of operations.

W2 & Q2: In our methodology, the diffusion model's output is the estimated image $\mathbf{x}_0$, and it is critical to recognize that this output cannot be directly compared with the measurement $\mathbf{y}$. To facilitate a meaningful comparison, we apply the forward function to the estimated image $\mathbf{x}_0$. This ensures that both the measurement $\mathbf{y}$ and the estimated image $\mathbf{x}_0$ are processed under comparable conditions, thus maintaining the integrity and relevance of the comparison.

Crucially, our proposed method is indeed capable of handling operations like inversion or flipping, which might cause $\mathbf{y}$ to appear dissimilar to $\mathbf{x}_0$. This is because our method does not rely on direct comparisons between $\mathbf{y}$ and $\mathbf{x}_0$, but rather compares images after they have been subjected to the same forward function. This method effectively accounts for transformations that might otherwise complicate direct comparisons, ensuring accurate and reliable assessments regardless of such operations.

W3: We appreciate your comment regarding the motivation of the problem presented in our paper. Our research introduces a new, challenging setting that expands the scope of traditional inverse problems and blind inverse problems. While we recognize that a specific real-world application was not explicitly detailed, the broader aim of our work is to establish a foundational approach that can be adapted to a wide range of practical applications in the future. The reduced restrictions in our proposed problem setting allow for more flexible and comprehensive solutions in complex scenarios where traditional methods may fall short. This has significant potential in fields such as medical imaging, signal processing, and other areas where accurate image reconstruction is crucial despite incomplete or altered data. Furthermore, our methodology's effectiveness and adaptability, particularly in the selection of forward measurement strategies, opens up avenues for extensive future research.

W4: Equation (10) is based on the application of Bayes' Theorem in the context of Maximum A Posteriori (MAP) estimation. The theorem states that the posterior probability is proportional to the likelihood of the data given the parameters multiplied by the prior probability of the parameters. While the equivalence may not be immediately obvious, it is grounded in established probabilistic principles and the specific assumptions of our model.

W5: The prior of $\mathcal{H}$ is omitted since we model it as a constant. This assumption is based on our decision to model the prior distribution over possible operators as a simple uniform distribution. This simplification is made for reasons of computational efficiency and simplicity in the modeling process.

W6: Thank you for your comment regarding the scope of our comparative analysis. It is indeed crucial to emphasize that the proposed framework is designed with a high degree of flexibility and adaptability. This design choice allows other methods, which are solutions to various inverse problems, to be potentially integrated into our framework. The versatility of our approach means that it is not limited to motion deblurring but can be extended to other types of inverse problems commonly encountered in blind settings as shown in section 4.3.