ODEdit: Blind Face Restoration through Ordinary Differential Equations

Thank you for the review and thoughtful feedback. Below we address specific questions.

Q: SDEdit can solve plenty of image editing and synthesis tasks. However, the proposed ODEdit is only for face restoration. Can the authors explain the reasons?

A: The current bottleneck of ODEdit lies in the instability of DDIM inversion. The knowledge that an image represents a face acts as a strong prior, which significantly stabilizes the DDIM inversion, thereby facilitating the application of our method. However, when applied to varied-domain images such as those in ImageNet, the inversion process proved too unstable even before performing any update steps, preventing a fair assessment of the method's performance.

Q: The advantage that replacing SDE with ODE is not clear. Why should the injection of extra noise be circumvented? Are there any disadvantages when using ODE?

A: In fact, injecting random noise into an input increases the complexity of the denoising task, especially in extreme cases. For instance, if the input is already clean, adding noise to further clean it paradoxically introduces additional stochasticity. A major challenge with SDEdit arises when the input is so severely damaged that a significant amount of noise must be injected to generate outputs that are within the expected distribution. In such cases, fidelity to the original image may be completely lost and such scenarios are the ones we mostly address. Some papers address this issue by adding extra guidance when using SDEdit for denoising. However, we propose a guidance-free method. There are certainly drawbacks to using ODEs. Primarily, the quality of images generated using SDEs appears to be superior. Additionally, our method heavily depends on the accuracy of ODE inversion; inaccuracies in this process can lead to failure, as explained previously. Finally, our method is slower as it requires to perform multiple steps to invert the image, whereas SDEdit simply injects noise.

Q: It is not clear how can we get Algorithm 1.

A: We have included more details and insights about algorithm 1.

Q: The experiments lack some details and some comparisons are unfair and incomplete. (1) What is the testing dataset? The authors seem not to describe it in the manuscript. (2) Why not compare different methods on real degraded face images? (3) The synthetic degradations are based on (Hendrycks & Dietterich, 2019) which is not considered in existing methods e.g. CodeFormer. This makes the comparison unfair. Why not compare different methods based on the testing set of CodeFormer?

A: Our method is designed to work on 64x64 images, which limits its applicability. Since we propose a blind restoration method and are comparing it with other methods that also claim to be blind, the type of corruption should not be a concern. Therefore, it's reasonable to compare blind restoration methods across various types of corruption. However, it's important to note that some methods in our benchmark specifically incorporate super resolution and blur during training. Consequently, we expect these methods to perform better with these specific types of noise. Our goal is to use corruptions that we believe were not present during training, in order to accurately assess the real robustness of these restoration approaches. We aim to test a wide range of corruptions to evaluate the versatility and robustness of the methods, rather than their exceptional performance in specific scenarios. Additionally, we have included an example of restoration on a real, old, damaged image in the supplementary material, despite the fact that this specific sort of monochromatic damaged images is a known limit of our approach.

Thank you for the review and thoughtful feedback. Below we address specific questions.

Q: Face fidelity is not only reflected in reconstruction metrics (PSNR,SSIM) but also in face recognition. Maybe the authors should provide some face recognition metrics to further verify the degree of face fidelity.

A: Thank you for your comment. It is indeed ideal to reconstruct a face that can be recognized as the same individual as in the source image by face recognition algorithms. However, since our focus is on cases with a high degree of corruption, the faces in the source images are often unrecognizable. In many instances, face recognition algorithms even fail to detect a face in the source images, let alone provide meaningful landmarks. In such cases of high corruptions, comparing directly with the target is not our goal per se. We strive to generate a realistic image as close as possible to the source image. As a result, in cases of severely corrupted inputs, generating a realistic, plausible face already constitutes an important step.

Q: Compared with existing BFR methods, the proposed method is unsupervised method without any prior. Then, how does this method perform on real-world face images?

A: Thank you for your comment. We have included an example of an old, damaged image in our appendix. However, the prior model we trained was based on a relatively small number of faces, all of which are centered, aligned, and have pleasing colorations. Consequently, the prior we are using is somewhat limited in directly addressing real-world scenarios. Therefore, our examples should be viewed more as proofs of concept rather than as a definitive model.

Q: The experimental part is insufficient and lack of analysis. The experimental results can only verify the superiority of the proposed method but cannot verify its robustness and adaptability. The authors should provide more convincing experiments and detailed analysis.

A: We emphasize that our method performs more consistently across a wide range of different corruptions. This consistency is what we refer to as robustness, as opposed to excelling in handling a specific type of corruption. Indeed, when considering one type of corruption at a time, there are methods that outperform ours. However, these methods often fail more dramatically when confronted with different types of corruption. It is this versatility that we specifically aim to achieve.

Q: The implementation details are missing. The authors should provide the degradation process of this paper in details.

A: We have included additional details in the paper.

Thank you for the review and thoughtful feedback. Below we address specific questions.

Q: If the diffusion model outputs the predicted data instead of the noise, you could potentially utilize face identity loss with a face recognition network. Would this be a feasible naive extension to address the identity leakage problem?

A: Thank you very much indeed for your suggestion. Introducing guidance into the generative process of ODEdit is certainly feasible, and employing a face recognition network with a face identity loss like cosine similarity, could effectively address the identity leakage issue. This approach has been successfully implemented in a recent paper. However, our aim is to design a method robust enough for extreme scenarios where other methods might fail. For guidance to be effective in such cases, the face recognition network itself must be equally robust. Generally, adding any guidance necessitates the guidance method to be as resilient as our method to add value. Practically, in low-noise conditions, this works quite well, but in such scenarios, our method does not outperform others anyway. Conversely, in high-noise situations, we observed that face detection might fail, and landmarks can be unreliable. Considering these extreme cases, we currently prefer a method that relies solely on itself. Furthermore, while identity leakage is a significant concern generally, in cases of severely corrupted inputs, being able to generate a realistic, plausible face is an important initial step.

Q: The model may work for only the celebrated face. One way to figure out is utilize FairFace [1] dataset for inference.

A: Thank you for your comment. It is absolutely right; however, our paper primarily focuses on using a pretrained model to restore images, rather than on the training procedure itself. As a result, if the pretrained model contains biases, they will indeed be reflected in our method. However, we do not believe this fundamentally challenges our method per se. Replacing the pretrained model with an unbiased one would be a straightforward solution.

Q: As seen in Table 1, the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) are worse than previous works.

A: Our method aims to make as few changes as possible to the source image in order to enhance its realism. Consequently, all pixel-based metrics remain close to those of the corrupted image, while the FID and LPIPS metrics show significant improvement. This is a direct result of our approach.

Thank you for the review and thoughtful feedback. Below we address specific questions.

Q: Algorithm 1. This is the core of the method. It's unclear what does it mean ODE_SOLVER( . ). Is this running one step of DDIM Eq(9)? This is used twice in the algorithm. The rationale of the algorithm is not clear or explained.

A: We clarified in the paper what is meant by ODE_solver. In our practical case, it consists indeed of running one step of DDIM. It is used twice because it is used backward and forward, to revert the image, and generate a new one, as an ODE can be run both ways. We added more explanations regarding algorithm 1.

Q: Results of the base methods are blurry. So this method doesn't seem to be doing a good job. Better results are obtained when plugging the method with another prior (CodeFormer, GFPGAN, etc). So in the end, it's more like with this ODEdit you can get a blurry reconstruction (Average) and then from there another prior is enforced to get a better reconstruction. So in the end, where is the gain coming from? Also it's not clear how this method is applied in conjunction with other methods.

A: Our method works with 64x64 images, whereas other methods perform super resolution in 512x512. As a result, ODEdit outputs do seem low resolution compared to others because they are. However, ODEdit is not simply producing blurry or averaged reconstruction. We proposed adding a super resolution model post-ODEdit to compare improvements more fairly. Furthermore, other models simply fail to work on severe corruption altogether, while ODEdit makes them work again. We chose to focus on 64x64 images because the ODE solver we used (DDIM) tends to be more and more unstable when moving to higher resolution. However, provided that DDIM inversion works, we added some results we obtained in 256x256 in Appendix that show a significantly higher degree of details. We also added in Appendix example comparing plugging blurred images, as suggested, to plugging ODEdit outputs to super resolution models. Finally, to combine the methods, we only need to input low-resolution ODEdit outputs to other approaches.

Q: Presentation is cumbersome.

A: We have made modifications in order to follow your comments, reducing the background to more closely related research and extending our description of SDEdit.

Q:The paper is about face restoration, but there's nothing about faces -the only thing about faces is that the base diffusion model is trained on faces.

A: Thank you for your comment. It is absolutely right, nothing seems to prevent us from transferring the proposed approach to other domains apart from the instability of the DDIM inversion. When we applied it to other domains such as Imagenet, even reconstructions on clean data could be inaccurate, hence our choice to first narrow down our focus on faces.