Human Body Restoration with One-Step Diffusion Model and A New Benchmark

Q4-1: However, the images restored from OSDHuman have some color shift. For example, in Figs. 5 and 6 of the supplementary materials, the teeth of the person in the 2nd and 5th sets of images are noticeably whiter.

A4-1: Thank you for pointing out the color shift issue. We address this by applying wavelet-based color correction [1], which combines the high-frequency details from the restored image with the low-frequency color components from the original input. This helps align color distributions while preserving structural details.

We plan to integrate this correction into the model for end-to-end learning of color consistency in future work. Your feedback is greatly appreciated.

[1] Mallat, Stephane G. "A theory for multiresolution signal decomposition: the wavelet representation." TPAMI 1989.

---

Q4-2: The LQ images in PERSONA-Val seem to have stronger Gaussian noise than those in PERSONA-Test. Could the degradation be made more realistic?

A4-2: In real-world scenarios like group photos, surveillance footage, or chat images, factors such as long shooting distance, compression, and low-end devices often lead to severe degradation. Our pipeline is designed to simulate such challenging cases, which may result in stronger noise than typical natural degradations.

Thank you for the suggestion. We will explore refining the pipeline by incorporating more realistic artifacts like motion blur in future work.

---

Q4-3: Some individuals in the dataset images occupy less than one-third of the 512×512 frame. Does this mean the resolution is too small for effective human body restoration?

A4-3: Thanks for raising this concern. Human body restoration is mainly used in portrait photography. In such cases, people care more about the harmony between the person and the background, not just the body or the scene alone. In everyday mobile photography, the human subject does not always take up a large portion of the image. Instead, good composition and visual balance are more important.

The PERSONA dataset uses square 512×512 images. This format works well for various poses, such as standing, sitting, crouching, or group interactions. It also helps blend the human subject with the background, requiring the model to restore both in a consistent and natural way. This is why we chose this design for the dataset.

---

Q4-4: Regarding the model, how scalable is it for images with larger resolutions?

A4-4: Thank you for your thoughtful questions. Our model scales well to high-resolution images using a tiled inference strategy. For example, we processed a 3472×4800 image by setting the VAE encoder tile size to 1024×1024 (32px overlap), latent tile size to 96×96 (32px overlap), and decoder tile size to 224×224. Inference ran on an A6000 GPU with a peak memory usage of 32 GB. You can visually inspect the detailed results using the anonymous link.

---

Q4-5:

• In Table 2, the DISTS column shows that OSEDiff* performs better than OSEDiff. OSEDiff* should be marked in cyan.
• In the second row of line 240, 1024 should be formatted as 1,024.

A4-5: Thank you for pointing these out. We will make the necessary corrections.

---

Q4-6: From Table 2 and Table 4, it seems that using HFIE as a Prompt Extractor does not result in better performance for the MANIQA metric. Could the authors provide an explanation?

A4-6: Thank you for the observation. MANIQA is a no-reference IQA metric sensitive to the dataset it was trained on. In our paper, we used the PIPAL-trained version, which yielded lower scores for HFIE in Table 4. However, when using the MANIQA model trained on the KonIQ dataset, HFIE achieved the best performance in our ablation study. The results are shown below:

This highlights how dataset bias affects MANIQA's judgment. When averaging across both metrics, HFIE performs best, further confirming its effectiveness for human body restoration.

---

Q4-7: The authors froze the timestep of the OSD model to 999. I wonder whether this parameter has any effect on the results. Could the authors clarify this?

A4-7: Thank you for your question. In one-step diffusion models, the timestep mainly determines the initial noise level. Since the model performs only a single denoising step, this parameter has limited impact on performance. Fixing it (e.g., to 999) is a common practice, and the model is then fine-tuned to adapt to this noise level.

Q3-1: The paper's innovation seems limited, as both the model and dataset build on existing work.

A3-1: Thanks for raising this concern. OSDHuman is the first one-step diffusion model applied to human body image restoration. Compared to traditional multi-step models, it achieves faster inference and lower computational cost, making it practical for real-world scenarios such as group photos or compressed chat images. To reduce bias introduced by external guidance modules, we introduce HFIE, which enables end-to-end training with lightweight and unbiased prompts. For more details on the novelty of HFIE, please refer to A1-1.

The PERSONA dataset is also the first large-scale benchmark for this task, addressing prior limitations such as single-person bias, fixed poses, and narrow aspect ratios. It covers diverse poses, interactions, and real-world scenarios and will be open-sourced to support the community.

---

Q3-2: The assumption that one denoising step can effectively restore complex human images lacks rigorous theoretical justification, and there is no formal analysis of the HFIE’s attention mechanism or its convergence properties under severe degradation.

A3-2: Thanks for raising this concern. One-step diffusion has been widely explored in image generation and restoration. Recent works [1, 2] distill multi-step models into one step, while methods like SinSR and OSEDiff use one-step strategies for super-resolution. Since body images share similar complexity with natural images, applying one-step diffusion is theoretically reasonable.

The attention mechanism of HFIE can be understood as follows: It encodes 145 embeddings into 77 vectors required by Stable Diffusion 2.1. A learnable query attends to these embeddings via softmax, producing a weighted sum that preserves both local and global information. As a convex combination in the original feature space, this ensures stable and effective guidance during training.

The low-quality images from our degradation pipeline are heavily degraded. OSDHuman with HFIE trains stably on them and achieves good convergence. Compared to DAPE, HFIE leads to faster loss reduction:

Loss visualizations are available at the anonymous GitHub link.

[1] Liu et al., Flow straight and fast: Learning to generate and transfer data with rectified flow, ICLR, 2023.

[2] Yin et al., One-step diffusion with distribution matching distillation, CVPR, 2023.

---

Q3-3: The article lacks more theoretical analysis of the one-step strategy. Is such a one-step strategy better than multiple steps? Can more theoretical proof be provided?

A3-3: Thanks for your feedback. One-step diffusion offers a practical trade-off between performance and efficiency, achieving results comparable to multi-step models with much lower latency (see A2-1). This is made possible by strong base models with good generalization, which are well-suited for tasks like human body restoration.

As for theoretical reasoning, our work focuses more on introducing a new benchmark and model design rather than formal theoretical proof. We are also eager to see further theoretical development on one-step diffusion models, which would benefit the machine learning community.

---

Q3-4: In addition, from a visual comparison, the visual effect of OSEDiff is obviously better than the proposed method, especially it looks more natural. Why is this?

A3-4: Thanks for your questions. In rare cases, our results may look less natural due to specific degradations, sometimes causing color shifts. This can be mitigated via post-processing like wavelet-based correction [3], as described in more detail in A4-1. We will continue to refine our model to improve visual consistency, especially in challenging scenarios.

Overall, our method outperforms OSEDiff in preserving fine facial details and natural tones. For example, in Fig. 2, expressions like subtle smiles are better retained, while OSEDiff may distort them and introduce unnatural reddish hues, indicating weaker perceptual consistency.

[3] Mallat, Stephane G., A theory for multiresolution signal decomposition: the wavelet representation, TPAMI, 1989.

---

Q3-5: The lack of ablations on varied degradations (e.g., motion blur, compression) weakens the evidence for HFIE’s effectiveness.

A3-5: The degradation model we used, Real-ESRGAN, includes several common degradation types, such as downsampling, noise, blur, and JPEG compression. We appreciate your suggestions, and we plan to explore additional degradation types to guide the model to perform better in more natural scenarios. We will experiment with motion blur and continue to explore the model’s effectiveness across a wider range of scenarios.

Q2-1: It is necessary to show the inference speed of different algorithms. This is the reason why one-step diffusion models are used.

A2-1: Thank you for your thoughtful suggestion. We have provided a detailed comparison of inference speed, parameter count, and computational cost for several recent diffusion models in the supplementary materials. For your convenience, the table below presents the same results:

---

Q2-2: In ablation experiments, the proposed components did not always improve on all metrics. It is recommended to add visual comparisons to show the effectiveness of the proposed components.

A2-2: Thank you for your suggestion. Regarding the observation that the proposed component (HFIE) did not always improve on all metrics in the ablation study (Table 4), we have discussed this in detail in our response to Q4-7. We believe that HFIE is not inferior to other methods under the MANIQA metric.

We appreciate your suggestion and will include additional visual comparisons to illustrate the effectiveness of each component. You can view the visual comparisons at the anonymous GitHub link.

---

Q2-3: This paper aims to present the dataset and the corresponding methodology for human body restoration. However, the comparison of the visualizations in Figs. 8 and 9 is still focusing on the face region and the gaps in other human body regions are not significant. I am concerned about the value of this study and how it differs from face restoration.

A2-3: Thanks for raising this concern. Firstly, the reason we focus on faces in Figs. 8 and 9 emphasizes our method's effectiveness in restoring small-scale facial details. Since humans are particularly sensitive to facial perception, improvements in facial regions are noteworthy. Secondly, our method does not solely focus on faces; it also achieves significant restoration results for other body regions and background areas. Additional visual comparisons demonstrating these broader improvements can be found in the supplementary materials.

---

Q2-4: Comparisons with non-diffusion restoration models could be added.

A2-4: Thank you for your suggestion. We will add additional comparisons with classic non-diffusion restoration models. The test set comparison results are shown in the table below:

These results demonstrate the effectiveness of our proposed method compared to existing non-diffusion approaches.

[1] Wang et al., Real-ESRGAN: Blind super-resolution with pure synthetic data, ICCV, 2021

[2] Zhang et al., Designing a practical degradation model for deep blind image super-resolution, ICCV, 2021

[3] Liang et al., SwinIR: Image restoration using Swin transformer, ICCVW, 2021

[4] Chen et al., Dual aggregation transformer for image super-resolution, ICCV, 2023

[5] Chen et al., Activating more pixels in image super-resolution transformer, CVPR, 2023

Q1-1: The novelty seems limited, as VSD is adapted from prior work and HFIE resembles an attention-based DAPE.

A1-1: Thank you for your valuable comments. Our model is the first to focus on human body restoration using a one-step diffusion framework. Our model's VSD module follows the OSEDiff design, which builds on DMD [2] to optimize distribution loss in the latent state space. The DAPE, which requires additional training, inevitably introduces errors and missed predictions, as discussed in our supplementary materials. In contrast, our proposed HFIE does not require separate training. Instead, HFIE utilizes the image encoder from the Recognize Anything Model and integrates a trainable multi-head attention layer. This approach improves performance and reduces computational costs by eliminating the need for extra training and tagging heads.

[1] Wu et al., One-step effective diffusion network for real-world image super-resolution, NeurIPS, 2024

[2] Yin et al., One-step diffusion with distribution matching distillation, CVPR, 2023

---

Q1-2: The dataset uses limited degradation types. If it mainly relies on blind SR from [4], should the task be framed as "super-resolution" rather than "restoration"?

A1-2: Thanks for raising this concern. The RealESRGAN [4] pipeline models a broad range of realistic degradations, including blur, noise, and JPEG compression, beyond simple downsampling. After applying this pipeline, we resample images to 512×512, preserving diverse artifacts.

In addition to synthetic validation data, our test set includes real-world degraded images with motion blur, noise, and compression. Examples are shown in Figs. 5 and 6 of our supplementary materials.

[4] Wang et al., Real-ESRGAN: Training real-world blind super-resolution with pure synthetic data, ICCV 2021

---

Q1-3: Since low-IQA images were filtered out, does this introduce evaluation bias and weaken the value of the HQ-ACF pipeline?

A1-3: Thanks for raising this concern. The IQA metrics used in our HQ-ACF pipeline are widely recognized and commonly applied in image restoration tasks, ensuring the high quality of our dataset. To address potential bias, we further evaluated the dataset using additional IQA metrics not involved in the selection and compared it with other human-related datasets. As shown in the table below, PERSONA achieves the best overall quality and the highest RAM++ category diversity, indicating not only consistently high data quality but also richer semantics.

[5] Mittal et al., Blind/referenceless image spatial quality evaluator, Asilomar Conference on Signals, Systems, and Computers, 2011

[6] Su et al., Blindly assess image quality in the wild guided by a self-adaptive hyper network, CVPR, 2020

[7] Chen et al., TOPIQ: A top-down approach from semantics to distortions for image quality assessment, IEEE TIP, 2024

[8] Zhang et al., Blind image quality assessment via vision-language correspondence: A multitask learning perspective, CVPR, 2023

---

Q1-4: Since the test set only uses VOC, does it fully reflect the diversity of the training data?

A1-4: The VOC dataset contains images that generally exhibit more severe degradations compared to more recent datasets. Thus, we consider it suitable for evaluating the model's restoration capabilities. We appreciate the suggestion and will include more diverse sources in the public release to enhance test set representativeness.

---

Q1-5: Evaluation is only on PERSONA. Would testing on existing datasets better demonstrate generalizability?

A1-5: Thank you for your thoughtful questions. Our proposed PERSONA dataset is the first publicly available benchmark specifically designed for human body restoration. Previous human body restoration methods [5, 6] have not provided public access to their test datasets. We hope that the release of the PERSONA dataset and benchmark will facilitate further contributions to the machine learning and computer vision communities.

[5] Zhang et al., Diffbody: Human body restoration by imagining with generative diffusion prior, arXiv:2404.03642, 2024

[6] Wang et al., Prior based pyramid residual clique network for human body image super-resolution, Pattern Recognition, 2024

---

Q1-6: There are some minor issues that need to be addressed:

1. The full name of SOTA is missing in Line 107.
2. The subcaption of Fig. 2 should be "SinSR" instead of "Sinsr."
3. Some citation formats should be revised.
4. Mathematical expressions should be consistent: "z_L" is used in Line 226, while "Z_L" appears in Fig. 4.
5. LoRA is not cited.

A1-6: Thank you for pointing out these issues. We'll carefully revise them.