One Diffusion Step to Real-World Super-Resolution via Flow Trajectory Distillation

Response to Reviewer mAgS (denoted as R5)

Q5-1: Could the authors provide more details on the scalability of FluxSR? For instance, how does the method perform on larger or more diverse datasets, and what are the implications for real-time applications?

A5-1: We generate 10k noise-image pairs on the 220k-GPT4Vision-captions-from-LIVIS dataset as new training data. After training with a larger dataset, there are no significant changes in the quantitative results. However, we observe that the model trained with the larger dataset generated fewer high-frequency artifacts in the actual generated images. For real-time applications, further optimization, such as model pruning, quantization or more efficient techniques, would be helpful to improve real-time efficiency without sacrificing performance.

Q5-2: Are there any plans to further refine the model to address this issue? Could the authors explore additional regularization techniques or architectural changes to mitigate these artifacts?

A5-2: Yes, we increase the weights of the anti-artifact losses. Specifically, we increase the weight of TV-LPIPS to 2 and the weight of ADL to 0.2. In practice, we do not observe the high-frequency artifacts anymore, which demonstrates the effectiveness of the proposed anti-artifacts losses. We will include these results into the paper.

Q5-3: While the paper compares FluxSR with other diffusion-based methods, it would be beneficial to include a more detailed comparison with non-diffusion-based approaches, especially in terms of computational efficiency and real-world applicability.

A5-3: Thank you for your suggestion. We compare the computational efficiency and quantitative comparison of FluxSR with non-diffusion methods in the table below. Compared to non-diffusion methods, FluxSR produces significantly better visual results with higher image quality. Although FluxSR has higher computational complexity, it generates more realistic images. In highly degraded scenarios, if the user aims to generate high-quality images and is not too strict about inference speed, we believe FluxSR holds greater value.

Complex Analysis

RealSet65

Q5-4: The paper does not explore how well the method performs when applied to different types of image degradation beyond the tested datasets.

A5-4: We evaluate FluxSR on face restoration task. Although no specific training set was used, our method still achieves good results. The specific quantitative results are shown in the table below.

WebPhoto-Test

Wider-Test

Response to Reviewer jQYF (denoted as R4)

Q4-1: How does the noise-to-image mapping in T2I models fundamentally differ from LR-to-SR degradations in practice? Could you provide a more detailed analysis to support the claim in Figure 2?

A4-1: Although $x_t$ in the diffusion process appears to be similar to LR, they are fundamentally different. First, the distributions of degradations on these mappings are different. The calculation of $x_t$ is given by

$$x_t = (1 - t) x_0 + t \epsilon,$$

which is equivalent to adding Gaussian noise to the image. In contrast, LR is a real-world low-resolution image, which undergoes complex and unknown degradation. In other words, LR images may not be on the T2I trajectory, which is also a condition for the validity of Figure 2. Second, the degradations/noises are added in different domains. T2I adds noise in the latent space, but LR-to-SR can be regarded as adding degradations/noise onto the HR images. We highlight that these differences directly motivates us to reduce the distribution shift and propose our FTD method.

Q4-2: Can you provide a quantitative comparison of inference time, MACs, and parameter count for FluxSR vs. other one-step and multi-step methods? This would help support the efficiency claims.

A4-2: Yes, we have compared FluxSR's computational complexity with other multi-step and single-step methods, including inference speed, MACs, and parameter count. Despite using a 12B parameter model, the inference speed of our method is less than twice that of the fastest one-step diffusion ISR method.

Q4-3: Can you provide a visual comparison of TV-LPIPS vs. LPIPS alone? This would help illustrate the effectiveness of the proposed perceptual loss.

A4-3: Thank you for your suggestion. We would like to show the qualitative comparison of the ablation studies. However, it is not allowed to upload any images according to the rules of rebuttal. In fact, our TV-LPIPS loss achieves better visual results than the LPIPS loss. As shown in Table 4 of the paper (the first two rows), this can be verified by the improved metrics that measure visual quality, including MUSIQ, ManIQA, and Q-Align. We will include more qualitative comparisons in the revised paper.

Q4-4: How sensitive is FluxSR to different training datasets? Would training on a different set of noise-image pairs alter its performance?

A4-4: We generate 10k noise-image pairs on the 220k-GPT4Vision-captions-from-LIVIS dataset as new training data. After training with a larger dataset, there are no significant changes in the quantitative results. However, we observe that the model trained with the larger dataset generated fewer high-frequency artifacts in the actual generated images.

Q4-5: Can the proposed FTD method be extended to video super-resolution? Would additional temporal constraints be needed?

A4-5: Our FTD is indeed a promising approach for image SR, and we believe it could be extended to video SR with some adjustments. Incorporating temporal constraints (e.g., optical flow, temporal smoothness, feature alignment and propagation) between frames is crucial to avoid introducing flickering or artifacts across consecutive frames, and helps preserve motion consistency and smoothness. Due to the time limit, we leave it for future work.

Q4-6: No Supplementary Material is provided.

A4-6: In fact, we have provided supplementary material.

Q4-7: A comparison to alternative single-step SR methods (e.g., GAN-based SR models like ESRGAN, Real-ESRGAN) would help contextualize the approach.

A4-7: We have provided a comparison with non-diffusion methods in the supplementary material. Here, we present a comparison with BSRGAN, Real-ESRGAN, SwinIR, LDL, and FeMASR, as shown in the table below.

RealSet65

Response to Reviewer uJUz (denoted as R2)

Q2-1: The paper lacks innovation. The TV-LPIPS and ADL proposed in the paper are both existing works.

A2-1: The main contribution of this paper is the introduction of FTD, and our method is the first work to distill a large-scale flow matching model like FLUX into a one-step model for image super-resolution. Existing one-step methods struggle with large diffusion models. Specifically, VSD (with optional GAN loss) requires two additional copies of the large model in GPU memory, exceeding the capacity of an 80GB A800 GPU. Our FTD is large-model-friendly, requiring only about 55GB of GPU memory per card for training, and 23.7GB for inference with 512px resolution.

Q2-2: The authors analyze the possible negative outcomes of VSD or GANs in Sec. 4.1, but lack experimental support.

A2-2: We add experiments to support our argument. Specifically, we generate images using teacher models of OSEDiff, AddSR, FluxSR and compute the FID between original T2I images and SR images. FluxSR’s FID is significantly lower than OSEDiff and AddSR, indicating that FTD avoids distribution shift.

Q2-3: The FTD lacks quantitative and qualitative analysis to verify the effectiveness of this improvement.

A2-3: We have presented quantitative results of the FTD ablation in the ablation study section, demonstrating its effectiveness. We will include qualitative comparisons in the supplementary material.

Q2-4: Is the outstanding performance of the proposed method mainly attributed to the FLUX model?

A2-4: We replace FluxSR's baseline with SD3-medium and retrained it, comparing it with TSD-SR that is also trained on SD3-medium . The results show that FluxSR outperforms TSD-SR, indicating that the performance improvement comes from our method.

Q2-5: Lack of corresponding comparison and analysis of model inference speed.

A2-5: We compare FluxSR’s computational complexity with other multi-step and one-step methods. Despite using a 12B parameter model, the inference time of FluxSR is no more than 0.1s longer than the current fastest method.

Q2-6: What advantages does Large-Model-Friendly Training bring to model training and inference?

A2-6: The Large-Model-Friendly Training (LMFT) strategy reduces memory usage and training time. During inference, it reduces memory usage and inference latency.

Q2-7: The color markings in Tab.1 and Tab.2 of the paper are confusing.

A2-7: In Tables 1 and 2, red and blue bold represent the best and second-best methods among all approaches. The best method in one-step diffusion will also be bolded separately. Red bold indicates it is the best overall, while blue bold indicates it is the best in one-step methods but second-best overall.

Q2-8: Lacks a comparison with SUPIR.

A2-8: We add a comparison with SUPIR on RealLQ250 datasets in the table below. And we have included an efficiency comparison with SUPIR in A5.

RealLQ250

Q2-9: The ablation study in the paper lacks a qualitative comparison.

A2-9: We would like to show the qualitative comparison of the ablation studies. However, it is not allowed to upload any images according to the rules of rebuttal. However, the metrics in our paper (MUSIQ, ManIQA, Q-Align) reflect image quality, showing that our method outperforms others in visual results, with TV-LPIPS and ADL effectively reducing high-frequency artifacts. We will include more qualitative comparisons in the revised paper.

Response to Reviewer ozHQ (denoted as R3)

Q3-1: Does not include the DrealSR that is widely used in various baselines.

A3-1: Thank you for pointing out this issue. The table below shows the quantitative comparison on the DRealSR dataset. From the table below, our FluxSR obtains significantly better results.

DRealSR

Q3-2: Concern about the over-sharp qualitative results shown in the Appendix. Also, the cost of the aforementioned full-reference metric like PSNR.

A3-2: We agree that our FTD is not very effective on PSNR, but the hallucination issue is also not severe since most images still retain high consistency with the LR image in terms of content. We highlight that PSNR does not always align well with human visual perception, which has been widely proved in existing works. Moreover, when the input images are severely degraded, the "hallucinated" details generated by our method are reasonable and realistic, and no-reference metrics are better at reflecting the perceptual quality of the images.

Q3-3: In line 035 of the appendix, there is a question mask during referencing that needs to be fixed.

A3-3: Thank you for pointing out this mistake. We will make the correction.

Q3-4: In Figure 1, the proposed FluxSR method seems to generate over-smooth results that do not align with the original low-res image. For example, the helmet in the bottom row ignores all the high-frequency details that can be observed from the LR image.

A3-4: We agree that the generated helmet is smooth, but still visually reasonable. We highlight that our FluxSR is able to adaptively adjust the generative ability according to the risk of generating artifacts, showing better robustness than existing methods. For example, the texture of helmet is very blurry and suffers a high risk of artifacts. Existing methods consistently produce very poor results with severe distortions. Nevertheless, as for the nose and mouth in the face, since the textures can be easily imagined, our FluxSR works very well to produce detailed textures. Moreover, our FluxSR produces better visual details for text (see the second row of Figure 5).

Q3-5: Also, the ablation studies in Tables 3 and 4 only compare four of the eight metrics. Thus, it can be quite incomplete to show the effectiveness of each component. Besides, the reported PSNR is always inferior after incorporating this paper’s designs, which is detrimental to the justification of the claims.

A3-5: We expand the comparison to include additional metrics to provide a more complete evaluation of the performance, as shown in the table below. We highlight that our FTD consistently obtains better results on the metrics that measure visual quality. Regarding the PSNR results, it is important to note that PSNR is not always the best indicator of perceptual image quality, especially in severely degraded images. Our method focuses on enhancing perceptual fidelity and more visually realistic output, and it is better captured by no-reference metrics. It may not always align with the traditional PSNR metric, but it contributes positively to the overall perceptual quality. We will clarify this trade-off in the revised manuscript, emphasizing that our method is optimized for visual realism rather than solely maximizing PSNR.

Ablation study on FTD

Ablation study on different loss functions

Response to Reviewer m3JT (denoted as R1)

We sincerely thank the reviewer for the constructive comments. We provide the detailed responses to all the concerns below.

Q1-1: Incomplete Ablation on FTD.

A1-1: We add the relevant experiments. The results are shown in the table below. In practice, training the entire flow trajectory on $[T_L,1]$ yields similar results with training on a single time step. The main reason is that it is extremely hard to enforce a one-step model to directly mimic the entire flow trajectory, without explicitly preserving the generative ability of the teacher model. Instead, our FTD obtains significantly better results in most metrics that are widely used to evaluate visual quality.

Ablation study on FTD

Q1-2: Insufficient Explanation for Artifacts.

A1-2: To investigate the artifact issue, we visualize the attention/feature maps and found that the similarity between tokens is quite high, and there are indeed repeated features in some dimensions. Additionally, we observe that using the official FLUX model for one-step inference also results in high-frequency artifacts, which we believe is a characteristic of FLUX itself. When using only the reconstruction loss, we observed even more severe periodic artifacts, so these artifacts are not caused by FTD. We hypothesize that this issue is mainly attributed to the limited training data, which are significantly smaller than the ones used to train the multi-step FLUX. To verify this, when using a larger dataset, such as 10k noise-image pairs generated from the 220k-GPT4Vision-captions-from-LIVIS dataset, we observed a clear reduction in periodic artifacts.

Q1-3: Limited Evaluation on Real Datasets.

A1-3: According to your suggestions, we add the comparison on these two datasets, and the results are shown in the table below. In fact, FluxSR achieves the best results. We will include these into the paper.

RealLQ250

RealLR200

Q1-4: There are some inconsistencies and typos regarding mathematical notations. For example, the formulation presented in Algorithm 1 (line 9) is inconsistent with the corresponding equation shown in Figure 3. Clarifying and correcting these discrepancies would improve readability.

A1-4: Thank you for noticing and pointing out this mistake. We did make an error while editing the image. Specifically, the formula in Figure 3 corresponding to line 9 of Algorithm 1 should have a "$+$" sign instead of a "$-$". We will correct this error.

Q1-5: Although Flux is described as a DiT-based diffusion model, Figure 3 visually depicts it as a U-Net structure, potentially misleading readers. Revising the figure to accurately represent Flux’s architecture (DiT-based transformer) would avoid confusion.

A1-5: Thank you for pointing this out. We will delete the U-Net structure and replace it with the DiT architecture in Figure 3.