I-Max: Maximize the Resolution Potential of Pre-trained Rectified Flow Transformers with Projected Flow

W1. Lack of Quantitative Comparisons.

Thanks! Relying solely on a single metric indeed weakens the persuasiveness of the paper. Referring to widely accepted works on high-resolution synthesis [1,2], we use 1,000 prompts randomly sampled from LAION-5B as the test set, along with 10,000 randomly sampled images as the reference set, to calculate FID, FID_crop, and CLIP Score. This allows us to comprehensively evaluate both the global and local image quality, as well as the text-image consistency.

In the table below, we report the evaluation results of Flux.1-dev + I-Max at different resolutions. We can observe that the results were highly consistent with those from the GPT-4o preference evaluation. Both two image quality metrics (FID and FID_crop) show improvement as the resolution increased, achieving optimal performance at 3K resolution, echoing the results in Fig. 6. For the text-image consistency metric, CLIP Score, we observed a very slight decrease as the resolution increased, which is understandable since I-Max guides high-resolution generation using the output at 1024×1024 resolution. Thus, the text-image consistency is not further enhanced.

In addition, the Black Forest Lab Team claims that Flux.1-dev can directly generate images at 2048×2048, but it has been noticed that the generation quality at 2048×2048 is not particularly good enough. Therefore, we also compared the results of its default settings against the results with I-Max at the 2048×2048 resolution. I-Max also achieves a significant margin of improvement, indicating its practical value in real-world applications.

As for the comparison between different low-resolution guidance approaches, we report the results in the table below. All metrics show strong consistency with the GPT-4o preference evaluation in Fig. 7, demonstrating the superiority of the proposed projected flow approach.

We will revise the paper and include these results.

W2. Need for User Study.

Good advice! Human preference evaluation is indeed the most direct method for assessing the performance of generative models. However, during the rebuttal phase, it is challenging to organize a sufficiently large-scale user study. Instead, we utilized GPT-4o to simulate preference evaluation in the paper. As demonstrated in existing work [3], GPT-4o preferences have been validated to align closely with human preferences. In the camera-ready version, we will also supplement the quantitative results derived during the rebuttal phase to make the paper more convincing.

W3. Comparison of Model Parameters and Runtime.

Well said! As a training-free resolution extrapolation algorithm, I-Max does not introduce any additional model parameters. Regarding efficiency, the table below presents the inference time costs at different resolutions under varying numbers of inference steps.

Additionally, we provide quantitative evaluation results at 4096×4096 resolution for different inference step numbers to demonstrate the trade-off between inference cost and generation quality in I-Max. What we observe is that, thanks to the extrapolation stability brought by projected flow, the quality of generated results shows only a slight degradation as the number of inference steps decreases, effectively reducing the high computational costs associated with high-resolution inference. Similar conclusions can also be drawn from the qualitative results shown in the Fig. 10 of our submission.

We will revise the paper and include these results.

[1] Du, Ruoyi, et al. "Demofusion: Democratizing high-resolution image generation with no $$$", CVPR, 2024.

[2] Yang, Zhuoyi, et al. "Inf-dit: Upsampling any-resolution image with memory-efficient diffusion transformer", ECCV, 2024.

[3] Chen, Junsong, et al. "Pixart-\sigma: Weak-to-strong training of diffusion transformer for 4k text-to-image generation." arXiv preprint arXiv:2403.04692 (2024).

Thanks. I understand that the reviewer expects us to compare I-Max with SOTA high-resolution generation methods in terms of performance, parameter count, and runtime. However, among the mentioned methods, Inf-DiT [2] is a super-resolution model, PixArt-Sigma [3] and UltraPixel [6] are tuning-based approaches, while the remaining methods — DemoFusion [1], ScaleCrafter [5], and FouriScale [4] — focus on resolution extrapolation (i.e., training-free high-resolution generation) and share the same motivation as I-Max. In the existing literature, it is uncommon to directly compare tuning-based methods with training-free methods, considering the advantage of tuning-free methods lies in the absence of tuning costs.. Therefore, we will prioritize discussing these training-free competitors [1,4,5].

1. Comparison of performance.

Factually, we conducted comparisons with SOTA training-free methods in the paper, albeit in a reimplementation manner. Considering that I-Max is the first approach tailored for rectified flow transformers, whereas previous works are primarily applied to UNet-based LDMs (e.g., SDXL), to avoid unfair comparisons, we first need to reimplement these methods on the same base model with the same inference techniques — we all know that Flux.1-dev significantly surpasses SDXL. In the table below, our comparative experiments on different guidance methods essentially include: (1) reimplementation of SOTA methods on Flux.1-dev, e.g., Skip-residual from DemoFusion; (2) advanced methods under the DiT architecture, e.g., Time-aware Scaled RoPE from Lumina-Next; (3) widely used model-agnostic method, e.g., SDEdit.

ScaleCrafter [5] and its variants (e.g., FouriScale [4]) rely on dilated convolution for resolution extrapolation, which is specifically tailored for UNet-based LDMs and not suitable for DiT-based LDMs. This further highlights the unique value of I-Max's exploration.

2. Comparison of parameter count and runtime.

The training-free nature of resolution extrapolation approaches ensures that they do not introduce additional parameters, meaning the model's parameter count is entirely determined by the base model used. As for runtime, here we provide a cross-model comparison of inference times between I-Max and the mentioned methods. Thanks to the stability of I-Max (please refer to our previous response), it requires fewer inference steps, achieving comparable inference times to methods based on 3.5B models while utilizing a 12B model. All the results are tested on a single A100 GPU.

3. Summary

Overall, I understand that a cross-model comparison can show how I-Max compares with other T2I Pipelines [1,2,3,4,5,6] in terms of effectiveness. However, as a research paper, I would rather divide the problem into several key techniques and provide useful insights to the community through fair experiments. In addition, resolution extrapolation focuses on unlocking the potential of pre-trained LDMs. Therefore, we argue that the vertical comparison in the paper are of particular value. Compared with existing works [1,4,5], we are the first to achieve that the model's overall performance at the extrapolated resolution can surpass that at the native resolution (e.g., the FID at 4096 resolution is better than the FID at 1024 resolution).

[1] Du, Ruoyi, et al. “Demofusion: Democratizing high-resolution image generation with no $$$”, CVPR, 2024.

[2] Yang, Zhuoyi, et al. “Inf-dit: Upsampling any-resolution image with memory-efficient diffusion transformer”, ECCV, 2024.

[3] Chen, Junsong, et al. “Pixart-σ: Weak-to-strong training of diffusion transformer for 4k text-to-image generation.” arXiv preprint arXiv:2403.04692 (2024).

[4] A frequency perspective on training-free high-resolution image synthesis.

[5] Scalecrafter: Tuning-free higher-resolution visual generation with diffusion models.

[6] UltraPixel: Advancing Ultra-High-Resolution Image Synthesis to New Peaks.

[7] Meng, Chenlin, et al. "SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations." ICLR.

[8] Zhuo, Le, et al. "Lumina-next: Making lumina-t2x stronger and faster with next-dit." NeurIPS, 2024.

W1. The presentation is at times not optimal.

Thanks for the advice!

• Regarding Lines 93-95, we revisit existing work from the perspectives of "How to guide?" and "How to infer?" because they primarily address the problem from two angles: (i) how to use low-resolution generation results to provide content guidance for high-resolution generation, and (ii) how to enhance the model's generalization ability for high-resolution inputs during the inference phase. Perhaps we could use more straightforward terms to distinguish these two aspects, such as "low-resolution guidance" and "extrapolated-resolution generalization."
• For Line 220, we greatly appreciate the reviewer pointing this out! What we intended to express was equivariance, not invariance. We will correct this oversight in the camera-ready version.

Currently, the presentation of this paper is admittedly less than perfect, and we will try to polish it as comprehensively as possible.

Q1. Provide visual examples of failures.

Good point! We should provide failure cases to qualitatively demonstrate the model's behaviour when approaching the limits of its capabilities. A common failure mode can be understood as the generation of highly grainy images with unexpected high-frequency textures when the model's prior knowledge fails to support extrapolation to higher resolutions. Considering that we cannot include images during the rebuttal phase, we will supplement the corresponding results and discussions in the camera-ready version.

Q2. Why is the projected flow implemented via the classifier-free guidance?

Apologies for the confusion. When we mentioned classifier-free guidance, we did not intend to imply a theoretical connection between projected flow and classifier-free guidance — we simply aimed to convey that projected flow is also implemented as a guidance term. We will revise the paper to clarify this point. As for whether this guidance can be implemented in other forms, we have supplemented some experimental results here to demonstrate the necessity of certain design choices.

• A guidance scale that decreases over time t is not a conventional approach. In the table below, we compare the cosine decay guidance scale (from 1 to 0) with two constant guidance scales (0.5 and 1.0). All three metrics (FID, FID_crop, CLIP Score) indicate the necessity of the cosine decay strategy, as it aligns with the working characteristics of diffusion models — restoring low-frequency structures in the early stages and high-frequency details in the later stages.

  Guidance Strategy	FID	FID_crop	Clip Score
  Cosine Decay	53.75	53.01	28.41
  Constant 0.5	54.98	54.32	28.22
  Constant 1.0	55.67	55.41	27.76

• The projection function we use is also customizable. In the paper, we by default use low-pass filtering based on Discrete Wavelet Transform. In the table below, we report the evaluation results for three types of filters: (1) Discrete Wavelet Transform, (2) Fast Fourier Transform, and (3) low-pass filtering based on downsampling-upsampling. We observe that different filters do not have a significant impact on the performance of I-Max, with DWT showing slightly better overall performance.

  Different Projection	FID	FID_crop	Clip Score
  DWT	53.75	53.01	28.41
  FFT	53.64	53.56	28.37
  Downsample-Upsample	53.98	53.57	28.43

We will include these results and the corresponding discussion in the camera-ready version.

Q3. Technical details about the transition to Eq. (5).

If the reviewer's question is about how Eq. (4) leads to Eq. (5), the theoretical details behind it are actually quite straightforward. Let us consider a simpler setting. When constructing a rectified flow model at a fixed resolution, we have:

$$dX_t = (X_1 - X_0) dt.$$

Similarly, we can express $X_t$ as:

$$X_t = X_0 + dX_t \cdot dt.$$

Then it follows that:

$$dX_t = \frac{X_1 - X_t}{1 - t} dt.$$

However, in this context, $X_t$ represents the intermediate state at time $t$ , while $dX_t$ denotes the increment, which might cause confusion. To prevent ambiguity, we can update the notation to:

$$dX_t = \frac{X_1 - X_\tau}{1 - \tau} dt.$$

Similarly, Eq. (5) can be rewritten as:

$$dP(X_t^\text{extra}) \approx \frac{\hat{X_1}^\text{native} - P(X_\tau^\text{extra})}{1-\tau} dt.$$

We'll modify all the corresponding notations in the camera-ready version.

W1. A single metric limits the objectivity of the results.

Thanks! Relying solely on a single metric indeed weakens the persuasiveness of the paper. Referring to widely accepted works on high-resolution synthesis [1,2], we found that a feasible approach to evaluate the image quality of high-resolution images is to consider both the FID metric for the entire images and for image crops (i.e., FID and FID_crop). Meanwhile, metrics for evaluating text-image consistency, such as CLIP Score, are not affected by resolution and can be applied directly to high-resolution images. Therefore, in this rebuttal, we follow the approach in [1], using 1,000 prompts randomly sampled from LAION-5B as the test set, along with 10,000 randomly sampled images as the reference set, to calculate FID, FID_crop, and CLIP Score. This allows us to comprehensively evaluate both the global and local image quality, as well as the text-image consistency.

In the table below, we report the evaluation results of I-Max at four different resolutions (1024×1024, 2048×2048, 3072×3072, and 4096×4096) when using Flux.1-dev as the foundation model. We can observe that the results were highly consistent with those from the GPT-4o preference evaluation. Both two image quality metrics (FID and FID_crop) show improvement as the resolution increased, achieving optimal performance at 3K resolution, echoing the results in Fig. 6. For the text-image consistency metric, CLIP Score, we observed a very slight decrease as the resolution increased, which is understandable since I-Max guides high-resolution generation using the output at 1024×1024 resolution. Thus, the text-image consistency is not further enhanced. We appreciate the reviewer for encouraging us to include these experimental results, as they further clarify that the gains of resolution extrapolation are primarily in image quality.

In addition, the Black Forest Lab Team claims that Flux.1-dev can directly generate images at 2048×2048, but it has been noticed that the generation quality at 2048×2048 is not particularly good enough. Therefore, we also compared the results of using Flux.1-dev with its default settings against Flux.1-dev enhanced with I-Max at the 2048×2048 resolution. I-Max also achieves a significant margin of improvement, indicating its practical value in real-world applications.

As for the comparison between different low-resolution guidance approaches, we report the results in the table below. All metrics show strong consistency with the GPT-4o preference evaluation in Fig. 7, demonstrating the superiority of the proposed projected flow approach.

Once again, we thank the reviewers for emphasizing the importance of using multiple quantitative metrics for evaluation. We will revise the paper and include these experimental results. Regarding the reviewers' suggestion to evaluate on low-resolution diffusion models and datasets, we acknowledge that this is an aspect often overlooked by current resolution extrapolation work. Due to time constraints, during the rebuttal phase, we mainly focus on supplementing experiments based on the existing base model. However, we will consider including results on other models in the camera-ready version.

W2. Exploring the impact of different low-pass filters.

Good point! In the paper, we by default use low-pass filtering based on Discrete Wavelet Transform. In the table below, we report the evaluation results for three types of filters: (1) Discrete Wavelet Transform, (2) Fast Fourier Transform, and (3) low-pass filtering based on downsampling-upsampling. We observe that different filters do not have a significant impact on the performance of I-Max, with DWT showing slightly better overall performance. We will include these results and the corresponding discussion in the camera-ready version.

[1] Du, Ruoyi, et al. "Demofusion: Democratizing high-resolution image generation with no $$$", CVPR, 2024.

[2] Yang, Zhuoyi, et al. "Inf-dit: Upsampling any-resolution image with memory-efficient diffusion transformer", ECCV, 2024.

Dear Reviewer jzk2,

Considering the discussion period is about to end, we sincerely look forward to your replies. We hope that we have addressed your concerns; if not, we are happy to listen to the follow-up questions. Thanks!

Bests,

Authors of Submission 2180

W1. The relationship with Classifier-free Guidance.

Sorry for the confusion. The guidance term in projected flow does not have a direct theoretical relationship to classifier-free guidance. We agree with the reviewer's point that this is simply a guidance function, and our mention of classifier-free guidance was intended to illustrate that projected flow is essentially a form of guidance. We will modify the statement that "in the form of classifier-free guidance" to "in a similar form of classifier-free guidance" to eliminate misunderstanding.

W2. How is term v_{theta} obtained?

Yes, Term v_{theta} is derived directly from inference using the model trained on native resolution applied to extrapolated resolution. To ensure the model achieves optimal generalization capability for extrapolated resolution, we incorporated several inference techniques, including scaled RoPE, time shifting, proportional attention, and text duplication, as illustrated in Fig. 4.

W3. Summarize the inference process in the form of an algorithm.

Thanks for the suggestion! In the revised version, we will present the complete inference process in the form of an algorithm to facilitate readers' understanding.

W4. Questions about evaluation.

Well said! Since additional images cannot be included during the rebuttal phase, we will supplement more qualitative results in the camera-ready version as per the reviewers' suggestions. Here, to further support the conclusions drawn from the GPT-4o preference evaluation, we supplement the evaluation results under the metrics of FID, FID_crop, and CLIP Score.

In the table below, we report the evaluation results of Flux.1-dev + I-Max at different resolutions. We can observe that the results were highly consistent with those from the GPT-4o preference evaluation. Both two image quality metrics (FID and FID_crop) show improvement as the resolution increased, achieving optimal performance at 3K resolution, echoing the results in Fig. 6. For the text-image consistency metric, CLIP Score, we observed a very slight decrease as the resolution increased, which is understandable since I-Max guides high-resolution generation using the output at 1K resolution.

As for the comparison between different low-resolution guidance approaches, we report the results in the table below. All metrics show strong consistency with the GPT-4o preference evaluation in Fig. 7, demonstrating the superiority of the proposed projected flow approach. It is worth noting that in Fig. 7, we present the win rate of projected flow compared to the baseline methods, where higher values indicate a more significant advantage of projected flow.

Besides, we will also provide detailed analysis about the trade-off of inference costs and generation quality, please refer to our response to the Reviewer eZ4Q.

W5. Typo.

Thanks! We will correct the typo and carefully check other parts of the paper.

W6. Some of the ideas incorporated are based on existing works.

Good point! Some of the ideas in this paper were indeed proposed in previous literature, and we have carefully cited them. What we have done in I-Max is to revisit existing resolution extrapolation methods and, specifically for rectified flow transformers, integrate useful ideas (e.g., scaled RoPE and time-shifting) while proposing new techniques (e.g., projected flow and text duplication) to achieve performance that is reliable for practical applications. Without emphasizing SOTA performance, the newly introduced techniques are undoubtedly our contributions. Additionally, our analysis and validation of existing ideas on advanced base models also provide meaningful insights for the field, e.g., as demonstrated in Fig. 4 and Fig. 12, these techniques exhibit different behaviours across various model architectures (DiT vs MMDiT).

We will further clarify our contributions in the revision.

Q1. Why not use Lumina-Next?

We use Lumina-Next-2K to validate that I-Max can be applied to: (i) different foundation models, (ii) different model architectures (MMDiT and standard DiT), and (iii) models with varying native resolutions (hence the fine-tuning of the model). In fact, a 2K fine-tuned SFT model allows us to generate 8K images through the same scale factor, demonstrating that the resolution extrapolation capability enabled by I-Max does not diminish as the native resolution increases. This makes it more valuable compared to models with lower native resolutions.