Model Collapse Analysis and Improvement for Rectified Flow Models

Thank you very much for acknowledging the theoretical and methodological contributions of our paper. We apologize for the oversight in the rushed writing and submission of the experimental section. In the revised version, we have supplemented all the missing content, making the experimental section—especially the image experiments—clearer and more complete. We have also adjusted some of the appendices and the order of the main text to improve readability.

We would like to address your specific questions and concerns:

1. The connection between the theoretical analysis and the proposed method is unclear.

   We have reformulated the relevant proof in Appendix A.3 and summarized it as Proposition 2 in the main text. Under mild conditions, we demonstrate that incorporating real data provides a theoretical lower bound on the maximum eigenvalue of the Denoising Autoencoder (DAE) trained in each iteration. This theoretical insight explains how integrating real data into the training process helps to break the bound established in Theorem 1, thus improving the stability and performance of the model.

2. The mixing scheme in Equation 11 is questionable.

   We appreciate your careful observation and apologize for the confusion caused by a typographical error in the original manuscript. In the revised version, see Equation 12, we clarify that we actually train using a proportional mixture of noise-image pairs generated in the forward process and image-noise pairs generated by the reverse flow. Specifically, we randomly sample and combine these pairs during training, as detailed in Algorithm 1. This approach avoids disrupting the target distribution, as we do not perform convex interpolation between independent synthetic and real data. Instead, we maintain the integrity of the data distributions while balancing the contributions of synthetic and real data.

3. The FID on CIFAR from Figure 1 contradicts with the FID in Figure 5.

   We apologize for the inconsistency. As noted in Appendix C.3, Table 3, we provide detailed parameter descriptions for our experiments. It is important to note that the figure on the first page was intended as a schematic illustration, and we did not perform standard parameter Reflow training for that figure. To prevent misunderstanding, we have removed the FID values from that figure in the revised manuscript.

4. Performance differences with and without RCA.

   Based on Figure 5 and Table 2, we can clearly see that the Reflow-RCA method improves the performance of Reflow by mitigating model collapse. While we acknowledge that completely eliminating model collapse is very challenging, our method significantly enhances the generation quality during Reflow training. This leads to a better trade-off between generation efficiency and quality, effectively pushing the Pareto frontier forward. Notably, our OCRA and OCRA-S methods outperform all existing Reflow methods on real datasets, demonstrating the effectiveness of our approach.

   Regarding your concern about the benefits of more than 1–3 Reflow iterations, our experiments show that while vanilla Reflow suffers from severe model collapse with increased iterations, our RCA method substantially reduces this effect. This allows for more iterations without significant degradation in performance, meaning that the model continues to benefit from additional Reflow iterations when using RCA, leading to improved results over the baseline.

Furthermore, our method is compatible with many techniques aimed at improving the quality of generative models. For example, we can incorporate the LPIPS loss used in [1], the Pseudo-Huber loss from [2], or the specialized probability flow design in [3]. Additionally, our approach can be applied to existing simulation-free generative models with higher generation quality, such as [3], to achieve even greater generation efficiency.

References:

[1] Consistency Models, ICML'23.

[2] Improved Techniques for Training Consistency Models, ICLR'24

[3] Scaling rectified flow transformers for high-resolution image synthesis, ICML'24

Dear Reviewer,

Thank you for your response. Due to space limitations, we did not provide detailed descriptions of our parameter settings in the main text, causing you inconvenience, and we apologize for that.

Firstly, the main purpose of our experiments is to verify the correctness of our theory in both Gaussian and image experiments and to demonstrate that RCA and OCRA can effectively alleviate the model collapse phenomenon during Reflow in Rectified Flow. We did not claim any state-of-the-art (SOTA) results. It is especially worth noting that our method is compatible with most training techniques that can improve model performance, such as the LPIPS loss mentioned in our previous response.

Secondly, in all our self-trained comparative experiments, we ensured fair parameter settings. Unless new parameters were introduced, we maintained the parameters stated in the Flow Matching paper [1], [2] and ensured parameter consistency across the same experiments. Based on this, we believe our experiments are fair. We will provide specific explanations below.

As shown in Figure 5(a), our RCA provides improvements in generation quality regardless of the number of sampling steps. Since we needed to perform 8 Reflow iterations to plot Figure 5—to showcase the model collapse phenomenon in Reflow and the improvements of our method—we had to adjust the parameters (e.g., using a half-scale U-Net, larger λ, and larger α) to reduce training resource consumption due to limited computational resources. We believe this is the reason why the effect in Figure 5(a) is not as pronounced at 1–2 steps, which is also reflected in our parameter ablation experiments (see Appendix C.4). Meanwhile, we provide very detailed parameter settings and the reasons for using these parameters in Appendix C.3. For better parameter tuning to demonstrate the advantages of RCA and OCRA, please refer to the experiments in Table 2, which better showcase our quantifiable advantages.

In the quantitative experiments in Table 2, we used a full-scale U-Net (with parameters consistent with [2]) and a more optimized α, as shown in Appendix Table 3. It is worth mentioning that we used 500k noise-image pairs because, according to the authors' blog (see our revision at line 1137 and footnote on page 22: https://zhuanlan.zhihu.com/p/603740431 ), in their response on 2023/02/07, they mentioned:

"On CIFAR-10, we probably need to generate at least 500,000 pairs of (noise, image) to train 2-Rectified Flow with FID < 5; to achieve the results in the article, we generated 4 million pairs of (noise, image)." (Translated by Google)

Since the comparative experiments for Reflow and Reflow-RCA are fair, we used 500k pairs to save computational resources. The training of 10-Reflow will take more than 300 hours using 6 GPUs.

Furthermore, in all our experiments on CIFAR-10, we used the codebase https://github.com/atong01/conditional-flow-matching, which is provided by [1]. This is because Flow Matching [2] does not provide publicly available source code, and this codebase integrates most of the well-known PyTorch implementations of Flow Matching and Rectified Flow. According to our observations, the neural network used in the [1] codebase has some minor differences from the Rectified Flow provided source code. The Rectified Flow source code utilizes PyTorch/TensorFlow/JAX code styles and uses NCSN inherited from [3], while the network in [1] is based on [4]. Based on this, we retrained 0-Rectified Flow, and Table 2 shows our results. To avoid misunderstanding, we also provide the values reported by the authors in the "Best NFE" column of the table.

Finally, we believe that although Rectified Flow does not utilize many advanced training and generation techniques—as partly reported by [5]—it may not match the most advanced distilled diffusion models, such as Consistency Models, in the 1–2 step setting. However, improvement techniques based on Rectified Flow remain meaningful because straighter flows bring significant benefits, such as ease of distillation [6]. We believe our method provides new directions for advancement.

Moreover, we consider that multiple Reflow iterations have significant mathematical significance, as they offer a theoretically rigorous approach to approximate optimal transport [7]. This effectively demonstrates the combination of theory and practice. Our focus should be on optimizing the methods, such as exploring better ways to avoid model collapse in future work.

We will include a more comprehensive Table 2 in the appendix of the revision, which will contain 1–2 step sampling results to better reflect our advantages. Additionally, we will add a more detailed description of the experimental section in the revision to eliminate any misunderstandings.

Once again, thank you for your careful reading. Your dedication plays a crucial role in driving the continuous development of our community.

Thank you sincerely for acknowledging the theoretical and methodological contributions of our paper. We apologize for the oversight in the experimental section caused by our rushed writing and submission process. In an effort to meet the 10-page limit, we mistakenly omitted some experimental tables and descriptions. In the revised version, we have restored all missing content, making the experimental section—especially the image experiments—more comprehensive and clearer. We have also adjusted the order of some appendices and the main text to improve coherence.

We would like to clarify an important point regarding the model collapse phenomenon studied in our paper, which differs from mode collapse. Mode collapse primarily occurs in GANs and refers to the generative model producing only a few modes or samples from the data distribution while neglecting others. In contrast, model collapse refers to the progressive degradation in performance of generative models trained iteratively on their own outputs. In our revision, we have included images generated by models trained with k-reflow that illustrate this gradual deterioration, helping to better understand model collapse. We apologize for any confusion this may have caused.

Weaknesses 1: Presentation

Regarding the areas that needed improvement:

1. Restored Table 2: Located at line 512 in the main text.

2. Redrawn Figure 5: We have updated the figure to present clearer visualization curves and included additional generated results to enhance understanding.

3. Precision/Recall Measurements: We greatly appreciate your suggestion to test recall. We conducted precision and recall evaluations, and the results are provided in Appendix C.4. Our findings indicate that iterative Reflow training reduces the recall of generated images, implying decreased diversity. Our RCA-Reflow method mitigates this decline effectively.

4. Sample Images on CIFAR-10: Provided in Figure 5(b) to illustrate our results more concretely.

5. Ablation Study on RCA Parameters: We included an ablation study of the RCA method's parameters, with results presented in Appendix C.4.

We sincerely appreciate your acknowledgment of the theoretical and methodological contributions of our paper. We apologize for the oversight in the experimental section, which was due to the rushed nature of our writing and submission process.

Weaknesses 1: Paper Writing

We sincerely apologize for the rushed submission, which led to issues such as unresolved hyperlinks. These have all been corrected in the revised version. The specific updates you mentioned have been addressed as follows:

1. Added content and clarifications:
   • Line 310: Appendix C1 and C2
   • Line 340: Figure 2
   • Line 372: Algorithm 1
   • Line 417: Appendix B.2
   • Line 514: Table 2

Weaknesses 2: Extra Quantitative Numbers and Qualitative Results

1. Regarding the CIFAR-10 and CelebA-HQ256 datasets, we regret that some content was mistakenly commented out during our adjustments to meet the page limit. We sincerely apologize for this oversight.

   • In the revision, we have restored the table (Table 2) and have also re-plotted the model collapse curves for CIFAR-10 (Figure 5 ) to provide a clearer visual comparison.
   • Additionally, we included sample images illustrating the model collapse process and the effectiveness of our RCA method in mitigating this issue. These additions provide both quantitative and qualitative evidence to support our claims.

Weaknesses 3: High-Resolution Images

We acknowledge the concerns regarding high-resolution image generation and address them as follows:

1. Our method is compatible with commonly used diffusion-based high-resolution image generation techniques, such as latent space modeling. This is evident in our experiments with CelebA-HQ 256.

2. While we agree that additional experiments with higher resolutions could enhance our results, we note that the complexity of our current experiments already surpasses most model collapse studies. For example:

   • [1] and [2] use datasets with resolutions of only 32 or 64 pixels.
   • Similarly, works in the generative modeling field, such as [3], report experiments with datasets capped at 256 pixels.

3. Regarding 10-RF CIFAR-10 experiments (Figure 5), we used a reduced channel size (num of channels = 128, which is half of the recommended size in Flow Matching [5]) to expedite training (details in Appendix C3). Despite these optimizations, training required over 7 days to complete 10 rounds of Reflow.

4. Due to our limited computational resources, we were unable to extend our experiments to resolutions like 1024×1024. However, our results with latent space techniques indicate that our approach is well-suited to enhancing high-resolution image generation performance in Rectified Flow [4].

Question 1

As detailed in Section 5.1 and Algorithm 1:

1. $x^{(i)}$ refers to a randomly sampled image from the training dataset, used to generate a matching noise vector $\hat{z}^{(i)}$.
2. $\hat{x}^{(i)}$ represents the generated image obtained by sampling with the random noise vector $z^{(i)}$.

References:

[1] On the Stability of Iterative Retraining of Generative Models on their own Data, ICLR'24.

[2] Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data, COLT'24.

[3] Consistency Models, ICML'23.

[4] Scaling Rectified Flow Transformers for High-Resolution Image Synthesis, ICML'24.

[5] Flow Matching for Generative Modeling, ICLR'23.