CatVTON: Concatenation Is All You Need for Virtual Try-On with Diffusion Models

Post-discussion period

I will maintain my current score of 5, as the issues I raised in my review remain unresolved:

• Discrepancies in Quantitative Results: There are significant discrepancies between the reported results in the submission and the original baseline papers. This inconsistency makes the comparison less convincing. To address this, I suggested that the authors use the original evaluation scripts from the baseline papers alongside their own settings, but it is disregarded. This would provide a fairer comparison, particularly given the considerable variance observed in metrics like LPIPS, FID, and KID.

• Architectural and Textual Inconsistencies: There is a minor but important inconsistency between the overall architecture figure and the description in the main text. Specifically, the process of input concatenation, whether in pixel-space or latent-space, is unclear. This discrepancy raises questions about whether the results are indeed identical to what was claimed.

For me, if there are unclear aspects in a submission and the authors disregard suggestions such as using original scripts for reevaluation, I cannot raise the score or recommend acceptance unless these concerns are fully addressed. The final decision I would let the effort of ACs.

This work introduces CatVTON, a diffusion-based virtual try-on model that enhances efficiency by removing complex modules and preprocessing steps typical in other approaches. Diffusion-based virtual try-on methods usually require additional encoding, extensive training, and preprocessing, which CatVTON addresses by simplifying the architecture to a VAE and denoising UNet, reducing parameters to 899.06M. Through analysis, the authors find that self-attention modules are essential for adapting diffusion models to this task, enabling high-quality outputs with only 49.57M training parameters. CatVTON further reduces inference complexity, eliminating pre-processings like pose estimation and human parsing, which cuts memory usage by over 49%. Extensive testing demonstrates CatVTON’s strong performance and generalization, even with a limited 73K sample dataset, achieving better results than baseline methods in both controlled and real-world scenarios. Experiments are conducted on two datasets: VITON-HD and DressCode.

This paper aims to efficiently adapt pre-trained diffusion models to high-quality virtual try-on. It introduces efficiency improvements in three key areas: model architecture, parameter-efficient training, and simplified inference, and analyzes the motivation and feasibility from the perspective of image representation interaction, supported by experimental validation. Extensive qualitative and quantitative experiments are conducted on VITON-HD and DressCode datasets, along with ablation studies on key components of proposed method. Compared to other virtual try-on methods, this method demonstrates significant advantages in model parameters, training efficiency, and inference costs while achieving better performance. Despite being trained on small-scale public datasets, results and comparisons in in-the-wild scenarios demonstrate the method's generalization capabilities.

CatVTON is a virtual try-on model that achieves high-quality and efficient garment transfer without relying on additional encoders or complex preprocessing. Unlike traditional virtual try-on approaches, CatVTON simplifies the architecture by using only a VAE and a single, lightweight UNet with a self-attention module, reducing the parameter count to 899.06M. Moreover, this model directly concatenates the person and garment images along spatial dimensions, requiring only 49.57M trainable parameters and cutting down inference memory usage by over 49% compared to other methods. Extensive experiments demonstrate that CatVTON achieves superior results in both controlled and in-the-wild settings.

The study re-evaluates the necessity of additional modules in virtual try-on methods based on diffusion models and proposes CatVTON, a simple and efficient model. CatVTON transfers garments of arbitrary categories to target individuals by concatenating them as inputs, removing redundant encoding modules and steps. It features a lightweight network with just 899.06M parameters, efficient training with only 49.57M parameters through self-attention modules, and simplified inference that requires only person image and garment reference, reducing memory usage by 49%+. Experiments show that CatVTON achieves superior results and generalizes well in real-world scenarios, despite being trained on a limited public dataset.

In this paper, the authors propose CatVTON, a simple and efficient virtual try-on diffusion model using a VAE and denoising UNet with spatial concatenation, reducing memory usage and training parameters. The paper demonstrated competitive performance on standard benchmarks and in real-world scenarios, with strong generalization and efficient inference. However, the reviewers raised concerns about reproducibility due to discrepancies in evaluation metrics compared to prior works, insufficient explanation of the concatenation's advantages, and a lack of detailed theoretical validation.

After the author-reviewer discussion period, most of the reviewers confirmed their concerns had been addressed. However, Reviewer iWwH still has two important remaining concerns:

• Discrepancies in quantitative results: there are significant discrepancies between the reported results in the submission and the original baseline papers.
• Architectural and textual inconsistencies: there is a minor but important inconsistency between the overall architecture figure and the description in the main text.

Considering most of the reviewers are positive towards this submission, and Reviewer iWwH is not strongly against accepting it, I recommend acceptance. However, the discussions between the reviewers and authors, especially the concerns from Reviewer iWwH, should be included in the final version.

Accept (Poster)