Vision Foundation Models as Effective Visual Tokenizers for Autoregressive Generation

Dear reviewer APhE,

Thank you for your time and efforts in helping us improve our paper! We hereby answer your questions and resolve your concerns as follows:

W1: The use of VFM for tokenizers is not novel. LightningDiT, VILA-U, VQGAN-LC, SoftVQ-VAE have explored similar directions.

Thank you for bringing these works to our attention. Despite using VFM for VAE/VQVAEs, we note clear distinctions from these works. We would like to clarify that our motivation is to show that frozen pre-trained VFM features alone are sufficient to build an image tokenizer. To achieve this, we leverage multi-level features from the VFM, and reduce redundancy via the region-adaptive quantization strategy. In contrast:

• Approaches like LightningDiT and SoftVQ-VAE use a pre-trained VFM only as a feature distillation target for optimizing its tokenizer.
• VQGAN-LC investigates initializing its codebook by heuristically clustering CLIP patch features, and its reconstruction and generation quality is substantially lower than VFMTok's.
• VILA-U initializes its tokenizer's encoder with a pre-trained VFM, whose parameters are explicitly updated during training.

In conclusion, all of them cannot build a tokenizer with frozen VFMs solely. We are sorry for the confusion will incorporate the discussion into the manuscript for a clearer relation with related works. Besides, we provide a more detailed discussion on the primary purpose of the VFM in our response to Reviewer vMYn's first question (vMYn-W1). We kindly ask you to refer to that for more information.

W2.1: Lack of comparison with PAR and RandAR.

Per your request, we integrated VFMTok into RandAR and also compared it with that of PAR. As shown in the table below, VFMTok achieves better image reconstruction and generation performance. We will incorporate the comparison to the manuscript for a more complete comparison with recent works.

W2.2: VFMTok does not outperform non-VFM-based tokenizers; linear probing accuracy is lower than original VFM features.

In Table 2 of the manuscript, the image reconstruction quality of VFMTok is inferior to that of ImageFolder [26]. This is because the experiment setting is aligned with LlamaGen [43] baseline for a fair comparison. Once the codebook size and the number of tokens are aligned with those of ImageFolder [26], VFMToks's image reconstruction and linear probing accuracy surpass ImageFolder's, while its generation quality is comparable. The results aligned with ImageFolder are shown below.

Besides, there is a concern that VFMToks's linear probing accuracy is lower than that of the original VFM. This occurs because the quantization introduces information loss, consequently compromising the semantic fidelity of the tokens.

W3: Deformable sampling can cause slower inference; lacking experimental comparison of inference speed.

• Please kindly note that deformable sampling is adopted only in the tokenizer's training stage. During inference (image generation), only the decoder of the tokenizer is used.
• The question might be whether deformable sampling makes VFMTok slower to train. Thanks to the use of frozen VFM features, these can be cached, and the trainable module in the encoder is only a lightweight deformable sampler. Together, the training of our tokenizer is also substantially faster.
• Below, we provide a complete comparison of the compute and throughput of different stages of the tokenizer across multiple resolutions. Both training and inference are efficient, especially as resolution increases.

W4: Generating images at 384×384 resolution and then resizing to 256×256 may lead to unfair comparisons.

• This follows a common setting proposed in LlamaGen[43] and PAR, where comparisons are often conducted by generating images at $384\times384$ resolution and subsequently resizing them to $256\times256$ for evaluation. This paradigm requires 576 image tokens to represent an image.
• Instead, VFMTok generates at a $336\times336$ resolution and resizes to $256\times256$ for evaluation, representing each image with just 256 tokens and thereby accelerating the AR generation speed.
• For fairness, we also report the $256\times256$ image generation performance in Appendix Table A1. Please kindly take a look for reference.

1. Comparison of fairness. VFMTok's novelty is in its tokenizer design, an approach orthogonal to advancements in AR generative frameworks. Thus, we used an identical generative framework (LlamaGen[43]) for all tokenizers for a fair comparison. Moreover, this decision ensure a fair comparison by strictly constraining the image token count to 256, maintaining the computational overhead consistent in the AR generation time. As shown in Table A1 in the appendix, the performance of RandAR in the rebuttal, and the table.(a) below, under the same computational budget, VFMTok improves the generation performance of LlamaGen, MaskGiT, RandAR, respectively.

(a). Incoperating different tokenizer with MaskGiT for image generation with CFG-free.

2. Comparison with SoTA AR generative frameworks: For fairness, we also integrate VFMTok with advanced AR generative frameworks, RandAR and RAR, in terms of the image resolution of $256\times256$. Now we are working on the results of RandAR and RAR, and will report them later in a separate thread. As depicted in the table below, VFMTok achieved SOTA results compared to the previous excellent works (including diffusion models and AR generative models) in both image generation with and without CFG. Again, VFMTok primarily focused on tokenizer design for AR generation instead of AR model architectural advancements. Besides, experiments illustrated improvements on different generative frameworks under the same computational overhead. We would like to learn from the reviewer specific superior tokenizers or systems to compare with. | Approach | #Res| #tok. |params.| gFID($\downarrow$, w/ cfg) | gIS($\uparrow$, w/ cfg)|gFID($\downarrow$, w/o cfg) | gIS($\uparrow$, w/o cfg)| | :--- |:---:| :---: | :---: | :---: | :---: |:---:|:---:| |DiT-XL/2|${256\times 256}$|--|675M|2.27|278.2|9.60|--| |SiT-XL/2|${256\times 256}$|--|675M|2.06|270.3|8.30|--| |SiT-XL/2+REPA|${256\times 256}$|--|675M|1.80|284.0|5.90|--| |ImageFolder|${256\times 256}$|${286\times2}$|362M|2.60|295.0|--|--| |RandAR-L+VQGAN[43]|${256\times 256}$|256|343M|2.55|288.8|11.59|103.9| |RandAR-L+VFMTok|${256\times 256}$|256|343M|2.19|301.1|2.87|216.1| |RAR-L+VQGAN|${256\times 256}$|256|461M|1.70|299.5|6.72|129.2| |RAR-L+VFMTok|${256\times 256}$|256|461M|1.47|316.8|2.14|213.1|

3. Evaluation on semantics: We would like to clarify that the goal is to transform VFMs into good tokenizers (for generation), rather than understanding ability alone. For image generation, the semantics enable faster convergence and notably better CFG-free image synthesis. For understanding, in our rebuttal, we demonstrated a clear superiority over ImageFolder[26] (74.9% vs. 58.1% top-1 accuracy), the other one tokenizer that incorporates DINOv2 with advanced quantization technique. This consolidates our tokenizer preserves the semantics in VFMs better. We will be glad to hear from the reviewer for suggestions on further comparisons.

4. Novelty of the method: VFMTok is the first to thoroughly validate that features from a single mainstream pre-trained VFMs alone, spanning both self-supervised and language-supervised paradigms, are effective for image reconstruction and generation. Leveraging the rich semantics of frozen pre-trained VFM features and novel quantization technique, VFMTok enables AR models to converge faster and achieve high-fidelity, CFG-free image synthesis without complex heuristic designs (Tab. 3 in the manuscript).

Following your suggestion, we have integrated VFMTok with current advanced AR generative frameworks following a practical setup. Both VFMTok and AR models are trained and evaluated with image size of ${256\times256}$. As presented in the above table, VFMTok achieved SOTA results compared to previous excellent works (both diffusion and AR generative models).

Dear reviewer SUoJ,

Thank you for reviewing our paper. Your concerns are answered as follows.

W1: The paper lacks novelty, as using VFMs as image tokenizers to improve VQGAN has already been explored in prior work (SemHiTok and TokenFlow).

• Our motivation is to show that frozen VFM features alone are sufficient to build an image tokenizer.
• Instead, both SemHiTok and TokenFlow adopt a dual-encoder approach, having one VQGAN branch for pixel-level features and a VFM branch for high-level semantics. VFMTok leverages the multi-level features from a single-branch VFM to capture both low-level and high-level features that benefit image reconstruction and generation.
• Moreover, we also designed a light-weight region-adaptive quantization strategy to reduce redundancy in irregular image regions.
• Besides, we provide a more detailed discussion on the primary purpose of the VFM in our response to Reviewer vMYn's first question (vMYn-W1). We kindly ask you to refer to that for more information.
• Together, VFMTok sets apart prior works in both motivation and model architecture.
• Thank you for bringing these works to our attention, and we will add a citation for discussion.

W2: Why does the model achieve better performance without classifier-free guidance than those that incorporate it?

• Our intuition is that introducing semantic information renders the latent tokens semantically aware and helps stabilize the sampling process of the autoregressive (AR) model, thereby reducing the reliance on classifier-free guidance (CFG).
• As shown in our results, the performance in both settings is comparable, with the CFG-free setup achieving slightly better gFID. However, we also observe that incorporating CFG slightly improves the gIS, indicating enhanced diversity. This reflects the typical trade-off between fidelity (gFID) and diversity (gIS) commonly seen in CFG-based approaches.
• We will clarify these observations and the underlying trade-offs in the paper.

W3.1: Whether the region-adaptive token generation module adaptively sample regions of similar patterns?

• Within the region-adaptive generation module, each region contains multiple sampling points. We utilize these points to sample from the last spatial features of the original DINOv2, creating a feature representation for each point.
• These features are then aggregated via average pooling to form the representation for the entire region. The average similarity between each intra-region sampling point and the region feature is then computed. Additionally, we randomly sample points from outside the region, extract their features, and compute their similarity to the same region feature.
• Below, $S_{in}$ and $S_{in}$ denote the sampling points within and outside a sampling region, respectively.

As presented in the table above, it indicates a significant similarity among points within the region, whereas points from outside the region exhibit comparatively low similarity, which consolidates our hypothesis.

W3.2: Which aspects of the multi-level features are most beneficial for reconstruction?

According to the suggested setup, we begin by conducting an ablation study to evaluate the impact of each layer on image reconstruction and generation. Subsequently, we cumulatively add each feature layer to observe the combined effect.

The results, as presented in the table, show that while a single feature layer offers no significant advantage, the quality of both image reconstruction and generation markedly improves as more layers are incorporated.

All tokenizers and AR generation models (VFMTok-L) are trained for 50 and 100 epochs, respectively. As shown in the tables, our results show that while individual layers provide little benefit, cumulatively adding more layers significantly improves both image reconstruction and generation quality. $F_i$ represents the indexed feature level.

(a) Single-level feature investigation

(b) Analysis of cumulatively adding multi-level features

W4: Has the author considered or experimented with MaskGIT decoding strategy for this work?

We added an experiment that integrates VFMTok into MaskGiT [4]. For fairness, the images are generated without Classifier-Free Guidance (CFG).

Could you kindly clarify which specific aspects of our approach or the related works led to the observation that VFMTok's novelty may be somewhat limited? It would enable us to better address your concerns more effectively in our response.

1. Specifically, the novelty of VFMTok can be summarized below:

• VFMTok is the first to throughly validated that features from mainstream pre-trained VFMs alone, spanning both self-supervised and language-supervised paradigms, are effective for image reconstruction and generation.
• To address the inherent redundancies in VFM's features, VFMTok introduces a novel region-adaptive quantization to achieve compact tokenization. This design substantially reduces the number of visual tokens while simultaneously enhancing performance, facilitating efficient AR generation without sacrificing quality (Tab. 1 in the manuscript).
• Leveraging the rich semantics of frozen pre-trained VFM features and novel quantization techique, VFMTok enables AR models to converge faster and achieve high-fidelity, CFG-free image synthesis without complex heuristic designs (Tab. 3 in the manuscript).

2. Empirically, none of prior works showed simply using VFM features can achieve superior image reconstruction and generation performance. A key example is SemHiTok (similar to TokenFlow in motivation, methodology), which finds that VFM tokens lack pixel-level detail and thus requires a separate pixel branch to compensate for this deficiency. VFMTok, in contrast, circumvents this issue, achieving excellent image reconstruction and generation quality.

Dear reviewer vMYn,

Thank you for your thorough review of our paper and detailed suggestions for improvement. Your questions and concerns are answered below:

W1: Using VFMs as part of tokenization pipeline is not a new idea. A dedicated related work section is needed.

Thanks for your suggestion. We note that our motivation is to show that frozen VFM features alone are sufficient to build an image tokenizer, while all previous works use VFM for other purposes or in other ways. To make a clear comparison, we provide a table on the purpose of using VFMs below:

Table (a). The primary purpose of VFM.

Table (b). Performance comparsion between VQGAN-LC[58] and VFMTok.

• As shown in the Table.(a), concerning the purpose of VFMs, prior methods have primarily used VFM for: 1) encoder initialization (e.g., EMU[C], VILA-U, SemHiTok, TokenFlow); 2) feature regularization (e.g., REPA[A], Lighting-DiT, SoftVQ-VAE, Flextok[D]); 3) conditional feature generation for diffusion models (e.g., EMU, SEED[B]); and 4) codebook initialization (e.g., VQGAN-LC).
• Moreover, approaches that utilize a VFM to initialize the tokenizer's encoder (such as EMU, VILA-U, and TokenFlow) all explicitly update the VFM's parameters during training.
• Empirically, the most related work is VQGAN-LC[58], which investigates initializing its codebook by heuristically clustering CLIP patch features. Similarly, its parameters in VFM also remain frozen during the training phase. Nevertheless, its performance in image reconstruction and generation is substantially lower than that of VFMTok. As indicated in the Table.(b) above.
• In conclusion, all of them cannot build a tokenizer with frozen VFMs solely. We will incorporate the discussion into the manuscript for a clearer relation with related works.

W2: REPA showed improved convergence by aligning to DINOv2. FlexTok used REPA to improve discrete image tokenizers. A comparison is needed.

Comparison with REPA. Here we convert VFMTok into a continuous VAE (VFMVAE) and integrate it into SiT/XL. Image reconstruction and generation performance are presented below. For simplicity and fairness, images are generated without classifier-free guidance(CFG).

Comparison with FlexTok. VFMTok achieves better reconstruction and generation quality with approximately half of the parameters in the AR generation model.

W3: Why do we think VFMs trained with masked prediction objective do better than CLIP? What about MAE then?

• In this setup, the encoder of VFMTok is substituted with MAE-L. Training setup: VFMTok (50 epochs), AR model, VFMTok-L (100 epochs). As shown in the table below, while the generation and reconstruction quality of VFMTok (MAE) does not match that of VFMTok (CLIP), it demonstrates a significant improvement over the baseline. Moreover, its reconstruction performance is better than that of VQGAN (CLIP).
• This suggests that VFMTok, integrating pre-trained models with a masked prediction objective, is advantageous for image reconstruction. Nevertheless, due to factors such as the VFM's model size, the volume of training data, and time constraints, VFMTok (MAE) is not yet comparable to VFMTok (CLIP).
• We will investigate this further and provide a more complete discussion in the paper.

W4.1: Total training FLOPs should be reported, as currently you operate at a different resolution.

• We presented the required training computation and throughput for VFMTok below.
• Thanks to the use of frozen VFM features, these can be cached, and the trainable module in the encoder is only a lightweight deformable sampler. Together, the training of our tokenizer is also substantially faster.
• Besides, we have also included 256x256 generation experiments in Appendix Table A1 for reference.

W4.2: TiTok reimplementation results require further clarification.

We used the official TiTok weights, but integrated with an AR model (LlamaGen) for fair comparison with other methods instead of the original MaskGiT. We’ll revise the text for clarity.

W5.1: Whether other VFMs show similar trends?

Other VFM, such as VFMTok (SigLIP), also show a similar trend, which is presented in Table A6 in the Appendix.

W5.2: What about w/o region-adaptive tokenization, and w/ & w/o reconstructing the features?

Below, we show the image reconstruction performance under the suggested settings, where the first two rows do not use region-adaptive tokenization. We train VFMTok for 50 epochs, while the AR models are trained for 100 epochs.

W5.3: If running the ablations with 50 epoch training lead to stable results?

We list the AR generation quality at both 50 and 100 epochs in the table below. The trend is consistent.

W5.4: Why linear probing accuracy of the randomly initialized encoder is much better than vanilla VQGAN?

• To analyze the cause of VQGAN [43]'s low linear probing accuracy, we begin by substituting the PatchGAN in VQGAN with DinoGAN. Subsequently, we incorporate the objective of feature reconstruction.
• The table below shows that adding DinoGAN, with its semantic discriminability, remarkably improves VQGAN's accuracy. VFM feature reconstruction further boosts this accuracy.
• We can therefore conclude that a semantically-informed loss function enhances the linear probing accuracy of VQGAN. |Tokenizer|top-1($\uparrow$,%)| |:---|:---:| |VQGAN[43]|15.0| |VQGAN[43]+DinoGAN|40.5| |+Semantic recon.|58.3|

W6: What would be the performance on other generative benchmarks such as COCO and GenEval?

Time constraints prevented us from completing the text-to-image generation task for evaluation on COCO and GenEval. We will continue to work on this and integrate VFMTok with a text-to-image AR generation model to conduct these evaluations in the future. Thanks for your suggestion!

W7: What are the limitations of the method? What are possible future directions to investigate?

The limitations section is presented in the Appendix (page 4), and we will elaborate more on this part in the next revision.

1. The table for W5.2 need be considered along with the table in W5.3. From these, we can summarize the following takeaways::

• More tokens in a tokenizer to represent an image can achieve a better reconstruction performance.
• Introducing stronger semantics improves generation quality.
• VFMTok, equipped with region-adaptive quantization, can better represent an image with fewer tokens, achieving faster convergence speed with a lesser computational budget, but obtaining better generation quality.

2. Those takeaways can be specified below:

• With the region-adaptive quantization removed from VFMTok, it degraded into a vanilla VQGAN, which utilizes 576 tokens to represent an image. As the token count representing an image increases, the reconstruction quality improves accordingly. However, the rFID of the tokenizer with VFM reconstruction is slightly worse than that without it. We hypothesize that VFM reconstruction loss could adversely affect pixel-level reconstruction during training in this setup.

• As shown in the table for W5.3, and as depicted by the gFID (an important metric that estimates image generation quality where a lower value indicated better performance), introducing stronger semantics significantly accelerates the convergence of the AR generative models and improves their generation quality.

• In contrast, VFMTok, which adopts only 256 tokens (Note: there is a mistaken writing in the first table for W5.2. its token count should align with that in Table 2 of the manuscript.) but is equipped with the region-adaptive quantization, significantly accelerates the training convergence speed and inference speed. It also reduces computational cost and improves generation quality compared to the other 2 tokenizers containing 576 tokens. This consolidates that the region-adaptive quantization is crucial to VFMTok's efficacy. |AR model|#tok.$\downarrow$|rFID$\downarrow$|rIS$\uparrow$|#epochs|gFID$\downarrow$|gIS$\uparrow$| |:---|:---:|:---:|:---:|:---:|:---:|:---:| |w/o VFM recon|576|0.63|207.3|100|3.43|273.5| |w/ VFM recon|576|0.71|212.8|100|3.20|274.0| |VFM Tok|256|0.89|215.4|100|3.09|274.2|

Thank you for your acknowledgement of our work, and we would like to provide more insights on the question "What makes a VFM a good visual tokenizer?" (for image reconstruction, generation, and possibly understanding).

The main takeaways are:

1. MIM (masked image modeling) objectives primarily help reconstruction and generation.
2. MIM in pixel space helps reconstruction more, but is less beneficial for generation than MIM in latent space.
3. CL (contrastive learning) objectives are less helpful for reconstruction & generation, but are important for understanding abilities (eg, top-1 accuracy on ImageNet).
4. Best VFMs for visual tokenization are trained with both MIM in latent space and CL objectives, namely DINOv2 and SigLIP2.

We begin with a brief discussion on the training objectives of different VFMs, and then how these VFMs perform as image tokenizers.

Training objectives of VFMs. CLIP and SigLIP apply CL to image-text pairs, whereas DINOv1 applies a CL-like clustering objective to images. MAE performs MIM on image patches in pixel space, and iBOT predicts DINO-like cluster assignments of image patches in latent space. DINOv2 is basically DINOv1+iBOT, and SigLIP2 is also essentially SigLIP+iBOT (termed TIPS loss in their paper), omitting other regularizing losses. We hereby provide a holistic comparison of the training objectives of different VFMs below:

Notably, iBOT/DINOv2/SigLIP2 are performing VQ-VAE-like codebook learning on image patches, despite codes being soft rather than one-hot, and codebook vectors being high-dimensional (eg, 256). Following DINOv1, they learn a set of 8192 or 65536 prototypes (l2-normalized 256-dim vectors), and require two augmented versions of the same patch (or image) to have identical prototype assignments (soft codes). They also predict the soft codes of masked patches, thus performing MIM in latent space. This formulation is termed self-distillation in DINO, and then extended to patches with MIM by iBOT, and then adopted by DINOv2 and SigLIP2.

The connection between Latent MIM and VQ-VAE makes it natural to convert these VFMs to tokenizers and expect good reconstruction/generation performance. Additionally, the global (image-level) CL objectives of DINOv2 and SigLIP2 also allow better high-level semantics (better performance on understanding tasks).

To better support the discussion above, we provide an ablation on VFMs with all results at hand below. We see that:

1. Pixel MIM (MAE) aligns best to reconstruction objectives, thus, MAE achieves the best rFID. However, this does not provide a feature space as friendly as codebook learning (latent MIM) for generation tasks. We also do not expect strong semantics (top-1 accuracy) in this model.
2. CL-only models (CLIP & SigLIP) achieve best semantics (top-1 accuracy), but modest gFID and worse rFID. Without MIM objectives, they lack good modeling of low-level structures.
3. Latent MIM & CL co-trained methods (SigLIP2 and DINOv2) achieve the best overall performance.

This discussion above provides a more systematic framework for what makes a VFM a good tokenizer, from the perceptive of reconstruction, generation, and understanding. We also connect their performance with specific pre-training objectives, and discover a strong association between some VFMs and VAE. Together, our work could provide a systematic answer to what and why VFMs can be transformed to strong tokenizers, and how to achieve this. We believe this discussion would be greatly beneficial for both pre-training and generation research.

We appreciate the reviewer for raising the question and look forward to further feedback. We will also incorporate the discussion into our manuscript to strengthen the theoretical framework.

We highlight the novelty of our approach as follows: VFMTok is the first to establish that features from a single, frozen pre-trained VFM—whether self-supervised or language-supervised—are sufficient for both image reconstruction and generation. Through a novel region-adaptive quantization that leverages these rich semantic features, VFMTok achieves several key advantages: it 1) accelerates AR model convergence, 2) enables high-fidelity CFG-free image synthesis, and 3) reduces the overall computational budget (Table 3 of the manuscript).

Dear reviewer KT8u,

Thank you for your thorough review of our paper and detailed suggestions for empirical results. We have made every effort to add the evaluations, which provided a more complete validation of our method, and will be incorporated to improve the paper. Your concerns and questions are answered as follows:

W1: Missing comparisons to tokenizers like VQ-VAE, MAGVIT-v2, and MAETok.

Thanks for your reminder. We are sorry that a fair comparison with the discrete tokenizers VQ-VAE and MAGVIT-v2 is not possible due to their absence of officially released checkpoints. Nevertheless, we are able to compare with the strongest suggested baseline MAETok. As MAETok is a continuous VAE, we also convert VFMTok into a continuous one (denoted as VFMVAE), train and measure its image reconstruction quality on ImageNet. A comparison is listed below. We’ll incorporate this into the manuscript for a more detailed comparison.

W2: The tokenizer is only tested within the LlamaGen framework, and its performance with other AR frameworks like EMU, Janus Pro, or MAR is unclear.

Please note that 1) both EMU and MAR utilize continuous VAEs, while VFMTok is a discrete tokenizer. Besides, 2) EMU and Janus Pro are only evaluated on text-to-image generation tasks, while related works [23, 26, 44, 52, 55], including ours, mainly verify their effectiveness through class-to-image generation tasks. Thus, it is hard to perform an apple-to-apple comparison with other tokenizers under these AR frameworks.

During the limited rebuttal period, we managed to integrate VFMTok into a recent AR framework, RandAR. As listed below, this integration allows VFMTok to achieve better image reconstruction and generation performance.

To better verify the compatibility of VFMTok, we also integrated it into the MaskGiT[4] framework. For fairness, the images are generated without Classifier-Free Guidance (CFG). VFMTok presents consistent improvement, further supporting its compatibility with AR and non-AR generation frameworks.

W3: Lack of comparison of latency or inference speed against baseline tokenizers.

Below, we show the decoding (inference) computation and throughput of tokenizers under different resolutions, where VFMTok is faster at higher resolutions thanks to the fixed-length tokens. Note that the comparison does not consider the time for AR generation of tokens. When considered, VFMTok requires a fixed length of 256 tokens, while VQGAN requires #token quadratic to the resolution, the inference efficiency would be more profound.

W4: Performance in text-to-image generation scenarios is missing.

Thanks for your suggestion. Due to the limited rebuttal period, we could only supplement with class-to-image generation experiments. Besides, highly related works such as ImageFolder [26], VAR [44], ViTVQGAN [52], TiTok [55], and RQ-VAE [23] also primarily validate their effectiveness through class-to-image generation tasks. We will continue to work towards the goal of text-to-image generation in the future, which, however, might be out of the scope of this paper.

Q1: What is the performance of VFMTok on higher-resolution images?

Per your request, we added an experiment on image generation at $512\times512$ resolution. All AR models (VFMTok-L) are trained for 100 epochs and tested without CFG. As shown below, our method is consistently stronger on higher resolution settings.

Q2: The region-adaptive quantization framework appears to be general. How would it perform if applied to other tokenizers?

For simplicity, we adapted VQGAN[43] by replacing its PatchGAN with DinoGAN and inserting a region-adaptive module, constructing VQGAN-D and VQGAN-D-RA. The training setup is: tokenizer (50 epochs), VFMTok-L (100 epochs). As shown in the table below, it also improves image generation quality.

Q3: Could you provide some direct qualitative comparisons?

We are sorry that due to the rebuttal policy, we cannot include new figures at present. We will incorporate them into the paper in a future revision following your suggestions.

1. For fairness and following the practical setup, we integrate VFMTok with advanced AR generative frameworks, RandAR and RAR, in terms of the image resolution of ${256\times256}$. As depicted in the table below, VFMTok achieved SOTA results compared to the previous excellent works (including diffusion models and AR generative models) in both image generation with and without CFG. This reveals VFMTok is a good tokenizer for AR image generation.

2. Our work focuses on studying VFM for visual generation. Extending it to unified models like Janus-Pro and Emu3 lies beyond the scope of the current study and will be pursued in future research. In addition, the short rebuttal period has limited the resources and time available for such an extension. Nevertheless, given the encouraging results obtained thus far, we believe this direction holds strong promise and will be explored in our future work.