Visual Autoregressive Modeling for Image Super-Resolution

A1. Implementation details.

Thank you for your suggestions. We will elaborate further on these details in our supplementary materials, for example, the MLP architecture of the diffusion refiner, which consists of linear and activation layers. Additionally, our code will be open-sourced to help readers double-check and verify our implementation details.

A2. User Study.

We conducted a larger-scale user study involving 20 participants to evaluate 100 images. The scale of this study is already sufficiently large compared to previous research (e.g., PASD in ECCV2024 involved 15 participants evaluating 40 images). The experimental results further validate our assertion that our VARSR attains the highest selection rate of 57.1%, significantly surpassing alternative methods. This underscores the potent capability of VARSR in real-world settings to produce lifelike images that harmonize with human aesthetics.

A3. Domain coverage of our large-scale dataset.

To achieve diversity and balance in images from different domains, we conducted semantic clustering and supplemented specific category data. As shown in the Tab below, we ensured that the dataset covers a wide range of category scenes and a relatively balanced proportion (Scenes with a broader semantic scope correspond to a higher proportion of images). This includes portraits, people, food, animals, natural landscapes, cartoons, cityscapes, indoor and outdoor scenes, ensuring comprehensive coverage of visual concepts and richness of scene content.

It is undeniable that due to the limited number of images, some rare scenes may not be covered well, such as the semantic scenes discussed in Sec. B.4 Limitations of our Supp..

A4. Highly degraded scenarios.

In Sec. B.2 of our Supp., we conducted evaluations on the RealLR200 dataset and achieved SOTA results compared to previous methods. The RealLR200 dataset already contains many highly degraded real-world images, such as historical photographs and extreme compression artifacts.

We further provided restoration results in these extreme degradation scenarios, which can be accessed at https://figshare.com/articles/figure/extreme_pdf/28668452?file=53243348. It can be observed that VARSR can still produce faithful and high-quality results in such extreme degradation scenarios, validating the robust capability of VARSR.

A5. Tiling for High-resolution images.

VARSR adopts a tiling approach in generating high-resolution images, which is completely consistent with the diffusion based method. Specifically, we uniformly divide the high-resolution image to be generated into overlapping grids, with each grid having a standard resolution of $512\times 512$. VARSR generates SR results for each grid separately and then tiles them together to obtain the complete image restoration result.

Therefore, the computational cost increases linearly with the increase in output resolution. However, we can batch process different grids in parallel to accelerate the inference process, significantly increasing the actual computational speed. Importantly, due to VARSR and the diffusion model using the same tiling approach, the tenfold efficiency advantage over the diffusion model still exists when handling images of various resolutions.

A6. Clarification of Diffusion Refiner.

Please see A4 to Reviewer yFzJ.

A7. Spatial Misalignment.

In the field of image restoration, our goal is to recover the original HR image from severely degraded LR images, where degradation refers to distortions such as blur, noise, compression, and other artifacts that may occur during transmission or capture processes, as defined in the classic work SRCNN (TPAMI2015) in ISR field.

The spatial distortions you mentioned, such as rotation or flipping, typically do not fall within the scope of distortions that need to be addressed in ISR field. When spatial distortions such as rotation occur, we can employ preprocessing methods (e.g., corner detection) to restore spatial positioning, followed by the image restoration process. Therefore, we can assume that LR and HR images are always aligned in spatial scale. Our proposed SA-RoPE effectively conveys this spatial structural consistency, thereby enhancing the ISR performance.

A1. Effectiveness of CFG.

CFG leads to a certain reduction in fidelity metrics but significantly improves the perceptual quality of the image. Existing fidelity metrics have limitations in accurately measuring human perceptual quality, especially when the original HR image quality is low. In Fig.4, generated images of higher perceptual quality for humans can lag behind in certain fidelity metrics, as overly smoothed low-quality images tend to score better on these metrics.

These limitations have been confirmed in many previous studies (e.g., SUPIR in CVPR2024 and Pipal in ECCV2020), and mathematical derivations have verified the inherent contradiction between fidelity and quality (The Perception-Distortion Tradeoff in CVPR2018). In Fig.9, the introduction of CFG results in significantly richer textures in the generated images, substantially improving perceptual quality to meet human preferences while maintaining correct semantics.

A2. Other IQA Metrics.

Thanks for your valuable comments. First, as mentioned in A1, fidelity and IQA metrics can exhibit certain contradictions. Using our large-scale datasets for training can generate images that retain semantics and have higher quality.

Second, the good performance of VARSR in IQA metrics does not originate from specialized data filtering but from its ability to generate high-quality images. We further conduct evaluation using other well-adopted IQA metrics that were not used for data filtering, including CNNIQA (CVPR2014), HyperIQA (CVPR2020), and TOPIQ (TIP2024). As shown in the Tab below, VARSR continues to achieve SOTA results, surpassing other methods.

A3. Complexity of the training setup.

Our model is no more complex than diffusion methods, both requiring the same three-step process for application in ISR tasks: (1) training the VAE, (2) pre-training on C2I/T2I tasks, and (3) fine-tuning on ISR. The apparent simplicity of training in previous diffusion methods stems from directly using pre-trained Stable-Diffusion and its VAE as base models. These base models have already undergone training in the first two steps, requiring only fine-tuning for the ISR task.

However, the open-source VAR base model falls short of our needs, as it could only generate $256*256$ images and is limited in generated image quality. Therefore, we need to conduct training in all three stages, making it appear more complex. We intend to open-source the base models trained in the first two steps to ease training burdens for future research and contribute more to the community.

A4. Clarification of Diffusion Refiner.

Our proposal of the Diffusion Refiner does not mean introducing an additional ISR process but offers a mapping of continuous residual distributions. As highlighted in lines 185-195, quantization of image discrete vectorization leads to loss, thereby restricting the upper bound of restoration, as VAR can only predict the quantized discrete vectors of the image. Therefore, we specifically proposed a refiner to convert predictions of categorical vector distribution into a continuous-valued space through a diffusion loss, thereby enhancing the upper bound of VAR's capacity. Such an idea has been validated in previous works: MAR (NIPS2025) and HART (ICLR2025).

The Refiner solely serves to map the probability distribution of quantized residuals with VAR features as a condition, and does not have the capabilities of an ISR model. A lightweight network (only 37M parameters accounting for 3% of the 1.1B model) suffices for the discrete-to-continuous mapping. In the Tab below, a larger refiner does not yield significant gains, which was also confirmed in MAR (NIPS2025). Thus, we believe that a standard SR model as the refiner will not lead to improvement.

A1. Comparison with more SOTA.

BSRGAN and Real-ESRGAN are still commonly used SOTA GAN-based models due to their excellent performance. Other ISR works (e.g., SeeSR, PASD) also chose them as GAN-based baselines for comparison.

GigaGAN does not provide open-source models or code for testing on its homepage. Therefore, we further conduct a comparison with more recent SOTA DASR (GAN-based, ECCV2022) and SinSR (diffusion-based, CVPR2024). As shown in the Tab, the results are consistent with the findings in the paper, with VARSR leading by a significant margin in perceptual quality metrics, validating the strong performance of VARSR.

A2. Necessity of Diffusion Refiner.

As mentioned in lines 043-051 of our paper, we believe that VAR is effective for ISR tasks. Similar to other improvements made to VAR-related components (e.g. Prefix Tokens, SA-RoPE), the Diffusion Refiner is also an attempt to enhance the upper bound of VAR in ISR tasks.

As highlighted in lines 185-195, quantization of image discrete vectorization leads to loss. Even if VAR accurately predicts all quantized discrete tokens, the generated image will still have quantization losses that limit its upper bound of restoration. Therefore, we specifically proposed a refiner to convert predictions of categorical discrete vector distribution into a continuous-valued space, thereby enhancing the upper bound of VAR's representational capacity. This Refiner utilizes the loss of diffusion form to model the probability of quantization residuals instead of the complete latents to accelerate convergence. The idea of using a diffusion refiner for discrete-to-continuous mapping has been validated in many previous works: MAR (NIPS2025) and HART (ICLR2025).

In Tab.5, the introduction of the diffusion refiner led to improvements in all metrics, especially in perceptual quality metrics. Notably, MANIQA achieved an average improvement of 2.2%, and SSIM improved by 0.82%. Fig. 8 shows that the diffusion refiner reintroduced many image details lost in quantization (e.g., textures of the clothes and the flowers). Furthermore, the diffusion refiner is an extremely lightweight module, with only 37M parameters accounting for just 3% of the 1.1B model. We believe that the Diffusion Refiner is effective in enhancing the representational capacity upper bound of VAR in ISR tasks with a very small param increase.

A3. Necessity of CFG.

Similarly, the introduction of CFG is also aimed at expanding the upper bound of the image quality generated by VAR, thereby generating more realistic images through guided sampling.

As mentioned in lines 152-156, VAR, GANs, and Diffusion models all target fidelity as the optimization objective in ISR tasks, which may result in generated images being overly smooth and lacking in detail. It tends to retain distortions such as blur from LR images, leading to lower human-perceived quality. To address this, we propose an image-based CFG that follows the principles of the standard CFG. By learning low-quality image distributions during training, it allows us to guide the probability distribution during sampling towards generating higher-quality images, thereby expanding the upper bound of the image quality.

Image-based CFG validates the introduction of a new form of CFG into the VAR framework, striking a balance between realism and fidelity similar to the diffusion model. Results in Tab.6 and Fig.9 confirm the effectiveness of Imaged-base CFG, resulting in significantly richer textures in the generated images, substantially improving perceptual quality to meet human preferences while maintaining correct semantics.

A1. Training with different datasets.

Thanks for your comments. In fact, baseline methods are trained on different datasets (e.g., SeeSR uses LSDIR and PASD uses DIV2K/FFHQ, with a tenfold difference in data quantity). There are also differences in the pretrained models, which are important in providing generative priors for ISR (e.g., diffusion methods using Stable Diffusion and GAN methods trained from scratch). In lines 258-270, the primary objective of using large-scale data is for pretraining to acquire generative priors. The open-source VAR base model falls short of our needs, as it could only generate $256*256$ images and is limited in generated image quality. However, diffusion methods leverage the powerful Stable Diffusion as their base model, which is pretrained on billions of image-text pairs, far surpassing our scale of 4 million. Thus, in evaluating ISR methods, previous works (e.g., SeeSR/PASD) typically compare based on the performance of the models themselves.

Based on your advice, we present VARSR results trained on the same datasets as baselines. In the Tab below, when pretraining with our large-scale data and fine-tuning with the same LSDIR as SeeSR, VARSR still performs exceptionally well, far surpassing other methods in perceptual quality metrics. This is consistent with our conclusion when training on large-scale data, validating the superiority of the VARSR framework.

A2. Advantages of VARSR in preserving semantics.

VAR has the advantage over diffusion models in better human-perceived semantic fidelity and quality.

(1) Firstly, in terms of fidelity metrics as PSNR and SSIM, VARSR performs similarly to SeeSR and outperforms all other diffusion methods on DIV2K and DrealSR datasets (Tab.1). However, existing fidelity metrics have limitations in measuring human perceptual quality. Fig.4 shows that images with higher human perceptual quality may not score well on fidelity metrics, as overly smoothed images tend to perform better. These limitations have been confirmed in previous studies (e.g., SUPIR in CVPR2024, Pipal in ECCV2020), and mathematical derivations verify the contradiction between fidelity and quality (The Perception-Distortion Tradeoff in CVPR2018).

(2) Secondly, user study and numerous examples support VARSR's superiority in maintaining spatial structure and preserving semantics. In Tab.10 (user study), the results of VARSR align more closely with human preferences. In Fig.5/15/16/17, examples illustrate that VARSR excels in generating textures faithful to the original image, outperforming diffusion models. In the last two rows of Fig.5, only VARSR accurately restores architectural semantics and smooth details. In the 2nd and 3rd cases of Fig.17, VARSR generates clear foliage and walnuts, while diffusion methods exhibit illusion issues (SeeSR produces fabricated content). These results highlight VARSR's superior preservation of semantics and spatial structure.

A3. Efficiency Comparison.

Thanks for your comments. We believe that the superior efficiency of VARSR lies in the advantages of the VAR framework over the standard diffusion framework. Therefore, there is no need to compare with models that are optimized to enhance efficiency for a specific framework. This is because these efficiency designs can also be applied to VARSR to further improve efficiency. For example, one-step diffusion models (e.g., OSEDiff in NIPS2025 and SinSR in CVPR2024) often undergo knowledge distillation to match the performance of multi-step generation. Our VARSR can also adopt this approach to simplify the inference steps (e.g., reduce the inference scales).

Tab. 3 and the explanations (lines 380-384/407-413) demonstrate VARSR's notable efficiency enhancements over diffusion models, validating our motivation. Compared to the one-step diffusion method SinSR (refer to A1 to Reviewer vbZD), VARSR outperformed in most metrics, showcasing its effectiveness.

A4. Motivation.

VAR provides a novel and effective approach for addressing ISR tasks, offering advantages over diffusion models rather than a straightforward substitute. Our work is just an initial attempt, and there is vast potential for further leveraging VAR in ISR tasks. In addition to the advantages in preserving structural features and efficiency, as you mentioned, VAR aligns the generative paradigm for both vision and text. Thus, we believe that VAR has the potential to be integrated with LLMs, enabling the direct utilization of human preferences to guide ISR through optimization forms such as DPO or GRPO, which is a promising avenue for future research.