PocketSR: The Super-Resolution Expert in Your Pocket Mobiles

We sincerely thank the reviewer for recognizing the value of our work and for providing constructive feedback. We will carefully address each of the identified weaknesses and concerns in the following responses. All discussions and additional experiments presented in this rebuttal will be incorporated into the camera-ready version of the paper.

[W1] Contributions of online annealing pruning and multi-layer feature distillation.

We sincerely thank the reviewer for the valuable comments. However, we believe there may be some misunderstandings regarding the core contributions of our work.

Our main contribution lies in the design of the LiteED architecture, and the integration of Online Annealing Pruning and Multi-layer Feature Distillation, which together enable effective compression for super-resolution (SR) while preserving the generative prior.

1. While general pruning strategies (as cited in [1,2]) have been explored for efficient text-to-image (T2I) generation, our Figure 6 and Table 4 show that these general strategies fail to preserve the pretrained prior when applied to SR—likely due to fundamental differences in training objectives between T2I and SR. To our knowledge, no prior work proposes a pruning strategy specifically tailored to retain image priors in the SR setting, as we do.

2. Our Multi-layer Feature Distillation further improves prior preservation by supervising the student with multi-scale features across multiple layers. In contrast, existing methods such as SnapGen[3] and [4] typically distill only from the output, which imposes a weak constraint. As shown in Table 4, our method outperforms a variant using output-only distillation (Strategy 3), validating its effectiveness under a lightweight setting.

[1] Learning sparse networks using targeted dropout, Arxiv 2019

[2] Structural Dropout for Model Width Compression, Arxiv 2022

[3] Snapgen: Taming high-resolution text-to-image models for mobile devices with efficient architectures and training, CVPR 2025

[4] Cross-Architecture Knowledge Distillation, ACCV 2022

[W2] “Older” base model: U-Net.

We sincerely thank the reviewer for the valuable suggestion. Our choice not to adopt DiT as the base model is primarily driven by deployment considerations. This work aims to reduce computational overhead and enable on-device SR, especially on mobile NPUs, which currently lack sufficient support for DiT-based architectures—making them less practical for edge deployment.

In addition, while some recent single-step SR models use DiT, most state-of-the-art methods—including OSEDiff, AdcSR, and PiSA-SR (as cited by the reviewer)—still rely on U-Net architectures. By using the same backbone, we ensure fair and meaningful comparisons with these widely accepted baselines.

[W3] Arbitrary choice of pruning.

We appreciate the reviewer’s thoughtful analysis. While we understand the concern, we would like to clarify the rationale behind our pruning strategy.

Our pruning design (i.e., where to prune) is guided by empirical observations: shallow modules have greater impact on SR quality, while deeper ones can be pruned with minimal degradation. We support this with ablation studies across different module types and depths, as shown in Tables 1–4 of the supplementary material.

We acknowledge that we did not conduct independent ablations for every single module, mainly for two reasons:

1. Such exhaustive experiments would require significant resources beyond the scope of this work.

2. More importantly, we aim to offer conceptual insights rather than purely engineering-driven optimizations, which we believe are more meaningful for the research community.

[W4] CLIPIQA metric.

We sincerely thank the reviewer for the professional and constructive feedback. We will include a comparison on CLIPIQA in the camera-ready version.

The table below reports the results on the DRealSR dataset. CLIPIQA leverages the pretrained image-text alignment model CLIP to evaluate the consistency between an image and a given textual prompt. This metric tends to favor results with richer texture details, regardless of their alignment with the ground truth.

While our method performs slightly worse than OSEDiff and AdcSR on this metric—both of which emphasize stronger generative capability—we achieve better results on perceptual fidelity metrics such as LPIPS and DISTS, which are more indicative of human-perceived visual quality. Importantly, all these improvements are achieved under significantly lower computational budgets, highlighting the efficiency and practicality of our approach.

[W5&W6] Missing citations.

We sincerely thank the reviewer for the helpful suggestion. We will cite the papers mentioned by the reviewer in the Related Work section of the camera-ready version.

[Q1] Could the proposed LiteED be used for generation?

We thank the reviewer for the thoughtful and forward-looking question.

The proposed lite decoder can be directly applied to efficient image generation tasks. However, the lite encoder includes components specifically tailored for super-resolution—such as Adaptive Skip Connections and Dual-Path Feature Injection—which may not transfer well to generative settings and would likely require adaptation.

In image generation, encoders and decoders are typically used asymmetrically: the encoder is only used during training, while the decoder is involved in both training and inference. This prevents the use of skip connections as employed in super-resolution.

Additionally, generative models often operate in compact latent spaces. The heavy-path features from our encoder may introduce redundancy that could hinder generation quality rather than help it.

[Q2] Network symmetry.

Thank you for the excellent question. We explicitly considered this asymmetry when designing LiteED. Empirically, we observed that in super-resolution, the decoder plays a more critical role than the encoder in determining performance.

As shown in Table 1, the decoder accounts for 8.7× more MACs than the encoder. Moreover, Table 3 shows that enlarging the decoder significantly improves reconstruction quality, while increasing encoder complexity yields only marginal gains and adds training overhead.

This likely stems from the inherent nature of SR: low-resolution inputs contain less information than the high-resolution outputs. Under a fixed computational budget, it is thus more effective to prioritize the decoder, which reconstructs the richer content.

[Q3] Can findings be extented to DiTs?

Thank you for the insightful question. We agree that extending our approach to a DiT-based backbone for lightweight image super-resolution is a promising direction. In principle, both components of our method—LiteED and online annealing pruning—are transferable to other base models. LiteED targets the VAE-style decoder, and the pruning strategy is architecture-agnostic.

However, certain pruning decisions—such as where to prune and which lightweight modules to use—would need to be revisited for DiT, taking its structural characteristics into account. Recent work [5] suggests that DiT exhibits layer-wise specialization, with intermediate layers contributing most to semantic alignment. Such insights could help guide pruning when adapting our method to DiT.

We appreciate the reviewer’s suggestion, which points to a valuable future direction for improving the generalizability of our framework.

[5] Revelio: interpreting and leveraging semantic information in diffusion models, Arxiv 2024

We sincerely thank the reviewer for the time and effort dedicated to evaluating our work. We are encouraged to see that our paper was described as “technically sound” and of “great research value.” We will address each of the reviewer’s concerns and identified weaknesses in detail. All additional experiments and discussions presented in this rebuttal will be incorporated into the camera-ready version of the paper.

[W1] Reproducibility.

We fully understand and agree with the reviewer’s concern. In response, we have decided to conduct a code reimplementation, within the boundaries of our company’s policy, and plan to release it publicly upon acceptance to better support the community.

[W2&Q1] Efficiency on mobile devices.

We fully understand the reviewer’s concern. However, we hope you can appreciate that migrating and deploying all existing super-resolution methods to mobile devices would require a significant amount of time and engineering effort. As an alternative, we provide the measured inference efficiency of PocketSR on a newly released smartphone model from 2025. For comparison, we also report the runtime of the uncompressed SD-Turbo backbone on the same device across corresponding modules.

The results are shown in the table below (measured on a single 512×512 image; unit: milliseconds). As can be observed, PocketSR achieves an inference time of only 140ms on-device, representing an over 80% reduction compared to the original, non-lightweight backbone—demonstrating the practical efficiency of PocketSR for real-world deployment.

[W3] Concern on novelty.

We sincerely thank the reviewer for the thoughtful question. However, we believe there may have been a misunderstanding regarding the core contributions of our work.

Our paper makes three primary contributions:

1. We propose a lightweight encoder-decoder framework, LiteED, specifically designed for the super-resolution task. LiteED integrates carefully designed components, including Adaptive Skip Connections and Dual-Path Feature Injection, and serves as an effective replacement for the original SD VAE. Remarkably, despite its significantly reduced parameter count, LiteED matches or even outperforms the original VAE in certain aspects.
2. We introduce Online Annealing Pruning, a pruning strategy tailored for low-level vision tasks. While similar pruning concepts have been explored in efficient text-to-image (T2I) generation, our Figure 6 and Table 4 show that such methods fail to retain the image priors encoded in pretrained generative models when applied to super-resolution. This discrepancy stems from the fundamental difference between T2I and SR training objectives.
3. Our proposed Multi-layer Feature Distillation enhances the transfer of generative knowledge by guiding the student model with multi-scale features across several layers. In contrast, prior methods typically distill only from the model’s output—a relatively weak constraint that may fail to preserve rich prior information after pruning. In Table 4, we compare our approach with a variant that distills solely from the U-Net output (Strategy 3). The results clearly demonstrate the superior performance of our method, validating its effectiveness in retaining the pretrained generative prior in a lightweight setting.

To the best of our knowledge, all contributions are novel and have not been addressed in prior work. While we do adopt step distillation, it is not the core innovation of our paper.

[Q2] Detailed structure for the “lightweight” block.

Thank you for the insightful question. We provide an explanation of the “Lightweight Block” in Figure 2 (lines 153–160), and would like to clarify the following:

1. Scope of Pruning: The Online Annealing Pruning strategy is applied only to the U-Net, not to LiteED, so the notion of “Lightweight Block” does not apply to LiteED.

2. Pruned Components: Pruning targets the four most computationally expensive U-Net components: residual blocks, cross-attention, self-attention, and FFNs. Lightweight layers (e.g., normalization, activation) remain unchanged.

3. Pruning Details:

   • Residual Blocks: All convolutions are replaced with depthwise separable convolutions (as in MobileNet), forming the “Lightweight Block.”
   • Cross-Attention: Since text input is not used in SR, we replace these layers with an MLP of two linear layers.
   • Self-Attention: Approximated using linear attention to reduce cost.
   • FFNs: Hidden dimensions are reduced to one-fourth.

We acknowledge that the term “ultra-lightweight FFN” may have caused confusion and will revise it for clarity in the camera-ready version.

[Q3] Questions on linear attention.

Thank you for the question. We adopt the basic form of linear attention as described in [1].

As shown in Table 3 of the supplementary material, replacing standard attention with linear attention involves performance trade-offs. Our experiments indicate that self-attention—especially in the shallow layers of the U-Net—is crucial for capturing fine-grained textures. Replacing it with linear attention in early layers leads to noticeable degradation, likely because these layers extract low-level features that lightweight alternatives cannot easily replicate.

To balance efficiency and performance, we apply linear attention only at depth 4 of the U-Net. This modification preserves model performance while improving training stability and efficiency.

[1] Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention, ICML 2020

We sincerely appreciate the time and effort you dedicated to reviewing our paper. Several of your comments were particularly insightful and thought-provoking. We have carefully responded to each of the weaknesses and suggestions you raised. If you have a moment, could you kindly let us know whether our responses have successfully addressed your concerns? Your feedback is very important to us. Thank you again for your thoughtful contributions.

We sincerely thank the reviewer for the insightful and constructive feedback, which has prompted us to further reflect on our approach. We also appreciate your recognition of our work’s effectiveness and novelty. Below, we address your concerns point by point (with related issues grouped for clarity). All discussions and new experiments will be included in the camera-ready version.

[W1&Q1] Explanation for Table 3.

We apologize for not clearly specifying the experimental setup of Table 3, and we will include this information in the camera-ready version. The experiments in Table 3 follow the first-stage training described in Section 3.3, where the unpruned SD U-Net is jointly trained with various encoder and decoder architectures for 80,000 steps.

Table 3 evaluates the impact of different encoder and decoder designs on super-resolution performance (not VAE reconstruction), serving as an ablation for our proposed LiteED. U-Net pruning is not applied at this stage. All settings share the same model and training configuration, differing only in encoder/decoder architecture. RealSR is used as the test dataset.

[W2&Q2] Explanation for SD version.

Thank you for the question. As mentioned in line 101 of the paper, PocketSR uses SD-Turbo as the pretrained model.

[W3&Q3] Performance on handling high-resolution SR.

Thank you for the valuable suggestion. To assess PocketSR at higher resolutions, we constructed a multi-resolution benchmark with three 4× SR tasks: upscaling to 1K, 2K, and 4K. Each setting includes 20 test images with ground-truths. The 1K and 2K samples are selected from RealSR, ensuring resolution and scene diversity; 4K samples are from DRealSR. Notably, this benchmark uses original full-resolution images, unlike Table 2 which evaluates on center-cropped patches—a common practice in SR [1].

As shown below, PocketSR consistently achieves superior LPIPS, DISTS, and NIQE scores, and competitive PSNR, SSIM, and MUSIQ results. Its stable performance across resolutions demonstrates strong robustness, making it well-suited for real-world use cases like mobile photography.

We adopt SD-Turbo for SD pretraining, a widely used default in many super-resolution frameworks [2,3]. The reviewer raises an insightful question about whether model performance across resolutions is influenced by the resolution of the pretrained SD. We believe the answer is likely yes, as SD models trained on different resolutions may encode distinct resolution-specific priors. We consider this a valuable direction for future work.

[1] Exploiting Diffusion Prior for Real-World Image Super-Resolution, IJCV 2024

[2] Arbitrary-steps image super-resolution via diffusion inversion, CVPR 2025

[3] Degradation-Guided One-Step Image Super-Resolution with Diffusion Priors, Arxiv 2024

[W4&Q4-1] Detailed ablation studies for the Adaptive Skip Connection (ASC).

We sincerely thank the reviewer for the constructive suggestions. In response, we provide additional ablation studies on the Adaptive Skip Connection (ASC) module in the table below. The experiments are conducted on the RealSR dataset, following the same settings as in Table 3 of the main paper for ease of comparison. We omit efficiency-related metrics here, as the differences are minimal.

1. We investigate the effect of removing the learnable control coefficients in ASC. We observe that using naive skip connections hinders the model’s ability to adaptively control the contribution of skip features based on the degradation level of the input. As a result, the model performs poorly when handling diverse real-world degradations.
2. We explore an alternative design where the control coefficients are predicted from features in the heavy path. However, we find that this strategy leads to instability during training, as heavy path features are uncompressed and contain redundant information, which interferes with the learning of accurate control signals. Although this variant performs relatively well on the no-reference metric NIQE, the significantly lower LPIPS and PSNR scores suggest compromised perceptual and pixel-level fidelity, with more noticeable artifacts in the output.

[W4&Q4-2] Characteristics of the learned control coefficients.

Thank you for the excellent question. Due to space constraints, we couldn’t include this analysis in the paper, but we provide more details below:

1. Control Coefficients vs. Outputs: Larger control coefficients make outputs more similar to the low-resolution (LR) input—preserving fidelity but reducing generative strength. Smaller coefficients shift reliance to the U-Net output, enhancing generative capacity at the cost of fidelity.
2. Control Coefficients vs. Inputs: For severely degraded LR inputs (e.g., high-scale SR, real-world degradations), control coefficients are generally smaller in early layers and slightly larger in deeper ones. In mild degradation cases, the pattern reverses, with higher coefficients in shallow layers.

This behavior aligns with intuition: mildly degraded inputs benefit from more skip features to preserve details, while heavily degraded ones require suppressing skips to rely more on generative priors for plausible reconstruction.

Regarding Q4-D, we agree that manually adjusting the control coefficients can reveal a similar fidelity–generation trade-off. However, such tuning is sensitive and may cause instability, so we recommend using the model-learned adaptive coefficients, which are more robust across varied degradations.

[W5&Q5] Explanation and ablation studies for the Dual-path Feature Injection (DFI).

Thank you for the thoughtful question. In our Dual-path Feature Injection (DFI) design, the lite path provides compressed, information-dense features that offer global structural guidance and align well with the VAE feature distribution in SD, making them easier for the pretrained U-Net to utilize. In contrast, the heavy path contains richer details but lower information density, making it harder for generative models to use directly.

The lite path serves as the primary guidance source, while the heavy path complements it with fine-grained textures. Both are essential for high-quality super-resolution. To validate this, we conducted ablation studies following the experimental settings in Table 3 of the main paper. We observe that removing the lite path leads to a significant drop in fidelity, as the model loses its primary information source. In this case, the model overly relies on the pretrained SD’s generative prior, often hallucinating textures, which results in perceptual artifacts and inflated non-reference scores like NIQE. On the other hand, removing the heavy path causes a moderate degradation in detail reconstruction, reflected by a noticeable drop in perceptual metrics such as LPIPS and DISTS.

In summary, DFI effectively balances structural guidance and detail injection, significantly improving fidelity with minimal computational overhead.

Thank you to the authors for the response. Most of my concerns are solved.

I still have a few questions for further discussion.

1. The output of the U-Net has a size of B×4×64×64, which matches the size of the output from the lite path in the encoder. Do the authors observe that this design may lead to some additional information introduced by the heavy path—through the increased channel width of 512—being ultimately discarded?

2. In the ablation study, the model without the lite path performs worse than the one without the heavy path. Could the authors elaborate on the underlying reasons for this result? Does this indicate that the lite path plays a more critical role than the heavy path, perhaps because its output directly matches the input dimensions of the decoder? In the authors' response, the lite path is described as the primary source of information—could the authors clarify why this is the case?

Additionally, in pre-trained diffusion models, the VAE often results in information loss, which can negatively impact restoration tasks such as super-resolution. What are the authors' thoughts on the future development direction of diffusion-based super-resolution models?

Thank you for the constructive discussion.

[Q1] The size of the U-Net ouput.

First, we would like to clarify the motivation behind using 4 channels at the U-Net output. This design choice was made to facilitate alignment with the pretrained decoder weights, making the learning process smoother. As mentioned in our previous response, we align the decoder input features with the latent space of Stable Diffusion (SD), and using a consistent feature dimension helps stabilize supervision during training.

Furthermore, we believe that introducing heavy-path features is meaningful—even though their spatial and channel dimensions are bigger than the U-Net output's. This is based on the observation that SD is capable of generating rich and detailed 512×512 images from 4-channel latent features. The transformation from heavy-path input to U-Net output only increases feature density, not information loss. This differs from the encoder side: the lightweight encoder design (only 0.8M parameters) may struggle to extract sufficient detail, and thus the heavy path serves as a necessary supplement.

Of course, we agree with your observation—increasing the number of latent channels can indeed be beneficial, and this aligns with the evolving trends in modern image generation models. For example, the more recent FLUX model uses a latent space with 16 channels. However, since our current design is based on Stable Diffusion (SD), we use 4 channels to remain compatible with the pretrained decoder. In the future, we plan to explore updated backbone models and consider using higher-dimensional latent representations.

[Q2] Explanation of the ablation study.

Regarding the ablation results where removing the lite path leads to a larger performance drop than removing the heavy path, we believe this is because our model builds upon pretrained T2I weights. After fine-tuning, the model mainly relies on the compressed information flow through the lite path. When the lite path is removed, the model becomes entirely dependent on the uncompressed heavy-path features, which contain redundant information and interfere with the learning of accurate control signals. This observation further validates our dual-path design, which enables complementary feature injection from both directions.

[Q3] Future direction.

As for the reviewer’s question on VAE-related information loss, we agree that this is a well-known issue in current SR models. On the one hand, recent progress in VAE design—such as the FLUX series—has significantly improved restoration capacity. On the other hand, strategies such as using VAEs with lower compression ratios (e.g., 4× downsampling instead of 8×), or leveraging dual-path feature injection as proposed in our work, may help mitigate this problem.

In terms of future directions, we believe that enhancing the generative capacity of models while achieving greater efficiency will be a key research direction in the future. Specific to the problem, we think semantic information remains a critical component in activating the generative potential of SR models. Existing semantic extractors, such as LLaVA (used in SUPIR), cognitive encoders (CoSeR), and DAPE (SeeSR), are often too heavy for practical use. Developing lightweight yet effective semantic extraction methods to guide texture generation in a perceptually aligned manner remains an important open challenge.

We sincerely thank the reviewer for taking the time to read our paper and provide valuable feedback. We are pleased to see that the reviewer finds our work both novel and practically valuable. We deeply appreciate your insights, and in the following, we address each of your concerns carefully. All discussions and additional experiments presented in this rebuttal will be incorporated into the camera-ready version of the paper.

[W1] Insufficient reproducibility.

We fully understand and agree with the reviewer’s concern. In response, we have decided to conduct a code reimplementation, within the boundaries of our company’s policy, and plan to release it publicly upon acceptance to better support the community.

Regarding the training details mentioned by the reviewer, we would like to clarify that they are indeed provided in Section 3.3 rather than Section 4.1. Nonetheless, we will further enrich the description of training settings and implementation details in the camera-ready version to improve clarity and reproducibility.

[W2] Performance trade-off (better LPIPS/NIPE but lower PSNR/SSIM than ResShift).

We sincerely thank the reviewer for the insightful and professional comments. We would like to clarify this concern from two perspectives:

1. PSNR and SSIM primarily reflect pixel-level similarity, which does not always align with human visual perception—a limitation that has been widely discussed in prior works on image super-resolution and quality assessment [1,2,3]. Moreover, it is possible for overly smoothed outputs to yield higher PSNR/SSIM scores. For example, we found that simply upsampling the low-resolution images in DRealSR using bicubic interpolation achieves a PSNR of 30.58 and SSIM of 0.8312—surpassing all super-resolution methods reported in Table 2. A similar phenomenon is observed on RealSR. This clearly contradicts the fundamental goal of super-resolution, which is to recover fine and realistic image details.
2. Given the limitations of PSNR/SSIM, we advocate for a multi-metric evaluation strategy rather than relying on a single indicator. From the perceptual similarity perspective, our method achieves the best performance in both LPIPS and DISTS (which measure different aspects from PSNR/SSIM), indicating that our results are most consistent with human perception. On the pixel-wise alignment side, while our method slightly lags behind ResShift and SinSR, it performs on par with previous multi-step diffusion-based methods such as StableSR and DiffBIR, as well as single-step approaches like OSEDiff and AdcSR—demonstrating comparable pixel-level fidelity. In terms of image detail and texture, our method achieves the best NIQE score and comparable MUSIQ, further reflecting its ability to produce high-quality outputs.

In summary, our method delivers competitive or even superior performance across multiple dimensions, while being significantly more efficient in computation, highlighting its practical value on common application scenarios.

[1] Scaling up to excellence: Practicing model scaling for photo-realistic image restoration in the wild, CVPR 2024

[2] CoSeR: Bridging Image and Language for Cognitive Super-Resolution, CVPR 2024

[3] Image quality assessment: Unifying structure and texture similarity, TPAMI 2020

[W3] Writing and clarity.

We sincerely appreciate the reviewer’s careful examination. The mention of LiteAE was a typographical error, and we will correct it in the camera-ready version.

Regarding the parameter counts reported in Table 2, we would like to clarify that the numbers are accurate. This is because SinSR is distilled from ResShift, and thus they share the same number of parameters. We respectfully invite the reviewer to refer to Table 2 in OSEDiff[4], where the reported values are consistent with ours.

[4] One-Step Effective Diffusion Network for Real-World Image Super-Resolution, NeurIPS 2024

[W4-1] Comparisons with non-diffusion SR models.

We sincerely thank the reviewer for the insightful and professional feedback. To address your concern, we have conducted additional comparisons on the DRealSR dataset with several representative non-diffusion-based methods, including a CNN-based method (DASR [5]) and a Transformer-based method (FeMaSR [6]). The comparison results are presented in the table below (FPS measured on an A100 server).

Our method significantly outperforms the competing approaches in terms of perceptual quality metrics such as LPIPS, DISTS, NIQE, and MUSIQ. While DASR shows stronger results in PSNR, SSIM, and efficiency, we believe this is partly due to its extremely lightweight design, which—while advantageous in terms of speed and model size—also limits its ability to recover fine-grained details from complex real-world degradations, resulting in smoother outputs. As discussed in [W2], such characteristics can sometimes lead to higher PSNR and SSIM scores that may not fully reflect perceptual quality.

[5] Efficient and Degradation-Adaptive Network for Real-World Image Super-Resolution, ECCV 2022

[6] Real-World Blind Super-Resolution via Feature Matching with Implicit High-Resolution Priors, ACMMM 2022

[W4-2] Inference speed on common GPUs.

We sincerely thank the reviewer for the valuable suggestion. The table below presents the inference time of different methods on an RTX 4090 GPU. We observe that the inference speed on the 4090 is comparable to that on the A100, demonstrating that our method maintains high efficiency even when deployed on consumer-grade GPUs—highlighting its practicality for real-world applications. If the reviewer is interested in the performance of PocketSR in more constrained edge-computing environments such as mobile NPUs, we kindly refer you to our response to Reviewer nGC4’s Weakness 2. Specifically, PocketSR takes approximately 140ms to super-resolve a 512×512 image on a mobile NPU.