One-Step Diffusion-Based Image Compression with Semantic Distillation

Many thanks for your valuable comments and the recognition on our contributions. We hope the following responses could address your concerns.

1. Wider bitrate range

We have trained additional OneDC models spanning a wider bitrate range (0.0034–0.1115) and compare them with PerCo (SD) on full-resolution Kodak images:

These results confirm that OneDC maintains strong performance across a wide bitrate range, including both extremely low and moderately high bitrate settings. We will include results on extended bitrate range in the final version.

2. Comparison with SOTA DDCM

To ensure a fair comparison with DDCM and enable FID calculation on Kodak dataset, we follow DDCM's evaluation protocol: center-cropping and resizing images to certain resolution. The test datasets include Kodak (512x512) and CLIC2020 (512x512,768x768). All DDCM data numbers are obtained from their paper.

• The table below shows the absolute numbers on the Kodak (512x512 resolution, FID calculated on 64x64 patches).

  Bpp(OneDC)	FID(OneDC)↓	LPIPS(OneDC)↓	PSNR(OneDC)↑	Bpp (DDCM)	FID(DDCM)↓	LPIPS(DDCM)↓	PSNR(DDCM)↑
  0.0034	54.292	0.394	16.756
  0.0109	36.862	0.223	20.259
  0.0177	33.075	0.183	21.219
  0.0257	30.947	0.154	21.970	0.030	32.031	0.222	22.066
  0.0369	29.103	0.132	22.718	0.038	29.117	0.190	22.551
  0.0524	27.428	0.113	23.363	0.050	25.647	0.161	23.013
  0.0795	25.452	0.095	24.202	0.095	24.215	0.138	23.606
  0.1139	23.448	0.082	25.021	0.149	23.199	0.124	24.069

  The absolute numbers on CLIC2020 are omitted here due to rebuttal character constraint.

• For more comprehensive comparison, we also calculate the relative BD-Rate with PerCo as an anchor:

  • Kodak (512x512 resolution, FID calculated on 64x64 patches):

    	BD-Rate (FID)↓	BD-Rate (LPIPS)↓	BD-Rate (PSNR)↓
    PerCo	0.00%	0.00%	0.00%
    DDCM	-60.18%	-57.32%	-73.76%
    OneDC	-65.30%	-81.92%	-77.13%

  • CLIC2020 (512x512 resolution, FID calculated on 128x128 patches):

    	BD-Rate (FID)↓	BD-Rate (LPIPS)↓	BD-Rate (PSNR)↓
    PerCo	0.00%	0.00%	0.00%
    DDCM	-65.97%	-61.58%	-77.37%
    OneDC	-69.94%	-84.51%	-86.11%

  • CLIC2020 (768x768 resolution, FID calculated on 128x128 patches):

    	BD-Rate (FID)↓	BD-Rate (LPIPS)↓	BD-Rate (PSNR)↓
    PerCo	0.00%	0.00%	0.00%
    DDCM	-52.20%	-53.04%	-86.23%
    OneDC	-93.37%	-85.98%	-89.16%

We will add all absolute numbers and the relative BD-rate comparisons in the revision.

3. LoRA parameter count

We appreciate the detailed observation of the reviewer and would like to clarify a misunderstanding. The base U-Net from SD1.5 indeed contains ~860M parameters. Our LoRA tuning (rank 64) adds only ~68M trainable parameters, bringing the total to ~928M during training. However, since LoRA can be merged into the base model at inference, the final U-Net can be ~860M. We will revise the paper to clarify this point.

4. Training time

Stage I is trained for 6 days, and Stage II for 12 days using 4×A100 GPUs, since we use high-resolution patches for training.

5. Comparison on model size && memory usage

We compare OneDC with PerCo and DDCM on the Kodak dataset under DDCM’s test setting (center crop and resize to 512×512, FID on 64×64 patches). The memory usage of PerCo (SD) and DDCM is calculated with their open-sourced code.

* Open-sourced PerCo (SD) includes additional 3.8B BLIP2 caption model and 340M CLIP text embedding model.

We will include the model size comparison and memory usage into our final version.

6. On training a separate model per bitrate

We appreciate the reviewer’s comment. This paper focuses on establishing an effective one-step diffusion framework for compression, so we currently use fixed-rate models for clarity and controlled analysis, like many other generative codecs [1][2][4][5]. We believe that OneDC is compatible with existing techniques for variable bitrate (e.g., bottleneck scaling[7] and adjustable quantization[8]), which we leave as future work.

7. Hyperprior Usage

The hyperprior is introduced by Ballé et al. [6] and widely adopted in methods such as HiFiC [1], MS-ILLM [2], GLC [3], and DiffEIC [4]. In OneDC, we extend its role to also provide semantic guidance for generative decoding. We will add a brief background and proper citation in the final version to improve clarity.

8. On the modification of Figure 3

Thanks for your great suggestion. We will separate $g_s$ into its own module and update Figure 3 accordingly in the final version.

9. About PerCo (SD)

We will clarify the details regarding the use of PerCo (SD) in the revised main paper.

10. On the rate–perception–distortion tradeoff

We fully agree reviewer's opinion. We will cite the rate–perception–distortion tradeoff when discussing the limitations of traditional codecs at low bitrates.

11. On missing PSNR in the main text

We will include PSNR results in the revised main paper to enable a fair comparison with pixel-domain methods like MS-ILLM does.

12. On missing citations

We will add the appropriate citations in the revision.

13. Regarding the results of fine-tuned MS-ILLM and PerCo

• MS-ILLM:

  Our paper focuses on the 0.015–0.05 bpp range on the Kodak dataset, while official MS-ILLM vlo1/vlo2 checkpoints target <0.01 bpp. To ensure a proper comparison under similar bitrate range, we fine-tuned MS-ILLM using the official code to produce models operating between 0.015–0.045 bpp. These fine-tuned models outperform the original vlo1/vlo2 checkpoints due to their higher bitrate. For bitrates above 0.045 bpp, we use the official MS-ILLM checkpoints.

  We will include BD-Rate comparisons against all official MS-ILLM checkpoints (vlo1/vlo2 and quality levels 1–4) in the revision, using OneDC extended to a broader bitrate range.

• PerCo (SD):

  Our results align with the PerCo (SD) technical report, which notes slightly degraded performance to the original PerCo (not publicly released) on the Kodak and MS-COCO 30K datasets. For results on CLIC2020 (full resolution), PerCo performs worse due to limited resolution adaptability.

  We will clarify the use of PerCo (SD) in the main paper and add the comparisons to the original PerCo using data extracted from their paper.

14. On focusing on the low bitrate range

We focus on the extremely low bitrate range because existing generative codecs often fail to preserve acceptable visual quality in this setting. It poses challenges in maintaining semantic content and synthesizing fine details, making it a key research frontier as explored in works like DiffEIC, PerCo, and GLC. Our aim is to advance compression capabilities specifically under these challenging conditions. To broaden applicability, we have also extended OneDC to cover higher bitrate range, and will include these results in the final version.

15. On adapting the model to additional bitrates

Yes, we fine-tune the extra bitrate points from the existing model, which converges faster than training from scratch.

16. On claims

Thanks for your reminders. In the revision, we will carefully modify these claims according to reviewer's comments. Specifically, we will:

• Revise our expression about bitrate reduction and the text usage of existing methods.
• Clarify the meaning of each number in tables and supplement concrete numbers.

Citations:

[1] High-fidelity generative image compression, NeurIPS 2020

[2] Improving statistical fidelity for neural image compression with implicit local likelihood models, ICML 2023

[3] Generative latent coding for ultra-low bitrate image compression, CVPR 2024

[4] Towards extreme image compression with latent feature guidance and diffusion prior, TCSVT 2024

[5] Towards image compression with perfect realism at ultra-low bitrates, ICLR 2024

[6] Variational image compression with a scale hyperprior, ICLR 2018

[7] Asymmetric gained deep image compression with continuous rate adaptation, CVPR 2021

[8] EVC: Towards Real-Time Neural Image Compression with Mask Decay, ICLR 2023

Many thanks for your valuable comments and the recognition on our contributions. We hope the following responses could address your concerns.

1. On the role and bitrate portion of $\hat{y}$

The hyperprior latent $\hat{z}$ provides high-level semantic guidance, while the main latent $\hat{y}$ preserves low-level details for reconstruction. To analyze their respective contributions, we conducted an ablation study on the CLIC2020 dataset. We fixed the bitrate for the semantic hyperprior $\hat{z}$ at 0.0035 bpp and progressively increased the bitrate allocated to the main latent $\hat{y}$.

The results demonstrate the division of roles:

• Semantic Foundation ($\hat{z}$): Even with zero bits for $\hat{y}$, the model reconstructs a coherent image (FID=14.89), confirming that $\hat{z}$ provides the core semantic structure.
• Fidelity Enhancement ($\hat{y}$): Allocating more bits to $\hat{y}$ leads to substantial improvements across all fidelity and perceptual metrics (e.g., LPIPS drops from 0.290 to 0.119). This confirms that $\hat{y}$ is crucial for encoding fine-grained details.

Thanks for your great suggestion; we will include this analysis in the revision.

2. Regarding the notation

Thanks for your great suggestion. To clarify, we will revise the notation $\tilde{y}_t, \tilde{y}_0$ to better reflect the actual data flow:

• subscript t→in : the latent decoded from the compressed bitstream, serving as the input to the one-step diffusion model.
• subscript 0→out : the denoised latent produced by the one-step diffusion model.

The new notation removes the unnecessary implication of a timestep and more clearly represents the flow of inference in our one-step framework. We will revise the figure and text accordingly in the revision.

3. Regarding how semantic context is injected via cross-attention

The semantic context $c\in R^{B \times N \times D}$, extracted from the quantized hyperprior $\hat{z}$ via a semantic decoder $h_{\text{sem}}$，is injected into the cross-attention layers of the one-step diffusion U-Net. Here $N = H \times W$ corresponds to the spatially flattened hyperprior feature map and $D$ is the embedding dimension.

In each cross-attention block, the latent feature serves as the query $Q$, while the semantic context $c$ provides the key $K$ and value $V$:

$\text{Attention}(\mathbf{Q}, \mathbf{K}, \mathbf{V}) = \text{softmax}\left( \frac{\mathbf{Q} \mathbf{K}^\top}{\sqrt{D}} \right) \mathbf{V}$

This design allows every spatial location in the latent to reference relevant semantic context adaptively, improving reconstruction quality.

Thanks for your great suggestion, we will revise the paper to clarify this process in the final version.

4. Open-sourcing

We fully agree with the reviewer's opinion on open-sourcing. We make the commitment that we will release the both training and testing code for our paper.

5. Minor issues

Thank you for these kind reminders.

• We will revise Fig. 4 to explicitly show the injection of semantic context $c$ into the one-step U-Net for clarity.

• We will revise Eq. (1) to use $P_{aux}(c)$ directly, avoiding the potentially misleading use of “logits.”

• We will include the “w/o second-stage training” setting in Table 1 to clarify the impact of hybrid-domain fine-tuning.

Many thanks for your valuable comments. We hope that the following responses can address your concerns.

1. Complexity of Architecture & Training

Our design philosophy balances performance with simplicity. Actually, many recent generative codecs require sophisticated design, but we have streamlined our architecture and training to be more integrated.

• Architectural Simplicity: Our OneDC integrates a one-step diffusion generator into a standard hyperprior codec framework. This design is self-contained and avoids the heavy external dependencies or complex add-ons seen in other methods. For instance:
  • PerCo [1] relies on large, separate vision-language (BLIP2) and text-embedding (CLIP) models for conditioning.
  • DiffEIC [2] incorporates a ControlNet and a custom Latent Feature-Guided Compression Module, increasing structural complexity.
  • DiffC [3] uses reverse-channel coding over many steps, requiring specialized CUDA kernels and complex scheduling.

In contrast, OneDC replaces external conditioning with a simple yet semantic-aware hyperprior.

• Training Strategy: We adopt a two-stage training strategy, a well-established and effective paradigm used by leading methods like HiFiC[4] / MS-ILLM[5].
  • Stage I focuses on learning the compression part.
  • Stage II refines the decoding for perceptual realism.

This principled approach avoids the more intricate multi-stage pipelines of other models like GLC[6], which involves a three-stage process of training a VQGAN, transform coding, and then end-to-end fine-tuning.

We believe our approach represents a proper trade-off between innovation and practicality. We will release our full training code to ensure transparency.

2. Novelty & comparison

We appreciate the opportunity to clarify our contributions. The primary novelty lies in the design of a new codec architecture that successfully integrates a one-step diffusion generator with a hyperprior-based latent compressor. This unified design achieves SOTA performance while being significantly more efficient and self-contained than previous diffusion-based codecs.

Our key technical contributions that enable this framework are:

• Hyperprior as Semantic Guidance: Unlike some prior works that rely on heavy external models (e.g., BLIP2, CLIP) for text-based conditioning, we are the first to leverage the hyperprior as a direct source of semantic guidance for the diffusion model. This approach is not only more efficient but also provides spatially-aware context that is better aligned with the image content, eliminating the need for textual prompts.

• Semantic distillation: To tap the potential of the hyperprior for its new role, we introduce a semantic distillation mechanism. During training, we distill knowledge from a powerful pretrained generative tokenizer into the hyperprior branch. This novel step enriches the hyperprior’s semantic representation, making it an effective conditioning signal for the generator.

• Integrated Architecture and Training: Our framework is designed for simplicity and reproducibility.

  • Architecture: While methods like DiffEIC or PerCo bolt on complex modules like ControlNet or large language models, our design is a self-contained system. It replaces complex conditioning with a learnable, integrated hyperprior.
  • Training: We adopt a principled two-stage training strategy (similar to HiFiC/MS-ILLM) that first establishes a robust latent representation and then refines for perceptual realism. This hybrid-domain approach (pixel and latent) ensures stable optimization and balances fidelity with realism, avoiding the more complex multi-stage pipelines of other generative codecs.

In summary, while we build upon established concepts like hyperpriors and diffusion models, our novelty stems from their synergistic integration into a new, streamlined framework specifically designed for efficient, high-fidelity, one-step generative compression.

3. Model size & parameter

We provide parameter counts, runtime comparisons (on 1024×1024 images), and BD-Rate results on the MS-COCO 30K dataset:

* Open-sourced PerCo (SD) includes additional 3.8B BLIP2 caption model and 340M CLIP text embedding model.

0.0 in BD-Rate means we use the OneDC as anchor for comparison. See our response for question 8 for more details.

Compared to MS-ILLM, diffusion-based methods typically use larger models but achieve superior perceptual quality (e.g., lower BD-Rate with FID) due to stronger generative capacity. Unlike other diffusion-based codecs, OneDC avoids external caption models and multi-step sampling, enabling over 20× faster decoding (0.34 s vs. 6.9~12.4 s) while also achieving better rate-distortion performance. We will include these parameter and runtime comparisons in the revision to support the claimed efficiency.

4. Equation input

Thanks for your suggestions. We will clarify the input of each equation in the revision.

5. Choice of Scalar quantization (SQ) / finite scalar quantization (FSQ)

Our model uses FSQ for the hyperprior branch and standard SQ for the main latent branch. This design is intentional and crucial for balancing semantic representation and reconstruction fidelity.

• FSQ for Hyperprior (Semantic Representation): The hyperprior is designed to capture high-level semantic information. FSQ, an improved variant of vector quantization, excels at learning a discrete, categorical representation from a large codebook. This makes it ideal for modeling the semantic-rich features in the hyperprior, providing effective guidance for the generative model.

• SQ for Main Latent (Low-Level Detail Preservation): The main latent must preserve fine-grained visual details for high-fidelity reconstruction. SQ provides a larger, more continuous-like quantization space that is better suited for this task, minimizing information loss for low-level textures and structures.

To validate it, we compare our approach against two alternatives: using SQ for both branches (Dual SQ) and FSQ for both branches (Dual FSQ). All models, including the compared OneDC, are trained from scratch under the same training duration in Stage I for fair comparison. The results below show performance degradation (BD-Rate increase) relative to our OneDC model.

The results confirm our hypothesis:

• Using SQ for the hyperprior (Dual SQ) weakens its semantic modeling capability, leading to a significant drop in perceptual quality (higher BD-Rate with FID).
• Using FSQ for the main latent (Dual FSQ) harms its ability to preserve fine details, resulting in a substantial loss of fidelity (higher BD-Rate with DISTS).

It verifies our hybrid quantization strategy, which effectively assigns the proper quantization method to each branch based on its specific role.

6. Connection between Fig. 3~4 & symbol definitions

The two figures are complementary and serve different purposes:

• Fig. 3 shows the inference pipeline about the interaction between latent compression module and the one-step diffusion generator. It highlights two key points:

  • The integration of a latent compression module and a one-step diffusion model.

  • The use of hyperprior-based semantic guidance instead of text-based conditioning for the diffusion U-Net.

• Fig. 4 presents the training pipeline, detailing the two-stage process and indicating which components are updated or used only during training:

  • Stage I trains the compression module and the one-step diffusion model for reconstruction, with a distillation mechanism to enhance the hyperprior’s semantic representation.

  • Stage II fine-tunes the one-step diffusion model using latent-domain diffusion distillation to improve generation realism, combined with a pixel-domain loss to preserve fidelity.

We will improve cross-referencing between the figures and explicitly define all symbols in the revision.

7. Input during inference

8. The 0.0 value in Table 1 / 2

In Table 1 / 2, we report Bjontegaard Delta-Rate (BD-Rate)[7] to reflect bitrate change under a specific quality metric when compared the anchor method. A negative BD-Rate value means bitrate savings over the anchor method, while a positive value means bitrate increase. The BD-Rate value over anchor itself is 0.0%.

In our paper, we use the final OneDC model as the anchor. Accordingly:

• In Table 1, BD-Rate reflects performance degradation when individual components of OneDC are ablated.
• In Table 2, BD-Rate reflects the degradation of other codecs compared to OneDC.

In the revision, we will clarify it and annotate the table headers as “BD-Rate (Quality Metric)” for improved clarity.

[1] Towards image compression with perfect realism at ultra-low bitrates, ICLR 2024.

[2] Towards extreme image compression with latent feature guidance and diffusion prior, TIP 2024.

[3] Lossy compression with pretrained diffusion models, ICLR 2025.

[4] High-fidelity generative image compression, NeurIPS 2020.

[5] Improving statistical fidelity for neural image compression with implicit local likelihood models, ICML 2023.

[6] Generative latent coding for ultra-low bitrate image compression, CVPR 2024.

[7] Calculation of average PSNR differences between RD-curves, VCEG-M33.

Many thanks for your valuable suggestions and the recognition on our contributions. We hope that the following responses can address your comments.

1. Discussion of training time

During training, image patches of size {512×512, 1024×1024} are randomly cropped with probabilities {0.6, 0.4} to enhance generalization to high-resolution inputs. We use a two-stage training strategy and 4×A100 80G GPUs to train our models.

• Stage I trains the latent compression module and adapts it to the pre-trained one-step diffusion model for the reconstruction. It takes approximately 6 days as we use high-resolution patches for training.

• Stage II further fine-tunes the one-step diffusion model to enhance the realism via latent-domain distillation and preserves fidelity through pixel-domain supervision. This stage requires around 12 days, as diffusion distillation introduces additional computational cost in addition to high-resolution training.

Thanks for your great suggestion. We will add these details into our revision, and will also release our training code for reproducibility. In addition, we will explore accelerated training strategies, such as the improved distillation method, to reduce training cost in the future.

2. Full fine-tuning vs. LoRA

We conducted additional experiments using full fine-tuning (FT) starting from the λ = 7.4 checkpoint originally used in our LoRA-based model. We denote this variant as LoRA + FT in below table. Owing to the significantly slower convergence of full fine-tuning compared to LoRA, we were limited to roughly 15% of the Stage II training iterations during the rebuttal period.

The preliminary results on the CLIC2020 dataset are below:

These early findings suggest that full fine-tuning can yield some improvements over LoRA under limited training conditions. In future work, we plan to explore more efficient training methods and further improve generation performance.