De-biasing Diffusion: Data-Free FP8 Quantization of Text-to-Image Models with Billions of Parameters

The primary similarity metric used by the authors for calibration is PSNR. They cite the SDXL [2] paper which states that FID has limitations due to Mean-Opinion-Scores. This is usually because the generates images are compared to real images (FID-GT). However, [3] propose FID-FP32, a variant of FID which compares the distribution of generated images from the quantized model, against those of the full precision model (which is treated as the baseline). If FID-32 is 0, the distribution of images from the quantized model is the same as that of the full precision model. Since this metric discards the weaknesses of the original FID, it is arguably a better metric for the objective than PSNR for what the authors aim to achieve, yet this paper does not seem to be aware of it.

Many thanks for this comment. We were not aware of this metric (It is a 2025 publication), and have gladly added it to the paper. This is a neat way to measure similarity while focusing on key features, as perceived by the Inception model. When evaluated over our COCO images (30K each), using pytorch-fid (dims=2048), the results are as follow:

The new results were added to the table in Figure 5 (d), next to the SSIM results. FID-FP32 is explained (with reference to the QDiffBench paper), in appendix E (renamed to: “Similarity Measurements via SSIM and FID-32”). Note the TensorRT results have a different FID baseline. We also used pytorch metrics to corroborate these values.

As expected, the FID-FP32 values are relatively small: The datasets we compare have much stronger likelihood than what we usually measure with FID-GT. In regard to the methods proposed by this paper, the FID-FP32 is consistent with the PSNR/SSIM scores: SR and WSR are beneficial for $T>20$, with WSR achieving almost as good a score as a naive bf16 quantization.

Most of the experimental results generate too few images (100 or 1k, e.g., Fig 1, Fig 3, Tab 2) to provide trustworthy and meaningful results.

The MS-COCO experiment uses 30K images (per FID value), not 1K. Other experiments used $100$ images (scanned over 10 different values of T), but the results were robust (i.e. the results did not significantly change with more images), as indicated by the very low p-values (Figure 9). Specifically, our methods show a clear statistically significant benefit ($p<0.05$) for each number of diffusion steps (except for $T=20$), which indicates we used enough images to verify our advantage.

Line 463-464 "the minimal recommended size" this is incorrect. See https://huggingface.co/docs/diffusers/en/using-diffusers/sdxl which states "anything below 512x512 is not likely to work".

These statements are not contradictory: 512x512 is likely to work, but it is not recommended, and will result in lower quality images (Special models were fine-tuned to address this domain [1]). However, we agree that the term “minimal” is problematic in relation to two-dimensional sizes and have removed this claim.

[1] https://huggingface.co/hotshotco/SDXL-512

The paper focuses on fp8 quantization, where performance drops relative to uncompressed model are not very pronounced. Would be interesting to see whether the conclusions are valid for the case of more aggressive quantization (say W4/A8) - i.e one should focus on reducing the bias, rather than the variance term. I suggest having at least one experiments with stronger compression to showcase the difference between RTN performance and SR.

Please note that the improvements we show in FP8 quantization are significant, as indicated by the very low p-values (Figure 9). Moreover, while it is correct that these improvements are mostly in the fine details of the output image, these details are important to the users. Diffusion models are tasked with producing art. Thus, a quantized model producing an image that is 'slightly blurry’ or `mostly similar to the original image’ is typically not good enough for real-world applications. Closing the gap for lower numerical precision might be more impressive in numerical terms, but this is not the focus of this work.

Specifically, for W4/A8, we were not able to get good results with 4-bit weights in our highly restrictive data-free setting. Doing so will most likely require the relaxation of some of our guiding principles, by using data for calibration and/or mixed-precision, which have significant implicit costs, as explained in our paper.

Noticeable decrease of FID metric relative to uncompressed model given high PSNR (small difference between the output of original and quantized model) looks suspicious. Do you have any explanation for this phenomenon? Do you adopt the same random seed for generation with compressed and original model? How robust is the FID value to the change of seed?

We agree that this is odd, but we did not find any good explanation for this phenomenon, except attributing it to the shortcomings of FID (especially when using a refiner). The FID value changes by $\pm 1$ when using different seeds (based on limited prior experiments), but we made sure to use the same seed in all experiments (this is mandatory for the SSIM/ PSNR evaluation).

What is the sampler used for image generation? Is it the default sampler in diffusers config or DDIM?

We used the default sampler (Euler) for all experiments in the main section. In appendix L, we also included results for UniPC and Heun, to measure the robustness of our method to changing hyperparameters. For Stable Diffusion 1.4 (Appendix G) we used PLMS.

The overall novelty is limited. Stochastic rounding is an established technique in various applications [1, 2] and the current work adopts it for diffusion models.

The main novelty of our paper is pinpointing quantization bias as the key issue in diffusion inference tasks, and showing the implications of this. Stochastic rounding (SR) is a tool for reducing bias, but it increases the quantization error (MSE), so in many cases using SR actually degrades the accuracy. For example, this happens often for weight and activations quantization in the forward pass in classification models (e.g. see Fig 1b in Chmiel et al. 2022, cited in our paper). Thus, the fact that SR has been used for other, vastly different applications (e.g., [1,2] quantized the gradient-based training process of image classification models) does not contradict the novelty of our findings. We have explicitly used stochastic rounding because it is a well-established method, allowing us to benefit from the already existing implementation in software and hardware.

The evaluation setup is not very compelling. Unless the drop of generation quality is very significant, metrics such as FID do not reflect it. PSNR/SSIM between the generations produced by the original model and quantized one may be not a proper measure, as a compressed model can generate samples different from the teacher, yet of high quality. I would suggest adding user preference study at least for some cases as a reliable assessment. Specifically, given two images an assesor should tell, whether one image is better than other (or both are equal in terms of quality).

As a quantization paper, our main goal was to produce models that make as little as possible changes to the non-quantized model’s output, with the underlying assumption that the output of the non-quantized model was good enough for the user. Having high PSNR images (high-similarity), means we are close to the original images, with the FID giving us some assurance that the changes maintain a reasonable coverage of the distribution. On the other hand, low PSNR images would mean that we have strayed far away from the original model, even if the quality of the model has somehow remained high. We note that none of the quantization studies we cited and made comparisons with have included a human-study.

The main challenge of user-study (apart from the direct financial cost), is that there is no clear candidate for comparison. Comparing naive quantization with SR+WSR may not be convincing enough. At the same time, few among the methods suggested by other papers offer official solutions that extend to SDXL (or beyond), and the ones we have checked are quite inflexible and are not sufficiently competitive to justify a user-study (Appendix A, G). Assuming a side-by-side user comparison is possible, what methods would the reviewer like to see compared? (and in which settings?)

WSR(p=4) increases amount of memory used for model storage by 50% relative to fp8. I would suggest to check whether mixed fp8/fp16 quantization is still inferior in terms of quality.

Our main reason for picking $p=4$ is that we wanted to err on the side of caution (The 30K COCO experiment can be quite long), and consider the weight memory cost to be a relatively minor concern (please see ``general comment’’ on this).

In response to the reviews, we initiated an experiment with lower values of $p$, with positive preliminary results. For $p=2 / 4, T=50$, we measured an FID of $18.84 / 18.88$ and a PSNR of $26.15, 26.07$ respectively, which is very similar to our result for $p=4$, and can help reduce the memory requirement by an additional 750MB/ 1GB. However, we stress again that for modern architectures, most of the on-device memory is reserved for activations, which can be fully quantized when using stochastic weights and standard 8-bit GeMM kernel, regardless of the value of $p$.

There is no-doubt that using mixed-precision can improve the results. However, the cost of changing between data-types is non-trivial and would be better handled by practical implementations that correspond to specific hardware accelerators.

Practical applicability of some of the proposed schemes. In the current form, only the stochastic rounding of activations can be deployed on devices; any other schemes need significant effort in software/hardware support. SR itself gives measurable improvements, but not as good as SR+WSR (based on table results)

If the reviewer’s main concern is the reduction of on-device memory, our methods (including WSR) can be used as is to produce better quality images than any other 8-bit quantized model, since software implementation and high values of $T$ may be slow, but will not cost additional memory. Also, please see ``general comment’’.

As for run-time, we do generally agree with this limitation with respect to current hardware, but believe that the main strength of the paper is the potential contribution to (near) future hardware. While no hardware implementation can be described as insignificant, Stochastic-Weights are far from being impractical: The operation’s complexity is linear with the size of weights, and can be performed preemptively with no dependency on the GeMM computation engine (e.g. Tensor Cores, in the case of Nvidia). We even made sure to use operations that have a smaller gate-count (this is why we used integer-increment, rather than addition). Modern AI accelerators include far more complex operations that offer significantly less benefit than what we proposed in this paper.

Assumptions on #diffusion steps used throughout manuscript (i.e., steps > 50) are unrealistic in my opinion simply due to the nature of discussion: if we are talking about efficiency and on-device deployment, such models are most probably will be deployed in few step regime

We acknowledge that there is some contradiction between long-inference tasks and the ambition to accelerate inference. That being said, our main results for both SDXL and FLUX have shown significant benefits for 50 steps diffusion, which we consider to be very important (if not the most important) for current applications, based on the default values, tutorials, and public use-cases [1]. Higher values are indeed less practical, but they are the best solution if the main concern is on-device-memory rather than run-time, since T=200 processes require the same amount of memory as T=30 runs. Also, in the context of our work, the behaviour of long diffusion processes is informative: Equation 7, the result of our theoretical analysis, has an exponential dependency on the number of steps.

Most prior works have focused on short/very short inference tasks, which we found to be a little disingenuous: they have specifically targeted easier quantization tasks with easier baselines (shorter processes have higher FID), with little to no acknowledgment of this limitation. We would also like to highlight the importance of the robustness of our method: Our networks can be used as is for shorter inference tasks, even though the benefit would be small. In contrast, existing methods, such as TensorRT, fail completely for a normal number of steps (Appendix A), even when calibrated.

p=4 for WSR scheme might be too big (in my opinion) for any practical deployment despite the shown advantage: wsr with p=4 means 12bit storage for every weight tensor, as such, there might be alternative allocation or scheme for same 12bit per weight storage. I think, for any WSR benefits, authors should try coming up with less taxing scheme on additional storage.

Our main reason for picking p=4 is that we wanted to err on the side of caution (as the 30K COCO experiment), and consider the weight memory cost to be a relatively minor concern. (See: general comment)

In response to the reviews, we initiated an experiment with lower values of $p$, with positive preliminary results. For $p=2 / 1, T=50$, we measured an FID of $18.84 / 18.88$ and a PSNR of $26.15, 26.07$ respectively, which is very similar to our result for $p=4$, and can help reduce the memory requirement by an additional 750MB/ 1GB. However, we stress again that for modern architectures, the main benefit of WSR is enabling 8bit$\times$8bit multiplication, and not the direct reduction in weight-memory, as we explained in the ``general comment’’.

User study. I think, having user study on perceived quality difference across proposed techniques would significantly strengthen the conclusions of the paper.

The main challenge of user-study (apart from the direct cost), is that there is no clear candidate for comparison. Comparing naive quantization with SR+WSR may not be convincing enough. At the same time, few of the feasible quantization methods suggested by other papers offer official solutions that extend to SDXL (or beyond), and the ones we have checked are quite inflexible and are not sufficiently competitive to justify a user-study (Appendix A). Assuming a side-by-side user comparison is possible, what methods would the reviewer like to see compared? (and in which settings?).

[1] https://prompthero.com

for a fair comparison, I suggest that the authors compare their method to W(8+p)A8 baselines. It is possible that the observed improvement primarily results from the additional weight bits.

Here are the results (PSNR scores) for W(8+p)A8 quantization, next to the matching WSR scores:

To be clear, we did expect W(8+p)A8 quantization to be (slightly) better than stochastic rounding, since it will have the same bias as stochastic rounding while also benefiting from smaller quantization noise. The downside (in comparison with WSR) is that the W(8+p)A8 format requires more complex, unbalanced computation kernels that are less likely to exist in any hardware and would be more expensive if they were implemented. In contrast, WSR enables matrix multiplication of FP8 formats, which is already implemented in some common accelerators. See also ``general comment’’.

Presentation

We thank the reviewer for the detailed comments. We addressed most of the comments in the revised paper.

A new figure with a visualized flow describing our methods was added as Figure 14. We would appreciate the reviewers’ feedback on whether it helps understand the paper.

We have not added visual examples to the main paper yet and will do so according to the available space in the final revision of the paper (but will make sure to have at least a single row). The examples we will use will match the images in the supplementary materials of this paper.

Improving figures 1,2 and 3 is more challenging. We agree that the information in these figures is dense, but we regard most of it as necessary (also, the other reviews have commented positively on them).

Questions

The concept of stochastic weights is unclear to me. Are the weights stored using bits while computations are still performed with 8-bit precision?

Correct. Stochastic weights essentially use the extra “p” bits for de-biasing. The computation will still be performed over 8-bit values (with higher quantization error). It works well because, according to our main argument in the paper, bias is more important than absolute error. We also leverage the fact that a common denoiser neural network has a lot of weight sharing. Consequently, the size of the activations can easily exceed the size of the weights by an order of magnitude (depending on batch size and image resolution). This means that the memory reserved for the $p$ additional bits is much smaller than the memory we “saved” by enabling 8-bit quantization for the activations. See also ``general comment’’.

In the paper, the authors apply their methods only to the 50-step FLUX.1-dev model. I wonder if the proposed method is also effective on the 4-step FLUX.1-schnell model.

Based on our theoretical analysis, we don’t recommend our method for turbo settings (<10 steps). From equation 7, the advantage of our method grows exponentially with the number of steps: with T=4, we will gain little from the reduction of the bias, while paying the full toll for the added quantization noise.

We thank the reviewer for their time and feedback.

We have updated the paper to discuss the differences between stochastic rounding and adaptive methods (AdaRound, AdaQuant or BREC-Q), after introducing Stochastic rounding.

Please note that we have made direct [Appendix G] and indirect [Appendix A, via TensorRT] comparisons with Diffusion-Q, which uses block-wise Adaptive Rounding (Calibrated Per-Timestamp).