Zero-shot Denoising via Neural Compression: Theoretical and algorithmic framework

We appreciate the reviewer’s positive feedback.

We thank the reviewer for their valuable feedback. We first address the specific questions raised, followed by a response to the points noted in the weaknesses section. In our response to Question 1, we also address some of the comments regarding the choice of settings in the theoretical results, as mentioned in the Strengths And Weaknesses.

Q1. Tightness of bounds in Section 3.

Our analysis is framed in a deterministic, high-probability setting: we provide uniform high-probability guarantees on reconstruction error for signals in a fixed class, without assuming any prior distribution. This framework is common in the information-theoretic literature on compression-based inference (e.g., compression-based compressed sensing), and it offers practical advantages, particularly in the zero-shot denoising scenario where the underlying data distribution may be unknown or difficult to model. The deterministic nature of our bounds ensures they hold uniformly over the signal class, rather than in expectation.

Regarding the tightness of the bounds, in the discussion following Corollary 1, we have compared our result with the optimal Bayesian performance (in terms of information dimension). In the revised version, we will also include a comparison to known minimax rates for $k$-sparse signal recovery under Gaussian noise. For the case $k=k_n$, such that $k_n/n\to 0$, as $n$ grows without bound, the minimax risk in known to scale as ${\sigma_z^2 k \log(n/k) \over n}$ [Johnstone2019] [Donoho1994] [Donoho1992]. This further confirms the tightness of our bound in the worst-case setting, and potential optimality of compression-based compressed sensing.

Finally, while in Table 1 we report average denoising performance, our method is designed to perform well on each individual instance, consistent with the worst-case theoretical guarantees we provide. This aligns with the goals of zero-shot denoising, where robustness to the specific realization is more critical than expected-case optimality.

[Johnstone2019] Johnstone, I. M. Gaussian Estimation: Sequence and Wavelet Models.

[Donoho1994] Donoho, D. L., et al. Minimax risk over $\ell_p$ balls for $\ell_q$ losses.

[Donoho1992] Donoho, D. L., et al. Maximum entropy and the nearly black object.

Q2. Selecting $\lambda$ based on theoretical upper bounds.

Theorem 1 provides a non-asymptotic upper bound on the reconstruction error, assuming that a compression code with a given rate $R$ and distortion $\delta$ is available. The constants appearing in the bound (such as $(8 \ln 2)(1 + 2\sqrt{\eta})^2$) characterize the relationship between rate and MSE, but they are not intended to prescribe a specific choice of the Lagrange multiplier $\lambda$ used in training.

Indeed, in principle, one could consider minimizing the upper bound in Theorem 1 as a function of $\delta$ (or $R$), which in turn could suggest an “optimal” rate-distortion operating point for denoising. However, solving such an optimization is highly non-trivial. Specifically, note that the upper bound in Theorem 1 can be written as

\frac{1}{2\sqrt{\delta}} + \sigma_z \zeta \cdot \frac{R'(\delta)}{2\sqrt{R(\delta)}} = 0. $$However, since the function $R(\delta)$ is generally not known in closed form for real-world data distributions or learned compressors, this optimization cannot be carried out analytically or even tractably in practice.

In contrast, during training, $\lambda$ is used as a Lagrange multiplier in a rate-distortion objective to implicitly select an operating point on the empirical RD curve of the single image. This RD curve is not analytically accessible and varies from image to image, making the mapping from $\lambda$ to $(R, \delta)$ nontrivial and data-dependent.

Our proposed heuristic, choosing $\lambda$ such that the distortion roughly matches the noise level, is based on signal recovery intuition and performs well empirically, as demonstrated in our experiments and in Section 4, "Setting the hyperparameter $\lambda$".

Q3. Seemingly surprising effectiveness of compression at denoising.

This is an insightful and intuitive question. While it may seem surprising that a compressor, which is not explicitly trained to denoise, can outperform methods designed specifically for denoising, the idea of using compression codes to solve inverse problems such as denoising and compressed sensing has a long history in information theory and signal processing. Prior work has shown that lossy compression codes can be used as a powerful implicit prior for solving inverse problems (see, e.g., [Donoho2002, Weissman2005, Jalali2016, Rezagah2017, Chang1997, Chang2000, Natarajan2002]).

In the zero-shot setting, where only a single noisy observation is available and no clean data is accessible, most learning-based denoisers rely on indirect forms of regularization to prevent overfitting. These include masking strategies (ZS-N2S), architectural bias (DIP), or underparameterized networks (Deep Decoder). While useful, these techniques are often heuristic and do not provide a principled way to control information flow from input to output.

In contrast, compression-based denoising imposes a fundamental information-theoretic constraint via the rate-distortion trade-off. This restricts the number of bits that can pass through the model's bottleneck, acting as a principled regularizer that prevents the model from memorizing noise. The model is therefore implicitly biased toward compressible (structured) reconstructions, which aligns well with the nature of clean signals.

Although SURE provides an unbiased estimate of the MSE under Gaussian noise with known variance, its effectiveness depends on accurate noise modeling and stable estimation of the divergence term. In practice, especially in the zero-shot setting with a single noisy image and no access to clean data, these conditions may not hold. Moreover, SURE implementations often rely on Monte Carlo approximations of the divergence, which can introduce significant variance, particularly in deep neural networks.

Thus, while traditional denoising methods are explicitly designed for the task, compression-based approaches like ZS-NCD offer a robust and principled alternative in data-scarce regimes, combining inductive bias with effective optimization.

[Chang1997] Chang, S. G., et al. Image denoising via lossy compression and wavelet thresholding.

[Chang2000] Chang, S. G., et al. Adaptive wavelet thresholding for image denoising and compression.

[Donoho2002] Donoho, D. L. The kolmogorov sampler.

[Natarajan2002] Natarajan, B. K. Filtering random noise from deterministic signals via data compression.

[Weissman2005] Weissman, T., et al. The empirical distribution of rate-constrained source codes.

[Jalali2016] Jalali, S., et al. From compression to compressed sensing.

[Rezagah2017] Rezagah, F. E., et al. Compression-based compressed sensing.

[Soltanayev2018] Soltanayev, S., et al. Training deep learning based denoisers without ground truth data.

[Metzler2018] Metzler, C. A., et al. Unsupervised learning with stein's unbiased risk estimator.

W1. Better bound in Theorem 3 over Theorem 2.

Note that while the bound in Theorem 2 is on MSE, the bound in Theorem 3 is on RMSE. To compare the results of the two theorems, we can square the bound in Theorem 3, which shows that while the MSE in Theorem 2 scales as $\sqrt{R/n}$, the MSE in Theorem 3 scales as $R^2/n$. This suggests that, unlike in Theorem 2, for the bound in Theorem 3 to be small, the rate $R$ cannot scale linearly with $n$. Therefore, minimizing the log-likelihood over a compression code should, in principle, yield better performance under Poisson noise. However, due to the highly non-convex nature of the log-likelihood loss, minimizing the MSE loss may lead to better performance in practice. A brief discussion of this observation, along with empirical results, is provided in Appendix B.2.

W2. Two concurrent zero-shot denoisers.

We thank the reviewer for bringing these results to our attention. The recent works advance the ZS-N2N framework. We will cite both references in the Related Work Section (Section 2) of our manuscript. Additionally, we have tested DS-N2N and Pixel2Pixel on the Kodak24 dataset under AWGN using the public code provided by the authors. Average PSNR/SSIM over the 24 images are reported in the table below.

[Bai2025] DS-N2N: Bai, J., et al. Dual-sampling noise2noise: Efficient single image denoising.

[Ma2025] Pixel2Pixel: Ma, Q., et al. Pixel2Pixel: A Pixelwise Approach for Zero-Shot Single Image Denoising.

W3. Patch size in ZS-NCD.

We thank the reviewer for pointing this out. We indeed use a patch size of $8 \times 8$ in all experiments and will clearly state this in the revised version of the paper.

Empirically, we observe that ZS-NCD is fairly robust to patch size, as long as the size is within a reasonable range, i.e., large enough to contain meaningful structure, but small enough to ensure enough training data per image.

To support this, we include below denoising results of ZS-NCD for different patch sizes, reported as PSNR (dB)/SSIM, on the kodim05 image from the Kodak24 dataset under AWGN.

We thank the reviewer for their valuable feedback. We first address the specific questions raised, followed by a response to the points noted in the weaknesses section.

Q1. Will it significantly degrade the quality of clean images?

ZS-NCD learns a lossy compression code using the patches extracted from the input noisy image. When the input is a noisy image, enforcing a rate-distortion trade-off via a penalty on the rate effectively suppresses noise and yield a denoised image. The denoising performance depends on the rate constraint. To achieve the best performance, one needs to set the rate penalty such that the operating rate-distortion point is consistent with the noise level. If the input image is a clean noise-free image, the output may or may not be distorted depending on the rate penalty. In the case of no rate penalty ($\lambda = 0$), ZS-NCD is able to recover an almost lossless reconstruction of the clean image, provided the network bottleneck has sufficient capacity for the required entropy (as distortion goes to zero, required rate grows without bound).

This contrasts with methods like ZS-N2N, ZS-N2S and S2S, which inherently cannot recover a clean image due to disjoint-pixel training strategies or masking.

To further clarify this point, we empirically evaluated different methods on a clean image (kodim05 from Kodak24 dataset). The results, reported as PSNR (dB)/SSIM, are shown in the following table. $n_b$ denotes the dimension of latent code in the bottleneck of ZS-NCD.

Q2. Is the computational cost acceptable since a model has to be trained for each individual image?

Training a model per noisy image is an inherent aspect of zero-shot learning-based denoisers, which assume no access to training datasets or noisy/clean image pairs. This is true for all zero-shot learning-based denoisers, and not just ZS-NCD. This implies that compared to semi-supervised and supervised methods, such algorithms require a higher computational complexity during test time. However, the trade-off is justified in cases where access to data is very limited. This is true in various biomedical imaging and scientific imaging applications.

We provide timing and hardware details in Appendix C (lines 638–643). Note that ZS-NCD has not yet been optimized for speed. We believe that there is considerable room for reducing the training time through more efficient architecture designs, weight reuse and adaptive training. The goal of this work is to both empirically and theoretically demonstrate the promise of compression-based zero-shot learning-based denoising, as a novel framework.

Q3. Does a model trained on a specific image possess generalizability? If not, would it be impossible to complete the denoising task for a large number of images within a reasonable timeframe?

This is an interesting question. The notion of generalizability is different in zero-shot learning-based denoisers compared to supervised methods. In zero-shot settings, the model is trained and evaluated on a single noisy image. Therefore, generalization beyond the observed image is not a goal. However, exploring generalization across images is an intriguing question. Zero-shot denoising methods, such as DIP and deep decoder, are image-dependent and fail completely when applied to an unseen noisy image. On the other hand, ZS-NCD, due to its compression-based structure and the fact that it is trained on image patches, shows promising generalizability when trained on a noisy image and applied to another unseen noisy image.

To empirically verify this point, in the following two tables, we report the performance of ZS-NCD trained on a noisy image and used to denoise another image, for two different noise levels ($\sigma=15$ and $\sigma=50$). In each case, we also report the performance achieved when ZS-NCD is used in its original form, i.e., using the same training and test image. It can be observed that, despite expected performance drop, still ZS-NCD shows reasonable and promising generalization. kodim01 and kodim05 are chosen from Kodak24 dataset.

W1. More recent baselines comparison.

The most recent state-of-the-art learning-based zero-shot denoiser we were aware of at the time of submission was ZS-N2N [Mansour2023], published in 2023. ZS-N2N is included in our comparisons as reported for example in Table 1 of the manuscript. We have since become aware of two concurrent zero-shot denoising methods: DS-N2N [Bai2025] and Pixel2Pixel [Ma2025], pointed to us by Reviewer FvQP. Both algorithms are variations of ZS-N2N [Mansour2023].

In the revised version of our manuscript we will cite both papers in the Related Work section. Additionally, we have tested DS-N2N and Pixel2Pixel on the Kodak24 dataset using the public code provided by the authors on our own machines, in line with our policy of reproducing all denoisers performances under the same condition for fairness. In the table below, we compare the performance of our proposed ZS-NCD under AWGN. The table reports the average PSNR (dB) and SSIM over the 24 images in Kodak24 dataset.

[Mansour2023] ZS-N2N: Mansour, Y., & Heckel, R. (2023). Zero-shot noise2noise: Efficient image denoising without any data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 14018-14027).

[Bai2025] DS-N2N: Bai, J., Zhu, D., & Chen, M. (2025). Dual-sampling noise2noise: Efficient single image denoising. IEEE Transactions on Instrumentation and Measurement.

[Ma2025] Pixel2Pixel: Ma, Q., Jiang, J., Zhou, X., Liang, P., Liu, X., & Ma, J. (2025). Pixel2Pixel: A Pixelwise Approach for Zero-Shot Single Image Denoising. IEEE Transactions on Pattern Analysis and Machine Intelligence.

W2. Clarification on performance claims of ZS-NCD.

The claim refers to ZS-NCD's performance among learning-based zero-shot denoisers. To ensure that the wording of the claim is clear, in the revised version, we will emphasize that the comparison is within learning-based zero-shot denoising methods. Specifically, we will update that sentence as follows: ZS-NCD achieves state-of-the performance among learning-based zero-shot denoisers for both Gaussian and Poisson noise, as supported by the empirical results reported in Table 1 and Table 2 of the manuscript. In both tables, we have included the performance of BM3D as a strong classical non-learning-based baseline to provide broader context.

W3. Clarification on overfitting claims.

The statement in the abstract is further elaborated in Section 5, ``Robustness to Overfitting'' (lines 282–290) and is empirically supported by Figure 2. Specifically, ZS-NCD leverages the entropy constraint of the compression-based architecture, which inherently limits amount of information that can pass through the bottleneck. As a result, the model avoids overfitting, despite being trained on all noisy pixels. This is a key advantage of zero-shot compression-based denoising compared to other zero-shot approaches.

This claim is further verified in Figure 2, which shows the PSNR of the denoised image over training iterations. It can be observed that the PSNR of ZS-NCD improves steadily and does not show degradation that is typical to overfitting. In contrast, in other zero-shot denoising methods overfitting is a known issue and is typically mitigated manually. For example,

• DIP and Deep Decoder rely on manual early stopping or under-parameterization.
• ZS-N2S, S2S and ZS-N2N modify the loss function to exclude some pixels, which is another form of manual regularization.

In contrast, ZS-NCD does not require any of these manual heuristics and architectural constrains. In our approach, overfitting is prevented naturally by the compression-based bottleneck that does not allow the model to reproduce the high-entropy noisy input image. We believe that this is a key advantage of our method.

Paper formatting concerns.

We will update the manuscript to address all the formatting issues and have detailed explanations for the figures in the appendix.

We thank the reviewer for recognizing the contribution of our work and deciding to raise the rating. Regarding generalizability, as we explained before, it is out of the scope of the zero-shot denoising problem. However, to address the reviewer’s comment, in our initial rebuttal we provided some simulation results that explored the generalization behavior of ZS-NCD by training the model on a noisy image and testing it on a different image.

In the simulation results presented in our initial rebuttal (in response to Q3), we picked two random images, namely kodim01 and kodim05, from the Kodak24 dataset. In the following tables, we provide a comprehensive comparison of the performance of ZS-NCD under two different settings.

The first table reports the performance achieved by ZS-NCD when applied in the original zero-shot setting, i.e., trained and tested on the same image. In other words, for each image in the Kodak24 dataset, we first created its noisy version (according to the desired noise power) and trained a ZS-NCD model. Then, we denoised the same image using the trained model. The reported number is the average performance (PSNR and SSIM) achieved across all 24 images for $\sigma=15$ and $\sigma=50$.

The following two tables present the average performance of ZS-NCD when there is a mismatch between the training and testing images. Column $i$ in the table, $i=1,\ldots,24$, corresponds to the average performance when a noisy version of image $i$ from the Kodak24 dataset is used for training. The reported average performance is computed on the remaining 23 images.

It can be observed that, while the performance drops when ZS-NCD is used in this alternative setting, typically the achieved performance remains reasonable.

Our final table is a $24\times 24$ table. Let $i$ and $j$ denote the row and column indices, respectively. The $(i,j)$-th entry in this table is the PSNR achieved in denoising the $i$-th image in Kodak24 dataset when ZS-NCD model is trained on image $j$ in the dataset. Here, the noise level is set as $\sigma = 15$. When comparing the entries within the $i$-th column, we observe that even when there is a mismatch between the training and test image (i.e., $i \neq j$), the model still achieves reasonable denoising performance. Also, for each image $i$, the highest PSNR in row $i$ corresponds to the model that is trained on image $i$, which aligns with the zero-shot setting.