We thank Reviewer Ws6m for recognizing the broader implications of our findings, especially in “correcting the common belief” about generalization in diffusion models and for noting that our method “improves upon previously proposed analytical diffusion models.” Other reviewers, such as 8EDr and 8kUK, also appreciated our benchmarks, experimental rigor, and the clarity of our comparative analysis.

We also appreciate reviewers' suggestions on how to further strengthen the paper. In the rest of this reply, we first address suggestions shared by multiple reviewers and then will specifically address comments in this review.

Shared comments

Computational Efficiency

Reviewers 8EDr and 8kUK suggested including a comparison of runtime between the baselines. The runtimes and hardware used for all baselines are reported in Section C.5 of the appendix, and we provide additional context below. Our algorithm is more computationally efficient than Kamb & Ganguli's model both asymptotically and in practice, as we are not assuming translation equivariance. As reviewer 8kUK correctly mentioned, creating an efficient analytical denoiser is not the main focus of this paper; however, we agree that additional analysis of computational efficiency will further strengthen the paper.

Below, we provide a detailed analysis of the algorithmic complexity for the Wiener filter, Kamb & Ganguli's analytical model, and ours.

Algorithmic Complexity and the Softmax

For small-resolution images, the Wiener filter remains the most efficient at $O(m^2)$, where $m$ is the flattened image resolution. Both our model and Kamb & Ganguli’s require a dataset pass per inference step, leading to scaling linear in $n$, where $n$ is the dataset size.

Kamb & Ganguli assume translation equivariance, so for each pixel, we compare its surrounding patch (size $p_t$ at denoising step $t$) to every patch in the dataset, resulting in $O(n p_t m^2)$ complexity. With approximate vector search (e.g., using $k$ clusters), this reduces to $O(n p_t m^2 / k)$. Our model forgoes translation equivariance as we did not observe any difference in quality (reported in the main paper). Additionally, we are using a distinct per-pixel mask pattern. This leads to the algorithmic complexity of our algorithm being $O(n p_t m)$. While this makes approximate search harder, our implementation (Section C.5) is faster than even Kamb & Ganguli’s approximate version on benchmark-scale datasets. For larger datasets, we can match their $O(n p_t m / k)$ complexity by indexing masks per timestep and pixel.

Softmax: As Reviewer 8kUK noted, computing softmax across a dataset is challenging. We address this with a numerically stable streaming implementation requiring just one pass and constant memory (see src/utils.py in the appendix files).

Algorithmic complexity table:

Choice of $A_t^q$: intuition, other alternatives and formatting

Motivation in Section A.3 (8kUK, Ws6m)

We agree with reviewers 8kUK and Ws6m that the main paper will benefit from some part of the motivation of our algorithm from the appendix to the main paper. We will update the main paper with a clearer intuition behind our algorithm: Kamb and Ganguli use rectangular binary masks fitted to the sensitivity fields of trained denoisers. In our paper, we show that the sensitivity fields of the trained denoisers come from the data covariance. In this paper, we are considering constant sensitivity fields, i.e., $A_t^q$ is not a function of $x$. If the sensitivity field is constant, the denoiser is linear, and we know that the optimal linear denoiser is the Wiener filter. Thus, the sensitivity field under the constant constraint is that of the Wiener filter.

We also revised the derivations for why the sensitivity fields should be binary:

• Generalized locality definition We first consider a generalized notion of locality, with the constraint $f^q(x) = f^q(A^q_t x)$, where $A^q_t$ can be any matrix. Please note that indeed, when $A^q_t$ is invertible, $f^q$ is effectively unconstrained, i.e., there is no locality. When $A^q_t = 0$, $f^q$ does not depend on $x$.

• Generalized locality imposes subspace projection Next we prove that imposing the generalized locality constraint is equivalent to adding a projection operator $\mathbf{P_t^q}=(A_t^q)^T (A_t^q (A_t^q)^T)^\dagger A_t^q$ (projection onto the row-space of $A^q_t$) to the optimal denoiser: $\hat{f}^q(x, t) = \sum_{i=1}^N (x^i_0)^q \; \text{softmax}_i\left(-\frac{1}{2\sigma_t^2} \|\mathbf{P_t^q}(x - \sqrt{\alpha_t}x_0^i)\|^2\right)$

• Projection = Orthogonal change of basis + Masking All projection operators can be written as an orthogonal change of basis $V_t^q$ followed by a masking operation $M^q_t$ (i.e., a diagonal 0-1 matrix): $\mathbf{P_t^q} = (V_t^q)^TM^q_tV_t^q$

This explains why the only interesting generalized locality matrices are binary masks. We will move motivation for our method and additional intuition from Appendix A.3 to Section 3 of the main paper, in case if the paper is accepted. We will update our paper's appendix with proofs of the above statements.

Individual comments

RE: What are other choices of $A_t^q$

Thank you for your thoughtful comments and suggestions on the choice of $A_t^q$. We provided extended motivation earlier in this reply. We also attempted to perform gradient-based optimization of the matrix $A_t^q$ pre-binarization; however, in the limited time of the rebuttal, we were not able to arrive at conclusive results.

Overall, our intuition is that for constant sensitivity fields (i.e., not a function of $x$), the denoiser is linear, and we know that the optimal linear denoiser is the Wiener filter. To further close the gap between analytical and trained denoisers, we will need to allow the sensitivity fields to depend on the input image, i.e., $A_t^q(x_t)$ should be a function of the noisy image. We agree that this is a very interesting direction to explore and plan to address it in future research.

RE: Low-Rank Covariance Matrix

We agree that a low-rank projection of the covariance matrix is a good method to prevent high-frequency artifacts in the generations produced by the Wiener filter. We performed additional experiments to see if removing the weakest principal components in the covariance matrix can help the analytical models better explain trained diffusers. For the Wiener filter, low-rank approximation leads to visually smoother outputs; however, it reduces the correlation with outputs from trained diffusion models. Our algorithm does not benefit from using a low-rank projection of the covariance matrix and still demonstrates higher correlation than the Wiener filter. This indicates that low-rank structure alone does not account for the performance of learned denoisers. We believe this result further supports the distinctiveness and relevance of our proposed model. Please refer to our detailed qualitative and quantitative analysis below.

Singular Values

First, analyzed the singular values of the covariance matrices for each of the datasets. We observe that most of the total energy is contained in the first 10% of the singular values for all datasets, leaving a long tail of principal components. We plan to include the plots in the appendix of the final version if accepted.

Quantitative Comparison

We zero out the smallest singular values of the covariance matrix based on different thresholds of total energy. Both for ours and for the Wiener filter we measure the $r^2$ between predictions of the analytic models and a trained DDPM model, as done in the paper. The results are averaged across 16 generations and presented in the table below. A 0% SVD Energy Cutoff corresponds to using the full-rank covariance matrix.

Qualitative Comparison

Additionally, we generate visual comparisons between all the baselines across the different energy cut-off thresholds. We observe that removing a small portion of the singular values (e.g., 0.01% in total energy) indeed smooths out the images generated by the Wiener filter and makes the results more visually appealing. However, as we can see from the table above, this does not lead to a better correlation with trained diffusers. For our method, projecting the covariance matrix to a low rank does not seem to visually improve the results—it only blurs the images. We plan to include the visual comparisons in the final version of the paper if accepted. Please note that we cannot attach the images to this response due to the new NeurIPS guidelines.

Conclusion

In conclusion, based on both qualitative and quantitative comparisons across multiple energy cut-off thresholds, we did not find evidence that low-rank projection of the covariance matrix in analytical models leads to a better explanation of trained denoisers.

RE: typo

Thank you for highlighting the typo. We fixed it.

Thank you for your thoughtful review, and please let us know if you have any additional suggestions on how we can further improve our paper.

Thank you for the thorough review of our paper, highlighting that it “surprisingly outperforms previous analytical models,” and for appreciating our “new quantitative benchmarks,” “detailed visual proofs,” and “very detailed discussion.” We're also encouraged that other reviewers, such as 8kUK and 4nQU, similarly praised the clarity, rigor, and completeness of our experiments and comparisons.

We also appreciate reviewers' suggestions on how to further strengthen the paper. In the rest of this reply, we first address suggestions shared by multiple reviewers and then will specifically address comments in this review.

Computational Efficiency

Reviewers 8EDr and 8kUK suggested including a comparison of runtime between the baselines. The runtimes and hardware used for all baselines are reported in Section C.5 of the appendix, and we provide additional context below. Our algorithm is more computationally efficient than Kamb & Ganguli's model both asymptotically and in practice, as we are not assuming translation equivariance. As reviewer 8kUK correctly mentioned, creating an efficient analytical denoiser is not the main focus of this paper; however, we agree that additional analysis of computational efficiency will further strengthen the paper.

Below, we provide a detailed analysis of the algorithmic complexity for the Wiener filter, Kamb & Ganguli's analytical model, and ours.

Algorithmic Complexity and the Softmax

For small-resolution images, the Wiener filter remains the most efficient at $O(m^2)$, where $m$ is the flattened image resolution. Both our model and Kamb & Ganguli’s require a dataset pass per inference step, leading to scaling linear in $n$, where $n$ is the dataset size.

Kamb & Ganguli assume translation equivariance, so for each pixel, we compare its surrounding patch (size $p_t$ at denoising step $t$) to every patch in the dataset, resulting in $O(n p_t m^2)$ complexity. With approximate vector search (e.g., using $k$ clusters), this reduces to $O(n p_t m^2 / k)$. Our model forgoes translation equivariance as we did not observe any difference in quality (reported in the main paper). Additionally, we are using a distinct per-pixel mask pattern. This leads to the algorithmic complexity of our algorithm being $O(n p_t m)$. While this makes approximate search harder, our implementation (Section C.5) is faster than even Kamb & Ganguli’s approximate version on benchmark-scale datasets. For larger datasets, we can match their $O(n p_t m / k)$ complexity by indexing masks per timestep and pixel.

Softmax: As Reviewer 8kUK noted, computing softmax across a dataset is challenging. We address this with a numerically stable streaming implementation requiring just one pass and constant memory (see src/utils.py in the appendix files).

Algorithmic complexity table:

Choice of $A_t^q$: intuition, other alternatives and formatting

Motivation in Section A.3 (8kUK, Ws6m)

We agree with reviewers 8kUK and Ws6m that the main paper will benefit from some part of the motivation of our algorithm from the appendix to the main paper. We will update the main paper with a clearer intuition behind our algorithm: Kamb and Ganguli use rectangular binary masks fitted to the sensitivity fields of trained denoisers. In our paper, we show that the sensitivity fields of the trained denoisers come from the data covariance. In this paper, we are considering constant sensitivity fields, i.e., $A_t^q$ is not a function of $x$. If the sensitivity field is constant, the denoiser is linear, and we know that the optimal linear denoiser is the Wiener filter. Thus, the sensitivity field under the constant constraint is that of the Wiener filter.

We also revised the derivations for why the sensitivity fields should be binary:

• Generalized locality definition We first consider a generalized notion of locality, with the constraint $f^q(x) = f^q(A^q_t x)$, where $A^q_t$ can be any matrix. Please note that indeed, when $A^q_t$ is invertible, $f^q$ is effectively unconstrained, i.e., there is no locality. When $A^q_t = 0$, $f^q$ does not depend on $x$.

• Generalized locality imposes subspace projection Next we prove that imposing the generalized locality constraint is equivalent to adding a projection operator $\mathbf{P_t^q}=(A_t^q)^T (A_t^q (A_t^q)^T)^\dagger A_t^q$ (projection onto the row-space of $A^q_t$) to the optimal denoiser: $\hat{f}^q(x, t) = \sum_{i=1}^N (x^i_0)^q \; \text{softmax}_i\left(-\frac{1}{2\sigma_t^2} \|\mathbf{P_t^q}(x - \sqrt{\alpha_t}x_0^i)\|^2\right)$

• Projection = Orthogonal change of basis + Masking All projection operators can be written as an orthogonal change of basis $V_t^q$ followed by a masking operation $M^q_t$ (i.e., a diagonal 0-1 matrix): $\mathbf{P_t^q} = (V_t^q)^TM^q_tV_t^q$

This explains why the only interesting generalized locality matrices are binary masks. We will move motivation for our method and additional intuition from Appendix A.3 to Section 3 of the main paper, in case if the paper is accepted. We will update our paper's appendix with proofs of the above statements.

Low-Rank Covariance Matrix

In response to Reviewer Ws6m’s suggestion, we conducted an additional comparison between the Wiener filter and our algorithm under low-rank projections of the covariance matrix. For the Wiener filter, low-rank approximation leads to visually smoother outputs; however, it reduces the correlation with outputs from trained diffusion models. Our algorithm does not benefit from using a low-rank projection of the covariance matrix and still demonstrates higher correlation than the Wiener filter. This indicates that low-rank structure alone does not account for the performance of learned denoisers. We believe this result further supports the distinctiveness and relevance of our proposed model. Please refer to our detailed qualitative and quantitative analysis in the personal response to Reviewer Ws6m.

Please let us know if you have any additional suggestions on how we can further improve our paper.

We sincerely thank Reviewer 4nQU for highlighting the strength of our approach, in particular, evaluating it as “very well-written”, as well as your remarks on the importance of the problem and the clarity of our experiments and proofs. We’re also grateful that other reviewers, including 8EDr and 8kUK, found the benchmarks and comparisons both novel and convincing.

We also appreciate reviewers' suggestions on how to further strengthen the paper. In the rest of this reply, we first address suggestions shared by multiple reviewers and then will specifically address comments in this review.

Shared comments

Computational Efficiency

Reviewers 8EDr and 8kUK suggested including a comparison of runtime between the baselines. The runtimes and hardware used for all baselines are reported in Section C.5 of the appendix, and we provide additional context below. Our algorithm is more computationally efficient than Kamb & Ganguli's model both asymptotically and in practice, as we are not assuming translation equivariance. As reviewer 8kUK correctly mentioned, creating an efficient analytical denoiser is not the main focus of this paper; however, we agree that additional analysis of computational efficiency will further strengthen the paper.

Below, we provide a detailed analysis of the algorithmic complexity for the Wiener filter, Kamb & Ganguli's analytical model, and ours.

Algorithmic Complexity and the Softmax

For small-resolution images, the Wiener filter remains the most efficient at $O(m^2)$, where $m$ is the flattened image resolution. Both our model and Kamb & Ganguli’s require a dataset pass per inference step, leading to scaling linear in $n$, where $n$ is the dataset size.

Kamb & Ganguli assume translation equivariance, so for each pixel, we compare its surrounding patch (size $p_t$ at denoising step $t$) to every patch in the dataset, resulting in $O(n p_t m^2)$ complexity. With approximate vector search (e.g., using $k$ clusters), this reduces to $O(n p_t m^2 / k)$. Our model forgoes translation equivariance as we did not observe any difference in quality (reported in the main paper). Additionally, we are using a distinct per-pixel mask pattern. This leads to the algorithmic complexity of our algorithm being $O(n p_t m)$. While this makes approximate search harder, our implementation (Section C.5) is faster than even Kamb & Ganguli’s approximate version on benchmark-scale datasets. For larger datasets, we can match their $O(n p_t m / k)$ complexity by indexing masks per timestep and pixel.

Softmax: As Reviewer 8kUK noted, computing softmax across a dataset is challenging. We address this with a numerically stable streaming implementation requiring just one pass and constant memory (see src/utils.py in the appendix files).

Algorithmic complexity table:

Choice of $A_t^q$: intuition, other alternatives and formatting

Motivation in Section A.3 (8kUK, Ws6m)

We agree with reviewers 8kUK and Ws6m that the main paper will benefit from moving some part of the motivation of our algorithm from the appendix to the main paper. We will update the main paper with a clearer intuition behind our algorithm: Kamb and Ganguli use rectangular binary masks fitted to the sensitivity fields of trained denoisers. In our paper, we show that the sensitivity fields of the trained denoisers come from the data covariance. In this paper, we are considering constant sensitivity fields, i.e., $A_t^q$ is not a function of $x$. If the sensitivity field is constant, the denoiser is linear, and we know that the optimal linear denoiser is the Wiener filter. Thus, the sensitivity field under the constant constraint is that of the Wiener filter.

We also revised the derivations for why the sensitivity fields should be binary:

• Generalized locality definition We first consider a generalized notion of locality, with the constraint $f^q(x) = f^q(A^q_t x)$, where $A^q_t$ can be any matrix. Please note that indeed, when $A^q_t$ is invertible, $f^q$ is effectively unconstrained, i.e., there is no locality. When $A^q_t = 0$, $f^q$ does not depend on $x$.

• Generalized locality imposes subspace projection Next we prove that imposing the generalized locality constraint is equivalent to adding a projection operator $\mathbf{P_t^q}=(A_t^q)^T (A_t^q (A_t^q)^T)^\dagger A_t^q$ (projection onto the row-space of $A^q_t$) to the optimal denoiser: $\hat{f}^q(x, t) = \sum_{i=1}^N (x^i_0)^q \; \text{softmax}_i\left(-\frac{1}{2\sigma_t^2} \|\mathbf{P_t^q}(x - \sqrt{\alpha_t}x_0^i)\|^2\right)$

• Projection = Orthogonal change of basis + Masking All projection operators can be written as an orthogonal change of basis $V_t^q$ followed by a masking operation $M^q_t$ (i.e., a diagonal 0-1 matrix): $\mathbf{P_t^q} = (V_t^q)^TM^q_tV_t^q$

This explains why the only interesting generalized locality matrices are binary masks. We will move motivation for our method and additional intuition from Appendix A.3 to Section 3 of the main paper, in case if the paper is accepted. We will update our paper's appendix with proofs of the above statements.

Low-Rank Covariance Matrix

In response to Reviewer Ws6m’s suggestion, we conducted an additional comparison between the Wiener filter and our algorithm under low-rank projections of the covariance matrix. For the Wiener filter, low-rank approximation leads to visually smoother outputs; however, it reduces the correlation with outputs from trained diffusion models. Our algorithm does not benefit from using a low-rank projection of the covariance matrix and still demonstrates higher correlation than the Wiener filter. This indicates that low-rank structure alone does not account for the performance of learned denoisers. We believe this result further supports the distinctiveness and relevance of our proposed model. Please refer to our detailed qualitative and quantitative analysis in the personal response to Reviewer Ws6m.

Individual comments

RE: the README in the code

Thank you for pointing this out. We updated our README with detailed installation and execution instructions. We will publicly release a cleaned-up and better-documented codebase together with our camera-ready version.

RE: Formatting of Figure 5

Since the main goal of the work is to study generalization mechanisms of trained diffusion models, we believe it is important to report the nearest image ("OursNN") in the dataset. By showing the closest image, we allow the readers to judge how novel the generated samples are and if the proposed analytical model indeed generalizes. To report the closest images for each baseline while preserving the space in the paper, we plan to change the format of Figure 5 and place a miniature version of the closest neighbor images at the top-right corner of each generated image. We will be happy to follow any recommendations on better formatting of Figure 5. We further agree that the label "OursNN" might be confusing and will rename it to "Dataset" by the camera-ready version.

RE: Table 1, moving variance to the main paper

We will be happy to move the variance for Table 1 from the appendix to the main paper. To comply with NeurIPS formatting, in the camera-ready version of the main paper, we are planning to provide the metrics only on 3 datasets out of 5 (CIFAR10, Celeba-HQ, and MNIST) with the respective variances, while moving the other two datasets (AFHQ and FashionMNIST) to the appendix. Please find the proposed formatting of Table 1 below:

RE: statistical significance

Our results are statistically significant. To elucidate this, we calculate the p-value by performing a two-sample t-test against the hypothesis that "Ours" is better than the other baselines. All the results are presented for the $r^2$ coefficient of determination. From the table below, with high statistical significance, our method explains trained diffusion models better than any of the baselines.

Please let us know if you have other suggestions on how to format it better.

We thank Reviewer 8kUK for their thoughtful review and for describing our paper as “a nice response” to recent work. We’re glad you found our experimental comparisons “very convincing” and appreciated the “quantitative and systematic comparison with the Wiener filter.” Reviewers 8EDr and Ws6m also found our model to significantly outperform prior analytical approaches, and 4nQU highlighted the clarity and accessibility of our paper.

We also appreciate reviewers' suggestions on how to further strengthen the paper. In the rest of this reply, we first address suggestions shared by multiple reviewers and then will specifically address comments in this review.

Computational Efficiency

Reviewers 8EDr and 8kUK suggested including a comparison of runtime between the baselines. The runtimes and hardware used for all baselines are reported in Section C.5 of the appendix, and we provide additional context below. Our algorithm is more computationally efficient than Kamb & Ganguli's model both asymptotically and in practice, as we are not assuming translation equivariance. As reviewer 8kUK correctly mentioned, creating an efficient analytical denoiser is not the main focus of this paper; however, we agree that additional analysis of computational efficiency will further strengthen the paper.

Below, we provide a detailed analysis of the algorithmic complexity for the Wiener filter, Kamb & Ganguli's analytical model, and ours.

Algorithmic Complexity and the Softmax

For small-resolution images, the Wiener filter remains the most efficient at $O(m^2)$, where $m$ is the flattened image resolution. Both our model and Kamb & Ganguli’s require a dataset pass per inference step, leading to scaling linear in $n$, where $n$ is the dataset size.

Kamb & Ganguli assume translation equivariance, so for each pixel, we compare its surrounding patch (size $p_t$ at denoising step $t$) to every patch in the dataset, resulting in $O(n p_t m^2)$ complexity. With approximate vector search (e.g., using $k$ clusters), this reduces to $O(n p_t m^2 / k)$. Our model forgoes translation equivariance as we did not observe any difference in quality (reported in the main paper). Additionally, we are using a distinct per-pixel mask pattern. This leads to the algorithmic complexity of our algorithm being $O(n p_t m)$. While this makes approximate search harder, our implementation (Section C.5) is faster than even Kamb & Ganguli’s approximate version on benchmark-scale datasets. For larger datasets, we can match their $O(n p_t m / k)$ complexity by indexing masks per timestep and pixel.

Softmax: As Reviewer 8kUK noted, computing softmax across a dataset is challenging. We address this with a numerically stable streaming implementation requiring just one pass and constant memory (see src/utils.py in the appendix files).

Algorithmic complexity table:

Choice of $A_t^q$: intuition, other alternatives and formatting

Motivation in Section A.3 (8kUK, Ws6m)

We agree with reviewers 8kUK and Ws6m that the main paper will benefit from some part of the motivation of our algorithm from the appendix to the main paper. We will update the main paper with a clearer intuition behind our algorithm: Kamb and Ganguli use rectangular binary masks fitted to the sensitivity fields of trained denoisers. In our paper, we show that the sensitivity fields of the trained denoisers come from the data covariance. In this paper, we are considering constant sensitivity fields, i.e., $A_t^q$ is not a function of $x$. If the sensitivity field is constant, the denoiser is linear, and we know that the optimal linear denoiser is the Wiener filter. Thus, the sensitivity field under the constant constraint is that of the Wiener filter.

We also revised the derivations for why the sensitivity fields should be binary:

• Generalized locality definition We first consider a generalized notion of locality, with the constraint $f^q(x) = f^q(A^q_t x)$, where $A^q_t$ can be any matrix. Please note that indeed, when $A^q_t$ is invertible, $f^q$ is effectively unconstrained, i.e., there is no locality. When $A^q_t = 0$, $f^q$ does not depend on $x$.

• Generalized locality imposes subspace projection Next we prove that imposing the generalized locality constraint is equivalent to adding a projection operator $\mathbf{P_t^q}=(A_t^q)^T (A_t^q (A_t^q)^T)^\dagger A_t^q$ (projection onto the row-space of $A^q_t$) to the optimal denoiser: $\hat{f}^q(x, t) = \sum_{i=1}^N (x^i_0)^q \; \text{softmax}_i\left(-\frac{1}{2\sigma_t^2} \|\mathbf{P_t^q}(x - \sqrt{\alpha_t}x_0^i)\|^2\right)$

• Projection = Orthogonal change of basis + Masking All projection operators can be written as an orthogonal change of basis $V_t^q$ followed by a masking operation $M^q_t$ (i.e., a diagonal 0-1 matrix): $\mathbf{P_t^q} = (V_t^q)^TM^q_tV_t^q$

This explains why the only interesting generalized locality matrices are binary masks. We will move motivation for our method and additional intuition from Appendix A.3 to Section 3 of the main paper, in case if the paper is accepted. We will update our paper's appendix with proofs of the above statements.

Low-Rank Covariance Matrix

In response to Reviewer Ws6m’s suggestion, we conducted an additional comparison between the Wiener filter and our algorithm under low-rank projections of the covariance matrix. For the Wiener filter, low-rank approximation leads to visually smoother outputs; however, it reduces the correlation with outputs from trained diffusion models. Our algorithm does not benefit from using a low-rank projection of the covariance matrix and still demonstrates higher correlation than the Wiener filter. This indicates that low-rank structure alone does not account for the performance of learned denoisers. We believe this result further supports the distinctiveness and relevance of our proposed model. Please refer to our detailed qualitative and quantitative analysis in the personal response to Reviewer Ws6m.

Thank you for your thoughtful review, and please let us know if you have any additional suggestions on how we can further improve our paper.

We appreciate the author’s detailed responses and extra work!

Algorithmic Complexity and the Softmax

The table comparing the complexities of different methods will be very nice reference for future readers for this line of work. Seems like table like this does not exist in the previous papers, e.g Kamb & Ganguli 2024 and Wang & Vastola 2024).

Motivation in Section A.3

Thanks for explaining the Sec. A.3. The generalized locality definition now makes much more sense! (although I still cannot see why this definition is termed “generalized locality” LOL, more like invariance to linear projection / linear transform… )

The intuition of projection equivalent to orthogonal transform + masking makes a lot of sense, and it’s indeed easier to follow.

I think the new logic flow can be briefed in main text to give more intuition!

Low rank covariance

Thanks for mentioning the new experiment on low rank covariance. The new experiments and table in response to Ws6m is very comprehensive and convincing, Kudos on the effort for pulling this off! :)

The graceful decay of correlation when cutting off spectrum for Wiener filter is kind of expected. Though the degradation of correlation for your new method is less intuitive and sometimes non-monotonic. Is there intuition why that is the case?

From the formula of Wiener filter or in Fig. 10 of [Wang, Vastola, 2024], diffusion at different noise scales should depend on different number PCs in the spectrum, i.e. higher noise scale depend on few number of PCs, and lower noise scales depend nearly all the PCs. So cutting off some lower PCs should only affect the lower noise regimes (at least for Gaussian score / Wiener filter). Not sure if there are some similar intuitions for your new method~

(Not sure keeping the same cutoff rank for all sampling steps will be good. But we also acknowledge that, it will be too much work to tune the rank for different noise levels, so a comprehensive study could be left for future work!)

We'd maintain/increase our supporting score and lean strongly towards acceptance!