Spectral Compressive Imaging via Unmixing-driven Subspace Diffusion Refinement

We thank the reviewer for recognizing the innovation of our approach, particularly the high-dimensional guidance mechanism with imaging consistency, as well as acknowledging our competitive performance on both standard and zero-shot datasets in terms of PSNR and SSIM, highlighting the practical value and effectiveness of our contributions. Please see below for our point-by-point responses to your comments.

• Comment: The motivation for using the unmixing-driven spectral embedding decomposition is not entirely clear. The paper does not sufficiently justify why this specific decomposition method is suitable for the SCI problem or how it leads to superior performance in terms of convergence speed, recovery accuracy, or error reduction. While the approach is innovative, its superiority over existing methods is not well-established.

@ The original intent behind designing the reversible unmixing module was to achieve dimensionality reduction of spectral information channels while ensuring accurate representation of spectral data and compatibility with the RGB space. Thus, we considered using an SVD-based approach for decomposition from the beginning. Our comparison results on KAIST dataset show that, compared to using the SVD method as described in the HIR-Diff paper, our learnable unmixing module achieves superior reconstruction performance and inference speed.

Method	Scene 1	Scene 2	Scene 3	Scene 4	Scene 5	Scene 6	Scene 7	Scene 8	Scene 9	Scene 10	Average
SVD-band-select (HIR-Diff [1])	40.65 0.9810	36.25 0.9667	39.20 0.9795	51.10 0.9948	38.70 0.9909	40.60 0.9873	43.12 0.9862	40.80 0.9867	33.00 0.9623	46.60 0.9913	41.00 0.9827
URSe (Ours)	50.54 0.9982	54.03 0.9990	49.61 0.9939	57.02 0.9994	49.28 0.9984	49.99 0.9984	47.75 0.9948	49.05 0.9977	50.57 0.9967	50.97 0.9988	50.88 0.9975

Specifically, SVD-based methods generally exhibit inferior reconstruction quality compared to our designed module. The reason for this is that SVD models the relationships between spectral bands as simple linear transformations, whereas, in reality, most spectral relationships are inherently nonlinear. Although SVD provides an option of spectral decomposition, its speed advantage diminishes in multispectral image. Conversely, our learnable module not only maintains competitive inference speed but also achieves superior compression and reconstruction performance.

Further investigation of URSe for spectral embedding (Our URSe shows robustness with noise)

To evaluate the robustness of our spectral embedding module URSe within the diffusion model, we conducted additional ablation experiments detailed in the table below. We introduced Gaussian perturbations to E and found that our method remains robust to zero-mean Gaussian noise with variances from 0 to 0.1, maintaining a PSNR exceeding 37 dB even under significant noise (variance 0.1). In contrast, the SVD-based method is sensitive to noise; with light noise of variance 0.01, its PSNR drops by 0.72 dB, and with noise of variance 0.1, it drops by 10.05 dB. These results highlight the superior noise robustness of our URSe module compared to SVD-based approaches.

Table. Ablation study on the robustness of spectral embedding for the diffusion model on KAIST dataset.

Type	Noise (Gaussian) perturbation	PSNR	SSIM
URSe (Ours)	0	38.14	0.9670
	0.01	38.1	0.9665
	0.1	37.04	0.9567
SVD-band-select (from HIR-diff [1])	0	36.87	0.9628
	0.01	36.15	0.9623
	0.1	26.82	0.9383

[1] Pang, Li, et al. "HIR-Diff: Unsupervised Hyperspectral Image Restoration Via Improved Diffusion Models." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.

Comment: Nevertheless, the inferior performance in PSNR and SSIM remains a major concern.

@: Thank you for your constructive feedback. We agree that acknowledging limitations is an important aspect of presenting scientific work transparently, it is indeed an area where improvement is always welcome.

In addition, we would like to respectfully point out that although our PSR-SCI achieves suboptimal PSNR on the KAIST dataset, our model was tested on four datasets, and on three of them—the ICVL, NTIRE, and Harvard datasets—it achieved the best PSNR and SSIM results. Additionally, our method provides superior reconstruction results on real data, retains capabilities for generation and local inpainting tasks, which non-diffusion methods cannot achieve. Therefore, while we acknowledge that there is room for improvement in some aspects, we respectfully disagree with the comment that "inferior performance in PSNR and SSIM remains a major concern." We will, however, make sure to present these clarifications and the strengths of our method more explicitly in the revised paper. Thank you again for your thoughtful comments.

Dear Reviewer Z3XV,

Thank you for raising an important point about the theoretical foundation of our spectral embedding and its advantages for SCI reconstruction. After thorough experimental verification, we appreciate the opportunity to share what we found and elaborate on how it may address your concerns.

1. Limitations of SVD-Based and Non-Learnable Approaches

SVD-based and band-selection methods, as shown in HIR-Diff and similar works, offer a linear and non-trainable means of spectral decomposition. However, such methods are inherently limited in two key ways:

• Forward-Decomposition Error Amplification: These approaches cannot adaptively fine-tune the reverse process, leading to amplified errors introduced during the forward decomposition when reconstructing the spectral image.
• Misalignment with Spectral Characteristics: SVD-based methods primarily focus on rough spectral-to-RGB mapping, which fails to capture the complex nonlinear relationships between spectral bands. This mismatch results in deviations between the diffusion model's outputs and the true spectral images due to differences in wavelength ranges (RGB: 460–650 nm vs. spectral imaging: 750–2500 nm).

2. Key Advantages of our URSe for SCI

Our learnable URSe module is specifically designed to overcome these limitations through the following innovations:

• Fine-Tunable Spectral Embedding: Unlike static SVD-based methods, URSe is fine-tuned on spectral image datasets, enabling it to learn task-specific nonlinear relationships and accurately align the spectral domain with pre-trained RGB diffusion models. This ensures that the model’s outputs are not just approximate reconstructions but are tailored to the spectral imaging domain. In addition, URSe is reversible, enabling high-quality spectral guidance for diffusion models.
• Noise Robustness: URSe is robust to noise, a crucial consideration in practical SCI scenarios where measurements often contain noise, as highlighted in literature such as DAUHST and MST. This robustness ensures consistent performance in real-world settings.
• Enhanced Convergence and Accuracy: By providing a more accurate and adaptive spectral embedding, URSe accelerates the diffusion process and achieves better reconstructions. For example, on the KAIST dataset, our method reduces inference time from 312.43s to 12.9s and improves PSNR from 33.42dB to 38.14dB compared to applying diffusion directly without URSe.

3. Compatibility with SCI Reconstruction Goals

The core contribution of URSe lies in its ability to bridge the gap between RGB diffusion models and spectral imaging tasks. By aligning the learned spectral embeddings with the spectral image space:

• It ensures that the diffusion model generates results that better represent the full spectral range.
• It mitigates domain mismatch issues between RGB-based models and spectral data, leveraging the pre-trained RGB model while adapting it to the unique characteristics of spectral imaging.

Summarize in short: Our approach extends beyond simple nonlinear modeling of spectral relationships. The learnable nature of URSe allows it to adapt dynamically, minimizing forward-reconstruction errors and ensuring robust, noise-resilient performance. This is particularly critical for SCI tasks where both accuracy and computational efficiency are paramount.

We appreciate your suggestion and we are updating our manuscript to include this detailed analysis to better explain the theoretical foundation and specific advantages of our approach for SCI reconstruction. Your feedback has helped us highlight this point, thank you!

Dear Reviewer Z3XV,

We truly appreciate the time you took to review our revised manuscript and for raising your score.

It means a lot to us that the revisions addressed your concerns and that the updates enhanced the clarity and completeness of the work. We’re really thankful for your contribution to making our paper better.

Thank you again for your support and for recognizing our efforts!

Best regards,

the authors

Dear Reviewer Z3XV,

Thank you for your continued discussion and for highlighting the need for clearer integration of our clarifications. We have carefully revised both the main manuscript and the appendix to fully incorporate the key clarifications you suggested (see the revised PDF).

Specifically, we have enhanced the discussion of our contributions, provided more detailed explanations of our methodology, and thoroughly addressed the limitations of our work. Due to space constraints, we have included most of the detailed clarifications, including supplementary tables and figures, in the appendix. We appreciate your understanding regarding these necessary adjustments to fit within the page limits. For your convenience, the appendix includes:

• Sec. A.1: Implementation Details
• Sec. A.2: Hyper-parameters Optimization
• Sec. A.3: Subspace Diffusion with High-dimensional Guidance
• Sec. A.4: Explanation of the Unmixing-driven Spectral Embedding Approach
• Sec. A.5: Framework Architecture Analysis and Performance Evaluation
• Sec. A.6: Limitation Analysis and Generalization Across Diverse Datasets
• Sec. A.7: Analysis of Training Datasets for Diffusion Models
• Sec. A.8: Additional Experimental Results

Thank you again for your feedback, which helped us to enhance our work!

Best regards,

the authors

We sincerely thank the reviewer for acknowledging the significance and novelty of our work in tackling the challenging problem of high-frequency detail reconstruction in spectral compressive imaging by leveraging pre-trained RGB generative models and introducing a reversible spectral unmixing strategy, which demonstrates promising results especially on difficult real-world SCI datasets. Please see below for our point-by-point responses to your comments.

• Comment：In lines 296-365 of the main paper and Section A.3 of the appendix, most of the mathematical derivations are easy to follow. However, some detailed notations are unclear in the context, which reduces readability. The motivation for designing the reversible unmixing module is not sufficiently clear. For example, why not use an SVD-based method? What advantages does the proposed unmixing module offer? For the reversible unmixing module, although the experiments show that it is lightweight and efficient, I am curious how it compares with SVD, PCA, or band selection-based methods.

@: We have revised the appendix and added more detailed explanations to clarify the connections between each equation, making the entire process clearer and easier to follow. Additionally, the original intent behind designing the reversible unmixing module was to achieve dimensionality reduction of spectral information channels while ensuring accurate representation of spectral data and compatibility with the RGB space. Thus, we considered using an SVD-based approach [1] for decomposition from the beginning. Our comparison results on KAIST dataset show that, compared to using the SVD method as described in the HIR-Diff paper, our learnable unmixing module achieves superior reconstruction performance and inference speed.

Method	Scene 1	Scene 2	Scene 3	Scene 4	Scene 5	Scene 6	Scene 7	Scene 8	Scene 9	Scene 10	Average
SVD-band-select (HIR-Diff [1])	40.65 0.9810	36.25 0.9667	39.20 0.9795	51.10 0.9948	38.70 0.9909	40.60 0.9873	43.12 0.9862	40.80 0.9867	33.00 0.9623	46.60 0.9913	41.00 0.9827
URSe(Ours)	50.54 0.9982	54.03 0.9990	49.61 0.9939	57.02 0.9994	49.28 0.9984	49.99 0.9984	47.75 0.9948	49.05 0.9977	50.57 0.9967	50.97 0.9988	50.88 0.9975

Specifically, SVD-based methods generally exhibit inferior reconstruction quality compared to our designed module. The reason for this is that SVD models the relationships between spectral bands as simple linear transformations, whereas, in reality, most spectral relationships are inherently nonlinear. Although SVD provides an option of spectral decomposition, its speed advantage diminishes in multispectral image. Conversely, our learnable module not only maintains competitive inference speed but also achieves superior compression and reconstruction performance.

• Comment: It seems the hyperparameters, scale sand timestep T, shown in Fig. 12, are sensitive to the model's reconstruction accuracy. How do they influence the visual quality of the reconstructed image? The PSNR is not always a reliable metric for measuring image quality, especially for spectral images.

@: We appreciate your insight regarding the sensitivity of hyperparameters s (scale) and T (timestep). It is worth noting that such parameters are inherently sensitive in most diffusion-based models, influencing generative performance and reconstruction quality. Recognizing that PSNR alone may not fully capture visual quality, particularly for spectral images, we have added visual hyperparameter maps in the revised appendix to supplement the PSNR and SSIM maps in the original submission. These visualizations illustrate how different hyperparameter settings affect perceptual quality, highlighting improvements or degradations that numerical metrics alone cannot capture, thus offering a more comprehensive understanding of their impact.

[1] Pang, Li, et al. "HIR-Diff: Unsupervised Hyperspectral Image Restoration Via Improved Diffusion Models." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.

Thank you very much for your highly positive feedback; we are delighted that you find our novel two-stage reconstruction framework both exciting and innovative, recognize its solid performance with significant advancements in diffusion-based methods for SCI reconstruction, and appreciate the clear and well-structured presentation of our work. Please see below for our point-by-point responses to your comments.

• Comment: Some modifications and explanations should be added. For example, why do you want to use an inexpensive predictor to produce an initial HSI instead of directly reconstructing the HSI results from the noisy measurements? Besides, I think it is better to discuss your work with latent space diffusion to highlight the differences and your contributions.

@: Thank you for the insightful suggestions. We opted to use an inexpensive predictor to produce an initial hyperspectral image (HSI) rather than directly reconstructing the HSI from noisy measurements because directly generating the spectral image from noise makes it challenging to leverage the prominent low-frequency information present in the measurements. In our two-stage framework, we recover the low-frequency and high-frequency components separately in two distinct stages. This approach allows the diffusion model to focus on high-frequency reconstruction, which reduces the number of diffusion steps needed and speeds up sampling.

Compared to generating HSIs entirely from noise, our two-stage approach achieves superior reconstruction quality with the same computational cost. The predictor efficiently estimates the low-frequency components, allowing the diffusion model to refine the high-frequency details effectively.

We have added this analysis to the ablation study section to further clarify these design decisions and highlight the advantages of our method. Additionally, we included a discussion on latent space diffusion to better differentiate our contributions and emphasize the benefits of our approach.

• Comment: The experiments are insufficient now. The ablation study only has a visual study (Figure 10). This is far from satisfactory. I suggest you add a break-down ablation like MST, MST++, CST, and DAUHST series works to demonstrate the effectiveness of each technical contribution. Or your claim of your work - the three modules are not convincing since their effects are unknown.

@：We conducted additional ablation experiments to investigate this issue, and the results are presented in the table below. From the table, we observe the following: when the diffusion model is removed from our PSR-SCI and only the initial predictor is used, the PSNR is 37.21 dB. When the pre-trained diffusion model, trained on a large RGB dataset, is used alone, the PSNR drops to 33.42 dB. This reduction indicates that the pre-trained diffusion model possesses inherent generative capabilities (as evidenced by the PSNR of 33.42 dB); however, its mismatch with the spectral imaging domain explains why the PSNR is lower compared to using the initial predictor.

According to our designed framework, after fine-tuning on a small spectral imaging dataset, the enhanced generative capabilities of the model become evident, resulting in an improvement of 2.83 dB in PSNR compared to directly using the diffusion model baseline. Additionally, our URSe module reduces inference time from 312.43 s to 13.79 s. The use of frequency decomposition yields a further improvement of 0.47 dB.

These results suggest that although the pre-trained diffusion model trained on a large RGB dataset shows inherent generative potential, fine-tuning, spectral embedding, and targeted generation of high-frequency details are essential to optimize its performance and efficiency in spectral imaging applications. | Initial Predictor | Freq-decomposition | URSe | Diffusion | PSNR↑ | SSIM↑ | LPIPS↓ | Best PSNR inference time (s) | | ----------------- | ------------------ | ---- | --------- | --------- | --------- | ----------- | ---------------------------- | | √ | × | × | × | 37.21 | 0.959 | 0.05718 | 0.26s | | × | × | × | √ | 33.42 | 0.883 | 0.06423 | 312.43s | | √ | × | √ | √ | 36.25 | 0.940 | 0.05375 | 13.79s | | √ | √ | × | √ | 37.67 | 0.962 | 0.04246 | 193.21s | | √ | √ | √ | √ | 38.14 | 0.967 | 0.02844 | 12.90s |

Thank you for your positive feedback; we are glad that you acknowledge our method's higher performance and reduced training and inference times compared to previous diffusion-based approaches in CASSI reconstruction. Please see below for our point-by-point responses to your comments.

• Comment: Comparsion with the spectral decomposition and refinement using pre-trained RGB diffusion models introduced in [1].

@: There are four differences between our work and [1]:

1. Spectral decomposition: Our work uses a learnable spectral embedding, whereas [1] selects 3 bands directly, our spectral decomposition gets higher reconstruction quality than SVD used in [1] (50.88dB vs. 41dB, 10 kaist scenes averaged), also with much faster speed (0.0009s vs. 0.001215s).
2. Refinement design: Our diffusion refinement design is specifically tailored for spectral images with a trainable framework, whereas [1] directly applies RGB diffusion without addressing the crucial spectral differences between RGB and multispectral images. As a result, our PSR-SCI achieves 38.14 dB, compared to 35.23 dB for [1].
3. Specific tasks: The tasks (applications) are also different—our work focuses on reconstructing N bands from 1 band, while [1] deals with N bands to N bands. Our task is more difficult due to the low sampling rate.
4. Framework flexibility and performance: Our work goes far beyond [1], offering a more flexible framework and significantly better performance (38.14dB vs. 35.23dB). We implemented the idea of [1] for SCI comparison, even though [1] was not designed for SCI.

Specifically, firstly, [1] is not trainable; it only uses a pre-trained RGB diffusion model and cannot be fine-tuned on spectral images. This makes the diffusion model's generated results still deviate from the spectral images due to the wavelengths of RGB images and spectral images being quite different. RGB imaging typically captures red (650 nm), green (560 nm), and blue (460 nm) wavelengths, whereas spectral imaging spans a broader range of wavelengths, extending into the near-infrared (750–2500 nm). We solved this problem by designing a fine-tunable diffusion framework, which is fine-tuned on a spectral image dataset, making full use of a pre-trained RGB diffusion model but also making it suitable for the spectral imaging task.

Secondly, the introduced diffusion method [1] is specifically designed for denoising or restoration of spectral images, focusing on the restoration from N bands to N bands. In contrast, our task involves reconstructing multispectral images from a single-shot snapshot compressed image (1 band to N bands), which is significantly more challenging due to the low sampling rate. These two tasks face fundamentally different challenges: denoising involves removing noise, while compressed sensing aims to recover full-spectral-spatial spectral image given a under-sampled measurement.

Meanwhile, our spectral compression and decomposition process reduces spectral images to three channels using trainable modules, achieving high reconstruction quality (41dB vs. 50.88dB, 10 kaist scenes averaged), also with much faster speed (0.0009s vs. *0.001215s).

Although [1] was not originally designed for the Snapshot Compressive Imaging (SCI) task we address in this paper, for a more comprehensive comparison, we incorporated it into the second stage of our two-stage framework (while keeping the first stage unchanged). We compared HIR-Diff with our proposed PSR-SCI, and the reconstruction performance for snapshot tasks is summarized below. The results indicate that HIR-Diff achieves reasonable reconstruction quality, but its performance metrics lag behind state-of-the-art methods. Our approach, leveraging guidance and additional trainable modules, demonstrates clear advantages over existing diffusion methods for this task, such as DiffSCI, in terms of performance metrics, visual reconstruction quality, and inference speed. .

Method	Reference	PSNR	SSIM
HIR-DIff (+our fisrt stage-to ensure it works for SCI) [1]	CVPR-2024	35.23	0.959
PSR-SCI-D	Ours	38.14	0.967

Dear Reviewer igVu,

We, the authors, sincerely appreciate you taking the time to read our response and for raising your score. Your support is incredibly encouraging and serves as significant inspiration for all of us working on this paper.

Thank you once again for your valuable feedback.

Kind regards,

The Authors