FRN: Fractal-Based Recursive Spectral Reconstruction Network

Thank you for your review.

1. In fact, since the FRN requires multiple recursive calls to the atomic network, the GFLOPs increase accordingly. Taking this into consideration, we designed the lightweight BAMamba. The comparison of GFLOPs among different methods is as follows: Method | GFLOPs$\\downarrow$ --------|-------- HSACS | 82.18 HSRNet | 3.00 SSRNet | 1.58 AWAN | 16.52 MST++ | 1.46 LTRN | $\**0.85**$ MSFN | 2.04 $\**FRN**$ | 1.45

2. The default setting for epochs is $300$.

3. At each stage, the input to $f(\\cdot)$ includes not only the RGB image but also the intermediate outputs from the previous stage, referred to as "Spectral Cues". In practice, we select several channels with the smallest Euclidean distances between all intermediate outputs from the previous stage and the current wavelength to be reconstructed by BAMamba, using them as additional cues. We found that the reconstruction results are optimal when the number of reference channels is $4$.

Thank you for your review.

1. In fact, the authors of [1] did not state that "fractals are generally adopted in two-dimensional and above spaces." The core of fractals lies in the "$\**self-similarity**$" inherent in both $\**the data and the model**$. For hyperspectral image (HSI) data, the strong correlation between adjacent spectral bands precisely indicates the presence of self-similarity along the spectral dimension. The HSI reconstruction task is essentially an ill-posed problem of generating many more bands (e.g., 31-channel HSI) from a few bands (e.g., RGB 3-channel image). This task can be decomposed into the reconstruction of atomic bands, and the networks involved in each sub-process also possess structural self-similarity. Therefore, the fractal concept is very well matched to the task addressed in this paper.

2. These Mamba-based spectral reconstruction methods have not explored the impact of SSM on spectral reconstruction from the perspective of $\**redundant information removal**$. Redundant information among spectral bands is one of the key factors hindering reconstruction performance. The band-aware mask introduced in BAMamba is a unique design in this paper aimed at addressing this issue. This scheme enables Mamba to learn more compact spectral and spatial correlations. Furthermore, existing studies typically stack multiple SSM layers to improve network performance, which weakens the inherent advantage of SSM’s low computational complexity. In contrast, the introduction of the mask reduces the involvement of irrelevant information in computations, further enhancing $\**inference efficiency**$. Experimental results also validate the effectiveness of this scheme in both reconstruction quality and efficiency. Therefore, BAMamba’s design demonstrates clear innovation over previous approaches from the perspective of $\**task understanding**$.

3. The progressively designed spectral reconstruction network based on fractal principles represents $\**a novel paradigm**$ for spectral reconstruction. It fundamentally reduces the difficulty of solving the ill-posed problem, which was $\**not addressed by previous methods**$. Experimentally, our method significantly outperforms prior approaches, and we believe it will provide valuable references for the community. On the other hand, by leveraging the physical or statistical characteristics of HSI data, this paper employs SSM as a backbone component to reduce the resource consumption required for processing high-dimensional data. Additionally, by introducing a mask, redundant information interference is partially eliminated, further improving the model’s inference efficiency and mitigating the negative effects caused by recursive calls.

[1] Fractal generative models, arvix 2025.

Thank you for your review. We will reply to your questions in order.

1. To address your concerns regarding the inference efficiency of FRN, we further provide a comparison of the inference time and memory cost required by each method for processing a single image (Taking a single sample from the CAVE dataset as an example.):

It can be observed that FRN has the second fastest inference speed after AWAN and the lowest memory cost.

2. Thank you for your constructive comments. Regarding the contributions of the fractal recursion, we provide a 'w/o' configuration in Table 3, which means reconstructing the HSI from the RGB image in one step (without using fractal recursion). It can be observed that all metrics drop significantly on the CAVE dataset (with PSNR decreasing by $1.18$ dB).

As for your suggestion on conducting a component-wise analysis of BAMamba, we attempted to replace the atomic generator with MSFN. The results on the CAVE dataset are as follows:

It can be found that BAMamba outperforms MSFN, achieving a $0.99$ dB higher PSNR on the CAVE dataset. When comparing the contributions of fractal recursion and BAMamba in a component-wise manner, we observe that the former contributes more significantly. We attribute this to the fact that fractal recursion serves as a general strategy that fundamentally reduces the difficulty of solving the ill-posed problem. In contrast, the latter (a deep neural network for HSI reconstruction) may be more prone to overfitting due to potential limitations in its design.

3. We trained BAMamba separately on the two datasets, and all experiments were conducted on a single NVIDIA RTX 4090 GPU. The specific training details are provided in Section 4.1. The default setting for epochs is $300$. The training time on the CAVE dataset was $1.58$ hours with a memory cost of $12.8$ GB, while the training time on the Harvard dataset was $13$ hours with a memory cost of $12.8$ GB.

4. In fact, we later identified this experimental gap and further conducted experiments by setting the recursion levels to $M=6$ and $n=2$. The results on the CAVE dataset are as follows: Config | PSNR$\\uparrow$ | RMSE$\\downarrow$ | UIQI$\\uparrow$ | SSIM$\\uparrow$ --------|--------|--------|--------|-------- $\**M=5**$ | $\**41.0522**$ | $\**3.6243**$ | $\**0.9010**$ | $\**0.9900**$ $M=6$ | 41.0233 | 3.6441 | 0.9005 | 0.9887

Deeper recursion introduced more redundant information into the reconstruction output, leading to a decline in reconstruction performance.

5. Same as Question 2.

6. We will release our code on GitHub.

Thank you for your review.

1. We will include an example to more concretely illustrate how the recursive levels M (in Section 4.5) influence the hyperspectral image generation process. As shown in Figure.2 (a), in this paper, we input a 3-channel RGB image into the FRN and generate a $K=31$-channel hyperspectral image. However, the generation process is not an end-to-end operation in the traditional sense. Instead, it requires $M$ stages to complete, with each stage employing the same U-Net style BAMamba model as the atomic reconstruction network. Assuming the default setting of $M=5$, the output size of BAMamba at each stage is fixed to $2 \\times H \\times W,$ that is, $n=2$ spectral channels (where $M=\\log_{n}{K}$). The complete process is as follows:

Stage 1 ($1$-call, ($n \\times 1=2$)-band generation):

$f(x_r, x_g, x_b) = (x_1, x_2)$;

Stage 2 ($2$-call, ($n \\times 2=4$)-band generation):

$f(x_r, x_g, x_b, x_1) = (x_1, x_2)$, $f(x_r, x_g, x_b, x_2) = (x_3, x_4)$;

Stage 3 ($4$-call, ($n \\times 4=8$)-band generation):

$f(x_r, x_g, x_b, x_1) = (x_1, x_2)$, $f(x_r, x_g, x_b, x_2) = (x_3, x_4)$, $f(x_r, x_g, x_b, x_3) = (x_5, x_6)$, $f(x_r, x_g, x_b, x_4) = (x_7, x_8)$;

Stage 4 ($8$-call, ($n \\times 8=16$)-band generation):

$f(x_r, x_g, x_b, x_1) = (x_1, x_2)$, $f(x_r, x_g, x_b, x_2) = (x_3, x_4)$, $f(x_r, x_g, x_b, x_3) = (x_5, x_6)$, $f(x_r, x_g, x_b, x_4) = (x_7, x_8)$, $f(x_r, x_g, x_b, x_5) = (x_9, x_{10})$, $f(x_r, x_g, x_b, x_6) = (x_{11}, x_{12})$, $f(x_r, x_g, x_b, x_7) = (x_{13}, x_{14})$, $f(x_r, x_g, x_b, x_8) = (x_{15}, x_{16})$;

Stage 5 ($16$-call, ($n \\times 16=32$)-band generation):

$f(x_r, x_g, x_b, x_1) = (x_1, x_2)$, $f(x_r, x_g, x_b, x_2) = (x_3, x_4)$, $\\cdots$, $f(x_r, x_g, x_b, x_{15}) = (x_{29}, x_{30})$, $f(x_r, x_g, x_b, x_{16}) = (x_{31}, x_{32})$.

The value of $M$ and the number of spectral channels in the HSI must satisfy the condition that $n^M$ is closest to $K$. This ensures that the FRN can generate a sufficient number of spectral bands without introducing excessive redundant outputs. For example, when $M=5$, the FRN ultimately produces $32$ channels ($>31$), and the extra final channel will be removed.

2. We further provide experimental results for the case of $M=6$ and $n=2$, where the FRN outputs $2^6=64$ channels. These were mapped to $32$ channels by computing the channel-wise mean, and the final extra channel was removed. The evaluation metrics are as follows: Config | PSNR$\\uparrow$ | RMSE$\\downarrow$ | UIQI$\\uparrow$ | SSIM$\\uparrow$ --------|--------|--------|--------|-------- $\**M=5**$ | $\**41.0522**$ | $\**3.6243**$ | $\**0.9010**$ | $\**0.9900**$ $M=6$ | 41.0233 | 3.6441 | 0.9005 | 0.9887

Deeper recursion introduced more redundant information into the reconstruction output, leading to a decline in reconstruction performance.

3. In each stage, the input to $f(\\cdot)$ includes not only the RGB image but also the intermediate outputs from the previous stage, referred to as "Spectral Cues" (as mentioned in Section 4.5). In practice, instead of simply adding one intermediate result from the previous stage as an additional condition to $f(\\cdot)$, we compute the Euclidean distances between all intermediate outputs of the previous stage and the current wavelength to be reconstructed by BAMamba. Then, we select the closest channels as additional cues. We found that the reconstruction achieves the best performance when the number of reference channels is set to $4$.

4. The values of $M$ and $n$ are determined such that $n^M$ approximates $31$ ($K$) as closely as possible.