FourierMamba: Fourier Learning Integration with State Space Models for Image Deraining

R1: Clarification on Motivation

Motivation: Mamba, a state-space model, offers global modeling and linear complexity, making it ideal for sequential data. Frequency information in the Fourier domain is inherently global, and Mamba’s sequence modeling efficiently captures inter-frequency dependencies. In contrast, Transformers incur high computational complexity (O(n²)) for long sequences, while 3×3 convolutions are limited by their receptive fields in capturing global frequency data. Thus, integrating Mamba with the Fourier domain is a targeted signal-processing design, leveraging frequency globality and computational efficiency to enhance deraining performance innovatively. Comparative Experiments: Addressing the suggestion, we conducted experiments comparing Transformer and 3×3 convolution on the Rain200L dataset (Table 1). Results demonstrate that Mamba’s scanning approach effectively balances performance and efficiency.

R2: Response to “Lack of Quantitative Evaluation in Real-World Scenarios”

We have included no-reference quantitative evaluation results for the Internet test set without ground truth (GT) from the SPA dataset, as presented in Table 2.

R3: Response to “Experimental Design and Analysis”

Real-world deraining results are presented in Figures 13–16, with quantitative outcomes in Tables 13–15, demonstrating adaptability and generalizability to real rain due to Fourier priors and Mamba’s efficient global modeling.

R4: Response to “Issues with Mamba in the Fourier Domain”

Mamba’s local pixel forgetting in the spatial domain stems from its sequence modeling, potentially overlooking local details, while spatial misalignment relates to scanning order. In the Fourier domain, however, information is globally represented as frequencies, rendering local forgetting less prominent, as frequency components reflect overall image properties. Moreover, Fourier priors decompose the image into frequencies, aiding Mamba in modeling global information comprehensively and mitigating misalignment. Experimental results (Figures 5 and 9) show FourierMamba preserves background details and removes rain streaks effectively, confirming these issues are alleviated in the Fourier domain. This analysis will be added to the revision.

R5: Response to “Unclear Connection Between Preliminary Equations and Figure 3 Modules”

We acknowledge the lack of clarity and will refine the Preliminary section in the revised manuscript. The equations therein introduce Fourier transform fundamentals, laying the theoretical groundwork for subsequent frequency modeling. The FFT operation in Figure 3 directly applies these principles, converting images from the spatial to the Fourier domain and separating amplitude and phase. We will explicitly link these equations to the FFT module in Figure 3, enhancing logical coherence and eliminating perceived redundancy.

R6: Response to “Issues with Feature Maps in Figure 7”

Figure 7 compares feature maps of FreqMamba and our method, revealing that our approach more effectively focuses on and localizes rain streak-related features. Consequently, our derain results (Fig5-6, Fig9-16) retains background information while minimizing residual rain streaks compared to FreqMamba.

R7: Response to “Missing Entries in Table 1”

Table 1 references the table in FreqMamba. Missing data primarily result from unavailable open-source code or discrepancies between open-source implementations and the corresponding papers, leading us to directly adopt reported results from the papers. In the future, we will strive to complete missing entries or provide explanations as feasible.

R8: Response to “Formatting and Presentation Issues”

In the revised manuscript, we will adjust the font sizes of tables and figures for consistency.

R9: Response to “Layout Issues in Figure 6”

Our method’s advantages over others in real-world scenarios are observable through enlarged patches. We will revise the layout of Figure 6 in the updated manuscript. Additional visual results for real-world deraining can be found in Supplementary Figures 13–16.

R1: Response to “Insufficient Novelty”

The distinctions between our method and FreqMamba are detailed in Appendix A.9.

Here, we further clarify the novelty:

Spatial-Domain Zigzag Scanning: Unlike FreqMamba’s simple wavelet-domain scanning, FourierMamba’s zigzag scanning, inspired by JPEG encoding, orders low-to-high frequencies in the Fourier domain, enhancing correlation modeling while preserving symmetry. This integrates signal processing knowledge into Mamba’s efficient structure, synergistically improving deraining outcomes.

Channel-Domain Fourier Scanning: Discussed in Appendix A.6, this design leverages varying degradation characteristics across channels, which collectively define global image information. By introducing Fourier transforms in the channel dimension, we capture inter-channel frequency dependencies, enhancing global representation while decoupling degradation (e.g., rain streaks) from background content using amplitude and phase spectra. Joint modeling with Mamba further strengthens channel interactions and global information modeling.

Overall Novelty: FourierMamba combines spatial zigzag scanning and channel Mamba scanning to form a versatile deraining framework, outperforming MambaIR and FreqMamba. This dual-domain frequency modeling represents a significant advancement in image restoration, far beyond a mere scanning improvement.

R2: Response to “Experiments Lack Convincing Evidence”

Joint training on Rain13k demands greater network generalization, often yielding lower performance than models trained separately on individual datasets. Except for FreqMamba, results in Table 1 reflect joint training, whereas FreqMamba’s paper conflates the two approaches. For a fair comparison with FreqMamba, we provide results under identical experimental settings in Appendix Tables 11 and 16, showing improvements of 0.54 dB and 0.71 dB on Rain13k and SPA datasets, respectively, with our method.

R3: Response to “Performance Lacking Competitiveness on Certain Datasets”

AST was trained on the enhanced SPAD version[1] of the SPA dataset, making direct comparison unfair. Thus, we retrained AST on the SPA dataset using parameters from its paper, yielding the following results in Tab1. For low-light enhancement, FourierMamba targets image deraining, with experiments in this task only validating its generalizability, lacking specialized designs like MambaLLIE’s Retinex prior. Hence, it is reasonable that FourierMamba underperforms MambaLLIE in low-light enhancement. Future work will explore tailoring FourierMamba for this task.

R4: Response to “Suboptimal Inference Speed” It was noted that FourierMamba’s inference time lags behind Restormer’s. This stems from PyTorch’s extensive CUDA optimizations for attention-based operators, which Mamba currently lacks. However, our method’s FLOPS (22.5) is significantly lower than Restormer’s (174.7), suggesting that further CUDA optimization of Mamba operators will enhance inference speed. Meanwhile, compared to other Mamba-based methods, ours achieves a superior balance of inference efficiency and performance.

R5: Response to “Issues with Supplementary Material”

The review highlighted a missing table reference in Section A.10 and an incorrect citation for FADformer. We apologize for these oversights and will correct them in the revised manuscript: Section A.10 will explicitly reference Table 12, and the FADformer citation will be updated to the correct source. We appreciate your meticulous feedback.

[1] Learning Weather-General and Weather-Specific Features for Image Restoration Under Multiple Adverse Weather Conditions [CVPR 2023]

R1: Details of Zigzag Scanning Implementation and Its Impact on Inference Speed

Section 3.2 briefly outlines the motivation and design of zigzag scanning, inspired by JPEG’s zigzag encoding, to order frequencies in the Fourier domain from low to high. For the spatial Fourier spectrum, it is divided into symmetric halves, with zigzag path scanning applied to one half, arranging frequencies from center (low) to edges (high), followed by Mamba sequence modeling. The other half is derived via Fourier symmetry, ensuring orderly frequency correlation without compromising symmetry, unlike full-spectrum scanning.

As a preprocessing step, zigzag scanning’s computational cost, mainly from frequency rearrangement and Mamba modeling, is minimal. Experiments show its additional time is negligible, involving a one-time index reorder—reusable as a dictionary for same-resolution images—while Mamba’s linear complexity ensures efficiency. FourierMamba’s inference time matches MambaIR and FreqMamba, indicating no significant burden. Revised Section 3.2 will detail the implementation and confirm its minimal impact on inference speed.

R2: Comparison with FreqMamba in Frequency-Domain Correlation Strategies

We further clarify the differences and similarities between FourierMamba and FreqMamba in frequency-domain strategies. While FreqMamba employs Mamba in the frequency domain, it primarily conducts spatial scanning in the wavelet domain, underutilizing the global properties of the Fourier domain. In contrast, FourierMamba applies Mamba directly in the Fourier domain, using zigzag encoding for orderly frequency correlation, enhancing rain streak capture. FreqMamba’s wavelet-domain scanning limits its global frequency modeling capacity.

Quantitative comparisons with FreqMamba on Rain13k and SPA datasets, shown in Tables 11 and 16, reveal our method’s improvements of 0.54 dB and 0.71 dB, respectively. Feature map visualizations in Figure 7 demonstrate that our approach more effectively targets rain degradation, yielding superior deraining results.

R3: Diversity of assessment indicators

We have augmented the evaluation with no-reference metric results for the unpaired Internet-Data test set from the SPA dataset, as shown in Table 1 above.

R4: Effectiveness and Generalizability of Mamba in Processing Fourier Frequencies

Mamba, a state-space model, excels with global modeling and linear complexity, ideal for sequential data. In the Fourier domain, where frequency information is inherently global and sequential, Mamba efficiently captures inter-frequency dependencies. Traditional convolutional or Transformer architectures, however, are less efficient or computationally costly for global frequency processing, while Mamba’s linear global attention effectively combines their strengths.

Regarding generalizability, supplementary results demonstrate FourierMamba’s strong performance in low-light enhancement and dehazing on datasets like LOL-V1 and Dense-Haze, indicating robust task adaptability. For deblurring, where frequency loss is key, its correlation modeling shows promise, to be explored further in future work.

Compared to wavelet transforms, which excel in local frequency and multi-scale analysis, Fourier transforms better support global frequency representation and degradation decoupling, particularly for deraining, where rain streaks are distinctly separable in the frequency domain. Thus, we prioritize Fourier transforms, but future work will explore wavelet integration. These points will be detailed in Sections 2 and 5 of the revised manuscript to fully address Mamba’s effectiveness and the method’s generalizability.

R5: Related Works Omissions

We appreciate your feedback and will include discussions of [1] and [2] in the revised manuscript. While [1] explores cross-modal fusion with Mamba for image restoration, our work focuses on frequency-domain modeling. Similarly, [2] employs linear models for weather degradation, which aligns partially with our approach. We will clarify these connections in Section 2 to better position our contributions.

[1] Cross-Modality Fusion Mamba for All-in-One Extreme Weather-Degraded Image Restoration, 2025. [2] Restoring images in adverse weather conditions via histogram transformer, 2024.

R1: The Necessity of Channel-Dimensional Fourier Transform

In Appendix A.6 of the paper, titled "Reasons for Using Channel-Dimensional Fourier," we note that different channels typically exhibit distinct degradation characteristics, which collectively determine the global information of an image when integrated across channels. This observation draws inspiration from style transfer research, such as the use of the Gram matrix to represent global style information [1]. Building on this insight, we introduce the Fourier transform in the channel dimension to enhance the representation of global information by capturing frequency dependencies across channels. Directly scanning the sequence (e.g., using Mamba) fails to effectively leverage the global properties inherent in the Fourier domain. In contrast, the channel-dimensional Fourier transform enables the decoupling of degradation information (e.g., rain streaks) from background content, thereby improving Mamba’s capacity to model frequency correlations. The effectiveness of this design is substantiated by ablation studies presented in Table 3 and Figure 17, which provide quantitative metrics and visualizations, respectively, demonstrating its impact.

R2: Ensuring Normal Spatial Features After Inverse Transformation from Fourier Space Scanning

You raised the concern that the paper should discuss how normal spatial features are preserved through inverse transformation following Fourier space scanning. We acknowledge that the current discussion on this aspect is insufficiently detailed and will address this in the revised manuscript by supplementing relevant content. The key to FourierMamba lies in its scanning strategy, which is designed to preserve the symmetry and global properties of the Fourier domain. Specifically, in the spatial dimension, we employ zigzag coding for scanning and process only half of the spectrum, leveraging the symmetry of the Fourier transform (amplitude centro-symmetry and phase anti-centro-symmetry) to deduce the other half (see Section 3.2). This approach ensures the integrity of the Fourier domain information. In the channel dimension, Mamba scanning is performed on a one-dimensional Fourier spectrum, adhering to similar symmetry principles. The inverse transformation relies on the standard Inverse Fast Fourier Transform (IFFT) algorithm, which theoretically guarantees perfect reconstruction from the Fourier domain to the spatial domain, provided that the scanning operation does not introduce irreversible information loss. Our experimental results (e.g., Figures 5 and 9) demonstrate that the inverse-transformed images retain normal spatial features, such as textures and details, owing to the orderly nature of the scanning design and the preservation of symmetry. We will enhance Sections 3.2 and 3.3 with these clarifications and may include a mathematical derivation in the appendix to further elucidate this process.

R3: Formatting Issues

We will address and correct spelling errors in the revised manuscript.

[1] Li, Y., Fang, C., Yang, J., Wang, Z., Lu, X., and Yang, M.-H.Universal style transfer via feature transforms. Advances in neural information processing systems, 30, 2017.