WKV-sharing embraced random shuffle RWKV high-order modeling for pan-sharpening

1, optimization loss.

The supplementary validation specifically assesses the efficacy of the random manifold loss against Fourier-transform and wavelet-based fixed-point losses under equivalent L1 constraints. Reference metrics demonstrate the proposed loss function's superior performance: while the L1 Fourier configuration achieves 41.9645 PSNR and 0.7823 Q_deep, and L1 Wavelet attains 41.9713 PSNR and 0.7821 Q_deep, the random manifold approach yields substantially enhanced results across critical fusion metrics. This empirical advantage stems from the fundamental distinction where traditional fixed-basis losses employ rigid orthogonal transformations that artificially constrain error distributions, whereas the random manifold framework dynamically decomposes errors through stochastic weight optimization without predefined spectral segmentation. The demonstrated performance differential - particularly improvement over Fourier-transform and versus wavelet-based alternatives - quantitatively validates the proposed method's capacity for more physically consistent optimization that intrinsically preserves both contextual information and spatial details.

Reference Metrics Evaluation

No Reference Metrics Evaluation

2, arrangement.

We include select formulas in the introduction to help readers navigate the complex theoretical framework developed in later sections. These equations provide a "conceptual map" that previews core ideas like the random weight manifold loss and Bayesian formulation, making the subsequent technical discussion more accessible. We recognize the reviewer's valid concern about repetition between sections and will streamline the introduction to retain only essential foundational formulas while moving detailed derivations to the methodology section. By providing early exposure to core operators and comparative formulations, we reduce cognitive barriers to later theoretical developments in Sections 3-4. We acknowledge the reviewer's valid concern regarding redundancy and will streamline this presentation by retaining only critical notation-defining equations while relocating derivations to methodological sections and adding explicit signposting to enhance conceptual flow without compromising necessary scaffolding for complex subsequent content.

First of all, we sincerely appreciate your recognition of our work's ​​novelty, efficacy, and completeness​​. Below are point-by-point responses to your review comments.

1, high-order.

The essence of high-order modeling lies in reinterpreting neural network stacking as functional composition – where successive layers form composite functions $f_L \circ \cdots \circ f_1(x)$ that progressively build higher-order representations. Traditional approaches stack multiple RWKV layers to achieve this, but incur $O(L \cdot M^2)$ complexity. Our key insight recognizes that RWKV's first-order property (capturing local feature interactions) can be augmented by strategically replacing subsequent layers with learnable 1D convolutions. These convolutional operations, while mathematically equivalent to Toeplitz matrix multiplication $Cx$, provide two critical advantages: (1). Computational efficiency ($O(M \log M)$ vs RWKV's $O(M^2)$) (2). Kernel fusion via associativity:
$(C_L \cdots C_1) x = C_L(C_{L-1} \cdots (C_1 x))$ allows absorption into RWKV's parameters through weight sharing.

The transformation occurs through three conceptual steps:

$ \text{Stacked RWKV} \quad & \rightarrow \quad \text{RWKV} + \text{Conv Layers} \\ & \downarrow \\ \text{Matrix equivalence} \quad & \rightarrow \quad W_{\text{total}} = W_{\text{RWKV}} \cdot \prod_{i} C_i \\ & \downarrow \\ \text{Parameter fusion} \quad & \rightarrow \quad W_{\text{shared}} = \sigma(W_{\text{RWKV}} \oplus \mathcal{F}( \{C_i\} )) $

where $\mathcal{F}$ denotes kernel fusion via associativity and $\oplus$ is parameter concatenation. This achieves:

• High-order capability: $W_{\text{shared}}^L x$ captures $L$-th order interactions
• Collapse prevention: Spectral constraint $\rho(W_{\text{shared}}) < 1$ ensures stability
• Complexity reduction: $O(M^2)$ vs stacked RWKV's $O(L\cdot M^2)$

This approach brilliantly rethinks depth not as sequential processing but as spectral expansion. Where traditional stacking builds depth through temporal computation (layer-by-layer processing), our method compresses depth into learned spectral components of a single transformation. The shared weights $W_{\text{shared}}$ essentially become a "depth compiler" – encoding what would be $L$ layers of computation into the eigenvalues and eigenvectors of a single matrix. This explains why $W^3x$ can capture shadow-texture-material relationships in urban imagery that normally require 3+ layers: the cubic expansion $(W-I)^3x$ directly models third-order feature covariances that emerge in multi-scale remote sensing data.

2, manifold loss.

The Random Weight Manifold Loss employs Taylor's unfolding manifold to intrinsically address the dual optimization objectives in pan-sharpening: preserving spectral signatures while enhancing spatial textures. Its core intuition stems from how Taylor series expansion naturally decomposes image characteristics – the zeroth-order terms maintain coarse-grained spectral consistency (critical for material identification), while first-order gradients capture fine-grained spatial structures (essential for texture details). As validated through our supplementary experiments comparing Fourier-transform and wavelet-based alternatives. As quantitatively demonstrated (Reference Metrics: L1+Wavelet PSNR=41.9713; Proposed PSNR=42.0945), Random Weight Manifold Loss achieves PSNR improvement over state-of-the-art frequency domain losses while simultaneously enhancing spectral-spatial correlation (SCC=0.9772 vs 0.9769). This advantage stems from three core mechanisms: Taylor decomposition intrinsically separates spectral consistency (preserved in 0th-order terms) from texture details (encoded in 1st-order gradients), creating adaptive regularization that dynamically responds to scene characteristics - unlike Fourier/wavelet's rigid frequency bands that introduce spectral distortions (SAM=0.0214 vs wavelet's 0.0216 despite superior PSNR). Crucially, Bayesian weight randomization expands the manifold's constraint boundaries to suppress two critical artifacts: 1) Nonlinear spectral distortions , and 2) spatial distoration, while preserving irregular textures that LPIPS oversimplifies due to perceptual uniformity bias. This physics-aware design - representing imaging modeling decompostion as linearly interpolatable manifold points - combined with stochastic weight sampling, yields the favorable results through optimal spectral-textural balance unattainable by fixed-basis losses.

Reference Metrics Evaluation

No Reference Metrics Evaluation

3, sampling.

Based on the stability analysis shown in Figure 9, we provide the following clarification regarding Monte Carlo (MC) sampling in our design. Key observation: Figure 9 demonstrates remarkable PSNR stability (fluctuations < 0.0006 dB) across 1→30 MC samples. The PSNR curve converges immediately to 42.0945 dB (variance < 0.0002 dB), empirically validating that increasing sample count yields clinically insignificant gains. Theoretical justification: The exceptional stability of our single-sample Monte Carlo estimation stems from two synergistic design principles. During inference, our Bayesian parameterization (Eq. 11) deterministically encodes the full posterior distribution into model weights, while randomized shuffle scanning (Section 3.2) trains the network to intrinsically capture ensemble expectations within a single forward pass. This achieves principled "implicit averaging" (Section 4.3) because attention map expectations computed by our WKV module exhibit minimal variation across stochastic paths – a property amplified by remote sensing imagery's intrinsic spatial homogeneity. The high pattern consistency in RS data (e.g., repetitive urban structures or agricultural textures) ensures attention maps converge to near-identical configurations regardless of scan order. Consequently, as Figure 9 demonstrates, the PSNR plateaus at 42.0945 dB after just 1-2 samples with clinically negligible fluctuations (ΔPSNR < 0.0003 dB). This domain-specific stability renders explicit multi-sample averaging redundant, allowing our approach to maintain theoretical rigor while eliminating 87.5% of computational overhead through optimized single-path inference.

4, details.

TheMapping() MLPs employ similar Vision-RWKV's gated MLP architecture, featuring a standardized configuration: two linear layers with fixed 4× expansion (hidden dimension = 4× input dimension), GeLU activation, and LayerNorm preceding each mapping operation. This design—identical to Equation 3 in Vision-RWKV—ensures architectural consistency. For the loss-balancing hyperparameter λ, both architectural implementation and hyperparameter settings are fully reproducible through our will-release source code.

1, about MC sampling.

Based on the stability analysis shown in Figure 9, we provide the following clarification regarding MC sampling in our design. Figure 9 demonstrates remarkable PSNR stability (fluctuations < 0.0006 dB) across 1→30 MC samples. The PSNR curve converges immediately to 42.0945 dB (variance < 0.0002 dB), empirically validating that increasing sample count yields clinically insignificant gains. The exceptional stability of our single-sample Monte Carlo estimation stems from two synergistic design principles. During inference, our Bayesian parameterization (Eq. 11) deterministically encodes the full posterior distribution into model weights, while randomized shuffle scanning (Section 3.2) trains the network to intrinsically capture ensemble expectations within a single forward pass. This achieves principled "implicit averaging" (Section 4.3) because attention map expectations computed by our WKV module exhibit minimal variation across stochastic paths – a property amplified by remote sensing imagery's intrinsic spatial homogeneity. The high pattern consistency in RS data (e.g., repetitive urban structures or agricultural textures) ensures attention maps converge to near-identical configurations regardless of scan order. Consequently, as Figure 9 demonstrates, the PSNR plateaus at 42.0945 dB after just 1-2 samples with clinically negligible fluctuations (ΔPSNR < 0.0003 dB). This domain-specific stability renders explicit multi-sample averaging redundant, allowing our approach to maintain theoretical rigor while eliminating 87.5% of computational overhead through optimized single-path inference.

2, about shuffle order.

Our method applies identical random shuffling patterns to the (R, K, V) components during training – a design choice grounded in the RWKV architecture's intrinsic capacity for learning order-agnostic representations. Over successive training epochs, the persistent reshuffling mechanism exposes all components to the maximal diversity of distinct scanning sequences, with identical patterns ensuring synchronized exposure to each permutation state. Over multiple epochs, repeated randomization ensures each component experiences diverse scanning orders—whether implemented identically or independently—ultimately converging to the same representational quality. This equivalence occurs because: (1) Bayesian weight manifolds intrinsically capture relationship invariance across scanning variants; (2) Remote sensing imagery's spatial homogeneity reduces sensitivity to component-specific scanning divergences. The identical pattern strategy simplifies implementation while preserving theoretical integrity. This coordinated randomization forces the model to disentangle feature relationships from sequential biases, while the Bayesian parameterization (Eq. 11) inherently collapses permutation variants into functionally equivalent representations. Crucially, remote sensing imagery's spatial homogeneity – characterized by repetitive structures (e.g., urban grids, agricultural textures) and spectral consistency – naturally suppresses sensitivity to component-specific scanning divergences. These domain-specific regularities, combined with the extended training exposure to diverse synchronization patterns, theoretically guarantee that identical shuffling yields identical functional convergence to independent permutation strategies.

1，about w and u.

Parameter w serves as a scaling factor in WKV attention calculation, adjusting the relative contribution of different tokens to the overall attention. It allows the model to control the strength of interactions between tokens, thus enabling more flexible learning of dependencies. Parameter u introduces a bias in the WKV attention score computation, influencing how tokens are weighted relative to one another. It helps the model emphasize certain tokens over others, depending on their relevance to the task. These parameters are learned during training to optimize WKV attention mechanism's behavior, and they are vital for improving the network's ability to model complex dependencies across sequences, enhancing the network's capacity to capture both local and long-range dependencies.

2, about shuflle and inverse shuffle.

(1) Motivation for Shuffle and Inverse Shuffle: The primary motivation behind using shuffle and inverse shuffle in Random Shuffle RWKV mechanism is to effectively model global attention over the entire input space while overcoming the limitations posed by traditional fixed-order attention calculation. In traditional RWKV mechanisms, attention is calculated with a fixed order, typically following a bidirectional scanning strategy. This fixed order introduces biases depending on the sequence's processing direction. When scanning the sequence row-by-row or column-by-column, the model tends to focus on local dependencies first, potentially overlooking global relationships. This fixed-order calculation can also lead to redundant computations and introduce biases when attempting to model long-range dependencies. The ideal WKV attention calculation should consider all possible sequence orderings to capture global dependencies effectively. However, in traditional RWKV, the global context is not fully explored due to the fixed scan order. The shuffle operation addresses this by randomly permuting the sequence, ensuring that the model does not make biased assumptions based on sequence order. By applying the shuffle, the model can explore a complete space of possible token arrangements, enabling it to learn global relationships without being constrained by the fixed order of the input. This approach is akin to MC sampling, where randomness is introduced to prevent the model from overfitting to specific patterns that arise due to a fixed order. In this way, shuffling ensures a more comprehensive global attention by guaranteeing that all possible sequence permutations are represented, achieving theoretical completeness in the attention calculation.

(2) Importance of Inverse Shuffle for Information Processing: While shuffle helps mitigate biases and redundant computations in WKV attention modeling, the final output of the attention mechanism must be mapped back to the original two-dimensional spatial layout. This is important because although WKV attention mechanism works as a filter to refine information, the spatial relationships between features in the input image (such as the relative positions of objects) must be preserved for the network to make meaningful predictions. The inverse shuffle plays a crucial role here. It restores the original layout of the image, ensuring that spatial prior information remains intact. After WKV attention mechanism computes the attention scores and refines the features, the inverse shuffle guarantees that the transformed features are re-mapped to the correct positions in the spatial grid. This is particularly important for image-related tasks, where the relative positioning of objects in the image is essential for accurate reconstruction. For example, consider an image where the top-left corner contains a house and the bottom-right corner contains a mountain. After shuffling, the feature relationships may become mixed, but with the inverse shuffle, the model restores the original spatial configuration, ensuring that the house remains in the top-left corner and the mountain stays in the bottom-right corner. This preservation of spatial distribution is essential for maintaining the integrity of the image layout.

(3) Theoretical Completeness: The shuffle mechanism, combined with inverse shuffle, ensures that the attention model explores the full range of possible feature interactions, without being limited by the inherent biases of fixed-order processing. This approach addresses the redundancy and inefficiency found in traditional RWKV. However, while the shuffle and inverse shuffle mechanism’s main motivation is to explore the full feature space globally, its most significant benefit lies in the preservation of spatial relationships through the inverse shuffle. This allows the model to achieve the best of both worlds: the completeness of the shuffled attention calculation and the spatial consistency ensured by the inverse shuffle.

In summary, the inverse shuffle is not just about preserving information invariance; its main purpose is to maintain the additivity of spatial information in the context of image processing. This ensures that, after WKV attention mechanism processes the features, the restored features are correctly positioned within the image grid, respecting the original layout and maintaining the spatial distribution of objects. Without this step, the model would struggle to make sense of the refined features within the context of the original image structure.

3, about WKV sharing.

The following supplementary experiment compares the training performance with and without KV sharing. Based on the experiment, KV sharing does not have a significant impact on convergence speed in terms of training time per epoch. However, there is a slight increase in total training time due to the extra epochs, the slight increase in total training time with KV sharing suggests that the additional epochs might be contributing to a more stable or refined convergence.

4, about manifold loss.

Our supplementary experiments demonstrates that the introduction of random manifold loss requires identical 500 training epochs to achieve full convergence as the baseline without random manifold loss. The added computational overhead manifests solely in modest per-epoch processing time - increasing from 20 seconds to 26 seconds per epoch due to the loss term's calculation. This translates to a manageable increase in total training duration, while preserving identical convergence iteration requirements. Crucially, this trade-off delivers significant performance benefits as documented in our Table 3. The marginal training time penalty is strategically justified given that: (1) random manifold loss’ computational contribution remains negligible at scale (constituting <0.1% of per-epoch operations); (2) It introduces zero overhead during deployment since random manifold loss is exclusively a training auxiliary module; and (3) The community prioritizes inference efficiency and final accuracy over marginal training extensions, with random manifold loss leaving inference architecture and latency completely unaffected. We maintain that modest training cost increases are acceptable for substantial performance gains.

5, noisy conditions.

While conventional pan-sharpening evaluation typically focuses on imagery acquired under standard World protocols (which inherently represent real-world imaging physics and sensor characteristics), we have now conducted supplemental experiments to specifically address low-SNR scenarios at the reviewer's suggestion. Due to time constraints and considering that noisy condition validation is not standard practice in mainstream pan-sharpening research, we restricted our analysis to two representative SNR levels (10 and 20). Following hyperspectral community protocols, we introduced additive Gaussian noise to multispectral imagery at these SNR levels. The results unequivocally demonstrate our method's superior robustness. At both noisy SNR conditions (detailed metrics provided in the table below), our approach achieves state-of-the-art performance across most critical metrics.

SNR = 10

SNR = 20

6, future discussion.

We will expand the Limitations section to address training efficiency, noise resilience, and hardware adaptability. Our supplemental noise experiments above further confirm inherent robustness. As you suggested, we will discuss lightweight optimizations as potential future work.

Dear Reviewer kXdp,

Thanks a lot for your contributions. Would you please send in your final recommendations? I think the authors could still have time to answer your questions, if you have.

Best, AC